--------------------------------------------------------------------------------
/config/initializers/inflections.rb:
--------------------------------------------------------------------------------
1 | # Be sure to restart your server when you modify this file.
2 |
3 | # Add new inflection rules using the following format. Inflections
4 | # are locale specific, and you may define rules for as many different
5 | # locales as you wish. All of these examples are active by default:
6 | # ActiveSupport::Inflector.inflections(:en) do |inflect|
7 | # inflect.plural /^(ox)$/i, "\\1en"
8 | # inflect.singular /^(ox)en/i, "\\1"
9 | # inflect.irregular "person", "people"
10 | # inflect.uncountable %w( fish sheep )
11 | # end
12 |
13 | # These inflection rules are supported but not enabled by default:
14 | # ActiveSupport::Inflector.inflections(:en) do |inflect|
15 | # inflect.acronym "RESTful"
16 | # end
17 |
--------------------------------------------------------------------------------
/app/assets/stylesheets/application.css:
--------------------------------------------------------------------------------
1 | /*
2 | * This is a manifest file that'll be compiled into application.css, which will include all the files
3 | * listed below.
4 | *
5 | * Any CSS (and SCSS, if configured) file within this directory, lib/assets/stylesheets, or any plugin's
6 | * vendor/assets/stylesheets directory can be referenced here using a relative path.
7 | *
8 | * You're free to add application-wide styles to this file and they'll appear at the bottom of the
9 | * compiled file so the styles you add here take precedence over styles defined in any other CSS
10 | * files in this directory. Styles in this file should be added after the last require_* statement.
11 | * It is generally better to create a new file per style scope.
12 | *
13 | *= require_tree .
14 | *= require_self
15 | */
16 |
--------------------------------------------------------------------------------
/test/models/comment_test.rb:
--------------------------------------------------------------------------------
1 | require "test_helper"
2 |
3 | class CommentTest < ActiveSupport::TestCase
4 | test "belongs to post" do
5 | comment = Comment.new(post: nil)
6 | comment.valid?
7 | assert_includes comment.errors[:post], "must exist"
8 | end
9 |
10 | test "belongs to user" do
11 | comment = Comment.new(user: nil)
12 | comment.valid?
13 | assert_includes comment.errors[:user], "must exist"
14 | end
15 |
16 | test "body cannot be nil" do
17 | comment = Comment.new(body: nil)
18 | comment.valid?
19 | assert_includes comment.errors[:body], "can't be blank"
20 | end
21 |
22 | test "body must be more than 5 characters" do
23 | comment = Comment.new(body: "1234")
24 | comment.valid?
25 | assert_includes comment.errors[:body], "is too short (minimum is 5 characters)"
26 | end
27 | end
28 |
--------------------------------------------------------------------------------
/test/models/post_test.rb:
--------------------------------------------------------------------------------
1 | require "test_helper"
2 |
3 | class PostTest < ActiveSupport::TestCase
4 | test "belongs to user" do
5 | post = Post.new(user: nil)
6 | post.valid?
7 | assert_includes post.errors[:user], "must exist"
8 | end
9 |
10 | test "title cannot be nil" do
11 | post = Post.new(title: nil)
12 | post.valid?
13 | assert_includes post.errors[:title], "can't be blank"
14 | end
15 |
16 | test "title must be unique" do
17 | one = posts(:one)
18 | post = Post.new(title: one.title)
19 | post.valid?
20 | assert_includes post.errors[:title], "has already been taken"
21 | end
22 |
23 | test "title must be more than 5 characters" do
24 | post = Post.new(title: "1234")
25 | post.valid?
26 | assert_includes post.errors[:title], "is too short (minimum is 5 characters)"
27 | end
28 | end
29 |
--------------------------------------------------------------------------------
/app/views/comments/edit.html.erb:
--------------------------------------------------------------------------------
1 |
38 | <% end %>
39 |
--------------------------------------------------------------------------------
/config/puma.rb:
--------------------------------------------------------------------------------
1 | # This configuration file will be evaluated by Puma. The top-level methods that
2 | # are invoked here are part of Puma's configuration DSL. For more information
3 | # about methods provided by the DSL, see https://puma.io/puma/Puma/DSL.html.
4 |
5 | # Puma starts a configurable number of processes (workers) and each process
6 | # serves each request in a thread from an internal thread pool.
7 | #
8 | # The ideal number of threads per worker depends both on how much time the
9 | # application spends waiting for IO operations and on how much you wish to
10 | # to prioritize throughput over latency.
11 | #
12 | # As a rule of thumb, increasing the number of threads will increase how much
13 | # traffic a given process can handle (throughput), but due to CRuby's
14 | # Global VM Lock (GVL) it has diminishing returns and will degrade the
15 | # response time (latency) of the application.
16 | #
17 | # The default is set to 3 threads as it's deemed a decent compromise between
18 | # throughput and latency for the average Rails application.
19 | #
20 | # Any libraries that use a connection pool or another resource pool should
21 | # be configured to provide at least as many connections as the number of
22 | # threads. This includes Active Record's `pool` parameter in `database.yml`.
23 | threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)
24 | threads threads_count, threads_count
25 |
26 | # Specifies the `port` that Puma will listen on to receive requests; default is 3000.
27 | port ENV.fetch("PORT", 3000)
28 |
29 | # Allow puma to be restarted by `bin/rails restart` command.
30 | plugin :tmp_restart
31 |
32 | # Specify the PID file. Defaults to tmp/pids/server.pid in development.
33 | # In other environments, only set the PID file if requested.
34 | pidfile ENV["PIDFILE"] if ENV["PIDFILE"]
35 |
--------------------------------------------------------------------------------
/app/controllers/concerns/authentication.rb:
--------------------------------------------------------------------------------
1 | module Authentication
2 | extend ActiveSupport::Concern
3 |
4 | included do
5 | before_action :require_authentication
6 | helper_method :authenticated?
7 | end
8 |
9 | class_methods do
10 | def allow_unauthenticated_access(**options)
11 | skip_before_action :require_authentication, **options
12 | before_action :resume_session, **options
13 | end
14 | end
15 |
16 | private
17 | def authenticated?
18 | Current.session.present?
19 | end
20 |
21 | def require_authentication
22 | resume_session || request_authentication
23 | end
24 |
25 | def resume_session
26 | if session = find_session_by_cookie
27 | set_current_session session
28 | end
29 | end
30 |
31 | def find_session_by_cookie
32 | if token = cookies.signed[Session::COOKIE_KEY]
33 | Session.find_signed(token)
34 | end
35 | end
36 |
37 | def request_authentication
38 | session[:return_to_after_authenticating] = request.url
39 | redirect_to new_session_url
40 | end
41 |
42 | def after_authentication_url
43 | session.delete(:return_to_after_authenticating) || user_path(Current.user)
44 | end
45 |
46 | def start_new_session_for(user)
47 | user.sessions.create!(user_agent: request.user_agent, ip_address: request.remote_ip).tap do |session|
48 | set_current_session session
49 | end
50 | end
51 |
52 | def set_current_session(session)
53 | Current.session = session
54 | cookies.signed.permanent[Session::COOKIE_KEY] = { value: session.signed_id, httponly: true, same_site: :lax }
55 | end
56 |
57 | def terminate_session
58 | Current.session.destroy
59 | cookies.delete(Session::COOKIE_KEY)
60 | end
61 | end
--------------------------------------------------------------------------------
/db/seeds.rb:
--------------------------------------------------------------------------------
1 | # This file should ensure the existence of records required to run the application in every environment (production,
2 | # development, test). The code here should be idempotent so that it can be executed at any point in every environment.
3 | # The data can then be loaded with the bin/rails db:seed command (or created alongside the database with db:setup).
4 |
5 | User.insert_all(
6 | 100.times.map do |n|
7 | { screen_name: [ Faker::Lorem.word, n.to_s.rjust(3, '0') ].join("_"),
8 | password_digest: BCrypt::Password.create("password", cost: 1),
9 | about: Faker::Lorem.paragraph,
10 | created_at: Faker::Time.between(from: 1.week.ago, to: DateTime.now) }
11 | end
12 | )
13 | user_ids_and_created_ats = User.pluck(:id, :created_at)
14 | 1_000.times do |n|
15 | user_id, created_at = user_ids_and_created_ats.sample
16 | results = Post.insert({
17 | user_id: user_id,
18 | title: Faker::Lorem.words(number: rand(2..5)).join(" "),
19 | content: rand(5..10).times.collect { rand(5..10).times.collect { Faker::Lorem.sentence(word_count: rand(5..10)) }.join(" ") }.join("\n\n"),
20 | created_at: Faker::Time.between(from: created_at, to: DateTime.now)
21 | }, returning: [ :id, :created_at ])
22 |
23 | next unless results.present?
24 |
25 | result = results.to_a[0]
26 | rand(5..15).times do |nn|
27 | timestamp = Faker::Time.between(from: result["created_at"], to: DateTime.now)
28 | Comment.insert({
29 | post_id: result["id"],
30 | user_id: user_ids_and_created_ats.sample.first,
31 | body: Faker::Lorem.paragraph,
32 | created_at: timestamp,
33 | updated_at: timestamp
34 | })
35 | end
36 | end
37 |
38 | Post.pluck(:id).each { |post_id| Post.reset_counters(post_id, :comments) }
39 | User.pluck(:id).each { |user_id| User.reset_counters(user_id, :posts) }
40 |
--------------------------------------------------------------------------------
/app/controllers/comments_controller.rb:
--------------------------------------------------------------------------------
1 | class CommentsController < ApplicationController
2 | include ActionView::RecordIdentifier
3 |
4 | # ----- authenticated actions -----
5 | before_action :set_post, only: %i[ create ]
6 | before_action :set_and_authorize_comment, only: %i[ edit update destroy ]
7 |
8 | # GET /comments/1/edit
9 | def edit
10 | end
11 |
12 | # POST /posts/:post_id/comments
13 | def create
14 | @comment = @post.comments.new(comment_params)
15 |
16 | if @comment.save
17 | redirect_to post_path(@comment.post, anchor: dom_id(@comment)), notice: "Comment was successfully created."
18 | else
19 | @post = @comment.post
20 | render "posts/show", status: :unprocessable_entity
21 | end
22 | end
23 |
24 | # PATCH/PUT /comments/1
25 | def update
26 | if @comment.update(comment_params)
27 | redirect_to post_path(@comment.post, anchor: dom_id(@comment)), notice: "Comment was successfully updated.", status: :see_other
28 | else
29 | render :edit, status: :unprocessable_entity
30 | end
31 | end
32 |
33 | # DELETE /comments/1
34 | def destroy
35 | @comment.destroy!
36 | redirect_to post_path(@comment.post), notice: "Comment was successfully destroyed.", status: :see_other
37 | end
38 |
39 | private
40 | # Use callbacks to share common setup or constraints between actions.
41 | def set_post
42 | @post = Post.find(params[:post_id])
43 | end
44 |
45 | def set_and_authorize_comment
46 | @comment = Comment.find(params[:id])
47 | raise ApplicationController::NotAuthorized, "not allowed to #{action_name} this comment" unless @comment.user == Current.user
48 | end
49 |
50 | # Only allow a list of trusted parameters through.
51 | def comment_params
52 | params.require(:comment).permit(:body).merge(user_id: Current.user.id)
53 | end
54 | end
55 |
--------------------------------------------------------------------------------
/test/fixtures/posts.yml:
--------------------------------------------------------------------------------
1 | # Read about fixtures at https://api.rubyonrails.org/classes/ActiveRecord/FixtureSet.html
2 |
3 | one:
4 | user: one
5 | title: Voluptatem id sequi cupiditate dignissimos
6 | content: Curabitur cursus semper condimentum. Mauris quis tincidunt magna. Nulla suscipit, ex at rhoncus consectetur, eros lectus auctor arcu, non consequat arcu eros quis tellus. Nunc posuere erat vel nisl eleifend, ac faucibus ipsum molestie. Integer venenatis, velit quis lacinia vestibulum, lacus magna sagittis diam, vel dignissim neque purus quis eros. Ut eu mi eu risus placerat porttitor. Aenean vel lacus eros. Duis placerat gravida arcu elementum viverra. Sed dapibus tellus sed eros efficitur, vitae semper dui eleifend. Phasellus aliquet euismod enim vitae rutrum. Nullam pharetra rutrum eros ac hendrerit. Etiam vel sem eu dui viverra tincidunt sit amet eu libero. Integer vitae sapien ex. Proin luctus leo cursus feugiat euismod. Nulla a nibh vitae quam blandit elementum.
7 | comments_count: 0
8 |
9 | two:
10 | user: two
11 | title: Assumenda sed
12 | content: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras nibh neque, tempus non varius quis, imperdiet consectetur massa. Maecenas non faucibus eros, sed vulputate quam. Aenean quis ipsum ultrices, commodo dui sit amet, interdum purus. Suspendisse sit amet condimentum massa, at faucibus purus. Praesent quis ex et purus ultricies dictum. Phasellus eu tempus erat. Sed interdum, risus pulvinar consequat malesuada, felis libero tristique neque, at sollicitudin libero quam sit amet sem. Donec at libero aliquam arcu rhoncus dictum sit amet et leo. Vestibulum aliquam rutrum dolor, vitae cursus nisl facilisis eu. Curabitur ut mi rhoncus arcu tristique elementum ac mattis sem. Pellentesque id mattis augue, nec euismod nisl. Donec tristique id ex non gravida. Nulla eget suscipit lectus, non dignissim dui.
13 | comments_count: 0
14 |
--------------------------------------------------------------------------------
/public/406-unsupported-browser.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 | Your browser is not supported (406)
5 |
6 |
55 |
56 |
57 |
58 |
59 |
59 | <% end %>
60 |
61 | <% if alert.present? %>
62 |
63 | <%= alert.html_safe %>
64 |
65 | <% end %>
66 |
67 |
68 |
69 |
--------------------------------------------------------------------------------
/workshop/08-adding-solid-cache.md:
--------------------------------------------------------------------------------
1 | ## Adding Solid Cache
2 |
3 | In addition to a job backend, a full-featured Rails application often needs a cache backend. Modern Rails provides a database-backed default cache store named [Solid Cache](https://github.com/rails/solid_cache).
4 |
5 | ### The Implementation
6 |
7 | We install it like any other gem:
8 |
9 | ```sh
10 | bundle add solid_cache
11 | ```
12 |
13 | Like with Solid Queue, we need to configure Solid Cache to use a separate database. We can do this by adding a new database configuration to our `config/database.yml` file:
14 |
15 | ```yaml
16 | cache: &cache
17 | <<: *default
18 | migrations_paths: db/cache_migrate
19 | database: storage/<%= Rails.env %>-cache.sqlite3
20 | ```
21 |
22 | We then need to ensure that our `production` environment uses this `cache` database, like so:
23 |
24 | ```yaml
25 | production:
26 | primary:
27 | <<: *default
28 | database: storage/production.sqlite3
29 | queue: *queue
30 | cache: *cache
31 | ```
32 |
33 | With the new cache database configured, we can install Solid Cache into our application with that database
34 |
35 | ```sh
36 | RAILS_ENV=production DATABASE=cache bin/rails solid_cache:install
37 | ```
38 |
39 | This will create the migration files in the `db/cache_migrate` directory and set Solid Cache to be the production cache store. You should see output like:
40 |
41 | ```
42 | gsub config/environments/production.rb
43 | create config/solid_cache.yml
44 | gsub config/database.yml
45 | create db/cache_schema.rb
46 | ```
47 |
48 | Once installed, we can then run the load the generated schema like so:
49 |
50 | ```sh
51 | RAILS_ENV=production rails db:prepare DATABASE=cache
52 | ```
53 |
54 | > [!NOTE]
55 | > We are doing all of this in the `production` environment for this workshop. If you were setting up Solid Cache in `development`, you would also need to enable the cache in the `development` environment. This is done by running the `dev:cache` task:
56 | >```sh
57 | >bin/rails dev:cache
58 | >```
59 | > You want to see the following output:
60 | >```
61 | >Development mode is now being cached.
62 | >```
63 |
64 | With Solid Cache enabled for the `production` environment, we can finally ensure that Solid Cache itself will use our new cache database. Luckily, the new default, is that Solid Cache expects you to a separate `cache` database. And this is precisely what we have setup. Check the configuration file at `config/solid_cache.yml` and ensure it looks like this:
65 |
66 | ```yaml
67 | default: &default
68 | database: cache
69 | store_options:
70 | # Cap age of oldest cache entry to fulfill retention policies
71 | # max_age: <%= 60.days.to_i %>
72 | max_size: <%= 256.megabytes %>
73 | namespace: <%= Rails.env %>
74 |
75 | development:
76 | <<: *default
77 |
78 | test:
79 | <<: *default
80 |
81 | production:
82 | <<: *default
83 | ```
84 |
85 | ### Using Solid Cache
86 |
87 | With Solid Cache now fully integrated into our application, we can use it like any other Rails cache store. Let's confirm that everything is working as expecting by opening the Rails console:
88 |
89 | ```sh
90 | RAILS_ENV=production bin/rails console
91 | ```
92 |
93 | write to the `Rails.cache` object:
94 |
95 | ```ruby
96 | Rails.cache.write(:key, "value")
97 | ```
98 |
99 | If we then read that key back from the cache:
100 |
101 | ```ruby
102 | Rails.cache.read(:key)
103 | ```
104 |
105 | You should see the value `"value"` returned.
106 |
107 | You can confirm that this entry was stored in the Solid Cache database by checking:
108 |
109 | ```ruby
110 | SolidCache::Entry.count
111 | ```
112 |
113 | and also:
114 |
115 | ```ruby
116 | SolidCache::Entry.first.attributes
117 | ```
118 |
119 | This output will confirm that Solid Cache is working as expected!
120 |
121 | With caching now enabled in our application, we can use Solid Cache to cache expensive operations, such as database queries, API calls, or view partials, to improve the performance of our application.
122 |
123 | We can cache the rendering of the posts partial in the `posts/index.html.erb` view like so:
124 |
125 | ```erb
126 |
127 | <% cache post do %>
128 | <%= render post %>
129 | <% end %>
130 |
131 | ```
132 |
133 | ### Conclusion
134 |
135 | In this step, we added the Solid Cache gem, backed by a separate SQLite database, to our application. With Solid Cache installed and setup, the next step is to consider how to enhance SQLite with extensions.
136 |
--------------------------------------------------------------------------------
/workshop/04-simplifying-fixes.md:
--------------------------------------------------------------------------------
1 | ## Simplifying Fixes
2 |
3 | In the last two steps, we have addressed the two major issues that we identified in our baseline load tests. We have resolved the `500` error responses by opening _immediate_ transactions in our `post_create` action. We have also resolved the 5+ second responses by ensuring that Ruby's GVL is released when queries are waiting to retry to acquire SQLite's write lock.
4 |
5 | Neither fix is profoundly combersome, and I wanted to ensure that you were comfortable understanding the problems and the solutions (especially _why_ the solutions work). But, you don't actually need to manually make these changes in your Rails application. You can simply add a gem to your project that will automatically make these changes for you.
6 |
7 | ### The Solution
8 |
9 | Instead of manually patching our application, we can address both of these pain points by simply bringing into our project the [`activerecord-enhancedsqlite3-adapter` gem](https://github.com/fractaledmind/activerecord-enhancedsqlite3-adapter). This gem is a zero-configuration drop-in enhancement for the `sqlite3` adapter that comes with Rails. It will automatically open transactions in immediate mode, and it will also ensure that whenever SQLite is waiting for a query to acquire the write lock that other Puma workers can continue to process requests. In addition, it will back port some nice ActiveRecord features that aren't yet in a point release, like deferred foreign key constraints, custom return columns, and generated columns.
10 |
11 | To add the `activerecord-enhancedsqlite3-adapter` gem to your project, simply run the following command:
12 |
13 | ```sh
14 | bundle add activerecord-enhancedsqlite3-adapter
15 | ```
16 |
17 | Simply by adding the gem to your `Gemfile` you automatically get all of the gem's goodies. You don't need to configure anything.
18 |
19 | So, we can now remove the manual changes we made to our application. First, delete the `config/initializers/sqlite3_busy_timeout_patch.rb` file
20 |
21 | ```sh
22 | rm config/initializers/sqlite3_busy_timeout_patch.rb
23 | ```
24 |
25 | Then, remove the `default_transaction_mode: IMMEDIATE` line from your `config/database.yml` file:
26 |
27 | ```sh
28 | sed -i '' '/default_transaction_mode: IMMEDIATE/d' config/database.yml
29 | ```
30 |
31 | The enhanced adapter gem will supply both fixes automatically for us.
32 |
33 | ### Running the Load Tests
34 |
35 | Let's rerun our load tests and ensure things have stayed improved. We first need to restart our application server so that it picks up and uses the enhanced adapter. So, `Ctrl + C` to stop the running server, then re-run the `bin/serve` command in that terminal window/tab.
36 |
37 | Then, in the other terminal window/tab, run the `post_create` load test again:
38 |
39 | ```sh
40 | oha -c 20 -z 10s -m POST http://localhost:3000/benchmarking/post_create
41 | ```
42 |
43 | which gave me these results:
44 |
45 |
46 | 1017 RPS (click to see full breakdown)
47 |
48 | ```
49 | Summary:
50 | Success rate: 100.00%
51 | Total: 10.0005 secs
52 | Slowest: 0.2814 secs
53 | Fastest: 0.0024 secs
54 | Average: 0.0197 secs
55 | Requests/sec: 1017.0507
56 |
57 | Total data: 86.05 MiB
58 | Size/request: 8.68 KiB
59 | Size/sec: 8.60 MiB
60 |
61 | Response time histogram:
62 | 0.002 [1] |
63 | 0.030 [8415] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
64 | 0.058 [1412] |■■■■■
65 | 0.086 [212] |
66 | 0.114 [54] |
67 | 0.142 [28] |
68 | 0.170 [10] |
69 | 0.198 [9] |
70 | 0.226 [5] |
71 | 0.254 [3] |
72 | 0.281 [2] |
73 |
74 | Response time distribution:
75 | 10.00% in 0.0054 secs
76 | 25.00% in 0.0086 secs
77 | 50.00% in 0.0147 secs
78 | 75.00% in 0.0248 secs
79 | 90.00% in 0.0382 secs
80 | 95.00% in 0.0503 secs
81 | 99.00% in 0.0902 secs
82 | 99.90% in 0.1971 secs
83 | 99.99% in 0.2785 secs
84 |
85 |
86 | Details (average, fastest, slowest):
87 | DNS+dialup: 0.0013 secs, 0.0007 secs, 0.0018 secs
88 | DNS-lookup: 0.0001 secs, 0.0000 secs, 0.0003 secs
89 |
90 | Status code distribution:
91 | [200] 10151 responses
92 |
93 | Error distribution:
94 | [20] aborted due to deadline
95 | ```
96 |
97 |
98 | We see that there are still no `500` errored responses, requests per second remains above 1,000, and the slowest request is still under 300ms. The `activerecord-enhancedsqlite3-adapter` gem has successfully addressed the two major issues we identified in our baseline load tests.
99 |
100 | ### Conclusion
101 |
102 | While it is important to understand _what_ the enhanced adapter is doing, and some of you may actually prefer to have the code that fixes the problems we have discussed in your repository, it is nice that we can simply add a gem to our project and have these issues automatically addressed.
103 |
--------------------------------------------------------------------------------
/workshop/10-controlling-sqlite-compilation.md:
--------------------------------------------------------------------------------
1 | ## Controling SQLite Compilation
2 |
3 | Because SQLite is simply a single executable, it is easy to control the actual compilation of SQLite. The `sqlite3-ruby` gem allows us to control the compilation flags used to compile the SQLite executable. We can set the compilation flags via the `BUNDLE_BUILD__SQLITE3` environment variable set in the `.bundle/config` file. Bundler allows you set set such configuration via the `bundle config set` command. To control SQLite compilation, you use the `bundle config set build.sqlite3` command passing the `--with-sqlite-cflags` argument.
4 |
5 | ### The Test
6 |
7 | Before we set the actual compilation flags we want for SQLite, let's first confirm that this process works. We will need a compilation flag that will be easy to check in the Rails console. Luckily, whether or not SQLite was compiled with thread-safety is a check that the `sqlite3-ruby` gem provides via `SQLite3.threadsafe?`.
8 |
9 | So, let's first check the current value of `SQLite.threadsafe?`. First, we need to start a Rails console:
10 |
11 | ```sh
12 | RAILS_ENV=production bin/rails c
13 | ```
14 |
15 | Then, check the current value:
16 |
17 | ```irb
18 | > SQLite3.threadsafe?
19 | => true
20 | ```
21 |
22 | Ok, so by default SQLite is compiled with thread-safety. Let's now try to compile SQLite with thread-safety off.
23 |
24 | As described above, we can set the compilation flags via the `bundle config set build.sqlite3` command:
25 |
26 | ```sh
27 | bundle config set build.sqlite3 \
28 | "--with-sqlite-cflags='-DSQLITE_THREADSAFE=0'"
29 | ```
30 |
31 | If it doesn't already exist, this command will create a `.bundle/config` file in your project directory. Its contents should look something like:
32 |
33 | ```yaml
34 | ---
35 | BUNDLE_BUILD__SQLITE3: "--with-sqlite-cflags=' -DSQLITE_THREADSAFE=0'"
36 | ```
37 |
38 | Finally, in order to ensure that SQLite is compiled from source, we need to specify in the `Gemfile` that the SQLite gem should use the `ruby` platform version.
39 |
40 | ```ruby
41 | gem "sqlite3", ">= 2.0", force_ruby_platform: true
42 | ```
43 |
44 | When you now run `bundle install`, you should see something like:
45 |
46 | ```
47 | Fetching sqlite3 2.0.4
48 | Installing sqlite3 2.0.4 with native extensions
49 | ```
50 |
51 | Compiling SQLite from source can take a while, so be patient. Once it is done, you can start a Rails console and check the value of `SQLite3.threadsafe?` now. You should hopefully see:
52 |
53 | ```irb
54 | > SQLite3.threadsafe?
55 | => false
56 | ```
57 |
58 | If you see `false`, this confirms that the compilation flags were set correctly, that SQLite was compiled with thread-safety off, and that our Rails app is using the custom compiled SQLite executable. Now, we want to set the actually desired compilation flags for SQLite, which requires undoing the thread-safety flag we just set.
59 |
60 | If you ever want to undo custom compilation, you will not only need to remove the `build.sqlite3` configuration and the `force_ruby_platform` option from the `Gemfile`, but you will also need to delete the `sqlite3` gem from your system. This can be done with the following command:
61 |
62 | ```sh
63 | rm -rf $(bundle show sqlite3)
64 | ```
65 |
66 | After that, simply run `bundle install` again to reinstall the `sqlite3` gem.
67 |
68 | ### The Implementation
69 |
70 | The [SQLite docs recommend 12 flags](https://www.sqlite.org/compile.html#recommended_compile_time_options) for a 5% improvement. The `sqlite3-ruby` gem needs some of the features recommended to be omitted, and some are useful for Rails apps. These 6 flags are my recommendation for a Rails app, and can be set using the following command:
71 |
72 | ```sh
73 | bundle config set build.sqlite3 \
74 | "--with-sqlite-cflags='
75 | -DSQLITE_DQS=0
76 | -DSQLITE_DEFAULT_MEMSTATUS=0
77 | -DSQLITE_LIKE_DOESNT_MATCH_BLOBS
78 | -DSQLITE_MAX_EXPR_DEPTH=0
79 | -DSQLITE_OMIT_SHARED_CACHE
80 | -DSQLITE_USE_ALLOCA'"
81 | ```
82 |
83 | Typically, the `.bundle/config` file is removed from source control, but we add it back to make this app more portable. Note, however, that this does restrict individual configuration of Bundler. This requires a change to the `.gitignore` file. Find this portion of your `.gitignore` file (should be near the top):
84 |
85 | ```
86 | # Ignore bundler config.
87 | /.bundle
88 | ```
89 |
90 | And add this line:
91 |
92 | ```
93 | !.bundle/config
94 | ```
95 |
96 | Again, make sure that the SQLite gem forces the `ruby` platform version:
97 |
98 | ```ruby
99 | gem "sqlite3", ">= 2.0", force_ruby_platform: true
100 | ```
101 |
102 | Run `bundle install` again to compile SQLite with the new flags. And we're done.
103 |
104 | ### Conclusion
105 |
106 | With SQLite custom compiled, the next step is to setup the repository to work with branch-specific databases.
--------------------------------------------------------------------------------
/config/environments/production.rb:
--------------------------------------------------------------------------------
1 | require "active_support/core_ext/integer/time"
2 |
3 | Rails.application.configure do
4 | # Settings specified here will take precedence over those in config/application.rb.
5 |
6 | # Code is not reloaded between requests.
7 | config.enable_reloading = false
8 |
9 | # Eager load code on boot. This eager loads most of Rails and
10 | # your application in memory, allowing both threaded web servers
11 | # and those relying on copy on write to perform better.
12 | # Rake tasks automatically ignore this option for performance.
13 | config.eager_load = true
14 |
15 | # Full error reports are disabled and caching is turned on.
16 | config.consider_all_requests_local = false
17 | config.action_controller.perform_caching = true
18 |
19 | # Ensures that a master key has been made available in ENV["RAILS_MASTER_KEY"], config/master.key, or an environment
20 | # key such as config/credentials/production.key. This key is used to decrypt credentials (and other encrypted files).
21 | # config.require_master_key = true
22 |
23 | # Disable serving static files from `public/`, relying on NGINX/Apache to do so instead.
24 | # config.public_file_server.enabled = false
25 |
26 | # Compress CSS using a preprocessor.
27 | # config.assets.css_compressor = :sass
28 |
29 | # Do not fall back to assets pipeline if a precompiled asset is missed.
30 | config.assets.compile = false
31 |
32 | # Enable serving of images, stylesheets, and JavaScripts from an asset server.
33 | # config.asset_host = "http://assets.example.com"
34 |
35 | # Specifies the header that your server uses for sending files.
36 | # config.action_dispatch.x_sendfile_header = "X-Sendfile" # for Apache
37 | # config.action_dispatch.x_sendfile_header = "X-Accel-Redirect" # for NGINX
38 |
39 | # Store uploaded files on the local file system (see config/storage.yml for options).
40 | config.active_storage.service = :local
41 |
42 | # Mount Action Cable outside main process or domain.
43 | # config.action_cable.mount_path = nil
44 | # config.action_cable.url = "wss://example.com/cable"
45 | # config.action_cable.allowed_request_origins = [ "http://example.com", /http:\/\/example.*/ ]
46 |
47 | # Assume all access to the app is happening through a SSL-terminating reverse proxy.
48 | # Can be used together with config.force_ssl for Strict-Transport-Security and secure cookies.
49 | # config.assume_ssl = true
50 |
51 | # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
52 | config.force_ssl = ENV["RELAX_SSL"].blank?
53 |
54 | # Skip http-to-https redirect for the default health check endpoint.
55 | # config.ssl_options = { redirect: { exclude: ->(request) { request.path == "/up" } } }
56 |
57 | # Log to STDOUT by default
58 | config.logger = ActiveSupport::Logger.new(STDOUT)
59 | .tap { |logger| logger.formatter = ::Logger::Formatter.new }
60 | .then { |logger| ActiveSupport::TaggedLogging.new(logger) }
61 |
62 | # Prepend all log lines with the following tags.
63 | config.log_tags = [ :request_id ]
64 |
65 | # "info" includes generic and useful information about system operation, but avoids logging too much
66 | # information to avoid inadvertent exposure of personally identifiable information (PII). If you
67 | # want to log everything, set the level to "debug".
68 | config.log_level = ENV.fetch("RAILS_LOG_LEVEL", "info")
69 |
70 | # Use a different cache store in production.
71 | # config.cache_store = :mem_cache_store
72 |
73 | # Use a real queuing backend for Active Job (and separate queues per environment).
74 | # config.active_job.queue_adapter = :resque
75 | # config.active_job.queue_name_prefix = "euruko_2024_production"
76 |
77 | # Disable caching for Action Mailer templates even if Action Controller
78 | # caching is enabled.
79 | config.action_mailer.perform_caching = false
80 |
81 | # Ignore bad email addresses and do not raise email delivery errors.
82 | # Set this to true and configure the email server for immediate delivery to raise delivery errors.
83 | # config.action_mailer.raise_delivery_errors = false
84 |
85 | # Enable locale fallbacks for I18n (makes lookups for any locale fall back to
86 | # the I18n.default_locale when a translation cannot be found).
87 | config.i18n.fallbacks = true
88 |
89 | # Don't log any deprecations.
90 | config.active_support.report_deprecations = false
91 |
92 | # Do not dump schema after migrations.
93 | config.active_record.dump_schema_after_migration = false
94 |
95 | # Enable DNS rebinding protection and other `Host` header attacks.
96 | # config.hosts = [
97 | # "example.com", # Allow requests from example.com
98 | # /.*\.example\.com/ # Allow requests from subdomains like `www.example.com`
99 | # ]
100 | # Skip DNS rebinding protection for the default health check endpoint.
101 | # config.host_authorization = { exclude: ->(request) { request.path == "/up" } }
102 | end
103 |
--------------------------------------------------------------------------------
/workshop/09-using-sqlite-extensions.md:
--------------------------------------------------------------------------------
1 | ## Using SQLite Extensions
2 |
3 | Beyond simply spinning up separate SQLite databases for IO-bound Rails components, there are a number of ways that we can enhance working with SQLite itself. One of the most powerful features of SQLite is its support for [loadable extensions](https://www.sqlite.org/loadext.html). These extensions allow you to add new functionality to SQLite, such as full-text search, JSON support, or even custom functions.
4 |
5 | ### The Implementation
6 |
7 | There is an unofficial SQLite extension package manager called [sqlpkg](https://sqlpkg.org/). We can use sqlpkg to install a number of useful SQLite extensions. View all 97 extensions available in sqlpkg [here](https://sqlpkg.org/all/).
8 |
9 | We can install the [Ruby gem](https://github.com/fractaledmind/sqlpkg-ruby) that ships with precompiled executables like so:
10 |
11 | ```sh
12 | bundle add sqlpkg
13 | ```
14 |
15 | And then we can install it into our Rails application like so:
16 |
17 | ```sh
18 | RAILS_ENV=production bin/rails generate sqlpkg:install
19 | ```
20 |
21 | This will create 2 files in our application:
22 |
23 | 1. `.sqlpkg`, which ensures that sqlpkg will run in "project scope"
24 | 2. `sqlpkg.lock`, where sqlpkg will store information about the installed packages
25 |
26 | The gem provides the `sqlpkg` executable, which we can use to install SQLite extensions. For example, to install the [`uuid` extension](https://github.com/nalgeon/sqlean/blob/main/docs/uuid.md), we can run:
27 |
28 | ```sh
29 | bundle exec sqlpkg install nalgeon/uuid
30 | ```
31 |
32 | Or, to install the [`ulid` extension](https://github.com/asg017/sqlite-ulid), we can run:
33 |
34 | ```sh
35 | bundle exec sqlpkg install asg017/ulid
36 | ```
37 |
38 | As you will see on the [sqlpkg website](https://sqlpkg.org/all/), each extension has an identifier made up of a namespace and a name. There are many more extensions available.
39 |
40 | When you do install an extension, you will see logs like:
41 |
42 | ```
43 | (project scope)
44 | > installing asg017/ulid...
45 | ✓ installed package asg017/ulid to .sqlpkg/asg017/ulid
46 | ```
47 |
48 | In order to make use of these extensions in our Rails application, we need to load them when the database is opened. The enhanced adapter gem can load any extensions installed via `sqlpkg` by listing them in the `database.yml` file. For example, to load the `uuid` and `ulid` extensions, we would add the following to our `config/database.yml` file:
49 |
50 | ```yaml
51 | extensions:
52 | - nalgeon/uuid
53 | - asg017/ulid
54 | ```
55 |
56 | If you want an extension to be loaded for each database (`primary`, `queue`, and `cache`), add this section to the `default` section of the `database.yml` file. If there are some extensions that you only want to load for a specific database, you can add this section to the specific database configuration.
57 |
58 | For example, if we only want to load the `uuid` extension for the `primary` database, we would add this section to the `primary` section of the `database.yml` file:
59 |
60 | ```yaml
61 | primary: &primary
62 | <<: *default
63 | database: storage/<%= Rails.env %>.sqlite3
64 | extensions:
65 | - nalgeon/uuid
66 | ```
67 |
68 | But, if we wanted to load the `ulid` extension for all databases, we would add this section to the `default` section of the `database.yml` file:
69 |
70 | ```yaml
71 | default: &default
72 | adapter: sqlite3
73 | pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
74 | timeout: 5000
75 | extensions:
76 | - asg017/ulid
77 | ```
78 |
79 | We can confirm that the extensions are loaded by opening the Rails console and running a query that uses the extension. For example, to generate a UUID, we can run:
80 |
81 | ```ruby
82 | ActiveRecord::Base.connection.execute 'select uuid4();'
83 | ```
84 |
85 | If you see a return value something like:
86 |
87 | ```ruby
88 | [{"uuid4()"=>"abf3946d-5e04-4da0-8452-158cd983bd21"}]
89 | ```
90 |
91 | then you know that the extension is loaded and working correctly.
92 |
93 | ### Using SQLite Extensions in CI and Production
94 |
95 | In order to ensure that your extensions are downloaded and installed in your production environment, you need to ensure that the `.sqlpkg` directory is present in your application's repository, but doesn't contain any files. Then, you need to call the `sqlpkg install` command as a part of your deployment process:
96 |
97 | ```sh
98 | bundle exec sqlpkg install
99 | ```
100 |
101 | To do this, let's first create a `.keep` file in the `.sqlpkg` directory:
102 |
103 | ```sh
104 | touch .sqlpkg/.keep
105 | ```
106 |
107 | Then, we can add the following to the `.gitignore` file:
108 |
109 | ```
110 | /.sqlpkg/*
111 | !/.sqlpkg/.keep
112 | ```
113 |
114 | This ignores all files in the `.sqlpkg` directory except for the `.keep` file. This way, the `.sqlpkg` directory will be present in the repository, but will not contain any files. This allows us to run the `sqlpkg install` command as a part of our deployment process.
115 |
116 | When you run the `sqlpkg install` command without specifying a package, it will install all packages listed in the `sqlpkg.lock` file. So, you can install SQLite extensions locally, commit the `sqlpkg.lock` file to your repository, and then run the `sqlpkg install` command as a part of your deployment process to ensure that the extensions are installed in your production environment.
117 |
118 | ### Conclusion
119 |
120 | With SQLite extensions integrated, the next step is to control how the SQLite executable is compiled.
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # README
2 |
3 | This is an app built for demonstration purposes for the [EuRuKo 2024 conference](https://2024.euruko.org) held in Sarajevo, Bosnia & Herzegovina on September 11-13, 2024.
4 |
5 | The application is a basic "Hacker News" style app with `User`s, `Post`s, and `Comment`s. The seeds file will create ~100 users, ~1,000 posts, and ~10 comments per post (so ~10,000 comments). Every user has the same password: `password`, so you can sign in as any user to test the app.
6 |
7 | This application runs on Ruby >= 3.1, Rails 7.2.1, and SQLite 3.46.1 (gem version 2.0.4).
8 |
9 | ## Setup
10 |
11 | First you need to clone the repository to your local machine:
12 |
13 | ```sh
14 | git clone git@github.com:fractaledmind/euruko-2024.git
15 | cd euruko-2024
16 | ```
17 |
18 | After cloning the repository, install the dependencies:
19 |
20 | ```sh
21 | bundle install
22 | ```
23 |
24 | I have built the repository to be usable out-of-the-box. It contains a seeded production database, precompiled assets, and a binscript to start the server in production mode. The only other things you will need besides the RubyGems dependencies are multiple different Ruby versions and the load testing tool `oha`.
25 |
26 | ### Setup Ruby Versions
27 |
28 | By default, this repository uses Ruby 3.1.6, which is the most recent point release on the 3.1 branch. As a part of the exploration of the performance impact of different Ruby versions on SQLite-backed Rails applications, we will be testing the following Ruby versions:
29 |
30 | ```
31 | ruby-3.3.5
32 | ruby-3.2.5
33 | ruby-3.1.6
34 | ```
35 |
36 | Please make sure you have each of these Ruby versions installed on your machine. If you are using `rbenv`, you can install them with commands like the following:
37 |
38 | ```sh
39 | rbenv install 3.3.5
40 | ```
41 |
42 | Or, if you are using `asdf`, you can install them like so:
43 |
44 | ```sh
45 | asdf install ruby 3.3.5
46 | ```
47 |
48 | If you manage Ruby versions some other way, I'm sure you know how to install new Ruby versions with your tool of choice.
49 |
50 | ### Setup Load Testing
51 |
52 | Load testing can be done using the [`oha` CLI utility](https://github.com/hatoo/oha), which can be installed on MacOS via [homebrew](https://brew.sh):
53 |
54 | ```sh
55 | brew install oha
56 | ```
57 |
58 | and on Windows via [winget](https://github.com/microsoft/winget-cli):
59 |
60 | ```sh
61 | winget install hatoo.oha
62 | ```
63 |
64 | or using their [precompiled binaries](https://github.com/hatoo/oha?tab=readme-ov-file#installation) on other platforms.
65 |
66 | ## Load Testing
67 |
68 | Throughout this workshop, we will be load testing the application to observe how our various changes impact the performance of the application. In order to perform the load testing, you will need to run the web server in the `production` environment. I have provided a binscript to make this easier. To start the production server, run the following command:
69 |
70 | ```sh
71 | bin/serve
72 | ```
73 |
74 | This simply a shortcut for the following command:
75 |
76 | ```sh
77 | RAILS_ENV=production RELAX_SSL=true RAILS_LOG_LEVEL=warn WEB_CONCURRENCY=10 RAILS_MAX_THREADS=5 bin/rails server
78 | ```
79 |
80 | The `RELAX_SSL` environment variable is necessary to allow you to use `http://localhost`. The `RAILS_LOG_LEVEL` is set to `warn` to reduce the amount of logging output. Set `WEB_CONCURRENCY` to the number of cores you have on your laptop. I am on an M1 Macbook Pro with 10 cores, and thus I set the value to 10. The `RAILS_MAX_THREADS` controls the number of threads per worker. I left it at the default of 5, but you can tweak it to see how it affects performance.
81 |
82 | With your server running in one terminal window, you can use the load testing utility to test the app in another terminal window. Here is the shape of the command you will use to test the app:
83 |
84 | ```sh
85 | oha -c N -z 10s -m POST http://localhost:3000/benchmarking/PATH
86 | ```
87 |
88 | `N` is the number of concurrent requests that `oha` will make. I recommend running a large variety of different scenarios with different values of `N`. Personally, I scale up from 1 to 256 concurrent requests, doubling the number of concurrent requests each time. In general, when `N` matches your `WEB_CONCURRENCY` number, this is mostly likely the sweet spot for this app.
89 |
90 | `PATH` can be any of the benchmarking paths defined in the app. The app has a few different paths that you can test. From the `routes.rb` file:
91 |
92 | ```ruby
93 | namespace :benchmarking do
94 | post "post_create"
95 | post "comment_create"
96 | post "post_destroy"
97 | post "comment_destroy"
98 | post "post_show"
99 | post "posts_index"
100 | post "user_show"
101 | end
102 | ```
103 |
104 | You can validate that the application is properly set up for load testing by serving the application in one terminal window/tab (via `bin/serve`) and then running the following `curl` command in another terminal window/tab:
105 |
106 | ```sh
107 | curl -s -o /dev/null -w "%{http_code}" -X POST http://localhost:3000/benchmarking/posts_index
108 | ```
109 |
110 | If this returns `200`, then next ensure that you can run an `oha` command like the following:
111 |
112 | ```sh
113 | oha -c 1 -z 1s -m POST http://localhost:3000/benchmarking/posts_index
114 | ```
115 |
116 | If this runs successfully, then you are ready to begin the workshop.
117 |
118 | ## Workshop
119 |
120 | You will find the workshop instructions in the `workshop/` directory. The workshop is broken down into a series of steps, each of which is contained in a separate markdown file. The workshop is designed to be self-guided, but I am available to help if you get stuck. Please feel free to reach out to me on Twitter at [@fractaledmind](https://twitter.com/fractaledmind) if you have any questions.
121 |
122 | The first step is to [run some baseline load tests](workshop/00-run-baseline-load-tests.md). Once you have completed that step, you can move on to the next step, which will be linked at the bottom of each step.
123 |
--------------------------------------------------------------------------------
/workshop/06-restoring-from-a-backup.md:
--------------------------------------------------------------------------------
1 | ## Restoring from a Backup
2 |
3 | With Litestream streaming updates to your MinIO bucket, you can now restore your database from a backup. To do this, you can use the `litestream:restore` Rake task:
4 |
5 | ```sh
6 | RAILS_ENV=production bin/rails litestream:restore -- --database=storage/production.sqlite3 -o=storage/restored.sqlite3
7 | ```
8 |
9 | This task will download the latest snapshot and WAL files from your MinIO bucket and restore them to your local database.
10 |
11 | When you run this task, you should see output like:
12 |
13 | ```
14 | time=YYYY-MM-DDTHH:MM:SS.000+00:00 level=INFO msg="restoring snapshot" db=~/path/to/railsconf-2024/storage/production.sqlite3 replica=s3 generation=e9885230835eaf8b index=0 path=storage/restored.sqlite3.tmp
15 | time=YYYY-MM-DDTHH:MM:SS.000+00:00 level=INFO msg="restoring wal files" db=~/path/to/railsconf-2024/storage/production.sqlite3 replica=s3 generation=e9885230835eaf8b index_min=0 index_max=0
16 | time=YYYY-MM-DDTHH:MM:SS.000+00:00 level=INFO msg="downloaded wal" db=~/path/to/railsconf-2024/storage/production.sqlite3 replica=s3 generation=e9885230835eaf8b index=0 elapsed=2.622459ms
17 | time=YYYY-MM-DDTHH:MM:SS.000+00:00 level=INFO msg="applied wal" db=~/path/to/railsconf-2024/storage/production.sqlite3 replica=s3 generation=e9885230835eaf8b index=0 elapsed=913.333µs
18 | time=YYYY-MM-DDTHH:MM:SS.000+00:00 level=INFO msg="renaming database from temporary location" db=~/path/to/railsconf-2024/storage/production.sqlite3 replica=s3
19 | ```
20 |
21 | You can inspect the contents of the `restored` database with the `sqlite3` console:
22 |
23 | ```sh
24 | sqlite3 storage/restored.sqlite3
25 | ```
26 |
27 | Check how many records are in the `posts` table:
28 |
29 | ```sql
30 | SELECT COUNT(*) FROM posts;
31 | ```
32 |
33 | and the same for the `comments` table:
34 |
35 | ```sql
36 | SELECT COUNT(*) FROM comments;
37 | ```
38 |
39 | You should see the same number of records in the `restored` database as in the `production` database.
40 |
41 | You can close the `sqlite3` console by typing `.quit`.
42 |
43 | ### Verifying Backups
44 |
45 | Running a single restoration like this is useful for testing, but in a real-world scenario, you would likely want to ensure that your backups are both fresh and restorable. In order to ensure that you consistently have a resilient backup strategy, the Litestream gem provides a `Litestream.verify!` method to, well, verify your backups. It is worth noting, to be clear, that this is not a feature of the underlying Litestream utility, but only a feature of the Litestream gem itself.
46 |
47 | The method takes the path to a database file that you have configured Litestream to backup; that is, it takes one of the `path` values under the `dbs` key in your `litestream.yml` configuration file. In order to verify that the backup for that database is both restorable and fresh, the method will add a new row to that database under the `_litestream_verification` table. It will then wait 10 seconds to give the Litestream utility time to replicate that change to whatever storage providers you have configured. After that, it will download the latest backup from that storage provider and ensure that this verification row is present in the backup. If the verification row is _not_ present, the method will raise a `Litestream::VerificationFailure` exception.
48 |
49 | Since we are using the Puma plugin, we can actually run the `Litestream.verify!` method directly from the Rails console:
50 |
51 | ```sh
52 | RAILS_ENV=production bin/rails console
53 | ```
54 |
55 | and run:
56 |
57 | ```ruby
58 | Litestream.verify!("storage/production.sqlite3")
59 | ```
60 |
61 | After 10 seconds, you will see it return `true`.
62 |
63 | If you want to force a verification failure, you will need to comment out the Puma plugin and stop the `bin/serve` process. To confirm that the replication process is not running, check your running processes:
64 |
65 | ```sh
66 | ps -a | grep litestream
67 | ```
68 |
69 | If you see a process running, you can kill it with:
70 |
71 | ```sh
72 | kill -9
73 | ```
74 |
75 | where `` is the process ID of the Litestream process (the first number in the set of three columns returned by the `ps` command).
76 |
77 | Once you are sure that the replication process is not running, open the Rails console again:
78 |
79 | ```sh
80 | RAILS_ENV=production bin/rails console
81 | ```
82 |
83 | This time, when you run:
84 |
85 | ```ruby
86 | Litestream.verify!("storage/production.sqlite3")
87 | ```
88 |
89 | After 10 seconds, you will see an exception raised:
90 |
91 | ```
92 | Verification failed for `storage/production.sqlite3` (Litestream::VerificationFailure)
93 | ```
94 |
95 | This demonstrates that the `verify!` method truly does only report success when the verification row is present in the backup, which requires the replication process to be running smoothly.
96 |
97 | ### Automating Backup Verification
98 |
99 | Even better than manually verifying your backups is to automate the process. In addition to the `verify!` method, the Litestream gem provides a background job to verify our backups for us. If you have an Active Job adapter that supports recurring jobs, you can configure it to run this job at regular intervals. Or, you could simply use `cron` to run the job at regular intervals. In this workshop, we will setup Solid Queue as our background job adapter in the next step. Once it is setup, we will configure this recurring job. Until then, let's explore what the job does and run it manually.
100 |
101 | If you inspec the gem's source code, you will find that the job is implemented like so:
102 |
103 | ```ruby
104 | module Litestream
105 | class VerificationJob < ActiveJob::Base
106 | queue_as Litestream.queue
107 |
108 | def perform
109 | Litestream::Commands.databases.each do |db_hash|
110 | Litestream.verify!(db_hash["path"])
111 | end
112 | end
113 | end
114 | end
115 | ```
116 |
117 | This job will allow us to verify our backup strategy for all databases we have configured Litestream to replicate. If any database fails verification, the job will raise an exception, which will be caught by Rails and logged.
118 |
119 | All we need now is a job backend that will allow us to schedule this job to run at regular intervals.
120 |
121 | ### Conclusion
122 |
123 | It is essential to have a backup strategy in place for your application's data. Litestream is a great tool for this purpose. In this step we have installed the gem, configured it, explored how the replication and restoration processes work, and learned how to verify our backups. With backup verification setup, the next step is to add a background job adapter.
124 |
--------------------------------------------------------------------------------
/workshop/00-run-baseline-load-tests.md:
--------------------------------------------------------------------------------
1 | ## Run Baseline Load Tests
2 |
3 | Before we start, let's establish a baseline. This is the starting point from which we will measure our progress. It's important to have a clear understanding of where we are now, so we can see how far we've come as we progress.
4 |
5 | We will run two load tests to assess the current state of the application's performance; one for the `post_create` action and one for the `posts_index` action. We will run each test with 20 concurrent requests for 10 seconds.
6 |
7 | We will run the read operation first since it can't have any effect on the write operation performance (while the inverse cannot be said). But first, it is often worth checking that the endpoint is responding as expected _before_ running a load test. So, let's make a single `curl` request first.
8 |
9 | In one terminal window, start the Rails server:
10 |
11 | ```sh
12 | bin/serve
13 | ```
14 |
15 | In another, make a single `curl` request to the `posts_index` endpoint:
16 |
17 | ```sh
18 | curl -s -o /dev/null -w "%{http_code}" -X POST http://localhost:3000/benchmarking/posts_index
19 | ```
20 |
21 | You should see a `200` response. If you see that response, everything is working as expected. If you don't, you will need to troubleshoot the issue before proceeding.
22 |
23 | Once we have verified that our Rails application is responding to the `benchmarking/posts_index` route as expected, we can run the load test and record the results.
24 |
25 | As stated earlier, we will use the `oha` tool to run the load test. We will send waves of 20 concurrent requests, which is twice the number of Puma workers that our application has spun up. We will run the test for 10 seconds. The command to run the load test is as follows:
26 |
27 | ```sh
28 | oha -c 20 -z 10s -m POST http://localhost:3000/benchmarking/posts_index
29 | ```
30 |
31 | Running this on my 2021 M1 MacBook Pro (32 GB of RAM running MacOS 14.6.1) against our Rails 7.2.1 app with Ruby 3.1.6, I get the following results:
32 |
33 |
34 | 242 RPS (click to see full breakdown)
35 |
36 | ```
37 | Summary:
38 | Success rate: 100.00%
39 | Total: 10.0014 secs
40 | Slowest: 0.1948 secs
41 | Fastest: 0.0053 secs
42 | Average: 0.0828 secs
43 | Requests/sec: 242.4653
44 |
45 | Total data: 153.67 MiB
46 | Size/request: 65.43 KiB
47 | Size/sec: 15.37 MiB
48 |
49 | Response time histogram:
50 | 0.005 [1] |
51 | 0.024 [53] |■■
52 | 0.043 [115] |■■■■■■
53 | 0.062 [507] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■
54 | 0.081 [578] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
55 | 0.100 [499] |■■■■■■■■■■■■■■■■■■■■■■■■■■■
56 | 0.119 [329] |■■■■■■■■■■■■■■■■■■
57 | 0.138 [189] |■■■■■■■■■■
58 | 0.157 [85] |■■■■
59 | 0.176 [43] |■■
60 | 0.195 [6] |
61 |
62 | Response time distribution:
63 | 10.00% in 0.0474 secs
64 | 25.00% in 0.0605 secs
65 | 50.00% in 0.0790 secs
66 | 75.00% in 0.1027 secs
67 | 90.00% in 0.1257 secs
68 | 95.00% in 0.1410 secs
69 | 99.00% in 0.1636 secs
70 | 99.90% in 0.1841 secs
71 | 99.99% in 0.1948 secs
72 |
73 |
74 | Details (average, fastest, slowest):
75 | DNS+dialup: 0.0022 secs, 0.0011 secs, 0.0032 secs
76 | DNS-lookup: 0.0002 secs, 0.0000 secs, 0.0007 secs
77 |
78 | Status code distribution:
79 | [200] 2405 responses
80 |
81 | Error distribution:
82 | [20] aborted due to deadline
83 | ```
84 |
85 |
86 | It is worth noting that when I ran this load test 4 months ago on the same machine, things were notably worse. The p99.99 response time was **over 5 seconds**, the RPS was only **~40**, and some responses simply errored out. The fixes and improvements continuously made to Rails and the SQLite gem are clearly having a positive impact.
87 |
88 | Now that we have the baseline for the `posts_index` action, we can move on to the `post_create` action. We will follow the same steps as above, but this time we will run the load test on the `post_create` endpoint.
89 |
90 | With the Rails server still running in one terminal window, we can make a single `curl` request to the `post_create` endpoint in another:
91 |
92 | ```sh
93 | curl -s -o /dev/null -w "%{http_code}" -X POST http://localhost:3000/benchmarking/post_create
94 | ```
95 |
96 | Again, you should see a `200` response. If you don't, you will need to troubleshoot the issue before proceeding.
97 |
98 | Once we have verified that our Rails application is responding to the `benchmarking/post_create` route as expected, we can run the load test and record the results.
99 |
100 | ```sh
101 | oha -c 20 -z 10s -m POST http://localhost:3000/benchmarking/post_create
102 | ```
103 |
104 | Running this on my 2021 M1 MacBook Pro (32 GB of RAM running MacOS 14.6.1) against our Rails 7.2.1 app with Ruby 3.1.6, I get the following results:
105 |
106 |
107 | 95 RPS (click to see full breakdown)
108 |
109 | ```
110 | Summary:
111 | Success rate: 100.00%
112 | Total: 10.0037 secs
113 | Slowest: 5.2195 secs
114 | Fastest: 0.0029 secs
115 | Average: 0.0387 secs
116 | Requests/sec: 94.9652
117 |
118 | Total data: 3.31 MiB
119 | Size/request: 3.65 KiB
120 | Size/sec: 339.07 KiB
121 |
122 | Response time histogram:
123 | 0.003 [1] |
124 | 0.525 [925] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
125 | 1.046 [0] |
126 | 1.568 [0] |
127 | 2.090 [0] |
128 | 2.611 [0] |
129 | 3.133 [0] |
130 | 3.655 [0] |
131 | 4.176 [0] |
132 | 4.698 [0] |
133 | 5.220 [4] |
134 |
135 | Response time distribution:
136 | 10.00% in 0.0037 secs
137 | 25.00% in 0.0062 secs
138 | 50.00% in 0.0094 secs
139 | 75.00% in 0.0166 secs
140 | 90.00% in 0.0397 secs
141 | 95.00% in 0.0620 secs
142 | 99.00% in 0.1307 secs
143 | 99.90% in 5.2195 secs
144 | 99.99% in 5.2195 secs
145 |
146 |
147 | Details (average, fastest, slowest):
148 | DNS+dialup: 0.0021 secs, 0.0012 secs, 0.0025 secs
149 | DNS-lookup: 0.0001 secs, 0.0000 secs, 0.0004 secs
150 |
151 | Status code distribution:
152 | [500] 661 responses
153 | [200] 269 responses
154 |
155 | Error distribution:
156 | [20] aborted due to deadline
157 | ```
158 |
159 |
160 | Immediately, it should jump out just how many `500` responses we are seeing. **71%** of the responses are returning an error status code. Suffice it to say, this is not at all what we want from our application. We also now see some requests taking over 5 seconds to complete, which is aweful. And our requests per second have plummeted to 2.5× to only **95**.
161 |
162 | Our first challenge is to fix these performance issues.
163 |
164 | > [!NOTE]
165 | > If you want to ensure that you are running your load tests from a clean slate each time, you can reset your database (drop the database, create it, migrate it, seed it) before running the tests. You can do this by running the following command:
166 | > ```sh
167 | > DISABLE_DATABASE_ENVIRONMENT_CHECK=1 RAILS_ENV=production bin/rails db:reset
168 | > ```
169 | > This isn't _necessary_ for the purposes of this workshop, as the load test exact results don't change anything, but it can be helpful if you want to run fairer, more direct comparisons.
170 |
171 | - - -
172 |
173 | The next step is to begin improving performance. You will find that step's instructions in the `workshop/01-improving-performance.md` file.
174 |
175 | There were no code changes in this step.
176 |
--------------------------------------------------------------------------------
/workshop/02-fixing-errored-responses.md:
--------------------------------------------------------------------------------
1 | ## Fixing Errored Responses
2 |
3 | As of today, a SQLite on Rails application will struggle with concurrency. Although Rails, since version 7.1.0, ensures that your SQLite databases are running in [WAL mode](https://www.sqlite.org/wal.html), this is insufficient to ensure quality performance for web applications under concurrent load.
4 |
5 | ### The Problem
6 |
7 | The first problem to fix are all of the `500` error responses that we saw in our `post_create` load test and occassionally in the `posts_index` test. If you look at the server logs, you will see the error being thrown is:
8 |
9 | ```
10 | ActiveRecord::StatementInvalid (SQLite3::BusyException: database is locked):
11 | ```
12 |
13 | This is the [`SQLITE_BUSY` exception](https://www.sqlite.org/rescode.html#busy).
14 |
15 | I will save you hours, if not days, of debugging and investigation and tell you that these errors are caused by Rails not opening transactions in what SQLite calls ["immediate mode"](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions). In order to ensure only one write operation occurs at a time, SQLite uses a write lock on the database. Only one connection can hold the write lock at a time. By default, SQLite interprets the `BEGIN TRANSACTION` command as initiating a _deferred_ transaction. This means that SQLite will not attempt to acquire the database write lock until a write operation is made inside that transaction. In contrast, an _immediate_ transaction will attempt to acquire the write lock immediately upon the `BEGIN IMMEDIATE TRANSACTION` command being issued.
16 |
17 | Opening deferred transactions in a web application with multiple connections open to the database _nearly guarantees_ that you will see a large number of `SQLite3::BusyException` errors. This is because SQLite is unable to retry the write operation within the deferred transaction if the write lock is already held by another connection because any retry would risk the transaction operating against a different snapshot of the database state.
18 |
19 | Opening _immediate_ transactions, on the other hand, is safer in a multi-connection environment because SQLite can safely retry the transaction opening command until the write lock is available, since the transaction won't grab a snapshot until the write lock is acquired.
20 |
21 | While future versions of Rails will address this issue by opening immediate transactions by default, we must fix this issue ourselves in the meantime.
22 |
23 | So, how do we ensure that our Rails application makes all transactions immediate?
24 |
25 | ### The Solution
26 |
27 | As of [version 1.6.9](https://github.com/sparklemotion/sqlite3-ruby/releases/tag/v1.6.9), the [`sqlite3-ruby` gem](https://github.com/sparklemotion/sqlite3-ruby) allows you to configure the default transaction mode with the `default_transaction_mode` option when initializing a new `SQLite3::Database` instance. Since Rails passes any top-level keys in your `database.yml` configuration directly to the `sqlite3-ruby` database initializer, you can easily ensure that Rails’ SQLite transactions are all run in IMMEDIATE mode by updating your `default` configuration in the `database.yml` file:
28 |
29 | ```yaml
30 | default: &default
31 | adapter: sqlite3
32 | pool: <%= ENV. fetch("RAILS_MAX_THREADS") { 5 }%›
33 | timeout: 5000
34 | default_transaction_mode: IMMEDIATE
35 | ```
36 |
37 | This will ensure that all transactions in your Rails application are run in immediate mode. Let's run our load tests again to see if this fixes the `500` errors.
38 |
39 | ### Running the Load Tests
40 |
41 | As always, let's restart our application server first. Go to your first terminal window/tab and use `Ctrl + C` to stop the server, then re-run `bin/serve` to restart it.
42 |
43 | Once you have the server running with the new Ruby version, you can run the `posts_index` load test again in another terminal window/tab:
44 |
45 | ```sh
46 | oha -c 20 -z 10s -m POST http://localhost:3000/benchmarking/posts_index
47 | ```
48 |
49 | You should not see any `500` errors this time:
50 |
51 |
52 | 294 RPS (click to see full breakdown)
53 |
54 | ```
55 | Summary:
56 | Success rate: 100.00%
57 | Total: 10.0004 secs
58 | Slowest: 0.7988 secs
59 | Fastest: 0.0032 secs
60 | Average: 0.0682 secs
61 | Requests/sec: 294.0875
62 |
63 | Total data: 186.72 MiB
64 | Size/request: 65.46 KiB
65 | Size/sec: 18.67 MiB
66 |
67 | Response time histogram:
68 | 0.003 [1] |
69 | 0.083 [2623] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
70 | 0.162 [247] |■■■
71 | 0.242 [22] |
72 | 0.321 [7] |
73 | 0.401 [2] |
74 | 0.481 [2] |
75 | 0.560 [9] |
76 | 0.640 [2] |
77 | 0.719 [5] |
78 | 0.799 [1] |
79 |
80 | Response time distribution:
81 | 10.00% in 0.0512 secs
82 | 25.00% in 0.0579 secs
83 | 50.00% in 0.0625 secs
84 | 75.00% in 0.0673 secs
85 | 90.00% in 0.0830 secs
86 | 95.00% in 0.1065 secs
87 | 99.00% in 0.2179 secs
88 | 99.90% in 0.6678 secs
89 | 99.99% in 0.7988 secs
90 |
91 |
92 | Details (average, fastest, slowest):
93 | DNS+dialup: 0.0012 secs, 0.0007 secs, 0.0017 secs
94 | DNS-lookup: 0.0002 secs, 0.0000 secs, 0.0006 secs
95 |
96 | Status code distribution:
97 | [200] 2921 responses
98 |
99 | Error distribution:
100 | [20] aborted due to deadline
101 | ```
102 |
103 |
104 | But, we have seen that the `posts_index` load test can sometimes run smoothly. Let's run the `post_create` load test again to see if the `500` errors are gone:
105 |
106 | ```sh
107 | oha -c 20 -z 10s -m POST http://localhost:3000/benchmarking/post_create
108 | ```
109 |
110 | No errors! The `500` errors are gone:
111 |
112 |
113 | 816 RPS (click to see full breakdown)
114 |
115 | ```
116 | Summary:
117 | Success rate: 100.00%
118 | Total: 10.0010 secs
119 | Slowest: 1.0928 secs
120 | Fastest: 0.0022 secs
121 | Average: 0.0245 secs
122 | Requests/sec: 816.3180
123 |
124 | Total data: 69.03 MiB
125 | Size/request: 8.68 KiB
126 | Size/sec: 6.90 MiB
127 |
128 | Response time histogram:
129 | 0.002 [1] |
130 | 0.111 [7821] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
131 | 0.220 [249] |■
132 | 0.329 [36] |
133 | 0.438 [27] |
134 | 0.547 [4] |
135 | 0.657 [4] |
136 | 0.766 [0] |
137 | 0.875 [0] |
138 | 0.984 [1] |
139 | 1.093 [1] |
140 |
141 | Response time distribution:
142 | 10.00% in 0.0033 secs
143 | 25.00% in 0.0050 secs
144 | 50.00% in 0.0116 secs
145 | 75.00% in 0.0245 secs
146 | 90.00% in 0.0540 secs
147 | 95.00% in 0.0968 secs
148 | 99.00% in 0.2148 secs
149 | 99.90% in 0.4419 secs
150 | 99.99% in 1.0928 secs
151 |
152 |
153 | Details (average, fastest, slowest):
154 | DNS+dialup: 0.0007 secs, 0.0006 secs, 0.0009 secs
155 | DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0002 secs
156 |
157 | Status code distribution:
158 | [200] 8144 responses
159 |
160 | Error distribution:
161 | [20] aborted due to deadline
162 | ```
163 |
164 |
165 | ### Conclusion
166 |
167 | We have confirmed that this simple configuration change has fixed the `500` errors in our Rails application. By setting the `default_transaction_mode` option to `IMMEDIATE` in the `database.yml` file, we can ensure that all transactions in our Rails application are run in immediate mode. And this will help prevent `500` errors caused by SQLite's default deferred transaction mode.
168 |
169 |
--------------------------------------------------------------------------------
/workshop/11-branch-specific-databases.md:
--------------------------------------------------------------------------------
1 | ## Branch-specific Databases
2 |
3 | Another enhancement that SQLite affords is a nice developer experience — branch-specific databases. If you have ever worked in team on a single codebase, you very likely have experienced the situation where you are working on a longer running feature branch, but then a colleague asks you to review or help on a feature branch that they had been working on. What happens when you had some migrations in your branch and they had some migrations in their branch? Because your database typically has no awareness of you changing git branches, your database ends up in a mixed state with both sets of migrations applied. When you return to your branch, your database is in an altered state than you left it.
4 |
5 | Because SQLite stores your entire database in literal files on disk and only runs embedded in your application process, databases are very cheap to create. So, what if we simply spun up a completely new database for each and every git branch you use in your application? Not only would this solve the mixed migrations issue, but it also opens up the ability to prepare branch-specific data that can then be shared with collegues or used in manual testing for that branch.
6 |
7 | ### The Implementation
8 |
9 | So, what all is entailed in getting such a setup for your Rails application? Well, the basic implementation is literally only 2 lines of code in 2 files!
10 |
11 | Firstly, in your `database.yml` file, we need to update how we set the database name for the `primary` database. Of course, if we wanted or needed to, we could do the same for our `queue` and `cache` databases as well, but I personally haven't yet needed that level of isolation. Instead of setting the name of the `primary` database to the current Rails environment, we want to set the name to the current git branch. Since we can execute shell commands easily in Ruby, this is nothing more than `git branch --show-current`. Because we can be in a detached state in git, we also need a fallback. You can either use `"development"` or `"detached"` or whatever else you'd like. In the end, our new configuration will look something like:
12 |
13 | ```yaml
14 | primary: &primary
15 | <<: *default
16 | database: storage/<%= `git branch --show-current`.chomp || "detached" %>.sqlite3
17 | ```
18 |
19 | This ensures that whenever Rails loads the database configuration, it will simply introspect the current git branch and use that as the database name. The second requirement is that this database file be properly prepared; that is, have the schema set and seeds ran.
20 |
21 | Rails provides a Rake task for precisely this use: `db:prepare`. More importantly for us, though, is that Rails provides a corresponding Ruby method as well: `ActiveRecord::Tasks::DatabaseTasks.prepare_all`. We simply need to ensure that this is run whenever Rails boots, and this is just what the `config.after_initialize` hook is for. Since this is only a development feature, we can simply add this to our `config/environments/development.rb` file:
22 |
23 | ```ruby
24 | # Ensure that the git branch database schema is prepared
25 | config.after_initialize do
26 | ActiveRecord::Tasks::DatabaseTasks.prepare_all
27 | end
28 | ```
29 |
30 | This hook ensures that whenever Rails boots, our database will definitely be prepared. This means when you open a console session, start the application server, or run a `rails runner` task. This is a very powerful feature that can save you a lot of time and headache when working on multiple branches simultaneously.
31 |
32 | ### The Enhancement
33 |
34 | But, what if you want to copy the table data from one branch to another? Well, that's a bit more involved, but it's still quite doable. The core piece of the implementation puzzle is SQLite's [`ATTACH` functionality](https://www.sqlite.org/lang_attach.html), which allows you to, well, attach another database to the current database connection. This allows you to run queries that span multiple databases. The basic idea is to attach the source database to the target database, and then copy the data from the source to the target. Mixin a bit of dynamic string generation, and you can craft a shell function that merges all table data from a source database into a target database:
35 |
36 | ```sh
37 | db_merge() {
38 | target="$1"
39 | source="$2"
40 |
41 | # Attach merging database to base database
42 | merge_sql="ATTACH DATABASE '$source' AS merging; BEGIN TRANSACTION;"
43 | # Loop through each table in merging database
44 | for table_name in $(sqlite3 $source "SELECT name FROM sqlite_master WHERE type = 'table';")
45 | do
46 | columns=$(sqlite3 $source "SELECT name FROM pragma_table_info('$table_name');" | tr '\n' ',' | sed 's/.$//')
47 | # Merge table data into target database, ignoring any duplicate entries
48 | merge_sql+=" INSERT OR IGNORE INTO $table_name ($columns) SELECT $columns FROM merging.$table_name;"
49 | done
50 | merge_sql+=" COMMIT TRANSACTION; DETACH DATABASE merging;"
51 |
52 | sqlite3 "$target" "$merge_sql"
53 | }
54 | ```
55 |
56 | What I like to do is add a script to the `bin/` directory that provides the ability to branch or merge databases easily. Let's create a `bin/db` script and make it executable:
57 |
58 | ```sh
59 | touch bin/db
60 | chmod u+x bin/db
61 | ```
62 |
63 | In addition to merging table data, we can provide the ability to clone a database's schema into a new database as well:
64 |
65 | ```sh
66 | db_branch() {
67 | target="$1"
68 | source="$2"
69 |
70 | sqlite3 "$source" ".schema --nosys" | sqlite3 "$target"
71 | }
72 | ```
73 |
74 | All our `bin/db` script will do is provided structured access to these functions. We want it to support both a `branch` and a `merge` command, and the `branch` command should default to copying both the schema and the table data, but you can specify to only copy the schema. The `merge` command should only copy the table data.
75 |
76 | The file is relatively long (~175 lines), so I won't copy it here, but you can find it in the repository at this commit. In addition to our `after_initialize` automated hook, we now have the ability to branch and merge whatever SQLite databases we like, whenever we like.
77 |
78 | To give one example of how we could use this script to automate branching and copying table data, we could create a post-checkout git hook:
79 |
80 | ```sh
81 | touch .git/hooks/post-checkout
82 | chmod u+x .git/hooks/post-checkout
83 | ```
84 |
85 | And then write some shell to ensure that we have checked out a new branch and call our `bin/db branch` command with the new branch and previous branch:
86 |
87 | ```sh
88 | # If this is a file checkout, do nothing
89 | if [ "$3" == "0" ]; then exit; fi
90 |
91 | # If the prev and curr refs don't match, do nothing
92 | if [ "$1" != "$2" ]; then exit; fi
93 |
94 | reflog=$(git reflog)
95 | prev_branch=$(echo $reflog | awk 'NR==1{ print $6; exit }')
96 | curr_branch=$(echo $reflog | awk 'NR==1{ print $8; exit }')
97 | num_checkouts=$(echo $reflog | grep -o $curr_branch | wc -l)
98 |
99 | # If the number of checkouts equals one, a new branch has been created
100 | if [ ${num_checkouts} -eq 1 ]; then
101 | bin/db branch "storage/$curr_branch.sqlite3" "storage/$prev_branch.sqlite3" --with-data
102 | fi
103 | ```
104 |
105 | With this in place, we wouldn't really need the `after_initialize` Rails hook as our new branch database would be created in this post-checkout git hook. Moreover, in this example that database would include all of the data from the original branch database as well.
106 |
107 | Depending on how your team works, this kind of automation may be a bit too heavy handed. I personally prefer to simply have the `bin/db` script and run `bin/db merge` whenever I want to populate a new database with table data from a pre-existing database. But, I wanted to at least demonstrate the power and flexibility possible with these tools.
108 |
109 | Regardless of how precisely you wire everything together, working with branch-specific databases has been a solid developer experience improvement for me.
110 |
111 | ### Conclusion
112 |
113 | With the repository working with branch-specific databases, the final step is to setup error monitoring.
--------------------------------------------------------------------------------