├── .gitignore
├── .yardopts
├── LICENSE
├── README.md
├── exeggutor.gemspec
├── lib
└── exeggutor.rb
├── misc
├── left.png
└── right.png
└── test
└── test.rb
/.gitignore:
--------------------------------------------------------------------------------
1 | *.gem
2 |
3 | ## Documentation cache and generated files:
4 | /.yardoc/
5 | /_yardoc/
6 | /doc/
7 | /rdoc/
8 |
--------------------------------------------------------------------------------
/.yardopts:
--------------------------------------------------------------------------------
1 | --no-private
2 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2025 Michael Eisel
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Exeggutor 🌴
2 |
3 | #### A Simple, Capable, and Unified Interface for Managing Subprocesses in Ruby
4 |
5 | Tired of juggling `system(...)`, `` `...` ``, and `Open3`? Exeggutor provides one simple method that handles many different use cases - safely spawn processes with real-time output, captured stdout/stderr, and sane error handling.
6 |
7 | From this:
8 |
9 | 
10 |
11 | To this:
12 |
13 | 
14 |
15 | #### Examples
16 |
17 | ```ruby
18 | # Copy old_file to #{new_dir}/foo, and raise an exception if it fails
19 | exeg(%W[cp #{old_file} #{new_dir}/foo]) # Exception raised by default on failure
20 |
21 | # Collect stdout from a long-running build task while showing the progress updates as they're
22 | # printed out to stderr
23 | output = exeg(%W[run_build.sh], show_stderr: true).stdout
24 |
25 | # Lots of overrides
26 | result = exeg(
27 | %W[foo bar],
28 | env: {SOME_OVERRIDE: "1"}, # Override env vars
29 | chdir: "/path/to/...", # Run the process in a different working directory
30 | can_fail: true, # Don't throw an exception, handle the error manually or ignore
31 | stdin: "foo", # Send this data to stdin as soon as the process starts
32 | )
33 |
34 | # Async execution - like popen3, but without its deadlock issue
35 | handle = exeg_async(%W[long_running_process.sh])
36 | handle.stdin.write("request")
37 | response = handle.stdout.gets
38 | ```
39 |
40 | #### Overview
41 |
42 | Although Ruby has many different ways of running a subprocess, they all have various drawbacks and quirks. Also, some of the most convenient ways of calling a process, e.g. with backticks, are the most dangerous, because they spawn a subshell. Here's an overview of how Exeggutor solves these shortcomings:
43 |
44 | |Problem with Standard Ruby APIs|Exeggutor Solution|
45 | |-|-|
46 | |Subshells are slow to spawn, error-prone, and insecure | Exeggutor never uses a subshell and always runs processes directly|
47 | |Non-subshells use ugly varargs syntax (e.g. `system('cp', old, "#{new}/foo")`) |Exeggutor encourages elegant %W syntax by taking an array for the arguments parameter (e.g. `exeg(%W[cp #{old} #{new}/foo])`)|
48 | |Process failures are silent, requiring manual checks|Exeggutor raises an exception on failure by default (with rich error context)|
49 | |No simple way to both capture stdout/stderr as strings afterwards and also print them to the shell in real-time |Exeggutor always captures stdout/stderr, and can optionally print them in real-time|
50 | |Different APIs for different use cases|Exeggutor provides a single method for blocking calls and another for non-blocking execution, with smart defaults and flexible named parameters|
51 |
52 | #### Installation
53 |
54 | ```
55 | gem install exeggutor
56 | ```
57 |
58 | #### Documentation
59 |
60 | Docs are available [here](https://www.rubydoc.info/gems/exeggutor/toplevel).
61 |
--------------------------------------------------------------------------------
/exeggutor.gemspec:
--------------------------------------------------------------------------------
1 | Gem::Specification.new do |s|
2 | s.name = "exeggutor"
3 | s.version = "0.1.5"
4 | s.summary = "A Simple, Capable, and Unified Interface for Running Subprocesses in Ruby"
5 | # s.description = "A longer description of your gem"
6 | s.authors = ["Michael Eisel"]
7 | s.email = "michael.eisel@gmail.com"
8 | s.files = Dir["lib/**/*.rb"]
9 | s.homepage = "https://github.com/michaeleisel/Exeggutor"
10 | s.license = "MIT"
11 |
12 | s.add_development_dependency "minitest"
13 | end
14 |
--------------------------------------------------------------------------------
/lib/exeggutor.rb:
--------------------------------------------------------------------------------
1 | require 'open3'
2 | require 'shellwords'
3 |
4 | module Exeggutor
5 |
6 | # A handle to a process, with IO handles to communicate with it
7 | # and a {ProcessResult} object when it's done. It's largely similar to the array
8 | # of 4 values return by Open3.popen3. However, it doesn't suffer from that library's
9 | # dead-locking issue. For example, even if lots of data has been written to stdout that hasn't been
10 | # read, the subprocess can still write to stdout and stderr without blocking
11 | class ProcessHandle
12 | # @private
13 | def initialize(args, env: nil, chdir: nil)
14 | @stdin_io, @stdout_io, @stderr_io, @wait_thread = Exeggutor::run_popen3(args, env, chdir)
15 |
16 | # Make the streams as synchronous as possible, to minimize the possibility of a surprising lack
17 | # of output
18 | @stdout_io.sync = true
19 | @stderr_io.sync = true
20 |
21 | @stdout_queue = Queue.new
22 | @stderr_queue = Queue.new
23 |
24 | @stdout_pipe_reader, @stdout_pipe_writer = IO.pipe
25 | @stderr_pipe_reader, @stderr_pipe_writer = IO.pipe
26 |
27 | @stdout_write_thread = Thread.new do
28 | loop do
29 | data = @stdout_queue.pop
30 | break if !data # Queue is closed
31 | @stdout_pipe_writer.write(data)
32 | end
33 | @stdout_pipe_writer.close
34 | end
35 |
36 | @stderr_write_thread = Thread.new do
37 | loop do
38 | data = @stderr_queue.pop
39 | break if !data # Queue is closed
40 | @stderr_pipe_writer.write(data)
41 | end
42 | @stderr_pipe_writer.close
43 | end
44 |
45 | # popen3 can deadlock if one stream is written to too much without being read,
46 | # so it's important to continuously read from both streams. This is why
47 | # we can't just let the user call .gets on the streams themselves
48 | @read_thread = Thread.new do
49 | remaining_ios = [@stdout_io, @stderr_io]
50 | while remaining_ios.size > 0
51 | readable_ios, = IO.select(remaining_ios)
52 | for readable_io in readable_ios
53 | begin
54 | data = readable_io.read_nonblock(100_000)
55 | if readable_io == @stdout_io
56 | @stdout_queue.push(data)
57 | else
58 | @stderr_queue.push(data)
59 | end
60 | rescue IO::WaitReadable
61 | # Shouldn't usually happen because IO.select indicated data is ready, but maybe due to EINTR or something
62 | next
63 | rescue EOFError
64 | if readable_io == @stdout_io
65 | @stdout_queue.close
66 | else
67 | @stderr_queue.close
68 | end
69 | remaining_ios.delete(readable_io)
70 | end
71 | end
72 | end
73 | end
74 | end
75 |
76 | # An object containing process metadata and which can be waited on to wait
77 | # until the subprocess ends. Identical to popen3's wait_thr
78 | def wait_thr
79 | @wait_thread
80 | end
81 |
82 | # An IO object for stdin that can be written to
83 | def stdin
84 | @stdin_io
85 | end
86 |
87 | # An IO object for stdout that can be written to
88 | def stdout
89 | @stdout_pipe_reader
90 | end
91 |
92 | # An IO object for stderr that can be written to
93 | def stderr
94 | @stderr_pipe_reader
95 | end
96 | end
97 |
98 | # Represents the result of a process execution.
99 | #
100 | # @attr_reader stdout [String] The standard output of the process.
101 | # @attr_reader stderr [String] The standard error of the process.
102 | # @attr_reader exit_code [Integer] The exit code of the process.
103 | class ProcessResult
104 | attr_reader :stdout, :stderr, :exit_code, :pid
105 |
106 | # @private
107 | def initialize(stdout:, stderr:, exit_code:, pid:)
108 | @stdout = stdout
109 | @stderr = stderr
110 | @exit_code = exit_code
111 | @pid = pid
112 | end
113 |
114 | # Checks if the process was successful.
115 | #
116 | # @return [Boolean] True if the exit code is 0, otherwise false.
117 | def success?
118 | exit_code == 0
119 | end
120 | end
121 |
122 | # Represents an error that occurs during a process execution.
123 | # The error contains a {ProcessResult} object with details about the process.
124 | #
125 | # @attr_reader result {ProcessResult} The result of the process execution.
126 | class ProcessError < StandardError
127 | attr_reader :result
128 |
129 | # @private
130 | def initialize(result)
131 | @result = result
132 | end
133 | end
134 |
135 | # @private
136 | def self.run_popen3(args, env, chdir)
137 | # Use this weird [args[0], args[0]] thing for the case where a command with just one arg is being run
138 | opts = {}
139 | opts[:chdir] = chdir if chdir
140 | if env
141 | Open3.popen3(env, [args[0], args[0]], *args.drop(1), opts)
142 | else
143 | Open3.popen3([args[0], args[0]], *args.drop(1), opts)
144 | end
145 | end
146 |
147 | end
148 |
149 | # Executes a command with the provided arguments and options. Waits for the process to finish.
150 | #
151 | # @param args [Array] The command and its arguments as an array.
152 | # @param can_fail [Boolean] If false, raises a ProcessError on failure.
153 | # @param show_stdout [Boolean] If true, prints stdout to the console in real-time.
154 | # @param show_stderr [Boolean] If true, prints stderr to the console in real-time.
155 | # @param chdir [String, nil] The working directory to run the command in. If nil, uses the current working directory.
156 | # @param stdin [String, nil] Input data to pass to the command's stdin. If nil, doesn't pass any data to stdin.
157 | # @param env [Hash{String => String}, nil] A hashmap containing environment variable overrides,
158 | # or `nil` if no overrides are desired
159 | #
160 | # @return {ProcessResult} An object containing process info such as stdout, stderr, and exit code.
161 | #
162 | # @raise {ProcessError} If the command fails and `can_fail` is false.
163 | def exeg(args, can_fail: false, show_stdout: false, show_stderr: false, env: nil, chdir: nil, stdin_data: nil)
164 | raise "args.size must be >= 1" if args.empty?
165 |
166 | stdin_io, stdout_io, stderr_io, wait_thr = Exeggutor::run_popen3(args, env, chdir)
167 | stdin_io.write(stdin_data) if stdin_data
168 | stdin_io.close
169 |
170 | # Make the streams as synchronous as possible, to minimize the possibility of a surprising lack
171 | # of output
172 | stdout_io.sync = true
173 | stderr_io.sync = true
174 |
175 | stdout = +''
176 | stderr = +''
177 |
178 | # Although there could be more code sharing between this and exeg_async, it would either complicate exeg_async's inner workings
179 | # or force us to pay the same performance cost that exeg_async does
180 | remaining_ios = [stdout_io, stderr_io]
181 | while remaining_ios.size > 0
182 | readable_ios, = IO.select(remaining_ios)
183 | for readable_io in readable_ios
184 | begin
185 | data = readable_io.read_nonblock(100_000)
186 | if readable_io == stdout_io
187 | stdout << data
188 | $stdout.print(data) if show_stdout
189 | else
190 | stderr << data
191 | $stderr.print(data) if show_stderr
192 | end
193 | rescue IO::WaitReadable
194 | # Shouldn't usually happen because IO.select indicated data is ready, but maybe due to EINTR or something
195 | next
196 | rescue EOFError
197 | remaining_ios.delete(readable_io)
198 | end
199 | end
200 | end
201 |
202 | result = Exeggutor::ProcessResult.new(
203 | stdout: stdout,
204 | stderr: stderr,
205 | exit_code: wait_thr.value.exitstatus,
206 | pid: wait_thr.pid
207 | )
208 | if !can_fail && !result.success?
209 | error_str = <<~ERROR_STR
210 | Command failed: #{args.shelljoin}
211 | Exit code: #{result.exit_code}
212 | stdout: #{result.stdout}
213 | stderr: #{result.stderr}
214 | pid: #{result.pid}
215 | ERROR_STR
216 | raise Exeggutor::ProcessError.new(result), error_str
217 | end
218 |
219 | result
220 | end
221 |
222 | # Executes a command with the provided arguments and options. Does not wait for the process to finish.
223 | #
224 | # @param args [Array] The command and its arguments as an array.
225 | # @param chdir [String, nil] The working directory to run the command in. If nil, uses the current working directory.
226 | # @param env [Hash{String => String}, nil] A hashmap containing environment variable overrides,
227 | # or `nil` if no overrides are desired
228 | #
229 | # @return {ProcessHandle}
230 | def exeg_async(args, env: nil, chdir: nil)
231 | Exeggutor::ProcessHandle.new(args, env: env, chdir: chdir)
232 | end
233 |
--------------------------------------------------------------------------------
/misc/left.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/michaeleisel/Exeggutor/79638c5609b99545e844c04fc251e3c4cccc6027/misc/left.png
--------------------------------------------------------------------------------
/misc/right.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/michaeleisel/Exeggutor/79638c5609b99545e844c04fc251e3c4cccc6027/misc/right.png
--------------------------------------------------------------------------------
/test/test.rb:
--------------------------------------------------------------------------------
1 | require "minitest/autorun"
2 | require_relative "../lib/exeggutor"
3 | require 'pry-byebug'
4 |
5 | class TestYourGem < Minitest::Test
6 | def test_various
7 | result = exeg(%W[echo hi])
8 | assert_equal "hi\n", result.stdout
9 | script = 'warn "this is stderr"; puts "this is stdout"; exit 1'
10 |
11 | error = assert_raises(Exeggutor::ProcessError) do
12 | exeg(%W[ruby -e #{script}])
13 | end
14 | assert_equal "this is stderr\n", error.result.stderr
15 | assert_equal "this is stdout\n", error.result.stdout
16 |
17 | result = exeg(%W[ruby -e #{script}], can_fail: true)
18 | assert_equal "this is stderr\n", error.result.stderr
19 | assert_equal "this is stdout\n", error.result.stdout
20 |
21 | exeg(%W[ruby -e #{script}], show_stdout: true, can_fail: true)
22 | exeg(%W[ruby -e #{script}], show_stderr: true, can_fail: true)
23 |
24 | result = exeg(%W[cat], stdin_data: "hi")
25 | assert_equal "hi", result.stdout
26 | assert_equal true, result.pid > 0
27 | end
28 |
29 | def test_async
30 | return if ENV["EXEG_SKIP_ASYNC"] == "1"
31 |
32 | handle = exeg_async(%W[ruby -e] + ["puts 'foo' ; sleep 1 ; warn 'bar' ; sleep 1 ; print 'done'"])
33 | assert_equal(handle.stdout.gets, "foo\n")
34 | assert_equal(handle.stderr.gets, "bar\n")
35 | assert_equal(handle.stdout.gets, "done")
36 | assert_equal(handle.stdout.gets, nil)
37 | assert_equal(handle.stderr.gets, nil)
38 | end
39 |
40 | def test_chdir
41 | handle = exeg_async(%W[false])
42 | assert_equal(handle.wait_thr.value.exitstatus, 1)
43 |
44 | Dir.mktmpdir do |dir|
45 | assert_equal(exeg(%W[pwd], chdir: dir).stdout, "#{File.realpath(dir)}\n")
46 | end
47 | end
48 | end
49 |
--------------------------------------------------------------------------------