├── .gitignore ├── .rspec ├── .travis.yml ├── Gemfile ├── Gemfile.lock ├── LICENSE ├── README.md ├── iso_latte.gemspec ├── lib ├── iso_latte.rb └── iso_latte │ └── version.rb └── spec ├── fork_spec.rb ├── spec_helper.rb └── tmp └── .keep /.gitignore: -------------------------------------------------------------------------------- 1 | spec/tmp/* 2 | -------------------------------------------------------------------------------- /.rspec: -------------------------------------------------------------------------------- 1 | --color 2 | --format progress 3 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: ruby 2 | cache: bundler 3 | rvm: 4 | - 2.2 5 | - 2.1 6 | - 2.0 7 | - 1.9 8 | - 1.8 9 | - ree-1.8.7 10 | script: bundle exec rspec spec 11 | sudo: false 12 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source "https://rubygems.org" 2 | 3 | gemspec :name => "iso_latte" 4 | 5 | gem "rspec" 6 | gem "segfault" 7 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | PATH 2 | remote: . 3 | specs: 4 | iso_latte (1.7) 5 | 6 | GEM 7 | remote: https://rubygems.org/ 8 | specs: 9 | diff-lcs (1.2.5) 10 | rspec (3.3.0) 11 | rspec-core (~> 3.3.0) 12 | rspec-expectations (~> 3.3.0) 13 | rspec-mocks (~> 3.3.0) 14 | rspec-core (3.3.1) 15 | rspec-support (~> 3.3.0) 16 | rspec-expectations (3.3.0) 17 | diff-lcs (>= 1.2.0, < 2.0) 18 | rspec-support (~> 3.3.0) 19 | rspec-mocks (3.3.1) 20 | diff-lcs (>= 1.2.0, < 2.0) 21 | rspec-support (~> 3.3.0) 22 | rspec-support (3.3.0) 23 | segfault (0.0.2) 24 | 25 | PLATFORMS 26 | ruby 27 | 28 | DEPENDENCIES 29 | iso_latte! 30 | rspec 31 | segfault 32 | 33 | BUNDLED WITH 34 | 1.10.6 35 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | copyright (c) 2013, Emcien Corporation 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | * Redistributions of source code must retain the above copyright 7 | notice, this list of conditions and the following disclaimer. 8 | * Redistributions in binary form must reproduce the above copyright 9 | notice, this list of conditions and the following disclaimer in the 10 | documentation and/or other materials provided with the distribution. 11 | * Neither the name of the nor the 12 | names of its contributors may be used to endorse or promote products 13 | derived from this software without specific prior written permission. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 16 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY 19 | DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 22 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # IsoLatte 2 | 3 | Sometimes you need to run background jobs that you can't make important 4 | guarantees about - they may run out of memory and get killed, or produce 5 | segmentation faults, or `exit!` directly - and you need to be able to clean 6 | up after such problems. 7 | 8 | IsoLatte is a gem that allows a block of code to be executed in a subprocess. 9 | Exceptions get passed back to the parent process through a pipe, and various 10 | exit conditions are handled via configurable callbacks. 11 | 12 | [![Build Status](https://travis-ci.org/emcien/iso_latte.svg?branch=master)](https://travis-ci.org/emcien/iso_latte) 13 | 14 | ## Simple Process Isolation 15 | 16 | ```ruby 17 | IsoLatte.fork do 18 | do_something_crazy! 19 | end 20 | ``` 21 | 22 | `do_something_crazy!` is now being invoked in a forked subprocess - if it 23 | crashes the interpreter, or gets killed by the OS, instead of taking down 24 | the original process, it will invoke the appropriate callback in the parent. 25 | 26 | ## Complex Example 27 | 28 | ```ruby 29 | IsoLatte.fork( 30 | stderr: "/tmp/suberr.txt", 31 | finish: ->(success, rc) { warn "Finished. Success? #{success}" }, 32 | success: ->() { warn "Was successful" }, 33 | kill: ->() { warn "Received a SIGKILL" }, 34 | fault: ->() { warn "Received a SIGABRT, probably a segmentation fault" }, 35 | exit: ->(rc) { warn "subprocess exited explicitly with #{rc}" } 36 | ) { do_something_crazy! } 37 | ``` 38 | 39 | ### Options 40 | 41 | * `stderr` - A path to write the subprocess' stderr stream into. Defaults to '/dev/null', 42 | supplying `nil` lets the subprocess continue writing to the parent's stderr stream. 43 | * `success` - a callable to execute if the subprocess completes successfully. 44 | * `fault` - a callable to execute if the subprocess receives a SIGABRT (segfault). 45 | * `kill` - a callable to execute if the subprocess receives a SIGKILL (from `kill -9` or oom-killer) 46 | * `exit` - a callable to execute if the subprocess explicitly exits with nonzero status. 47 | Receives the exit status value as its argument. 48 | * `finish` - a callable to execute when the subprocess terminates in any way. It receives 49 | a boolean 'success' value and an exit status as its arguments. 50 | * `timeout` - a number of seconds to wait - if the process has not terminated by then, 51 | the parent will kill it by issuing a SIGKILL signal (triggering the kill callback) 52 | 53 | ## Supported Platforms 54 | 55 | IsoLatte requires `Process.fork`, `Process.waitpid2`, and `IO.pipe`, and also requires 56 | `Timeout.timeout` and `Process.kill` to function properly. That means that Jruby is 57 | unsupported (no `fork`), and that Windows is certainly unsupported (no anything). 58 | 59 | Currently tested in travis and supported: MRI 2.2, 2.1, 2.0, 1.9, 1.8.7, and REE 1.8.7 60 | 61 | The *tests* (and the Gemfile) requires that a C extension be compiles for the `segfault` 62 | gem - that gem is not required for functioning, but it's an important test to run - if 63 | you know of a good way to compatibly test that on other platforms, I'm interested in a 64 | pull request. 65 | 66 | ## Roadmap 67 | 68 | 1. Add a convenient mechanism for sending a single marshaled object back to the parent afterward. 69 | 2. Allow the `stderr` to accept a callback to call for each line instead. 70 | 3. Improve compatibility with various gems that modify Exception, like `better_errors`. 71 | -------------------------------------------------------------------------------- /iso_latte.gemspec: -------------------------------------------------------------------------------- 1 | require File.expand_path("../lib/iso_latte/version", __FILE__) 2 | 3 | Gem::Specification.new do |spec| 4 | spec.name = "iso_latte" 5 | spec.version = IsoLatte::VERSION 6 | spec.date = Time.now.utc.strftime("%Y-%m-%d") 7 | spec.summary = "A gem for isolating execution in a subprocess" 8 | spec.description = "IsoLatte allows execution to be forked from the main process for safety" 9 | 10 | spec.authors = ["Emcien Engineering", "Eric Mueller"] 11 | spec.email = ["engineering@emcien.com"] 12 | spec.license = "BSD-3-Clause" 13 | spec.homepage = "https://github.com/emcien/iso_latte" 14 | 15 | spec.files = %w(lib/iso_latte.rb lib/iso_latte/version.rb) 16 | spec.require_paths = ["lib"] 17 | end 18 | -------------------------------------------------------------------------------- /lib/iso_latte.rb: -------------------------------------------------------------------------------- 1 | require "ostruct" 2 | require "timeout" 3 | 4 | module IsoLatte 5 | NO_EXIT = 122 6 | EXCEPTION_RAISED = 123 7 | 8 | # Available options: 9 | # stderr: a path to write stderr into. Defaults to '/dev/null' 10 | # nil means "do not change stderr" 11 | # finish: a callable to execute when the subprocess terminates (in any way). 12 | # receives a boolean 'success' value, and an exitstatus as arguments 13 | # success: a callable to execute if the subprocess completes successfully. 14 | # note: this can exit ALONGSIDE the exit callback, if the subprocess 15 | # exits with zero explicitly! 16 | # kill: a callable to execute if the subprocess is killed (SIGKILL). 17 | # fault: a callable to execute if the subprocess segfaults, core dumps, etc. 18 | # exit: a callable to execute if the subprocess voluntarily exits with nonzero. 19 | # receives the exit status value as its argument. 20 | # timeout: after this many seconds, the parent should send a SIGKILL to the child. 21 | # 22 | # It is allowable to Isolatte.fork from inside an IsoLatte.fork block (reentrant) 23 | # 24 | # We are using the exit statuses of 122 and 123 as sentinels that mean 25 | # 'the code did not exit on its own' and 'the code raised an exception'. 26 | # If you have code that actually uses those exit statuses.. change the special 27 | # statuses I guess. 28 | # 29 | def self.fork(options = nil, &block) 30 | defaults = { :stderr => "/dev/null", :exit => nil } 31 | opts = OpenStruct.new(defaults.merge(options || {})) 32 | 33 | read_ex, write_ex = IO.pipe 34 | 35 | child_pid = Process.fork do 36 | read_ex.close 37 | begin 38 | if opts.stderr 39 | File.open(opts.stderr, "w") do |stderr_file| 40 | STDERR.reopen(stderr_file) 41 | STDERR.sync = true 42 | $stderr = STDERR 43 | block.call 44 | end 45 | else 46 | block.call 47 | end 48 | rescue StandardError => e 49 | marshal(e) # To check if it works before writing any of it to the stream 50 | marshal(e, write_ex) 51 | write_ex.flush 52 | write_ex.close 53 | exit!(EXCEPTION_RAISED) 54 | end 55 | 56 | exit!(NO_EXIT) 57 | end 58 | 59 | write_ex.close 60 | 61 | pid, rc = 62 | begin 63 | if opts.timeout 64 | Timeout.timeout(opts.timeout) { Process.wait2(child_pid) } 65 | else 66 | Process.wait2(child_pid) 67 | end 68 | rescue Timeout::Error 69 | kill_child(child_pid) 70 | end 71 | 72 | fail(Error, "Wrong child's exit received!") unless pid == child_pid 73 | 74 | if rc.exited? && rc.exitstatus == EXCEPTION_RAISED 75 | e = Marshal.load read_ex 76 | read_ex.close 77 | fail e 78 | else 79 | read_ex.close 80 | end 81 | 82 | success = rc.success? || rc.exitstatus == NO_EXIT 83 | code = rc.exitstatus == NO_EXIT ? 0 : rc.exitstatus 84 | 85 | if success 86 | opts.success.call if opts.success 87 | else 88 | opts.fault.call if opts.fault && (rc.termsig == 6 || rc.coredump?) 89 | opts.kill.call if opts.kill && rc.termsig == 9 90 | end 91 | 92 | # This can execute on success OR failure - it indicates that the subprocess 93 | # *explicitly* exited, whether with zero or nonzero. 94 | opts.exit.call(rc.exitstatus) if opts.exit && rc.exited? && rc.exitstatus != NO_EXIT 95 | 96 | # This should execute regardless of the outcome 97 | # (unless some other hook raises an exception first) 98 | opts.finish.call(success, code) if opts.finish 99 | 100 | rc 101 | end 102 | 103 | def self.kill_child(pid) 104 | Process.kill("KILL", pid) 105 | Process.wait2(pid) 106 | rescue Errno::ESRCH 107 | # Save us from the race condition where it exited just as we decided to kill it. 108 | end 109 | 110 | def self.marshal(e, io = nil) 111 | begin 112 | return io ? Marshal.dump(e, io) : Marshal.dump(e) 113 | rescue NoMethodError 114 | rescue TypeError 115 | end 116 | 117 | begin 118 | e2 = e.class.new(e.message) 119 | e2.set_backtrace(e.backtrace) 120 | return io ? Marshal.dump(e2, io) : Marshal.dump(e2) 121 | rescue NoMethodError 122 | rescue TypeError 123 | end 124 | 125 | e3 = IsoLatte::Error.new("Marshalling error with: #{e.message}") 126 | e3.set_backtrace(e.backtrace) 127 | io ? Marshal.dump(e3, io) : Marshal.dump(e3) 128 | end 129 | 130 | class Error < StandardError; end 131 | end 132 | -------------------------------------------------------------------------------- /lib/iso_latte/version.rb: -------------------------------------------------------------------------------- 1 | module IsoLatte 2 | VERSION = "1.7" 3 | end 4 | -------------------------------------------------------------------------------- /spec/fork_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | require "fileutils" 3 | require "segfault" 4 | require "English" 5 | 6 | describe "IsoLatte.fork" do 7 | before(:each) do 8 | @exit_status = nil 9 | @killed = @faulted = @exited = @finished = @success = false 10 | end 11 | 12 | def kill_self!; `kill -9 #{$PROCESS_ID}`; end 13 | 14 | def segfault!; Segfault.dereference_null; end 15 | 16 | let(:on_kill) { lambda { @killed = true } } 17 | let(:on_fault) { lambda { @faulted = true } } 18 | let(:on_exit) { lambda { |rc| @exited, @exit_status = [true, rc] } } 19 | let(:on_finish) { lambda { |_s, c| @finished, @exit_status = [true, c] } } 20 | let(:on_success) { lambda { @success = true } } 21 | 22 | let(:opts) do 23 | { :success => on_success, 24 | :kill => on_kill, 25 | :exit => on_exit, 26 | :finish => on_finish, 27 | :fault => on_fault } 28 | end 29 | 30 | it "should run the block in a subprocess" do 31 | @ran_here = nil 32 | IsoLatte.fork do 33 | @ran_here = true 34 | FileUtils.touch tmp_path("ran_at_all", :clean => true) 35 | end 36 | 37 | expect(@ran_here).to be_nil 38 | expect(File.exist?(tmp_path "ran_at_all")).to be_truthy 39 | end 40 | 41 | it "should write stderr to the specified location" do 42 | IsoLatte.fork(:stderr => tmp_path("fork.err", :clean => true)) do 43 | warn "line 1" 44 | warn "line 2" 45 | end 46 | 47 | expect(File.exist? tmp_path("fork.err")).to eq(true) 48 | expect(File.read(tmp_path "fork.err")).to eq("line 1\nline 2\n") 49 | end 50 | 51 | it "should not redirect stderr if opts.stderr is nil" do 52 | $stderr = s = StringIO.open("", "w") 53 | IsoLatte.fork(:stderr => nil) do 54 | warn "again" 55 | File.open(tmp_path("fork2.stderr", :clean => true), "w") { |f| f.puts s.string } 56 | end 57 | $stderr = STDERR 58 | 59 | expect(File.read(tmp_path "fork2.stderr").strip).to eq("again") 60 | end 61 | 62 | it "should handle a clean finish correctly" do 63 | IsoLatte.fork(opts) { warn "do nothing" } 64 | expect(@killed).to eq(false) 65 | expect(@faulted).to eq(false) 66 | expect(@exited).to eq(false) 67 | expect(@finished).to eq(true) 68 | expect(@success).to eq(true) 69 | end 70 | 71 | it "should handle an explicit successful exit correctly" do 72 | IsoLatte.fork(opts) { exit! 0 } 73 | expect(@killed).to eq(false) 74 | expect(@faulted).to eq(false) 75 | expect(@exited).to eq(true) 76 | expect(@finished).to eq(true) 77 | expect(@success).to eq(true) 78 | expect(@exit_status).to eq(0) 79 | end 80 | 81 | it "should handle an explicit nonzero exit correctly" do 82 | IsoLatte.fork(opts) { exit! 12 } 83 | expect(@killed).to eq(false) 84 | expect(@faulted).to eq(false) 85 | expect(@exited).to eq(true) 86 | expect(@finished).to eq(true) 87 | expect(@success).to eq(false) 88 | expect(@exit_status).to eq(12) 89 | end 90 | 91 | it "should handle a SIGKILL correctly" do 92 | IsoLatte.fork(opts) { kill_self! } 93 | expect(@killed).to eq(true) 94 | expect(@faulted).to eq(false) 95 | expect(@exited).to eq(false) 96 | expect(@finished).to eq(true) 97 | expect(@success).to eq(false) 98 | end 99 | 100 | it "should handle a segfault correctly" do 101 | IsoLatte.fork(opts) { segfault! } 102 | expect(@killed).to eq(false) 103 | expect(@faulted).to eq(true) 104 | expect(@exited).to eq(false) 105 | expect(@finished).to eq(true) 106 | expect(@success).to eq(false) 107 | end 108 | 109 | it "should allow recursive isolation" do 110 | IsoLatte.fork(opts) do 111 | IsoLatte.fork(opts) do 112 | IsoLatte.fork(opts) do 113 | segfault! 114 | end 115 | kill_self! 116 | end 117 | 118 | exit(15) 119 | end 120 | 121 | expect(@killed).to eq(false) 122 | expect(@faulted).to eq(false) 123 | expect(@exited).to eq(true) 124 | expect(@exit_status).to eq(15) 125 | end 126 | 127 | it "should raise exceptions out of the isolated process" do 128 | expect do 129 | IsoLatte.fork(opts) do 130 | fail ArgumentError, "Foo bar bar bar" 131 | end 132 | end.to raise_exception(ArgumentError, "Foo bar bar bar") 133 | end 134 | end 135 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require "iso_latte" 2 | 3 | RSpec.configure do |config| 4 | config.run_all_when_everything_filtered = true 5 | config.filter_run :focus 6 | config.order = "random" 7 | end 8 | 9 | TMPDIR = File.expand_path("../tmp", __FILE__) 10 | def tmp_path(name, opts = {}) 11 | path = File.join(TMPDIR, name) 12 | File.unlink(path) if opts[:clean] && File.exist?(path) 13 | path 14 | end 15 | -------------------------------------------------------------------------------- /spec/tmp/.keep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emcien/iso_latte/379c898765b3b9d2ae7d3b9289851cdbb66c1efb/spec/tmp/.keep --------------------------------------------------------------------------------