├── .editorconfig ├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── benchmarks ├── baseline.cr └── nested_libevent.cr ├── shard.yml ├── spec ├── error_propagator_spec.cr ├── examples │ └── example.cr ├── libevent_context_spec.cr ├── main_spec.cr ├── nested_scheduler_spec.cr ├── result_collector_spec.cr ├── silent_spec.cr └── spec_helper.cr └── src ├── nested_scheduler.cr └── nested_scheduler ├── io_context.cr ├── libevent_context.cr ├── linked_list2.cr ├── monkeypatch ├── event_loop.cr ├── evented.cr ├── exception.cr ├── fiber.cr ├── fiber_channel.cr ├── file_descriptor.cr ├── main.cr ├── scheduler.cr └── socket.cr ├── result.cr ├── results ├── error_propagator.cr ├── result_collector.cr └── silent.cr └── thread_pool.cr /.editorconfig: -------------------------------------------------------------------------------- 1 | root = true 2 | 3 | [*.cr] 4 | charset = utf-8 5 | end_of_line = lf 6 | insert_final_newline = true 7 | indent_style = space 8 | indent_size = 2 9 | trim_trailing_whitespace = true 10 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /docs/ 2 | /lib/ 3 | /bin/ 4 | /.shards/ 5 | *.dwarf 6 | 7 | # Libraries don't need dependency lock 8 | # Dependencies will be locked in applications that use them 9 | /shard.lock 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2020 your-name-here 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY : clean run all 2 | 3 | all : clean run 4 | 5 | clean : 6 | rm -rf ./test/* 7 | 8 | s : 9 | crystal spec -Dpreview_mt --error-trace spec/io_uring_context_spec.cr 10 | le : 11 | crystal spec -Dpreview_mt --error-trace spec/libevent_context_spec.cr:30 12 | 13 | run : 14 | mkdir -p test 15 | CRYSTAL_LOAD_DWARF=1 crystal spec -Dpreview_mt --error-trace --stats 16 | 17 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Recent updates 2 | 3 | With the amount of changes in these parts of the source and the dependency on monkeypatching things, I'll probably won't update this to work with recent versions of crystal before it stabilizes again. 4 | 5 | The upside is that the changes that are made is generally great and will allow for better implementations. 6 | 7 | # nested_scheduler 8 | 9 | Nested Scheduler is an expansion and/or replacement for the built in 10 | fiber scheduler of Crystal. It allows setting up thread pools with one 11 | or more dedicated threads that will handle a set of fibers. It draws 12 | inspiration from [Notes on Structured 13 | Concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/) 14 | and from [Structured Concurrency](https://250bpm.com/blog:71/). Both 15 | articles are very much recommended read. 16 | 17 | As such, it follows a couple of core ideas: 18 | 19 | * Don't break abstraction. All fibers have a predefined lifetime 20 | defined by a section of code it can exist in and fiber lifetimes are 21 | strictly hierarchical - no partial overlaps. This helps by 22 | preventing many race conditions, but also by making silent leaks of 23 | fibers that never finishes executing a bit more visible. 24 | * Simplify resource cleanup. This may sound counterintuitive in a 25 | language like Crystal which has a garbage collector but it turns out 26 | to be very nice to be able to use local variables defined in the 27 | surrounding scope inside a fiber without fear of it going out of 28 | scope. This is especially true for file handles or other resources 29 | that often are closed when they go out of scope (using an `ensure` 30 | statement) 31 | * If there is an exception in a fiber, then the exception will by 32 | default be propagated and reraised in the originating context. This 33 | creates debuggable stacktraces with information about both what went 34 | wrong and how to get there. 35 | 36 | The constructs this library provide are related to constructs like 37 | supervisors in Erlang (a bit less powerful) and waitgroups/errgroups 38 | in Go (a bit more powerful). 39 | 40 | ## Installation 41 | 42 | 1. Add the dependency to your `shard.yml`: 43 | 44 | ```yaml 45 | dependencies: 46 | nested_scheduler: 47 | github: yxhuvud/nested_scheduler 48 | ``` 49 | 50 | 2. Run `shards install` 51 | 52 | ## Usage 53 | 54 | All usage of Nested Scheduler is assuming the program is compiled with 55 | -Dpreview_mt. It will not work without it. Additionally, 56 | `nested_scheduler` is implemented by monkeypatching many classes that 57 | is very much core to Crystal, so beware that this can be quite fragile 58 | if the crystal implementation changes. 59 | 60 | ### Basics 61 | 62 | The basics of the nested scheduler is the 63 | `NestedScheduler::ThreadPool.nursery` block. The point of that is to 64 | spawn fibers from the yielded pool. 65 | 66 | ```crystal 67 | require "nested_scheduler" 68 | 69 | NestedScheduler::ThreadPool.nursery do |pool| 70 | pool.spawn { sleep 5 } 71 | pool.spawn { sleep 2 } 72 | end 73 | ``` 74 | 75 | These fibers will execute concurrently (and potentially in parallel), 76 | and the `nursery` method will not return unless **all** fibers in the 77 | pool have finished. Calling `spawn` inside the block will generate a 78 | new fiber in the *current* pool. So the following would end up 79 | generating two fibers in the created pool: 80 | 81 | ```crystal 82 | require "nested_scheduler" 83 | 84 | NestedScheduler::ThreadPool.nursery do |pool| 85 | pool.spawn { spawn { sleep 5 } } 86 | end 87 | ``` 88 | 89 | WARNING: `nested_scheduler` replaces the built in scheduler with 90 | itself, which means that PROGRAMS THAT SPAWN FIBERS WILL NOT EXIT 91 | UNTIL ALL FIBERS HAVE STOPPED RUNNING. This is in general a very good 92 | thing, but it may be disruptive for programs not built assuming that. 93 | 94 | ### Threads 95 | Nested Scheduler defaults to spawning a single thread to process 96 | fibers, but it supports spawning more. To create a nursery with 97 | multiple worker threads, instantiate it like 98 | 99 | ```crystal 100 | require "nested_scheduler" 101 | 102 | NestedScheduler::ThreadPool.nursery(thread_count: 4) do |pool| 103 | pool.spawn { .. } 104 | end 105 | ``` 106 | 107 | The root nursery (that replaces the builtin scheduler at upstart) is 108 | instantiated with 4 threads, just as the original. 109 | 110 | Since `nested_scheduler` will create a pool of new threads, it is 111 | possible to use it to spawn many threads and use it as a poor mans 112 | replacement for asynchronous file IO. Doing blocking file IO in the 113 | pool while continuing execution in the root pool is totally possible. 114 | 115 | ### (Experimental) Exceptions 116 | 117 | The first exception raised will by bubble up the pool hiearchy. 118 | 119 | Assume the following code: 120 | 121 | ```crystal 122 | require "nested_scheduler" 123 | 124 | NestedScheduler::ThreadPool.nursery do |pool| 125 | pool.spawn { raise "Err" } 126 | end 127 | ``` 128 | 129 | What will happen here is that the pool will catch the error and then 130 | re-raise the exception in the outerlying scope. No more silent 131 | exceptions in fibers (unless you want them. People rarely do). Only 132 | the first exception is kept. 133 | 134 | ### (Experimental) Result collection 135 | 136 | By default only errors are kept track of. Something else that is 137 | common is to want to keep the results of the execution. 138 | 139 | That can be done using the following: 140 | 141 | ```crystal 142 | require "nested_scheduler" 143 | 144 | values = NestedScheduler::ThreadPool.collect(Int32) do |pool| 145 | pool.spawn { 4711 } 146 | pool.spawn { 13 } 147 | end 148 | values.sort! 149 | ``` 150 | 151 | After executing `values` will have the value `[13, 4711]`. If there is 152 | an exception, or if one of the spawned fibers return something that is 153 | of an incorrect type then there will be an exception raised from the 154 | `collect` block. 155 | 156 | ### Cancelation 157 | 158 | Currently only cooperative cancellation of a pool is supported. Example: 159 | 160 | ```crystal 161 | count = 0 162 | NestedScheduler::ThreadPool.nursery do |pl| 163 | pl.spawn do 164 | sleep 0.01 165 | pl.cancel 166 | end 167 | pl.spawn do 168 | loop do 169 | break if pl.canceled? 170 | count += 1 171 | sleep 0.001 172 | end 173 | end 174 | end 175 | # count will be > 7 when this point is reached. 176 | ``` 177 | 178 | Not supported are things like limited noncooperative canceling or 179 | canceling of independent fibers without the surrounding pool. 180 | 181 | ## Future work 182 | 183 | Eventually it would be nice to have more stream lined ways of creating 184 | nurseries, but as long as creating one always will create at least one 185 | dedicated new thread to process the work, it hasn't been necessary. It 186 | will be more relevant once there are cheaper ways to create nurseries 187 | that work gracefully within the current thread pool instead of having 188 | to create at least one new thread for every nursery. 189 | 190 | Also, cancelation, timeouts and perhaps grace periods needs a lot more 191 | thought and work. 192 | 193 | ## Development 194 | 195 | TODO: Write development instructions here 196 | 197 | ## Contributing 198 | 199 | 1. Fork it () 200 | 2. Create your feature branch (`git checkout -b my-new-feature`) 201 | 3. Commit your changes (`git commit -am 'Add some feature'`) 202 | 4. Push to the branch (`git push origin my-new-feature`) 203 | 5. Create a new Pull Request 204 | 205 | ## Contributors 206 | 207 | - [Linus Sellberg](https://github.com/yxhuvud) - creator and maintainer 208 | -------------------------------------------------------------------------------- /benchmarks/baseline.cr: -------------------------------------------------------------------------------- 1 | require "http/server" 2 | 3 | server = HTTP::Server.new do |context| 4 | context.response.content_type = "text/plain" 5 | context.response.print "Hello world!" 6 | end 7 | 8 | address = server.bind_tcp 8080 9 | puts "Listening on http://#{address}" 10 | server.listen 11 | -------------------------------------------------------------------------------- /benchmarks/nested_libevent.cr: -------------------------------------------------------------------------------- 1 | require "../src/nested_scheduler" 2 | require "http/server" 3 | 4 | threads = ARGV.any? ? ARGV[0].to_i : 4 5 | 6 | NestedScheduler::ThreadPool.nursery(threads) do |pl| 7 | pl.spawn(name: "serv") do 8 | server = HTTP::Server.new do |context| 9 | context.response.content_type = "text/plain" 10 | context.response.print "Hello world!" 11 | end 12 | 13 | address = server.bind_tcp 8080 14 | puts "Listening on http://#{address}" 15 | server.listen 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /shard.yml: -------------------------------------------------------------------------------- 1 | name: nested_scheduler 2 | version: 0.6.2 3 | 4 | authors: 5 | - Linus Sellberg 6 | 7 | crystal: ">= 1.11.0" 8 | 9 | license: MIT 10 | -------------------------------------------------------------------------------- /spec/error_propagator_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | # Use methods to get something sensible in the stack traces. 4 | private def raiser 5 | raise "wat" 6 | end 7 | 8 | private def will_raise 9 | NestedScheduler::ThreadPool.nursery(result_handler: NestedScheduler::Results::ErrorPropagator.new) do |pl| 10 | pl.spawn { raiser } 11 | end 12 | end 13 | 14 | describe NestedScheduler::Results::ErrorPropagator do 15 | it "on success" do 16 | executed = false 17 | res = 18 | NestedScheduler::ThreadPool.nursery(result_handler: NestedScheduler::Results::ErrorPropagator.new) do |pl| 19 | pl.spawn { executed = true } 20 | end 21 | res.should be_nil 22 | executed.should be_true 23 | end 24 | 25 | it "on error" do 26 | ex = expect_raises Exception do 27 | will_raise 28 | end 29 | ex.inspect_with_backtrace 30 | .should match( 31 | # unfortunately the will_raise ends up in the wrong file. Dunno why.. 32 | /from error_propagator_spec.cr:5:3 in 'raiser'(.*\n)*.*in 'will_raise'/ 33 | ) 34 | ex.message.should eq "wat" 35 | end 36 | end 37 | -------------------------------------------------------------------------------- /spec/examples/example.cr: -------------------------------------------------------------------------------- 1 | require "../../src/nested_scheduler" 2 | 3 | spawn do 4 | sleep 0.001 5 | puts "done!" 6 | end 7 | 8 | puts "passed" 9 | -------------------------------------------------------------------------------- /spec/libevent_context_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | private def nursery 4 | NestedScheduler::ThreadPool.nursery(1) do |pl| 5 | yield pl 6 | end 7 | end 8 | 9 | describe NestedScheduler::LibeventContext do 10 | it "works with enclosing scope" do 11 | run = false 12 | 13 | nursery &.spawn { run = true } 14 | run.should be_true 15 | end 16 | 17 | describe "#wait_readable" do 18 | it "yields when file becomes readable" do 19 | str = "hello world!" 20 | left, right = UNIXSocket.pair 21 | readable = false 22 | nursery do |pl| 23 | pl.spawn do 24 | sleep 0.001 25 | right.write(str.to_slice) 26 | end 27 | pl.spawn do 28 | left.wait_readable 29 | readable = true 30 | end 31 | pl.spawn do 32 | sleep 0.0005 33 | readable.should eq false 34 | sleep 0.0011 35 | readable.should eq true 36 | end 37 | end 38 | end 39 | 40 | it "supports timeouts" do 41 | str = "hello world!" 42 | left, right = UNIXSocket.pair 43 | has_timed_out = false 44 | nursery do |pl| 45 | pl.spawn do 46 | left.wait_readable(0.001.seconds) do 47 | has_timed_out = true 48 | end 49 | end 50 | pl.spawn do 51 | sleep 0.0005 52 | has_timed_out.should eq false 53 | sleep 0.0015 54 | has_timed_out.should eq true 55 | end 56 | end 57 | end 58 | end 59 | 60 | describe "write" do 61 | it "Can write to stdout" do 62 | nursery &.spawn { puts } 63 | end 64 | 65 | it "write" do 66 | filename = "test/write1" 67 | nursery &.spawn { File.write filename, "hello world" } 68 | File.read("test/write1").should eq "hello world" 69 | end 70 | end 71 | 72 | it "works with channels" do 73 | done = Channel(Nil).new(1) 74 | 75 | nursery &.spawn { done.send nil } 76 | done.receive.should be_nil 77 | 78 | done2 = Channel(Nil).new 79 | nursery do |pl| 80 | pl.spawn { done2.send nil } 81 | pl.spawn { done2.receive.should be_nil } 82 | end 83 | end 84 | 85 | it "#sleep" do 86 | sleep_time = 0.01 87 | spent_time = Time.measure do 88 | nursery do |pl| 89 | 100.times do |i| 90 | pl.spawn do 91 | sleep sleep_time 92 | end 93 | end 94 | end 95 | end.to_f 96 | 97 | spent_time.should be > sleep_time 98 | spent_time.should be < 5 * sleep_time 99 | end 100 | 101 | describe "#accept" do 102 | it "can accept" do 103 | port = unused_local_port 104 | server = Socket.new(Socket::Family::INET, Socket::Type::STREAM, Socket::Protocol::TCP) 105 | server.bind("0.0.0.0", port) 106 | server.listen 107 | 108 | spawn { TCPSocket.new("127.0.0.1", port).close } 109 | 110 | client = nil 111 | nursery &.spawn { client = server.accept } 112 | 113 | # expectations outside spawn block just to be sure it runs. 114 | client.not_nil!.family.should eq(Socket::Family::INET) 115 | client.not_nil!.type.should eq(Socket::Type::STREAM) 116 | client.not_nil!.protocol.should eq(Socket::Protocol::TCP) 117 | 118 | client.not_nil!.close 119 | server.close 120 | end 121 | 122 | it "can wait for acceptance" do 123 | port = unused_local_port 124 | server = Socket.new(Socket::Family::INET, Socket::Type::STREAM, Socket::Protocol::TCP) 125 | server.bind("127.0.0.1", port) 126 | server.listen 127 | client = nil 128 | nursery do |n| 129 | n.spawn { sleep 0.001; TCPSocket.new("127.0.0.1", port).close } 130 | n.spawn { client = server.accept } 131 | end 132 | 133 | # expectations outside spawn block just to be sure it runs. 134 | client.not_nil!.family.should eq(Socket::Family::INET) 135 | client.not_nil!.type.should eq(Socket::Type::STREAM) 136 | client.not_nil!.protocol.should eq(Socket::Protocol::TCP) 137 | 138 | client.not_nil!.close 139 | server.close 140 | end 141 | 142 | # pending "handles timeout" 143 | end 144 | 145 | describe "#connect" do 146 | it "can connect" do 147 | port = unused_local_port 148 | server = Socket.new(Socket::Family::INET, Socket::Type::STREAM, Socket::Protocol::TCP) 149 | server.bind("127.0.0.1", port) 150 | server.listen 151 | client = nil 152 | 153 | nursery do |n| 154 | n.spawn { sleep 0.001; TCPSocket.new("127.0.0.1", port).close } 155 | end 156 | 157 | client = server.accept 158 | client.not_nil!.family.should eq(Socket::Family::INET) 159 | client.not_nil!.type.should eq(Socket::Type::STREAM) 160 | client.not_nil!.protocol.should eq(Socket::Protocol::TCP) 161 | 162 | client.not_nil!.close 163 | server.close 164 | end 165 | 166 | # pending "handles timeout" 167 | end 168 | 169 | it "can wait for acceptance" do 170 | port = unused_local_port 171 | server = Socket.new(Socket::Family::INET, Socket::Type::STREAM, Socket::Protocol::TCP) 172 | server.bind("0.0.0.0", port) 173 | server.listen 174 | 175 | client = nil 176 | nursery do |n| 177 | n.spawn { client = server.accept } 178 | n.spawn do 179 | sleep 0.001 180 | TCPSocket.new("127.0.0.1", port).close 181 | end 182 | end 183 | 184 | # expectations outside spawn block just to be sure it runs. 185 | client.not_nil!.family.should eq(Socket::Family::INET) 186 | client.not_nil!.type.should eq(Socket::Type::STREAM) 187 | client.not_nil!.protocol.should eq(Socket::Protocol::TCP) 188 | 189 | client.not_nil!.close 190 | server.close 191 | end 192 | 193 | it "sends messages" do 194 | port = unused_local_port 195 | server = Socket.tcp(Socket::Family::INET6) 196 | server.bind("::1", port) 197 | server.listen 198 | address = Socket::IPAddress.new("::1", port) 199 | socket = Socket.tcp(Socket::Family::INET6) 200 | socket.connect(address) 201 | client = server.not_nil!.accept 202 | 203 | nursery do |pl| 204 | pl.spawn do 205 | client.gets.should eq "foo" 206 | client.puts "bar" 207 | end 208 | pl.spawn do 209 | socket.puts "foo" 210 | socket.gets.should eq "bar" 211 | end 212 | end 213 | ensure 214 | client.try &.close 215 | socket.try &.close 216 | server.try &.close 217 | end 218 | 219 | each_ip_family do |family, address, unspecified_address| 220 | it "sends and receives messages" do 221 | port = unused_local_port 222 | 223 | server = UDPSocket.new(family) 224 | server.bind(address, port) 225 | server.local_address.should eq(Socket::IPAddress.new(address, port)) 226 | 227 | client = UDPSocket.new(family) 228 | client.bind(address, 0) 229 | 230 | nursery do |pl| 231 | pl.spawn { client.send "message", to: server.local_address } 232 | pl.spawn { server.receive.should eq({"message", client.local_address}) } 233 | end 234 | 235 | client.connect(address, port) 236 | client.local_address.family.should eq(family) 237 | client.local_address.address.should eq(address) 238 | client.remote_address.should eq(Socket::IPAddress.new(address, port)) 239 | 240 | nursery do |pl| 241 | pl.spawn { client.send "message" } 242 | pl.spawn { server.receive.should eq({"message", client.local_address}) } 243 | end 244 | 245 | buffer = uninitialized UInt8[256] 246 | 247 | nursery do |pl| 248 | pl.spawn { client.send("laus deo semper") } 249 | pl.spawn do 250 | bytes_read, client_addr = server.receive(buffer.to_slice) 251 | message = String.new(buffer.to_slice[0, bytes_read]) 252 | message.should eq("laus deo semper") 253 | end 254 | end 255 | 256 | nursery do |pl| 257 | pl.spawn { client.send("laus deo semper") } 258 | pl.spawn do 259 | bytes_read, client_addr = server.receive(buffer.to_slice[0, 4]) 260 | message = String.new(buffer.to_slice[0, bytes_read]) 261 | message.should eq("laus") 262 | end 263 | end 264 | 265 | client.close 266 | server.close 267 | end 268 | end 269 | end 270 | -------------------------------------------------------------------------------- /spec/main_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | describe "main" do 4 | it "will wait for all fibers to complete" do 5 | command = "crystal run -Dpreview_mt spec/examples/example.cr" 6 | output = "passed\ndone!\n" 7 | result = `#{command}` 8 | result.should eq output 9 | end 10 | end 11 | -------------------------------------------------------------------------------- /spec/nested_scheduler_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | describe NestedScheduler do 4 | context ".nursery" do 5 | it "doesn't impact current thread pool" do 6 | worker_count = Thread.current.scheduler.pool.not_nil!.workers.size 7 | NestedScheduler::ThreadPool.nursery do |_| 8 | Thread.current.scheduler.pool.not_nil!.workers.size 9 | .should eq worker_count 10 | end 11 | end 12 | 13 | it "spawns a fiber" do 14 | NestedScheduler::ThreadPool.nursery { |pl| typeof(pl.spawn { }).should eq Fiber } 15 | end 16 | 17 | it "kills any started fibers" do 18 | 100.times do 19 | NestedScheduler::ThreadPool.nursery { } 20 | end 21 | i = 0 22 | Fiber.unsafe_each do |f| 23 | i += 1 24 | end 25 | i.should be < 10 26 | end 27 | 28 | it "does not leak file descriptors" do 29 | 5.times do 30 | NestedScheduler::ThreadPool.nursery(thread_count: 10) { } 31 | end 32 | pid = `pgrep crystal`.split.last 33 | fds = `lsof -p #{pid}`.split("\n") 34 | fds.size.should be < 50 35 | end 36 | 37 | it "exits immediately if not spawned" do 38 | # Unfortunately closing all fds take time. 39 | Time.measure do 40 | NestedScheduler::ThreadPool.nursery { |_| } 41 | end.to_f.should be < 0.0002 42 | end 43 | 44 | it "starts the pool and waits for it to finish" do 45 | sleep_time = 0.01 46 | spent_time = Time.measure do 47 | NestedScheduler::ThreadPool.nursery name: "t" do |pl| 48 | 100.times do |i| 49 | pl.spawn(name: "fiber#{i}") do 50 | sleep sleep_time 51 | end 52 | end 53 | end 54 | end.to_f 55 | 56 | spent_time.should be > sleep_time 57 | spent_time.should be < 5 * sleep_time 58 | end 59 | 60 | it "executes" do 61 | # capacity needed as otherwise the pool won't ever exit as the 62 | # channel isn't consumed. Why, because nursery doesn't return 63 | # until all the fibers are done.. 64 | chan = Channel(Int32).new capacity: 10 65 | NestedScheduler::ThreadPool.nursery do |pl| 66 | 10.times { |i| pl.spawn(name: "fiber: #{i}") { chan.send i } } 67 | end 68 | values = Array(Int32).new(10) { |i| chan.receive } 69 | values.sort.should eq [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] 70 | end 71 | 72 | it "can be soft canceled" do 73 | count = 0 74 | NestedScheduler::ThreadPool.nursery do |pl| 75 | pl.spawn do 76 | sleep 0.01 77 | pl.cancel 78 | end 79 | pl.spawn do 80 | pl.canceled?.should be_false 81 | loop do 82 | break if pl.canceled? 83 | count += 1 84 | sleep 0.001 85 | end 86 | end 87 | end 88 | count.should be > 7 # allow for some overhead.. 89 | end 90 | 91 | it "re-raise fiber exceptions by default" do 92 | ex = expect_raises NotImplementedError do 93 | NestedScheduler::ThreadPool.nursery do |pl| 94 | pl.spawn { raise NotImplementedError.new("fail") } 95 | end 96 | end 97 | 98 | ex.message.should eq "Not Implemented: fail" 99 | end 100 | end 101 | 102 | context "spawning original worker threads" do 103 | it "spawns the correct number" do 104 | pool = Thread.current.scheduler.pool.not_nil! 105 | pool.@workers.size.should eq ENV["CRYSTAL_WORKERS"]?.try(&.to_i) || 4 106 | end 107 | 108 | it "runs in the root pool" do 109 | Thread.current.scheduler.pool.not_nil! 110 | .name.should eq "Root Pool" 111 | end 112 | 113 | it "doesn't break basic spawning" do 114 | chan = Channel(Int32).new 115 | 116 | spawn { chan.send 1 } 117 | 118 | res = chan.receive 119 | res.should eq 1 120 | end 121 | 122 | it "doesn't break basic spawning1" do 123 | chan = Channel(Int32).new 124 | 125 | spawn { chan.send 1 } 126 | spawn { chan.send 2 } 127 | 128 | res = chan.receive + chan.receive 129 | res.should eq 3 130 | end 131 | 132 | it "doesn't break basic spawning2" do 133 | chan = Channel(Int32).new 134 | 135 | spawn { chan.send 1 } 136 | spawn { chan.send 2 } 137 | 138 | res = chan.receive + chan.receive 139 | res.should eq 3 140 | end 141 | 142 | it "doesn't break basic spawning3" do 143 | chan = Channel(Int32).new 144 | 145 | spawn chan.send(1) 146 | 147 | res = chan.receive 148 | res.should eq 1 149 | end 150 | 151 | it "doesn't break basic spawning4" do 152 | chan = Channel(Int32).new 153 | 154 | spawn chan.send(1) 155 | spawn chan.send(2) 156 | 157 | res = chan.receive + chan.receive 158 | res.should eq 3 159 | end 160 | 161 | it "can nest spawns" do 162 | executed = false 163 | NestedScheduler::ThreadPool.nursery do |pl| 164 | pl.spawn { spawn { executed = true } } 165 | end 166 | executed.should eq true 167 | end 168 | end 169 | end 170 | -------------------------------------------------------------------------------- /spec/result_collector_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | describe NestedScheduler::Results::ResultCollector do 4 | it "on success" do 5 | res = 6 | NestedScheduler::ThreadPool.collect(Int32) do |pl| 7 | pl.spawn { 1 } 8 | pl.spawn { 2 } 9 | end 10 | typeof(res).should eq Array(Int32) 11 | res.not_nil!.sort.should eq [1, 2] 12 | 13 | res2 = 14 | NestedScheduler::ThreadPool.collect(Float64) do |pl| 15 | pl.spawn { 1.0 } 16 | pl.spawn { 2.0 } 17 | end 18 | typeof(res2).should eq Array(Float64) 19 | res2.not_nil!.sort.should eq [1.0, 2.0] 20 | end 21 | 22 | # ok, it would be really nice if this was possible to catch compile-time. 23 | it "on mismatching type" do 24 | res = expect_raises ArgumentError do 25 | NestedScheduler::ThreadPool.nursery(result_handler: NestedScheduler::Results::ResultCollector(Int32).new) do |pl| 26 | pl.spawn { "wat" } 27 | end 28 | end 29 | 30 | res.message.should eq "Expected block to return Int32, but got String" 31 | end 32 | 33 | it "on error" do 34 | ex = expect_raises NotImplementedError do 35 | NestedScheduler::ThreadPool.nursery(result_handler: NestedScheduler::Results::ResultCollector(Int32).new) do |pl| 36 | pl.spawn { raise NotImplementedError.new "wat" } 37 | end 38 | end 39 | ex.message.should eq "Not Implemented: wat" 40 | end 41 | end 42 | -------------------------------------------------------------------------------- /spec/silent_spec.cr: -------------------------------------------------------------------------------- 1 | require "./spec_helper" 2 | 3 | describe NestedScheduler::Results::Silent do 4 | it "on success" do 5 | res = 6 | NestedScheduler::ThreadPool.nursery(result_handler: NestedScheduler::Results::Silent.new) do |pl| 7 | pl.spawn { 1 } 8 | pl.spawn { 2 } 9 | end 10 | res.should eq nil 11 | end 12 | 13 | it "on error" do 14 | res = 15 | NestedScheduler::ThreadPool.nursery(result_handler: NestedScheduler::Results::Silent.new) do |pl| 16 | pl.spawn { raise NotImplementedError.new "wat" } 17 | end 18 | res.should eq nil 19 | end 20 | end 21 | -------------------------------------------------------------------------------- /spec/spec_helper.cr: -------------------------------------------------------------------------------- 1 | require "spec" 2 | require "../src/nested_scheduler" 3 | 4 | def unused_local_port 5 | TCPServer.open("::", 0) do |server| 6 | server.local_address.port 7 | end 8 | end 9 | 10 | def each_ip_family(&block : Socket::Family, String, String ->) 11 | describe "using IPv4" do 12 | block.call Socket::Family::INET, "127.0.0.1", "0.0.0.0" 13 | end 14 | 15 | describe "using IPv6" do 16 | block.call Socket::Family::INET6, "::1", "::" 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /src/nested_scheduler.cr: -------------------------------------------------------------------------------- 1 | require "./nested_scheduler/monkeypatch/**" 2 | 3 | module NestedScheduler 4 | 5 | # TODO: Put your code here 6 | end 7 | -------------------------------------------------------------------------------- /src/nested_scheduler/io_context.cr: -------------------------------------------------------------------------------- 1 | module NestedScheduler # struct? 2 | abstract class IOContext 3 | abstract def new : self 4 | 5 | module IO 6 | @[AlwaysInline] 7 | protected def context 8 | scheduler = Thread.current.scheduler 9 | io = scheduler.io || raise "BUG: No io context when required" 10 | {io, scheduler} 11 | end 12 | end 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /src/nested_scheduler/libevent_context.cr: -------------------------------------------------------------------------------- 1 | require "./io_context" 2 | 3 | module NestedScheduler 4 | class LibeventContext < IOContext 5 | def new : self 6 | self 7 | end 8 | 9 | def wait_readable(io, scheduler, timeout) 10 | readers = io.@readers.get { Deque(Fiber).new } 11 | readers << Fiber.current 12 | # add_read_event inlined: 13 | event = io.@read_event.get { Crystal::Scheduler.event_loop.create_fd_read_event(io) } 14 | event.add timeout 15 | 16 | scheduler.actually_reschedule 17 | 18 | if io.@read_timed_out 19 | io.read_timed_out = false 20 | yield 21 | end 22 | end 23 | 24 | def wait_writable(io, scheduler, timeout) 25 | writers = io.@writers.get { Deque(Fiber).new } 26 | writers << Fiber.current 27 | # add_write_event inlined. 28 | event = io.@write_event.get { Crystal::Scheduler.event_loop.create_fd_write_event(io) } 29 | event.add timeout 30 | 31 | scheduler.actually_reschedule 32 | 33 | if io.@write_timed_out 34 | io.write_timed_out = false 35 | yield 36 | end 37 | end 38 | 39 | def accept(socket, _scheduler, timeout) 40 | loop do 41 | client_fd = LibC.accept(socket.fd, nil, nil) 42 | 43 | if client_fd == -1 44 | if socket.closed? 45 | return 46 | elsif Errno.value == Errno::EAGAIN 47 | wait_readable(socket, _scheduler, timeout) do 48 | raise Socket::TimeoutError.new("Accept timed out") 49 | end 50 | return if socket.closed? 51 | else 52 | raise Socket::ConnectError.from_errno("accept") 53 | end 54 | else 55 | return client_fd 56 | end 57 | end 58 | end 59 | 60 | def connect(socket, _scheduler, addr, timeout) 61 | loop do 62 | if LibC.connect(socket.fd, addr, addr.size) == 0 63 | return 64 | end 65 | case Errno.value 66 | when Errno::EISCONN 67 | return 68 | when Errno::EINPROGRESS, Errno::EALREADY 69 | wait_writable(socket, _scheduler, timeout: timeout) do 70 | return yield ::IO::TimeoutError.new("connect timed out") 71 | end 72 | else 73 | return yield Socket::ConnectError.from_errno("connect") 74 | end 75 | end 76 | end 77 | 78 | def send(socket, _scheduler, slice : Bytes, errno_message : String) : Int32 79 | socket.evented_send(slice, errno_message) do |slice| 80 | LibC.send(socket.fd, slice.to_unsafe.as(Void*), slice.size, 0) 81 | end 82 | end 83 | 84 | def send_to(socket, _scheduler, message, to addr : Socket::Address) : Int32 85 | slice = message.to_slice 86 | bytes_sent = LibC.sendto(socket.fd, slice.to_unsafe.as(Void*), slice.size, 0, addr, addr.size) 87 | raise Socket::Error.from_errno("Error sending datagram to #{addr}") if bytes_sent == -1 88 | # to_i32 is fine because string/slice sizes are an Int32 89 | bytes_sent.to_i32 90 | end 91 | 92 | def socket_write(socket, _scheduler, slice : Bytes, errno_message : String) : Nil 93 | socket.evented_write(slice, errno_message) do |slice| 94 | LibC.send(socket.fd, slice, slice.size, 0) 95 | end 96 | end 97 | 98 | def recv(socket, _scheduler, slice : Bytes, errno_message : String) 99 | socket.evented_read(slice, errno_message) do 100 | LibC.recv(socket.fd, slice, slice.size, 0).to_i32 101 | end 102 | end 103 | 104 | def recvfrom(socket, _scheduler, slice, sockaddr, addrlen, errno_message) 105 | socket.evented_read(slice, errno_message) do |slice| 106 | LibC.recvfrom(socket.fd, slice, slice.size, 0, sockaddr, pointerof(addrlen)) 107 | end 108 | end 109 | 110 | def read(io, _scheduler, slice : Bytes) 111 | io.evented_read(slice, "Error reading file") do 112 | LibC.read(io.fd, slice, slice.size).tap do |return_code| 113 | if return_code == -1 && Errno.value == Errno::EBADF 114 | raise ::IO::Error.new "File not open for reading" 115 | end 116 | end 117 | end 118 | end 119 | 120 | def write(io, _scheduler, slice : Bytes) 121 | io.evented_write(slice, "Error writing file") do |slice| 122 | LibC.write(io.fd, slice, slice.size).tap do |return_code| 123 | if return_code == -1 && Errno.value == Errno::EBADF 124 | raise ::IO::Error.new "File not open for writing" 125 | end 126 | end 127 | end 128 | end 129 | 130 | def sleep(scheduler, fiber, time) : Nil 131 | fiber.resume_event.add(time) 132 | scheduler.actually_reschedule 133 | end 134 | 135 | def yield(fiber : Fiber, to other) 136 | fiber.resume_event.add(0.seconds) 137 | end 138 | 139 | def prepare_close(file) 140 | file.evented_close 141 | end 142 | 143 | def close(fd, _scheduler) 144 | if LibC.close(fd) != 0 145 | case Errno.value 146 | when Errno::EINTR, Errno::EINPROGRESS 147 | # ignore 148 | else 149 | raise ::IO::Error.from_errno("Error closing file") 150 | end 151 | end 152 | end 153 | 154 | def reschedule(_scheduler) 155 | loop do 156 | if runnable = yield 157 | unless runnable == Fiber.current 158 | runnable.resume 159 | end 160 | return 161 | else 162 | Crystal::Scheduler.event_loop.run_once 163 | end 164 | end 165 | end 166 | 167 | def stop 168 | Crystal::Scheduler.event_loop.stop 169 | end 170 | end 171 | end 172 | -------------------------------------------------------------------------------- /src/nested_scheduler/linked_list2.cr: -------------------------------------------------------------------------------- 1 | module NestedScheduler 2 | # Same as Thread::LinkedList(T) but with different field names. 3 | # There is a need of a list per thread pool in addition to the 4 | # global one. Potentially it could be changed so that there isn't 5 | # any global list of fibers but only one list per pool (probably a 6 | # good idea - if there is many fibers then deletion becomes a 7 | # bottleneck due to it being O(n)). In that case it might be a good 8 | # idea to instead have Thread::LinkedList of pools for fiber 9 | # iteration purpose. It would probably make it easier to get an idea 10 | # of what a system is doing. 11 | class LinkedList2(T) 12 | @mutex = Thread::Mutex.new 13 | @head : T? 14 | @tail : T? 15 | 16 | # Iterates the list without acquiring the lock, to avoid a deadlock in 17 | # stop-the-world situations, where a paused thread could have acquired the 18 | # lock to push/delete a node, while still being "safe" to iterate (but only 19 | # during a stop-the-world). 20 | def unsafe_each : Nil 21 | node = @head 22 | 23 | while node 24 | yield node 25 | node = node.next2 26 | end 27 | end 28 | 29 | # Appends a node to the tail of the list. The operation is thread-safe. 30 | # 31 | # There are no guarantees that a node being pushed will be iterated by 32 | # `#unsafe_each` until the method has returned. 33 | def push(node : T) : Nil 34 | @mutex.synchronize do 35 | node.previous2 = nil 36 | 37 | if tail = @tail 38 | node.previous2 = tail 39 | @tail = tail.next2 = node 40 | else 41 | @head = @tail = node 42 | end 43 | end 44 | end 45 | 46 | # Removes a node from the list. The operation is thread-safe. 47 | # 48 | # There are no guarantees that a node being deleted won't be iterated by 49 | # `#unsafe_each` until the method has returned. 50 | def delete(node : T) : Nil 51 | @mutex.synchronize do 52 | if previous = node.previous2 53 | previous.next2 = node.next2 54 | else 55 | @head = node.next2 56 | end 57 | 58 | if _next = node.next2 59 | _next.previous2 = node.previous2 60 | else 61 | @tail = node.previous2 62 | end 63 | end 64 | end 65 | end 66 | end 67 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/event_loop.cr: -------------------------------------------------------------------------------- 1 | require "crystal/system/event_loop" 2 | 3 | abstract class Crystal::EventLoop 4 | def stop 5 | end 6 | end 7 | 8 | class Crystal::LibEvent::EventLoop 9 | def stop 10 | event_base.stop 11 | end 12 | end 13 | 14 | struct Crystal::LibEvent::Event::Base 15 | def stop : Nil 16 | LibEvent2.event_base_free(@base) 17 | end 18 | end 19 | 20 | lib LibEvent2 21 | fun event_base_free(event : EventBase) : Nil 22 | end 23 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/evented.cr: -------------------------------------------------------------------------------- 1 | # Ideally, this class should not need to be monkeypatched but simply used by the libevent_context, 2 | require "io/evented" 3 | 4 | module IO::Evented 5 | setter write_timed_out 6 | setter read_timed_out 7 | 8 | def wait_readable(timeout = @read_timeout, *, raise_if_closed = true, &) : Nil 9 | io, scheduler = context 10 | io.wait_readable(self, scheduler, timeout) do 11 | yield 12 | end 13 | 14 | check_open if raise_if_closed 15 | end 16 | 17 | # :nodoc: 18 | def wait_writable(timeout = @write_timeout, &) : Nil 19 | io, scheduler = context 20 | 21 | io.wait_writable(self, scheduler, timeout) do 22 | yield 23 | end 24 | 25 | check_open 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/exception.cr: -------------------------------------------------------------------------------- 1 | class Exception 2 | def prepend_current_callstack 3 | @callstack = Exception::CallStack.new(self) 4 | end 5 | end 6 | 7 | struct Exception::CallStack 8 | def initialize(template : Exception) 9 | if callstack = template.callstack 10 | @callstack = callstack.@callstack 11 | unwind = CallStack.unwind 12 | # skip lines related to stitching the stack itself. 13 | unwind.shift(5) 14 | @callstack.concat unwind 15 | else 16 | @callstack = CallStack.unwind 17 | end 18 | end 19 | end 20 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/fiber.cr: -------------------------------------------------------------------------------- 1 | require "fiber" 2 | require "./scheduler" 3 | 4 | class Fiber 5 | # A helper fiber is a fiber which don't block thread pool exit. 6 | property helper_fiber : Bool 7 | @helper_fiber = false 8 | 9 | # For thread pool fiber list 10 | property next2 : Fiber? 11 | property previous2 : Fiber? 12 | 13 | def run 14 | GC.unlock_read 15 | @proc.call 16 | rescue ex 17 | with_pool { |pool| pool.result_handler.register_error(pool, self, ex) } 18 | ensure 19 | @alive = false 20 | cleanup 21 | Crystal::Scheduler.reschedule 22 | end 23 | 24 | def cleanup 25 | Fiber.inactive(self) 26 | with_pool { |pool| pool.unregister_fiber(self) } 27 | # Delete the resume event if it was used by `yield` or `sleep` 28 | @resume_event.try &.free 29 | @timeout_event.try &.free 30 | @timeout_select_action = nil 31 | # Sigh, scheduler.enqueue_free_stack is protected. 32 | Crystal::Scheduler.enqueue_free_stack @stack 33 | end 34 | 35 | private def with_pool 36 | scheduler = Thread.current.scheduler 37 | # monkeypatch: Is this safe with regards to thread lifecycle? Dunno. 38 | if pool = scheduler.pool 39 | yield pool 40 | end 41 | end 42 | end 43 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/fiber_channel.cr: -------------------------------------------------------------------------------- 1 | require "crystal/fiber_channel" 2 | 3 | struct Crystal::FiberChannel 4 | def close 5 | @worker_in.write_bytes 0u64 6 | end 7 | 8 | def receive : Fiber? 9 | oid = @worker_out.read_bytes(UInt64) 10 | if oid.zero? 11 | @worker_out.close 12 | @worker_in.close 13 | nil 14 | else 15 | Pointer(Fiber).new(oid).as(Fiber) 16 | end 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/file_descriptor.cr: -------------------------------------------------------------------------------- 1 | module Crystal::System::FileDescriptor 2 | include NestedScheduler::IOContext::IO 3 | 4 | # TODO: Half the file belongs in libevent_context.. 5 | 6 | private def unbuffered_read(slice : Bytes) 7 | io, scheduler = context 8 | io.read(self, scheduler, slice) 9 | end 10 | 11 | private def unbuffered_write(slice : Bytes) 12 | io, scheduler = context 13 | 14 | io.write(self, scheduler, slice) 15 | end 16 | 17 | # private def system_info 18 | # TODO - this is supported by uring so it needs to be delegated right 19 | # stat = uninitialized LibC::Stat 20 | # ret = File.fstat(fd, pointerof(stat)) 21 | 22 | # if ret != 0 23 | # raise IO::Error.from_errno("Unable to get info") 24 | # end 25 | 26 | # ::File::Info.new(stat) 27 | 28 | # # FileInfo.new(stat) 29 | # end 30 | 31 | private def system_close 32 | io, scheduler = context 33 | # Perform libevent cleanup before LibC.close. Using a file 34 | # descriptor after it has been closed is never defined and can 35 | # always lead to undefined results as the system may reuse the fd. 36 | # This is not specific to libevent. 37 | 38 | # However, will io_uring automatically cancel all outstanding ops or 39 | # would that be a race condintion? Who knows, not I. 40 | io.prepare_close(self) 41 | 42 | # Clear the @volatile_fd before actually closing it in order to 43 | # reduce the chance of reading an outdated fd value 44 | _fd = @volatile_fd.swap(-1) 45 | io, scheduler = context 46 | io.close(_fd, scheduler) 47 | end 48 | 49 | def self.pread(fd, buffer, offset) 50 | # raise "TODO" 51 | bytes_read = LibC.pread(fd, buffer, buffer.size, offset) 52 | 53 | if bytes_read == -1 54 | raise IO::Error.from_errno "Error reading file" 55 | end 56 | 57 | bytes_read 58 | end 59 | 60 | def self.pipe(read_blocking, write_blocking) 61 | pipe_fds = uninitialized StaticArray(LibC::Int, 2) 62 | if LibC.pipe(pipe_fds) != 0 63 | raise IO::Error.from_errno("Could not create pipe") 64 | end 65 | 66 | r = IO::FileDescriptor.new(pipe_fds[0], read_blocking) 67 | w = IO::FileDescriptor.new(pipe_fds[1], write_blocking) 68 | r.close_on_exec = true 69 | w.close_on_exec = true 70 | w.sync = true 71 | 72 | {r, w} 73 | end 74 | end 75 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/main.cr: -------------------------------------------------------------------------------- 1 | def spawn(*, name : String? = nil, same_thread = false, &block) : Fiber 2 | if pool = Thread.current.scheduler.pool 3 | pool.spawn(name: name, same_thread: same_thread, &block) 4 | else 5 | # Fiber Clean Loop and Signal Loop are set up before any pool is 6 | # initiated. Handle these separately. 7 | fiber = Fiber.new(name, &block) 8 | fiber.helper_fiber = true 9 | Crystal::Scheduler.enqueue fiber 10 | fiber 11 | end 12 | end 13 | 14 | module Crystal 15 | def self.main(&block) 16 | GC.init 17 | 18 | status = 19 | begin 20 | yield 21 | 0 22 | rescue ex 23 | 1 24 | end 25 | 26 | main_exit(status, ex) 27 | end 28 | 29 | def self.main_exit(status : Int32, exception : Exception?) : Int32 30 | status = Crystal::AtExitHandlers.run status, exception 31 | # Exit handlers can (and do! For example the whole test suite) spawn new fibers 32 | Thread.current.scheduler.pool.not_nil!.wait_until_done 33 | 34 | if exception 35 | STDERR.print "Unhandled exception: " 36 | exception.inspect_with_backtrace(STDERR) 37 | end 38 | 39 | ignore_stdio_errors { STDOUT.flush } 40 | ignore_stdio_errors { STDERR.flush } 41 | 42 | status 43 | end 44 | end 45 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/scheduler.cr: -------------------------------------------------------------------------------- 1 | require "crystal/system/thread" 2 | require "crystal/scheduler" 3 | require "../thread_pool" 4 | require "../io_context" 5 | require "../libevent_context" 6 | require "../result" 7 | require "../results/*" 8 | 9 | class ::Crystal::Scheduler 10 | property pool : ::NestedScheduler::ThreadPool? 11 | 12 | def pool! 13 | pool || raise "BUG" 14 | end 15 | 16 | # TODO: Move io_context to Thread? 17 | property io_context : ::NestedScheduler::IOContext? 18 | 19 | def io : NestedScheduler::IOContext 20 | # Unfortunately I havn't figured out exactly where this is called 21 | # the first time (it doesn't help that the stacktrace I get don't 22 | # have line numbers), and as it is called sometime *before* 23 | # init_workers, I have no choice but to have a fallback :(. 24 | if context = io_context 25 | return context 26 | end 27 | self.io_context = NestedScheduler::LibeventContext.new 28 | # raise "IO Context Not yet initialized, BUG" 29 | end 30 | 31 | protected def find_target_thread 32 | pool.try { |p| p.next_thread! } || Thread.current 33 | end 34 | 35 | # doesn't seem to be possible to monkey patch visibility status. 36 | def actually_enqueue(fiber : Fiber) : Nil 37 | enqueue fiber 38 | end 39 | 40 | # doesn't seem to be possible to monkey patch visibility status. 41 | def actually_reschedule : Nil 42 | reschedule 43 | end 44 | 45 | def self.init_workers 46 | NestedScheduler::ThreadPool.new( 47 | NestedScheduler::LibeventContext.new, 48 | NestedScheduler::Results::ErrorPropagator.new, 49 | worker_count, 50 | bootstrap: true, 51 | name: "Root Pool" 52 | ) 53 | end 54 | 55 | def run_loop 56 | fiber_channel = self.fiber_channel 57 | loop do 58 | @lock.lock 59 | if runnable = @runnables.shift? 60 | @runnables << Fiber.current 61 | @lock.unlock 62 | resume(runnable) 63 | else 64 | @sleeping = true 65 | @lock.unlock 66 | unless fiber = fiber_channel.receive 67 | # Thread pool has signaled that it is time to shutdown in wait_until_done. 68 | # Do note that wait_until_done happens in the nursery origin thread. 69 | io.stop 70 | return 71 | end 72 | @lock.lock 73 | @sleeping = false 74 | @runnables << Fiber.current 75 | @lock.unlock 76 | resume(fiber) 77 | end 78 | end 79 | end 80 | 81 | def populate_fiber_channel 82 | fiber_channel 83 | end 84 | 85 | protected def reschedule : Nil 86 | io.reschedule(self) { @lock.sync { @runnables.shift? } } 87 | 88 | release_free_stacks 89 | end 90 | 91 | protected def sleep(time : Time::Span) : Nil 92 | io.sleep(self, @current, time) 93 | end 94 | 95 | protected def yield(fiber : Fiber) : Nil 96 | io.yield(@current, to: fiber) 97 | resume(fiber) 98 | end 99 | 100 | # Expected to be called from outside and that the scheduler is 101 | # waiting to receive fibers through the channel. Assumes there is no 102 | # work left. 103 | def shutdown 104 | fiber_channel.close 105 | end 106 | end 107 | -------------------------------------------------------------------------------- /src/nested_scheduler/monkeypatch/socket.cr: -------------------------------------------------------------------------------- 1 | require "socket" 2 | 3 | module Crystal::System::Socket 4 | include NestedScheduler::IOContext::IO 5 | 6 | def system_accept 7 | io, scheduler = context 8 | io.accept(self, scheduler, @read_timeout) 9 | end 10 | 11 | def system_send(bytes : Bytes) : Int32 12 | io, scheduler = context 13 | io.send(self, scheduler, bytes, "Error sending datagram") 14 | end 15 | 16 | def system_send_to(bytes : Bytes, addr : ::Socket::Address) : Int32 17 | io, scheduler = context 18 | io.send_to(self, scheduler, bytes, addr) 19 | end 20 | 21 | private def unbuffered_read(slice : Bytes) 22 | io, scheduler = context 23 | io.recv(self, scheduler, slice, "Error reading socket") 24 | end 25 | 26 | private def unbuffered_write(slice : Bytes) 27 | io, scheduler = context 28 | io.socket_write(self, scheduler, slice, "Error writing to socket") 29 | end 30 | 31 | protected def system_receive(slice) 32 | io, scheduler = context 33 | # we will see if these will have to be moved into the context 34 | sockaddr = Pointer(LibC::SockaddrStorage).malloc.as(LibC::Sockaddr*) 35 | # initialize sockaddr with the initialized family of the socket 36 | copy = sockaddr.value 37 | copy.sa_family = family 38 | sockaddr.value = copy 39 | 40 | addrlen = LibC::SocklenT.new(sizeof(LibC::SockaddrStorage)) 41 | 42 | bytes_read = io.recvfrom(self, scheduler, slice, sockaddr, addrlen, "Error receiving datagram") 43 | 44 | {bytes_read, sockaddr, addrlen} 45 | end 46 | 47 | def system_connect(addr, timeout = nil) 48 | timeout = timeout.seconds unless timeout.is_a? ::Time::Span | Nil 49 | io, scheduler = context 50 | 51 | io.connect(self, scheduler, addr, timeout) do |error| 52 | yield error 53 | end 54 | end 55 | 56 | private def system_close 57 | io, scheduler = context 58 | # Perform libevent cleanup before LibC.close. Using a file 59 | # descriptor after it has been closed is never defined and can 60 | # always lead to undefined results as the system may reuse the fd. 61 | # This is not specific to libevent. 62 | 63 | # However, will io_uring automatically cancel all outstanding ops or 64 | # would that be a race condintion? Who knows, not I. 65 | io.prepare_close(self) 66 | 67 | # Clear the @volatile_fd before actually closing it in order to 68 | # reduce the chance of reading an outdated fd value 69 | _fd = @volatile_fd.swap(-1) 70 | io, scheduler = context 71 | io.close(_fd, scheduler) 72 | end 73 | 74 | # def system_bind 75 | # def system_listen 76 | # def system_close_read 77 | # def system_close_write 78 | # def system_reuse_port? 79 | # def system_reuse_port= 80 | # private def shutdown(how) # doesnt 81 | end 82 | -------------------------------------------------------------------------------- /src/nested_scheduler/result.cr: -------------------------------------------------------------------------------- 1 | module NestedScheduler 2 | abstract class Result 3 | abstract def initialize 4 | abstract def register_error(pool, fiber, error) 5 | abstract def result 6 | abstract def init(&block : -> _) : -> 7 | 8 | def reraise_on_error 9 | if error = @error 10 | error.prepend_current_callstack 11 | # TODO: Add a line to callstack that explains that it is a nested 12 | # raise. 13 | # TODO: Inject fiber name etc in exception. 14 | 15 | raise error 16 | else 17 | yield 18 | end 19 | end 20 | end 21 | end 22 | -------------------------------------------------------------------------------- /src/nested_scheduler/results/error_propagator.cr: -------------------------------------------------------------------------------- 1 | module NestedScheduler::Results 2 | # Returns nil. Will cancel the pool on any errors and then it will 3 | # re-raise the error when the pool is done. 4 | class ErrorPropagator < NestedScheduler::Result 5 | property error : Exception? 6 | property name : String? 7 | 8 | def initialize 9 | @lock = Crystal::SpinLock.new 10 | end 11 | 12 | # TODO: Figure out how to handle cancellation of pool/fiber? 13 | def register_error(pool, fiber, error) 14 | return unless error 15 | @lock.sync do 16 | return if @error 17 | 18 | pool.cancel 19 | 20 | @error = error 21 | @name = fiber.name 22 | end 23 | end 24 | 25 | def result 26 | reraise_on_error { } 27 | end 28 | 29 | def init(&block : -> _) : -> 30 | block 31 | end 32 | end 33 | end 34 | -------------------------------------------------------------------------------- /src/nested_scheduler/results/result_collector.cr: -------------------------------------------------------------------------------- 1 | module NestedScheduler::Results 2 | # Returns an array of the specified type, in unspecified order. 3 | # Will cancel the pool on any errors and then it will re-raise the 4 | # error when the pool is done. 5 | class ResultCollector(T) < NestedScheduler::Result 6 | property error : Exception? 7 | property name : String? 8 | 9 | def initialize 10 | @lock = Crystal::SpinLock.new 11 | @results = [] of T 12 | end 13 | 14 | # TODO: Figure out how to handle cancellation of pool/fiber? 15 | def register_error(pool, fiber, error) 16 | @lock.sync do 17 | return if @error 18 | 19 | pool.cancel 20 | @error = error 21 | @name = fiber.name 22 | end 23 | end 24 | 25 | def result : Array(T) 26 | reraise_on_error { @results } 27 | end 28 | 29 | def init(&block : -> _) : -> 30 | Proc(Void).new do 31 | res = block.call 32 | @lock.sync do 33 | unless @error 34 | # If statement necessary due to type inference seemingly being borked in this case. 35 | # typeof(res) => T, but res.as(T) raises, thinking res is nilable. 36 | 37 | # Unfortunately I havn't been able to isolate the issue. 38 | if res.is_a?(T) 39 | @results << res 40 | else 41 | raise ArgumentError.new "Expected block to return #{T}, but got #{typeof(res)}" 42 | end 43 | end 44 | end 45 | end 46 | end 47 | end 48 | end 49 | -------------------------------------------------------------------------------- /src/nested_scheduler/results/silent.cr: -------------------------------------------------------------------------------- 1 | module NestedScheduler::Results 2 | # Returns nil. Will ignore any exceptions in fibers. 3 | class Silent < NestedScheduler::Result 4 | def initialize 5 | end 6 | 7 | def register_error(pool, fiber, error) 8 | end 9 | 10 | def result 11 | end 12 | 13 | def init(&block : -> _) : -> 14 | block 15 | end 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /src/nested_scheduler/thread_pool.cr: -------------------------------------------------------------------------------- 1 | require "concurrent" 2 | require "./linked_list2" 3 | require "./monkeypatch/scheduler" 4 | 5 | module NestedScheduler 6 | class ThreadPool 7 | enum State 8 | Ready 9 | Canceled 10 | Finishing 11 | Done 12 | end 13 | 14 | WORKER_NAME = "Worker Loop" 15 | 16 | property workers 17 | property done_channel : Channel(Nil) 18 | property name : String? 19 | property fibers : NestedScheduler::LinkedList2(Fiber) 20 | property spawned 21 | 22 | property io_context : ::NestedScheduler::IOContext 23 | property result_handler : ::NestedScheduler::Result 24 | 25 | def self.nursery( 26 | thread_count = 1, 27 | name = "Child pool", 28 | io_context = nil, 29 | result_handler = NestedScheduler::Results::ErrorPropagator.new 30 | ) 31 | if thread_count < 1 32 | raise ArgumentError.new "No support for nested thread pools in same thread yet" 33 | end 34 | 35 | unless io_context 36 | if p = Thread.current.scheduler.pool 37 | io_context ||= p.io_context 38 | end 39 | raise "Pool missing IO Context" unless io_context 40 | end 41 | pool = new(io_context, result_handler, thread_count, name: name) 42 | begin 43 | yield pool 44 | # TODO: Better exception behavior. Needs to support different 45 | # kinds of failure modes and stacktrace propagation. 46 | ensure 47 | pool.wait_until_done 48 | end 49 | # Unfortunately the result type is a union type of all possible 50 | # result_handler results, which can become arbitrarily big. If 51 | # there was some way to ground the type to only what the given 52 | # result gives, then that would be nice.. 53 | pool.result_handler.result 54 | end 55 | 56 | # Collects the return values of the fiber blocks, in unspecified order. 57 | # If an exception happen, it is propagated. 58 | macro collect(t, **options) 59 | NestedScheduler::ThreadPool.nursery( 60 | result_handler: NestedScheduler::Results::ResultCollector({{t.id}}).new, 61 | {{**options}} 62 | ) do |pl| 63 | {{ yield }} 64 | end.as(Array({{t.id}})) 65 | end 66 | 67 | def initialize( 68 | @io_context : NestedScheduler::IOContext, 69 | @result_handler : NestedScheduler::Result, 70 | count = 1, 71 | bootstrap = false, 72 | @name = nil 73 | ) 74 | @done_channel = Channel(Nil).new capacity: 1 75 | @rr_target = 0 76 | @workers = Array(Thread).new(initial_capacity: count) 77 | @fibers = NestedScheduler::LinkedList2(Fiber).new 78 | @spawned = Atomic(Int32).new(0) 79 | @state = Atomic(State).new(State::Ready) 80 | # Not using the state, as there would be many different waiting 81 | # states, ie regular waiting and cancelled waiting. 82 | @waiting_for_done = Atomic(Int32).new(0) 83 | 84 | if bootstrap 85 | count -= 1 86 | @workers << bootstrap_worker(io_context) 87 | end 88 | pending = Atomic(Int32).new(count) 89 | count.times do 90 | @workers << new_worker(io_context.new) { pending.sub(1) } 91 | end 92 | 93 | # Wait for all worker threads to be fully ready to be used 94 | while pending.get > 0 95 | Fiber.yield 96 | end 97 | end 98 | 99 | private def bootstrap_worker(io_context) 100 | # original init_workers hijack the current thread as part of the 101 | # bootstrap process. 102 | thread = Thread.current 103 | scheduler = thread.scheduler 104 | scheduler.pool = self 105 | # unfortunately, io happen before init_workers is run, so the 106 | # bootstrap scheduler needs a context. 107 | if ctx = scheduler.io_context 108 | raise "Mismatching io context" if ctx.class != io_context.class 109 | else 110 | scheduler.io_context = io_context.new 111 | end 112 | worker_loop = Fiber.new(name: WORKER_NAME) { thread.scheduler.run_loop } 113 | scheduler.actually_enqueue worker_loop 114 | thread 115 | end 116 | 117 | private def new_worker(io_context, &block) 118 | Thread.new do 119 | scheduler = Thread.current.scheduler 120 | scheduler.pool = self 121 | scheduler.io_context = io_context 122 | fiber = scheduler.@current 123 | fiber.name = WORKER_NAME 124 | scheduler.populate_fiber_channel 125 | block.call 126 | scheduler.run_loop 127 | end 128 | end 129 | 130 | def next_thread! 131 | @rr_target &+= 1 132 | workers[@rr_target % workers.size] 133 | end 134 | 135 | def spawn(*, name : String? = nil, same_thread = false, &block : -> _) : Fiber 136 | unless state.in?({State::Ready, State::Finishing}) 137 | raise "Pool is #{state}, can't spawn more fibers at this point" 138 | end 139 | 140 | @spawned.add 1 141 | fiber = Fiber.new(name: name, &result_handler.init(&block)) 142 | 143 | thread = resolve_thread(same_thread) 144 | thread.scheduler.pool!.register_fiber(fiber) 145 | fiber.@current_thread.set(thread) 146 | 147 | Crystal::Scheduler.enqueue fiber 148 | fiber 149 | end 150 | 151 | private def resolve_thread(same_thread) 152 | if same_thread 153 | th = Thread.current 154 | unless th.scheduler.pool == self 155 | raise "It is not possible to spawn into a different thread pool but keeping the same thread." 156 | else 157 | th 158 | end 159 | else 160 | # There is a need to set the thread before calling enqueue 161 | # because otherwise it will enqueue on calling pool. 162 | next_thread! 163 | end 164 | end 165 | 166 | # Cooperatively cancel the current pool. That means the users of 167 | # the pool need to actively check if it is canceled or not. 168 | 169 | # TODO: Investigate cancellation contexts, a la 170 | # https://vorpus.org/blog/timeouts-and-cancellation-for-humans/ 171 | def cancel 172 | # TBH, not totally certain it actually needs to be atomic.. 173 | return if done? 174 | 175 | self.state = State::Canceled 176 | end 177 | 178 | # Has the pool been canceled? 179 | def canceled? 180 | state.canceled? 181 | end 182 | 183 | def finishing? 184 | state.finishing? 185 | end 186 | 187 | def done? 188 | state.done? 189 | end 190 | 191 | def state 192 | @state.get 193 | end 194 | 195 | private def state=(new_state : State) 196 | @state.set new_state 197 | end 198 | 199 | def register_fiber(fiber) 200 | fibers.push(fiber) 201 | end 202 | 203 | def unregister_fiber(fiber) 204 | fibers.delete(fiber) 205 | return if fiber.helper_fiber 206 | 207 | previous_running = @spawned.sub(1) 208 | 209 | # This is probably a race condition, but I don't know how to 210 | # properly fix it. Basically, if several fibers are created and 211 | # then all finish before waiting_for_done is reached there could 212 | # be trouble? It is hard to think about unfortunately.. 213 | 214 | # If @waiting_for_done == 0, then .nursery block hasn't finished yet, 215 | # which means there can still be new fibers that are spawned. 216 | if previous_running == 1 && @waiting_for_done.get > 0 217 | done_channel.send(nil) 218 | end 219 | end 220 | 221 | def inspect 222 | res = [] of String 223 | fibers.unsafe_each do |f| 224 | res << f.inspect 225 | end 226 | <<-EOS 227 | Threadpool #{name}, in #{Fiber.current.name}: 228 | type: #{@io_context.class} 229 | jobs: #{@spawned.get} 230 | passed_block: #{@waiting_for_done.get} 231 | canceled: #{canceled?} 232 | \t#{res.join("\n\t")} 233 | EOS 234 | end 235 | 236 | def wait_until_done 237 | @waiting_for_done.set(1) 238 | self.state = State::Finishing 239 | # Potentially a race condition together with unregister_fiber? 240 | done_channel.receive if @spawned.get > 0 241 | self.state = State::Done 242 | current = Thread.current 243 | @workers.each do |th| 244 | th.scheduler.shutdown unless th == current 245 | end 246 | end 247 | end 248 | end 249 | --------------------------------------------------------------------------------