├── .gitignore ├── .rspec ├── .travis.yml ├── CHANGES.md ├── Gemfile ├── README.md ├── Rakefile ├── benchmarks └── pipeline_bench.rb ├── enumerating.gemspec ├── lib ├── enumerating.rb └── enumerating │ ├── concatenating.rb │ ├── filtering.rb │ ├── merging.rb │ ├── mixing.rb │ ├── prefetching.rb │ ├── threading.rb │ ├── version.rb │ └── zipping.rb └── spec ├── enumerating ├── concatenating_spec.rb ├── filtering_spec.rb ├── merging_spec.rb ├── prefetching_spec.rb ├── threading_spec.rb └── zipping_spec.rb └── spec_helper.rb /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | .bundle 3 | Gemfile.lock 4 | pkg/* 5 | -------------------------------------------------------------------------------- /.rspec: -------------------------------------------------------------------------------- 1 | --color 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: ruby 2 | rvm: 3 | - 1.8.7 4 | - 1.9.3 5 | - 2.0.0 6 | - jruby-18mode 7 | - jruby-19mode 8 | -------------------------------------------------------------------------------- /CHANGES.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## 1.2.0 (2013-04-16) 4 | 5 | * Add `#threading`. 6 | * Add `#prefetching`. 7 | 8 | ## Previously 9 | 10 | * Stuff happened 11 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source "https://rubygems.org" 2 | 3 | gemspec 4 | 5 | gem "rake", "~> 10.0.0" 6 | gem "rspec", ">= 2" 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Enumerating [![Build Status](https://secure.travis-ci.org/mdub/enumerating.png?branch=master)](http://travis-ci.org/mdub/enumerating) 2 | =========== 3 | 4 | Lazy "filtering" and transforming 5 | --------------------------------- 6 | 7 | Enumerating extends Enumerable with "lazy" versions of various common operations: 8 | 9 | * `#selecting` selects elements that pass a test block (like `Enumerable#select`) 10 | * `#finding_all` is an alias for `#selecting` (like `Enumerable#find_all`) 11 | * `#rejecting` selects elements that fail a test block (like `Enumerable#reject`) 12 | * `#collecting` applies a transforming block to each element (like `Enumerable#collect`) 13 | * `#mapping` is an alias for `#collecting` (like `Enumerable#map`) 14 | * `#uniqing` discards duplicates (like `Enumerable#uniq`) 15 | * `#taking`, `#taking_while`, `#dropping` and `#dropping_while` also do what you expect 16 | * '#[]' gets the nth element of a collection 17 | 18 | We say the "...ing" variants are "lazy", because they defer per-element processing until the result is used. They return Enumerable "result proxy" objects, rather than Arrays, and only perform the actual filtering (or transformation) as the result proxy is enumerated. 19 | 20 | Perhaps an example would help. Consider the following snippet: 21 | 22 | >> (1..10).collect { |x| puts "#{x}^2 = #{x*x}"; x*x }.take_while { |x| x < 20 } 23 | 1^2 = 1 24 | 2^2 = 4 25 | 3^2 = 9 26 | 4^2 = 16 27 | 5^2 = 25 28 | 6^2 = 36 29 | 7^2 = 49 30 | 8^2 = 64 31 | 9^2 = 81 32 | 10^2 = 100 33 | => [1, 4, 9, 16] 34 | 35 | Here we use plain old `#collect` to square a bunch of numbers, and then grab the ones less than 20. We can do the same thing using `#collecting`, rather than `#collect`: 36 | 37 | >> (1..10).collecting { |x| puts "#{x}^2 = #{x*x}"; x*x }.take_while { |x| x < 20 } 38 | 1^2 = 1 39 | 2^2 = 4 40 | 3^2 = 9 41 | 4^2 = 16 42 | 5^2 = 25 43 | => [1, 4, 9, 16] 44 | 45 | Same result, but notice how only the first five inputs were ever squared; just enough to find the first result above 20. 46 | 47 | Lazy pipelines 48 | -------------- 49 | 50 | By combining two or more of the lazy operations provided by Enumerating, you can create an efficient "pipeline", e.g. 51 | 52 | # enumerate all users 53 | users = User.to_enum(:find_each) 54 | 55 | # where first and last names start with the same letter 56 | users = users.selecting { |u| u.first_name[0] == u.last_name[0] } 57 | 58 | # grab their company (weeding out duplicates) 59 | companies = users.collecting(&:company).uniqing 60 | 61 | # resolve 62 | companies.to_a #=> ["Disney"] 63 | 64 | Because the steps in the pipeline operate in parallel, without creation of intermediate collections (Arrays), you can efficiently operate on large (or even infinite) Enumerable collections. 65 | 66 | Multi-threaded processing 67 | ------------------------- 68 | 69 | `Enumerable#threading` is a multi-threaded version of `Enumerable#collecting`, allowing multiple Ruby Thread's to be used to process elements of an collection. It requires a numeric argument specifying the maximum number of Threads to use. 70 | 71 | start = Time.now 72 | [5,6,7,8].threading(4) do |delay| 73 | sleep(delay) 74 | end.to_a 75 | (Time.now - start).to_i #=> 8 76 | 77 | Outputs will be yielded in the expected order, making it a drop-in replacement for `#collecting`. 78 | 79 | Unlike some other "parallel map" implementations, the output of `#threading` is lazy, though it does need to pre-fetch elements from the source collection as required to start Threads. 80 | 81 | Lazy combination of Enumerables 82 | ------------------------------- 83 | 84 | Enumerating also provides some interesting ways to combine several Enumerable collections to create a new collection. Again, these operate "lazily". 85 | 86 | `Enumerating.zipping` pulls elements from a number of collections in parallel, yielding each group. 87 | 88 | array1 = [1,3,6] 89 | array2 = [2,4,7] 90 | Enumerating.zipping(array1, array2) # generates: [1,2], [3,4], [6,7] 91 | 92 | `Enumerating.concatenating` concatenates collections. 93 | 94 | array1 = [1,3,6] 95 | array2 = [2,4,7] 96 | Enumerating.concatenating(array1, array2) 97 | # generates: [1,3,6,2,4,7] 98 | 99 | `Enumerating.merging` merges multiple collections, preserving sort-order. The inputs are assumed to be sorted already. 100 | 101 | array1 = [1,4,5] 102 | array2 = [2,3,6] 103 | Enumerating.merging(array1, array2) # generates: 1, 2, 3, 4, 5, 6 104 | 105 | Variant `Enumerating.merging_by` uses a block to determine sort-order. 106 | 107 | array1 = %w(a dd cccc) 108 | array2 = %w(eee bbbbb) 109 | Enumerating.merging_by(array1, array2) { |x| x.length } 110 | # generates: %w(a dd eee cccc bbbbb) 111 | 112 | Same but different 113 | ------------------ 114 | 115 | There are numerous similar implementations of lazy operations on Enumerables. A nod, in particular, to: 116 | 117 | * Greg Spurrier's gem "`lazing`" (from which Enumerating borrows the convention of using "..ing" to name lazy methods) 118 | * `Enumerable#defer` from the Ruby Facets library 119 | 120 | In the end, though, I felt the world deserved another. Enumerating's selling point is that it's basic (filtering/transforming) operations work on any Ruby, whereas most of the other implementations depend on the availablity of Ruby 1.9's "`Enumerator`". Enumerating has been tested on: 121 | 122 | * MRI 1.8.6 123 | * MRI 1.8.7 124 | * MRI 1.9.2 125 | * JRuby 1.5.3 126 | * JRuby 1.6.0 127 | * Rubinius 1.2.3 128 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require "rake" 2 | 3 | require 'bundler' 4 | Bundler::GemHelper.install_tasks 5 | 6 | require "rspec/core/rake_task" 7 | 8 | RSpec::Core::RakeTask.new do |t| 9 | t.pattern = 'spec/**/*_spec.rb' 10 | t.rspec_opts = ["--colour", "--format", "nested"] 11 | end 12 | 13 | task "default" => "spec" 14 | -------------------------------------------------------------------------------- /benchmarks/pipeline_bench.rb: -------------------------------------------------------------------------------- 1 | require 'benchmark' 2 | 3 | $: << File.expand_path("../../lib", __FILE__) 4 | 5 | array = (1..100000).to_a 6 | 7 | # Test scenario: 8 | # - filter out even numbers 9 | # - square them 10 | # - grab the first thousand 11 | 12 | printf "%-30s", "IMPLEMENTATION" 13 | printf "%12s", "take(10)" 14 | printf "%12s", "take(100)" 15 | printf "%12s", "take(1000)" 16 | printf "%12s", "to_a" 17 | puts "" 18 | 19 | def measure(&block) 20 | begin 21 | printf "%12.5f", Benchmark.realtime(&block) 22 | rescue 23 | printf "%12s", "n/a" 24 | end 25 | end 26 | 27 | def benchmark(description, control_result = nil) 28 | result = nil 29 | printf "%-30s", description 30 | measure { yield.take(10).to_a } 31 | measure { yield.take(100).to_a } 32 | measure { result = yield.take(1000).to_a } 33 | measure { yield.to_a } 34 | puts "" 35 | unless control_result.nil? || result == control_result 36 | raise "unexpected result from '#{description}'" 37 | end 38 | result 39 | end 40 | 41 | @control = benchmark "conventional (eager)" do 42 | array.select { |x| x.even? }.collect { |x| x*x } 43 | end 44 | 45 | def can_require?(library) 46 | require(library) 47 | true 48 | rescue LoadError 49 | printf "%-36s ----------------- N/A -----------------\n", library 50 | false 51 | end 52 | 53 | if can_require?("enumerating") 54 | 55 | benchmark "enumerating", @control do 56 | array.selecting { |x| x.even? }.collecting { |x| x*x } 57 | end 58 | 59 | end 60 | 61 | if defined?(Fiber) && can_require?("lazing") 62 | 63 | module Enumerable 64 | alias :lazing_select :selecting 65 | alias :lazing_collect :collecting 66 | end 67 | 68 | benchmark "lazing", @control do 69 | array.lazing_select { |x| x.even? }.lazing_collect { |x| x*x } 70 | end 71 | 72 | end 73 | 74 | if array.respond_to?(:lazy) 75 | 76 | benchmark "ruby2 Enumerable#lazy", @control do 77 | array.lazy.select { |x| x.even? }.lazy.collect { |x| x*x } 78 | end 79 | 80 | elsif can_require?("backports/2.0.0/enumerable") 81 | 82 | benchmark "backports Enumerable#lazy", @control do 83 | array.lazy.select { |x| x.even? }.lazy.collect { |x| x*x } 84 | end 85 | 86 | end 87 | 88 | if can_require? "facets" 89 | 90 | benchmark "facets Enumerable#defer", @control do 91 | array.defer.select { |x| x.even? }.collect { |x| x*x } 92 | end 93 | 94 | end 95 | -------------------------------------------------------------------------------- /enumerating.gemspec: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | $:.push File.expand_path("../lib", __FILE__) 3 | require "enumerating/version" 4 | 5 | Gem::Specification.new do |s| 6 | 7 | s.name = "enumerating" 8 | s.version = Enumerating::VERSION.dup 9 | s.platform = Gem::Platform::RUBY 10 | s.authors = ["Mike Williams"] 11 | s.email = "mdub@dogbiscuit.org" 12 | s.homepage = "http://github.com/mdub/enumerating" 13 | 14 | s.summary = %{Lazy filtering/transforming of Enumerable collections} 15 | s.description = %{Enumerating extends Enumerable with "lazy" versions of various operations, allowing streamed processing of large (or even infinite) collections. Even in Ruby 1.8.x.} 16 | 17 | s.files = `git ls-files`.split("\n") 18 | s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n") 19 | s.require_paths = ["lib"] 20 | 21 | end 22 | -------------------------------------------------------------------------------- /lib/enumerating.rb: -------------------------------------------------------------------------------- 1 | require 'enumerating/filtering' 2 | require 'enumerating/mixing' 3 | require 'enumerating/prefetching' 4 | require 'enumerating/threading' 5 | -------------------------------------------------------------------------------- /lib/enumerating/concatenating.rb: -------------------------------------------------------------------------------- 1 | 2 | module Enumerating 3 | 4 | class Concatenator 5 | 6 | include Enumerable 7 | 8 | def initialize(enumerables) 9 | @enumerables = enumerables 10 | end 11 | 12 | def each(&block) 13 | @enumerables.each do |enumerable| 14 | enumerable.each(&block) 15 | end 16 | end 17 | 18 | end 19 | 20 | end 21 | 22 | class << Enumerating 23 | 24 | def concatenating(*enumerables) 25 | Enumerating::Concatenator.new(enumerables) 26 | end 27 | 28 | end -------------------------------------------------------------------------------- /lib/enumerating/filtering.rb: -------------------------------------------------------------------------------- 1 | require 'set' 2 | 3 | module Enumerating 4 | 5 | class Filter 6 | 7 | include Enumerable 8 | 9 | def initialize(&generator) 10 | @generator = generator 11 | end 12 | 13 | DONE = "done-#{self.object_id}".to_sym 14 | 15 | def each 16 | return to_enum unless block_given? 17 | yielder = proc { |x| yield x } 18 | catch DONE do 19 | @generator.call(yielder) 20 | end 21 | end 22 | 23 | end 24 | 25 | end 26 | 27 | module Enumerable 28 | 29 | def collecting 30 | Enumerating::Filter.new do |output| 31 | each do |element| 32 | output.call yield(element) 33 | end 34 | end 35 | end 36 | 37 | alias mapping collecting 38 | 39 | def selecting 40 | Enumerating::Filter.new do |output| 41 | each do |element| 42 | output.call(element) if yield(element) 43 | end 44 | end 45 | end 46 | 47 | alias finding_all selecting 48 | 49 | def rejecting 50 | Enumerating::Filter.new do |output| 51 | each do |element| 52 | output.call(element) unless yield(element) 53 | end 54 | end 55 | end 56 | 57 | def uniqing 58 | Enumerating::Filter.new do |output| 59 | seen = Set.new 60 | each do |element| 61 | output.call(element) if seen.add?(element) 62 | end 63 | end 64 | end 65 | 66 | def uniqing_by 67 | Enumerating::Filter.new do |output| 68 | seen = Set.new 69 | each do |element| 70 | output.call(element) if seen.add?(yield element) 71 | end 72 | end 73 | end 74 | 75 | def taking(n) 76 | Enumerating::Filter.new do |output| 77 | if n > 0 78 | each_with_index do |element, index| 79 | output.call(element) 80 | throw Enumerating::Filter::DONE if index + 1 == n 81 | end 82 | end 83 | end 84 | end 85 | 86 | def taking_while 87 | Enumerating::Filter.new do |output| 88 | each do |element| 89 | throw Enumerating::Filter::DONE unless yield(element) 90 | output.call(element) 91 | end 92 | end 93 | end 94 | 95 | def dropping(n) 96 | Enumerating::Filter.new do |output| 97 | each_with_index do |element, index| 98 | next if index < n 99 | output.call(element) 100 | end 101 | end 102 | end 103 | 104 | def dropping_while 105 | Enumerating::Filter.new do |output| 106 | taking = false 107 | each do |element| 108 | taking ||= !yield(element) 109 | output.call(element) if taking 110 | end 111 | end 112 | end 113 | 114 | def [](n) 115 | dropping(n).first 116 | end 117 | 118 | end 119 | -------------------------------------------------------------------------------- /lib/enumerating/merging.rb: -------------------------------------------------------------------------------- 1 | module Enumerating 2 | 3 | class Merger 4 | 5 | include Enumerable 6 | 7 | def initialize(enumerables, &transformer) 8 | @enumerables = enumerables 9 | @transformer = transformer 10 | end 11 | 12 | def each(&block) 13 | return to_enum unless block_given? 14 | Generator.new(@enumerables.map(&:to_enum), @transformer).each(&block) 15 | end 16 | 17 | class Generator 18 | 19 | def initialize(enumerators, transformer) 20 | @enumerators = enumerators 21 | @transformer = transformer 22 | end 23 | 24 | def each 25 | loop do 26 | discard_empty_enumerators 27 | break if @enumerators.empty? 28 | yield next_enumerator.next 29 | end 30 | end 31 | 32 | private 33 | 34 | def discard_empty_enumerators 35 | @enumerators.delete_if do |e| 36 | begin 37 | e.peek 38 | false 39 | rescue StopIteration 40 | true 41 | end 42 | end 43 | end 44 | 45 | def next_enumerator 46 | @enumerators.min_by { |enumerator| transform(enumerator.peek) } 47 | end 48 | 49 | def transform(item) 50 | return item unless @transformer 51 | @transformer.call(item) 52 | end 53 | 54 | end 55 | 56 | end 57 | 58 | end 59 | 60 | class << Enumerating 61 | 62 | def merging(*enumerables) 63 | Enumerating::Merger.new(enumerables) 64 | end 65 | 66 | def merging_by(*enumerables, &block) 67 | Enumerating::Merger.new(enumerables, &block) 68 | end 69 | 70 | end 71 | -------------------------------------------------------------------------------- /lib/enumerating/mixing.rb: -------------------------------------------------------------------------------- 1 | require 'enumerating/concatenating' 2 | require 'enumerating/merging' 3 | require 'enumerating/zipping' 4 | -------------------------------------------------------------------------------- /lib/enumerating/prefetching.rb: -------------------------------------------------------------------------------- 1 | module Enumerating 2 | 3 | class Prefetcher 4 | 5 | include Enumerable 6 | 7 | def initialize(source, buffer_size) 8 | @source = source.to_enum 9 | @buffer_size = buffer_size 10 | end 11 | 12 | def each(&block) 13 | return @source.each(&block) if @buffer_size <= 0 14 | buffered_elements = [] 15 | i = 0 16 | @source.each do |element| 17 | slot = i % @buffer_size 18 | if i >= @buffer_size 19 | yield buffered_elements[slot] 20 | end 21 | buffered_elements[slot] = element 22 | i += 1 23 | end 24 | buffered_elements.size.times do 25 | slot = i % buffered_elements.size 26 | yield buffered_elements[slot] 27 | i += 1 28 | end 29 | end 30 | 31 | end 32 | 33 | end 34 | 35 | module Enumerable 36 | 37 | def prefetching(size) 38 | Enumerating::Prefetcher.new(self, size) 39 | end 40 | 41 | end 42 | -------------------------------------------------------------------------------- /lib/enumerating/threading.rb: -------------------------------------------------------------------------------- 1 | require 'enumerating/filtering' 2 | require 'enumerating/prefetching' 3 | 4 | module Enumerable 5 | 6 | def threading(max_threads, &block) 7 | collecting do |item| 8 | Thread.new { block.call(item) } 9 | end.prefetching(max_threads - 1).collecting do |thread| 10 | thread.join; thread.value 11 | end 12 | end 13 | 14 | end 15 | -------------------------------------------------------------------------------- /lib/enumerating/version.rb: -------------------------------------------------------------------------------- 1 | module Enumerating 2 | VERSION = "1.2.1".freeze 3 | end 4 | -------------------------------------------------------------------------------- /lib/enumerating/zipping.rb: -------------------------------------------------------------------------------- 1 | module Enumerating 2 | 3 | class Zipper 4 | 5 | include Enumerable 6 | 7 | def initialize(enumerables) 8 | @enumerables = enumerables 9 | end 10 | 11 | def each 12 | enumerators = @enumerables.map(&:to_enum) 13 | while true 14 | chunk = enumerators.map do |enumerator| 15 | begin 16 | enumerator.next 17 | rescue StopIteration 18 | nil 19 | end 20 | end 21 | break if chunk.all?(&:nil?) 22 | yield chunk 23 | end 24 | end 25 | 26 | end 27 | 28 | end 29 | 30 | class << Enumerating 31 | 32 | def zipping(*enumerables) 33 | Enumerating::Zipper.new(enumerables) 34 | end 35 | 36 | end 37 | -------------------------------------------------------------------------------- /spec/enumerating/concatenating_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | 3 | describe Enumerating do 4 | 5 | describe ".concatenating" do 6 | 7 | it "concatenates multiple Enumerables" do 8 | @array1 = [1,5,3] 9 | @array2 = [2,9,4] 10 | @zip = Enumerating.concatenating(@array1, @array2) 11 | @zip.to_a.should == [1,5,3,2,9,4] 12 | end 13 | 14 | it "is lazy" do 15 | @zip = Enumerating.concatenating([3,4], [1,2].with_time_bomb) 16 | @zip.take(3).should == [3,4,1] 17 | end 18 | 19 | end 20 | 21 | end -------------------------------------------------------------------------------- /spec/enumerating/filtering_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | 3 | module Enumerable 4 | 5 | unless method_defined?(:first) 6 | def first 7 | each do |first_item| 8 | return first_item 9 | end 10 | end 11 | end 12 | 13 | end 14 | 15 | describe Enumerable do 16 | 17 | describe "#collecting" do 18 | 19 | it "transforms items" do 20 | [1,2,3].collecting { |x| x * 2 }.to_a.should == [2,4,6] 21 | end 22 | 23 | it "is lazy" do 24 | [1,2,3].with_time_bomb.collecting { |x| x * 2 }.first.should == 2 25 | end 26 | 27 | end 28 | 29 | describe "#selecting" do 30 | 31 | it "excludes items that don't pass the predicate" do 32 | (1..6).selecting { |x| x%2 == 0 }.to_a.should == [2,4,6] 33 | end 34 | 35 | it "is lazy" do 36 | (1..6).with_time_bomb.selecting { |x| x%2 == 0 }.first == 2 37 | end 38 | 39 | end 40 | 41 | describe "#rejecting" do 42 | 43 | it "excludes items that do pass the predicate" do 44 | (1..6).rejecting { |x| x%2 == 0 }.to_a.should == [1,3,5] 45 | end 46 | 47 | it "is lazy" do 48 | (1..6).with_time_bomb.rejecting { |x| x%2 == 0 }.first == 1 49 | end 50 | 51 | end 52 | 53 | describe "#uniqing" do 54 | 55 | it "removes duplicates" do 56 | [1,3,2,4,3,5,4,6].uniqing.to_a.should == [1,3,2,4,5,6] 57 | end 58 | 59 | it "is lazy" do 60 | [1,2,3].with_time_bomb.uniqing.first.should == 1 61 | end 62 | 63 | end 64 | 65 | describe "#uniqing_by" do 66 | 67 | it "uses the block to derive identity" do 68 | @array = %w(A1 A2 B1 A3 C1 B2 C2) 69 | @array.uniqing_by { |s| s[0,1] }.to_a.should == %w(A1 B1 C1) 70 | end 71 | 72 | end 73 | 74 | describe "#taking" do 75 | 76 | it "includes the specified number" do 77 | @array = [1,2,3,4] 78 | @array.taking(3).to_a.should == [1,2,3] 79 | end 80 | 81 | it "is lazy" do 82 | [1,2].with_time_bomb.taking(2).to_a.should == [1,2] 83 | end 84 | 85 | it "is stackable" do 86 | [1,2].taking(2).taking(2).to_a.should == [1,2] 87 | end 88 | 89 | it "copes with 0" do 90 | [].with_time_bomb.taking(0).to_a.should == [] 91 | end 92 | 93 | end 94 | 95 | describe "#taking_while" do 96 | 97 | it "takes elements as long as the predicate is true" do 98 | @array = [2,4,6,3] 99 | @array.taking_while(&:even?).to_a.should == [2,4,6] 100 | end 101 | 102 | it "is lazy" do 103 | [2,3].with_time_bomb.taking_while(&:even?).to_a.should == [2] 104 | end 105 | 106 | end 107 | 108 | describe "#dropping" do 109 | 110 | it "excludes the specified number" do 111 | @array = [1,2,3,4] 112 | @array.dropping(2).to_a.should == [3,4] 113 | end 114 | 115 | it "is lazy" do 116 | [1,2,3,4].with_time_bomb.dropping(2).taking(1).to_a.should == [3] 117 | end 118 | 119 | end 120 | 121 | describe "#dropping_while" do 122 | 123 | it "drops elements as long as the predicate is true" do 124 | @array = [2,4,6,3,4] 125 | @array.dropping_while(&:even?).to_a.should == [3,4] 126 | end 127 | 128 | it "is lazy" do 129 | [2,3].with_time_bomb.dropping_while(&:even?).taking(1).to_a.should == [3] 130 | end 131 | 132 | end 133 | 134 | describe "#[]" do 135 | 136 | before do 137 | @evens = [1,2,3,4,5].collecting { |x| x * 2 } 138 | end 139 | 140 | it "finds the specified element" do 141 | @evens[2].should == 6 142 | end 143 | 144 | it "is lazy" do 145 | @evens.with_time_bomb[4].should == 10 146 | end 147 | 148 | end 149 | 150 | end 151 | -------------------------------------------------------------------------------- /spec/enumerating/merging_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | 3 | describe Enumerating, :needs_enumerators => true do 4 | 5 | describe ".merging" do 6 | 7 | it "merges multiple Enumerators" do 8 | @array1 = [1,3,6] 9 | @array2 = [2,4,7] 10 | @array3 = [5,8] 11 | @merge = Enumerating.merging(@array1, @array2, @array3) 12 | @merge.to_a.should == [1,2,3,4,5,6,7,8] 13 | end 14 | 15 | it "is lazy" do 16 | @enum1 = [1,3,6] 17 | @enum2 = [2,4,7].with_time_bomb 18 | @merge = Enumerating.merging(@enum1, @enum2) 19 | @merge.take(4).should == [1,2,3,4] 20 | end 21 | 22 | end 23 | 24 | describe ".merging_by" do 25 | 26 | it "uses the block to determine order" do 27 | @array1 = %w(cccc dd a) 28 | @array2 = %w(eeeee bbb) 29 | @merge = Enumerating.merging_by(@array1, @array2) { |s| -s.length } 30 | @merge.to_a.should == %w(eeeee cccc bbb dd a) 31 | end 32 | 33 | end 34 | 35 | end 36 | -------------------------------------------------------------------------------- /spec/enumerating/prefetching_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | 3 | describe Enumerable do 4 | 5 | Counter = Class.new do 6 | 7 | include Enumerable 8 | 9 | def initialize(source) 10 | @source = source 11 | @number_yielded = 0 12 | end 13 | 14 | def each 15 | @source.each do |item| 16 | @number_yielded += 1 17 | yield item 18 | end 19 | end 20 | 21 | attr_reader :number_yielded 22 | 23 | end 24 | 25 | describe "#prefetching" do 26 | 27 | let(:source) { [1, 2, 3, 4, nil, false, 7] } 28 | 29 | it "yields all items" do 30 | source.prefetching(2).to_a.should eq(source) 31 | source.prefetching(3).to_a.should eq(source) 32 | end 33 | 34 | it "is stateless" do 35 | source.prefetching(2).first.should eq(source.first) 36 | source.prefetching(2).first.should eq(source.first) 37 | end 38 | 39 | it "is lazy" do 40 | source.with_time_bomb.prefetching(2).first.should eq(source.first) 41 | end 42 | 43 | it "pre-computes the specified number of elements" do 44 | counter = Counter.new(source) 45 | counter.prefetching(2).take(1) 46 | counter.number_yielded.should eq(3) 47 | end 48 | 49 | context "with a buffer size of zero" do 50 | 51 | it "does not pre-fetch anything" do 52 | counter = Counter.new(source) 53 | counter.prefetching(0).take(1) 54 | counter.number_yielded.should eq(1) 55 | end 56 | 57 | end 58 | 59 | context "with a buffer bigger than the source Enumerable" do 60 | 61 | it "yields all items" do 62 | source.prefetching(20).to_a.should eq(source) 63 | end 64 | 65 | end 66 | 67 | end 68 | 69 | end 70 | -------------------------------------------------------------------------------- /spec/enumerating/threading_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | 3 | describe Enumerable do 4 | 5 | describe "#threading" do 6 | 7 | it "acts like #collect" do 8 | [1,2,3].threading(5) { |x| x * 2 }.to_a.should == [2,4,6] 9 | end 10 | 11 | it "runs things in separate threads" do 12 | [1,2,3].threading(5) { Thread.current.object_id }.to_a.uniq.size.should eq(3) 13 | end 14 | 15 | it "is lazy" do 16 | [1,2,3].with_time_bomb.threading(2) { |x| x * 2 }.first.should == 2 17 | end 18 | 19 | def round(n, accuracy = 0.02) 20 | (n / accuracy).round.to_f * accuracy 21 | end 22 | 23 | it "runs the specified number of threads in parallel" do 24 | delays = [0.03, 0.03, 0.03] 25 | start = Time.now 26 | delays.threading(2) do |delay| 27 | sleep(delay) 28 | end.to_a 29 | round(Time.now - start).should eq(0.06) 30 | end 31 | 32 | it "acts as a sliding window" do 33 | delays = [0.1, 0.08, 0.06, 0.04, 0.02] 34 | start = Time.now 35 | elapsed_times = delays.threading(3) do |delay| 36 | sleep(delay) 37 | round(Time.now - start) 38 | end 39 | elapsed_times.to_a.should eq([0.1, 0.08, 0.06, 0.14, 0.12]) 40 | end 41 | 42 | it "surfaces exceptions" do 43 | lambda do 44 | [1,2,3].threading(5) { raise "hell" }.to_a 45 | end.should raise_error(RuntimeError, "hell") 46 | end 47 | 48 | end 49 | 50 | end 51 | -------------------------------------------------------------------------------- /spec/enumerating/zipping_spec.rb: -------------------------------------------------------------------------------- 1 | require "spec_helper" 2 | 3 | describe Enumerating, :needs_enumerators => true do 4 | 5 | describe ".zipping" do 6 | 7 | it "zips together multiple Enumerables" do 8 | @array1 = [1,3,6] 9 | @array2 = [2,4,7] 10 | @array3 = [5,8] 11 | @zip = Enumerating.zipping(@array1, @array2, @array3) 12 | @zip.to_a.should == [[1,2,5], [3,4,8], [6,7,nil]] 13 | end 14 | 15 | it "is lazy" do 16 | @zip = Enumerating.zipping(%w(a b c), [1,2].with_time_bomb) 17 | @zip.take(2).should == [["a", 1], ["b", 2]] 18 | end 19 | 20 | end 21 | 22 | end 23 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require "rubygems" 2 | 3 | require 'rspec' 4 | 5 | RSpec.configure do |config| 6 | unless defined?(::Enumerator) 7 | config.filter_run_excluding :needs_enumerators => true 8 | end 9 | end 10 | 11 | class Boom < StandardError; end 12 | 13 | class WithTimeBomb 14 | 15 | include Enumerable 16 | 17 | def initialize(source) 18 | @source = source 19 | end 20 | 21 | def each(&block) 22 | @source.each(&block) 23 | raise Boom 24 | end 25 | 26 | end 27 | 28 | module Enumerable 29 | 30 | # extend an Enumerable to throw an exception after last element 31 | def with_time_bomb 32 | WithTimeBomb.new(self) 33 | end 34 | 35 | end 36 | 37 | require "enumerating" 38 | --------------------------------------------------------------------------------