├── README.md ├── haskell ├── Fusion.hs ├── Makefile └── README └── racket ├── Makefile └── transducer.rkt /README.md: -------------------------------------------------------------------------------- 1 | # Program Fusion 2 | 3 | This repository demonstrates two approaches to program fusion: 4 | 5 | 1. Stream Fusion, an example of compile-time metaprogramming, and 6 | 2. Transducers, an example of runtime metaprogramming. 7 | 8 | ## Stream Fusion - GHC Rewrite rules 9 | 10 | The first implementation is a set of GHC rewrite rules that makes up 11 | an implementation of [Stream Fusion](http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.7401). 12 | 13 | The high level idea of Stream Fusion is to transform nested operations 14 | over lists into a single pass with no intermediate data structures. 15 | For example 16 | 17 | ```haskell 18 | filter even $ map (+1) [1, 2, 3 ,4] 19 | ``` 20 | 21 | must build an intermediate representation for the result of the inner `map`. 22 | 23 | Note that Haskell implements the call-by-need evaluation strategy, which means that the `filter` and `map` 24 | operations are only realized as the caller requires them. 25 | So Stream Fusion does not (need to) avoid redundant passes over lists in Haskell, but 26 | in a call-by-value language with eager implementations of `filter` and `map`, this approach 27 | could provide considerable speedup over large lists. 28 | 29 | 30 | ### From Lists, to Streams ... 31 | 32 | Stream Fusion first converts list operations to be over Streams. 33 | 34 | ```haskell 35 | data Stream a = forall s. Stream (s -> Step a s) s 36 | data Step a s = Done 37 | | Yield a s 38 | | Skip s 39 | ``` 40 | 41 | For our purposes, `s` is a type of list and `a` is one of its elements. 42 | 43 | Streams are converted back and forth from lists via `stream` and `unstream`. 44 | 45 | ```haskell 46 | stream :: [a] -> Stream a 47 | stream xs0 = (Stream next xs0) 48 | where 49 | next [] = Done 50 | next (x : xs) = Yield x xs 51 | 52 | unstream :: Stream a -> [a] 53 | unstream (Stream next0 s0) = (unfold s0) 54 | where 55 | unfold s = case next0 s of 56 | Done -> [] 57 | Skip s' -> unfold s' 58 | Yield x s' -> x : unfold s' 59 | ``` 60 | 61 | We now define `map` and `filter` in terms of these primitives via static rewrite rules. 62 | 63 | ```clojure 64 | {-# RULES 65 | "map -> fusible" [1] 66 | forall f xs. map f xs = (unstream (mapS f (stream xs))) 67 | "filter -> fusible" [1] 68 | forall f xs. filter f xs = (unstream (filterS f (stream xs))) 69 | #-} 70 | ``` 71 | 72 | ### ... to Nothing at all 73 | 74 | But now our running example is pointlessly converting betwenn intermediate *stream* data structures. 75 | By our rewrite rules, we have now inlined: 76 | 77 | ```haskell 78 | -- vvvvvvvvvvvvvvvvv 79 | unstream . filterS even . stream . unstream . mapS . stream 80 | ``` 81 | 82 | One extra rewrite rule can fix this for us. 83 | 84 | ```haskell 85 | {-# RULES 86 | "stream/unstream fusion" [0] 87 | forall s. stream (unstream s) = s 88 | #-} 89 | ``` 90 | 91 | This removes any such redundant operations. 92 | 93 | ```haskell 94 | unstream . filterS even . mapS . stream 95 | ``` 96 | 97 | ### Inlining 98 | 99 | Haskell functions do their own inlining automatically. 100 | This interacts in unpredictable ways with GHC rewrite rules. 101 | 102 | In order to ensure some rules fired, I had to suppress inlining 103 | for some functions. For example, to get the `"stream/unstream fusion"` 104 | rule firing, these directives were necessary. 105 | 106 | ```haskell 107 | {-# NOINLINE [1] stream #-} 108 | {-# NOINLINE [1] unstream #-} 109 | ``` 110 | 111 | ### Evaluation 112 | 113 | I failed to install Criterion via Cabal, and didn't complete any benchmarking. 114 | 115 | To enable rewrite rules, I used `-fenable-rewrite-rules`. 116 | 117 | To ensure the rewrite rules fired, I did some crude testing debugging flags. 118 | 119 | ```bash 120 | -ddump-simpl-stats -ddump-rule-firings -dppr-debug 121 | ``` 122 | 123 | I'm fairly confident the `"map -> fusible"` rule is firing, however I'm unsure 124 | about `"stream/unstream fusion"`. 125 | 126 | ## Transducers - Racket 127 | 128 | Transducers are a way to build composable data transformations designed to work over `foldl`/`reduce`. 129 | 130 | They were [first introduced](http://clojure.org/transducers) by Rich Hickey as a Clojure 1.7 feature. 131 | 132 | ## Basics 133 | 134 | A *reducing function* is a function passed to `reduce`, that takes an accumulator and a value, then 135 | returns another accumulator. 136 | 137 | `conj` is a reducing function (argument order like `snoc` for Schemers). 138 | 139 | ```racket 140 | ; (: conj (All (a b) (-> (List a) a (List a)))) 141 | (define conj (lambda (r v) (append r `(,v)))) 142 | ``` 143 | 144 | Notice the type of conj: the first parameter and return types are identical, and it takes some input 145 | as second parameter. It is a reducing function. 146 | 147 | ```racket 148 | (conj '() 1) 149 | ;=> '(1) 150 | 151 | (conj '(1) 2) 152 | ;=> '(1 2) 153 | ``` 154 | 155 | We can use this over `reduce`: 156 | 157 | ```racket 158 | (reduce (lambda (a v) 159 | (conj a v)) 160 | '() '(1 2 3 4)) 161 | ;=> '(1 2 3 4) 162 | ``` 163 | 164 | ### Mapping transducer 165 | 166 | Say we want to `map` `add1` over this data before reducing, like: 167 | 168 | ```racket 169 | (reduce (lambda (a v) 170 | (conj a v)) 171 | '() (map add1 '(1 2 3 4))) 172 | ;=> '(2 3 4 5) 173 | ``` 174 | 175 | To save walking the list twice, we can push the `map` into the reducing 176 | function. 177 | 178 | ```racket 179 | (reduce (lambda (a v) 180 | (conj a (add1 v))) 181 | '() '(1 2 3 4)) 182 | ;=> '(2 3 4 5) 183 | ``` 184 | 185 | Transducers have a pattern for this shape of reducing function via a *mapping transducer*. 186 | 187 | ```racket 188 | (reduce ((Tmap add1) conj) 189 | '() '(1 2 3 4)) 190 | ;=> '(2 3 4 5) 191 | ``` 192 | 193 | `transduce` expresses this fold more readably. It has the same signature as `reduce` but 194 | takes a transducer as the first argument. 195 | 196 | ```racket 197 | (transduce (Tmap add1) conj 198 | '() '(1 2 3 4)) 199 | ;= > '(2 3 4 5) 200 | ``` 201 | 202 | ### What *is* a transducer? 203 | 204 | A transducer is a function that takes a reducing function and returns a reducing function. 205 | 206 | For example, `(Tmap add1)` is a transducer. Applying it to a reducing function 207 | as `((Tmap add1) conj)` results in another reducing function. 208 | 209 | `Tmap` is a transducer generator that generates mapping transducers. 210 | It takes a function that can perform one "step" of the mapping (here `add1`) 211 | and returns a transducer. 212 | 213 | There isn't any trickery here: this reducing function is simply used in every step of the 214 | reduce. Here is each step of the `reduce` explicitly. 215 | 216 | ```racket 217 | (define trans ((Tmap add1) conj)) 218 | 219 | (trans '() 1) 220 | ;=> '(2) 221 | 222 | (trans '(2) 2) 223 | ;=> '(2 3) 224 | 225 | (trans '(2 3) 3) 226 | ;=> '(2 3 4) 227 | 228 | (trans '(2 3 4) 4) 229 | ;=> '(2 3 4 5) 230 | ``` 231 | 232 | Notice the how if we defined `trans` as follows, we would get the same answers. 233 | 234 | ```racket 235 | (define trans (lambda (a v) 236 | (conj a (add1 v)))) 237 | ``` 238 | 239 | ### Filtering transducer 240 | 241 | Let's implement a `filter` over `even?` as a `reduce`. 242 | 243 | ```racket 244 | (reduce (lambda (a v) 245 | (if (even? v) 246 | (conj a v) 247 | a)) 248 | '() '(1 2 3 4)) 249 | ;=> '(2 4) 250 | ``` 251 | 252 | This can be expressed with a filtering transducer. 253 | 254 | ```racket 255 | (reduce ((Tfilter even?) conj) 256 | '() '(1 2 3 4)) 257 | ;=> '(2 4) 258 | ``` 259 | 260 | The trick that `filter` requires is to *skip* processing a member based on a predicate. 261 | Notice that the filtering transducer achieves this. 262 | 263 | ```racket 264 | (define Ftrans ((Tfilter even?) conj)) 265 | 266 | (Ftrans '() 1) 267 | ;=> '() 268 | 269 | (Ftrans '() 2) 270 | ;=> '(2) 271 | 272 | (Ftrans '(2) 3) 273 | ;=> '(2) 274 | 275 | (Ftrans '(2) 4) 276 | ;=> '(2 4) 277 | ``` 278 | 279 | ### Composing transducers 280 | 281 | Consider this reduction whose input has already been walked twice. 282 | 283 | ```racket 284 | (reduce (lambda (a v) 285 | (conj a v)) 286 | '() (filter even? (map sub1 '(1 2 3 4)))) 287 | ;=> '(0 2) 288 | ``` 289 | 290 | We can manually push both the `map` and `filter` inside 291 | the reducing function, saving execution time and intermediate 292 | lists. 293 | 294 | ```racket 295 | (reduce (lambda (a v) 296 | (let ([v (sub1 v)]) 297 | (if (even? v) 298 | (conj a v) 299 | a))) 300 | '() '(1 2 3 4)) 301 | ;=> '(0 2) 302 | ``` 303 | 304 | We can *compose* transducers to build a transducing pipeline. 305 | 306 | ```racket 307 | (reduce ((compose (Tmap sub1) (Tfilter even?)) 308 | conj) 309 | '() '(1 2 3 4)) 310 | ;=> '(0 2) 311 | ``` 312 | 313 | This is much nicer with `transduce`. 314 | 315 | ```racket 316 | (transduce (compose (Tmap sub1) (Tfilter even?)) 317 | conj 318 | '() '(1 2 3 4)) 319 | ;=> '(0 2) 320 | ``` 321 | 322 | Given `compose` composes right-to-left, this is surprising! 323 | 324 | ```racket 325 | (define subfilter 326 | (compose (curry filter even?) 327 | (curry map sub1))) 328 | 329 | (subfilter '(1 2 3 4)) 330 | ;=> '(0 2) 331 | ``` 332 | 333 | Transducers compose *backwards*. 334 | 335 | ```racket 336 | (define Tsubfilter 337 | ((compose (Tmap sub1) 338 | (Tfilter even?)) 339 | conj)) 340 | 341 | (Tsubfilter '() 1) 342 | ;=> '(0) 343 | 344 | (Tsubfilter '(0) 2) 345 | ;=> '(0) 346 | 347 | (Tsubfilter '(0) 3) 348 | ;=> '(0 2) 349 | 350 | (Tsubfilter '(0 2) 4) 351 | ;=> '(0 2) 352 | ``` 353 | 354 | To understand this, we first expand the definitions of `(Tmap sub1)` 355 | and `(Tfilter even?)` independently. 356 | 357 | #### Expanding `(Tmap sub1)` 358 | 359 | This is the expansion of a mapping transducer. It's just a function that takes 360 | a reducing function and returns a reducing function. 361 | 362 | ```racket 363 | (Tmap sub1) 364 | ;---> 365 | (lambda ([rf : ((Listof Number) Number -> (Listof Number))]) 366 | (lambda ([result : (Listof Number)] 367 | [input : Number]) 368 | (rf result (sub1 input)))) 369 | ``` 370 | 371 | If we actually apply the mapping transducer to a reducing function, we get back 372 | a new reducing function. Notice this does exactly what we want for a single step 373 | of a `map`: perform the operation (subtraction), then append to the list. 374 | 375 | ```racket 376 | ((Tmap sub1) conj) 377 | ;---> 378 | (lambda ([result : (Listof Number)] 379 | [input : Number]) 380 | (conj result (sub1 input))) 381 | ``` 382 | 383 | #### Expanding `(Tfilter sub1)` 384 | 385 | The filtering transducer is similar: it takes a reducing function and returns a reducing 386 | function that might call the passed in function if the predicate passes. 387 | 388 | ```racket 389 | (Tfilter even?) 390 | ;---> 391 | (lambda ([rf : ((Listof Number) Number -> (Listof Number))]) 392 | (lambda ([result : (Listof Number)] 393 | [input : Number]) 394 | (if (even? input) 395 | (rf result input) 396 | result))) 397 | ``` 398 | 399 | Providing a reducing function as `conj` gives us the stepper function for a `filter` defined 400 | with `reduce`. 401 | 402 | ```racket 403 | ((Tfilter even?) conj) 404 | ;---> 405 | (lambda ([result : (Listof Number)] 406 | [input : Number]) 407 | (if (even? input) 408 | (conj result input) 409 | result)) 410 | ``` 411 | 412 | #### Expanding `(compose (Tmap sub1) (Tfilter even?))` 413 | 414 | Here's where it gets interesting. 415 | Composing transducers works out left-to-right, so we should end up 416 | with a reducing function that first performs a step of `(Tmap sub1)` 417 | then a step of `(Tfilter even?)`. 418 | 419 | By composing two transducers we get another transducer: again, simply a function 420 | that takes a reducing function and returns one. 421 | (The definition of `compose` is inlined). 422 | 423 | ```racket 424 | (compose (Tmap sub1) (Tfilter even?)) 425 | ;----> 426 | (lambda ([rf : ((Listof Number) Number -> (Listof Number))]) 427 | ((Tmap sub1) 428 | ((Tfilter even?) 429 | rf))) 430 | ``` 431 | 432 | Observe what happens when we apply a reducing function. 433 | 434 | ```racket 435 | ((compose (Tmap sub1) (Tfilter even?)) 436 | conj) 437 | ;----> 438 | ((Tmap sub1) 439 | ((Tfilter even?) 440 | conj)) 441 | ;----> 442 | ;; expand Tfilter 443 | ((Tmap sub1) 444 | (lambda ([result : (Listof Number)] 445 | [input : Number]) 446 | (if (even? input) 447 | (conj result input) 448 | result))) 449 | ;----> 450 | ;; expand Tmap 451 | ((lambda ([rf : ((Listof Number) Number -> (Listof Number))]) 452 | (lambda ([result : (Listof Number)] 453 | [input : Number]) 454 | (rf result (sub1 input)))) 455 | (lambda ([result : (Listof Number)] 456 | [input : Number]) 457 | (if (even? input) 458 | (conj result input) 459 | result))) 460 | ;----> 461 | ;; beta reduction 462 | (lambda ([result : (Listof Number)] 463 | [input : Number]) 464 | ((lambda ([result : (Listof Number)] 465 | [input : Number]) 466 | (if (even? input) 467 | (conj result input) 468 | result)) 469 | result (sub1 input))) 470 | ;----> 471 | ;; convert inner lambda -> let 472 | (lambda ([result : (Listof Number)] 473 | [input : Number]) 474 | (let ([input (sub1 input)]) 475 | (if (even? input) 476 | (conj result input) 477 | result))) 478 | ``` 479 | 480 | Wow! The exact reducing function that we want appears. 481 | 482 | Now we can follow exactly why 483 | 484 | ```racket 485 | (reduce ((compose (Tmap sub1) 486 | (Tfilter even?)) 487 | conj) 488 | '() '(1 2 3 4)) 489 | ``` 490 | 491 | is equivalent to 492 | 493 | ```racket 494 | (reduce (lambda ([result : (Listof Number)] 495 | [input : Number]) 496 | (let ([input (sub1 input)]) 497 | (if (even? input) 498 | (conj result input) 499 | result))) 500 | '() '(1 2 3 4)) 501 | ``` 502 | 503 | ### Benchmarks 504 | 505 | I performed some microbenchmarks. I consistently observed speedup as I moved nested list operations 506 | into a reducing function. 507 | 508 | This is unsurprising, since `map` and `filter` are eager in Racket, so for large lists we are saving 509 | considerable time from traversing them multiple times. 510 | 511 | I ran each benchmark over 100 iterations, with a 100,000 element list, on a Ubuntu VM with 1gb RAM 512 | on 2.4GHz Intel Core i5, 2012 MacBook Pro. 513 | 514 | #### All transducers 515 | 516 | All transformations are performed in one pass. 517 | 518 | ```racket 519 | (transduce (compose (Tmap add1) (Tfilter even?)) + 0 big-list) 520 | ;; cpu time: 1300 real time: 1274 gc time: 100 521 | ``` 522 | 523 | #### Partial transducers 524 | 525 | Moving the map outside means one extra pass. 526 | 527 | ```racket 528 | (transduce (Tfilter even?) + 0 (map add1 big-list)) 529 | cpu time: 1624 real time: 1573 gc time: 116 530 | ``` 531 | 532 | This was 300ms slower than if `map` was a transducer. 533 | 534 | #### No transducers 535 | 536 | Moving the map and filter outside means two extra passes. 537 | 538 | ```racket 539 | (reduce + 0 (filter even? (map add1 big-list))) 540 | cpu time: 2008 real time: 1963 gc time: 92 541 | ``` 542 | 543 | This was 600ms slower than if `map` and `filter were transducers. 544 | 545 | ## Takeaways 546 | 547 | Metaprogramming comes in all shapes and sizes, depending on the context and 548 | the goals of a project. 549 | 550 | Stream Fusion was designed to leverage existing idioms by statically rewriting 551 | common list operations to fuse them together without intermediate data structures. 552 | 553 | Transducers were first invented for Clojure as a new way of compositing data transformations 554 | generically. This form of runtime metaprogramming did not leverage existing idioms. Outside 555 | of downstream operations internally using transducers, there is no automatic speedup for user-code. 556 | -------------------------------------------------------------------------------- /haskell/Fusion.hs: -------------------------------------------------------------------------------- 1 | import Prelude hiding (map, filter) 2 | import Debug.Trace 3 | 4 | {-# LANGUAGE ExistentialQuantification #-} 5 | 6 | {-# RULES 7 | "map -> fusible" [1] 8 | forall f xs. map f xs = (unstream (mapS f (stream xs))) 9 | "filter -> fusible" [1] 10 | forall f xs. filter f xs = (unstream (filterS f (stream xs))) 11 | "stream/unstream fusion" [0] 12 | forall s. stream (unstream s) = s 13 | #-} 14 | 15 | data Stream a = forall s. Stream (s -> Step a s) s 16 | data Step a s = Done 17 | | Yield a s 18 | | Skip s 19 | 20 | stream :: [a] -> Stream a 21 | stream xs0 = (Stream next xs0) 22 | where 23 | next [] = Done 24 | next (x : xs) = Yield x xs 25 | {-# NOINLINE [1] stream #-} 26 | 27 | unstream :: Stream a -> [a] 28 | unstream (Stream next0 s0) = (unfold s0) 29 | where 30 | unfold s = case next0 s of 31 | Done -> [] 32 | Skip s' -> unfold s' 33 | Yield x s' -> x : unfold s' 34 | {-# NOINLINE [1] unstream #-} 35 | 36 | 37 | mapS :: (a -> b) -> Stream a -> Stream b 38 | mapS f (Stream next0 s0) = Stream next s0 39 | where 40 | next s = case next0 s of 41 | Done -> Done 42 | Skip s' -> Skip s' 43 | Yield x s' -> Yield (f x) s' 44 | {-# NOINLINE [1] mapS #-} 45 | 46 | map :: (a -> b) -> [a] -> [b] 47 | map f = unstream . mapS f . stream 48 | {-# NOINLINE [1] map #-} 49 | 50 | filterS :: (a -> Bool) -> Stream a -> Stream a 51 | filterS p (Stream next0 s0) = Stream next s0 52 | where 53 | next s = case next0 s of 54 | Done -> Done 55 | Skip s' -> Skip s' 56 | Yield x s' | p x -> Yield x s' 57 | | otherwise -> Skip s' 58 | 59 | filter :: (a -> Bool) -> [a] -> [a] 60 | filter p = unstream . filterS p . stream 61 | {-# NOINLINE [1] filter #-} 62 | 63 | main :: IO () 64 | main = 65 | putStrLn . show 66 | $ (map (+1) (filter even (map (+1) [1, 2, 3 ,4]))) 67 | 68 | -- unstream . mapS (+1) . stream $ (unstream . mapS (+1) . stream $ [1,2]) 69 | -------------------------------------------------------------------------------- /haskell/Makefile: -------------------------------------------------------------------------------- 1 | all: 2 | ghc -XExistentialQuantification -fenable-rewrite-rules -ddump-simpl-stats -ddump-rule-firings -dppr-debug Fusion.hs 3 | 4 | clean: 5 | rm Fusion.o Fusion 6 | -------------------------------------------------------------------------------- /haskell/README: -------------------------------------------------------------------------------- 1 | # Stream Fusion in GHC 2 | 3 | Stream fusion is a form of deforestation introduced by [Coutts et al.](http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.7401). 4 | -------------------------------------------------------------------------------- /racket/Makefile: -------------------------------------------------------------------------------- 1 | all: 2 | racket transducer.rkt 3 | -------------------------------------------------------------------------------- /racket/transducer.rkt: -------------------------------------------------------------------------------- 1 | #lang typed/racket/no-check 2 | 3 | (require rackunit) 4 | (provide conj 5 | identity 6 | Tmap 7 | Tfilter 8 | reduce 9 | transduce) 10 | 11 | (define (conj xs x) (append xs `(,x))) 12 | 13 | ;; utils 14 | (define identity (lambda (x) x)) 15 | 16 | ;; reduce 17 | (struct Reduced ([val : Any])) 18 | 19 | (define reduced 20 | (lambda (val) 21 | (Reduced val))) 22 | 23 | (define reduced? 24 | Reduced?) 25 | 26 | (define deref-reduced 27 | (lambda (d) 28 | (Reduced-val d))) 29 | 30 | (define (reduce f val coll) 31 | (if (pair? coll) 32 | (let ([v (f val (car coll))]) 33 | (if (reduced? v) 34 | (deref-reduced v) 35 | (reduce f v (cdr coll)))) 36 | val)) 37 | 38 | (define-type Reducer (All (a) 39 | (All (r) 40 | (r a -> r)))) 41 | 42 | ;; A transducer is a transformation from one reducing function to another. 43 | 44 | (define 45 | #:forall (a b) 46 | (Tmap [f : (a -> b)]) : ((Reducer b) -> (Reducer a)) 47 | (lambda (rf) 48 | (lambda ([result : r] 49 | [input : a]) 50 | (rf result (f input))))) 51 | 52 | (define 53 | #:forall (a) 54 | (Tfilter pred) 55 | (lambda (rf) 56 | (lambda ([result : r] 57 | [input : a]) 58 | (if (pred input) 59 | (rf result input) 60 | result)))) 61 | 62 | (define (transduce xform f init coll) 63 | (reduce (xform f) init coll)) 64 | 65 | 66 | ;; equivalent to (+ (+ (+ 0 (add1 (add1 1))) 67 | ;; (add1 (add1 2))) 68 | ;; (add1 (add1 3))) 69 | (check-equal? (transduce (compose (Tmap add1) (Tmap add1)) + 0 '(1 2 3)) 70 | 12) 71 | 72 | 73 | ;; (equivalent to (+ (+ (+ 0 (identity 1)) 74 | ;; (identity 2)) 75 | ;; (identity 3)) 76 | (check-equal? (transduce (Tmap identity) + 0 '(1 2 3)) 77 | 6) 78 | 79 | ;; equivalent to (+ 0 2) 80 | (check-equal? (transduce (compose (Tfilter even?) (Tmap add1)) + 0 '(1 2 3)) 81 | 3) 82 | 83 | ;; equivalent to (+ (+ 0 2) 4) 84 | (check-equal? (transduce (compose (Tmap add1) (Tfilter even?)) + 0 '(1 2 3)) 85 | 6) 86 | 87 | (define small-list '(1 2 3 4)) 88 | (define big-list (for/list : Integer ([e : Integer 100000]) e)) 89 | 90 | ;; one traversal over big-list 91 | (define (one-trav) (transduce (compose (Tmap add1) (Tfilter even?)) + 0 big-list)) 92 | 93 | (time (for ([_ 100]) (one-trav))) 94 | 95 | ;; two traversals over big-list 96 | (define (two-trav) (transduce (Tfilter even?) + 0 (map add1 big-list))) 97 | 98 | (time (for ([_ 100]) 99 | (two-trav))) 100 | 101 | ;; three traversals over big-list 102 | (define (three-trav) (reduce + 0 (filter even? (map add1 big-list)))) 103 | 104 | (time (for ([_ 100]) (three-trav))) 105 | 106 | (check-equal? #t (= 2500050000 (one-trav) (two-trav) (three-trav))) 107 | 108 | (((Tmap add1) (lambda (r v) (cons v r))) '() 1) 109 | 110 | ;; Tmap is a transducer generator 111 | (define step1 Tmap) 112 | 113 | ;; (Tmap add1) is a transducer (takes a reducing function and returns a reducing function) 114 | (define step2 (step1 add1)) 115 | 116 | ;; ((Tmap add1) conj) is a reducing function (that takes a list and an accumulator and returns a list) 117 | (define step3 (step2 conj)) 118 | 119 | ;; (((Tmap add1) conj) '() 1) 120 | (define step4.1 (step3 '() 1)) 121 | (define step4.2 (step3 step4.1 2)) 122 | (define step4.3 (step3 step4.2 3)) 123 | (define step4.4 (step3 step4.3 4)) 124 | --------------------------------------------------------------------------------