├── .gitignore ├── README.md ├── package.json ├── src ├── chapters │ ├── 00-intro.md │ ├── 02-for-of.md │ ├── 06-iteration-everywhere.md │ ├── 01-pull-push.md │ ├── 04-generators.md │ ├── 05-polymorphic-iterators.md │ ├── 07-recursion.md │ ├── 03-iterators.md │ └── 08-advanced-topics.md ├── page.html └── compile.js └── dist ├── assets └── styles.css ├── 00-intro.html ├── 02-for-of.html ├── 06-iteration-everywhere.html ├── 01-pull-push.html ├── 04-generators.html ├── 05-polymorphic-iterators.html ├── 07-recursion.html ├── 03-iterators.html └── 08-advanced-topics.html /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules/ 2 | .DS_Store 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # gen 2 | 3 | A brief tutorial covering the basics of JavaScript iterators and generators as introduced in ES6/ES2015. 4 | 5 | Start here: http://greim.github.io/gen/dist/00-intro.html 6 | 7 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "gen", 3 | "version": "1.0.0", 4 | "description": "Generator articles", 5 | "main": "index.js", 6 | "scripts": { 7 | "test": "echo \"Error: no test specified\" && exit 1", 8 | "compile": "node src/compile.js" 9 | }, 10 | "author": "", 11 | "license": "ISC", 12 | "dependencies": { 13 | "co": "^4.6.0", 14 | "handlebars": "^4.0.5", 15 | "marked": "^0.3.5", 16 | "ugly-adapter": "^1.3.1" 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /src/chapters/00-intro.md: -------------------------------------------------------------------------------- 1 | # Generators and Iterators 2 | 3 | Generators are an under-appreciated feature of modern JavaScript. I believe this is because generators are just one half of a two-part whole comprising *both generators and iterators*, and that learning one without the other paints an incomplete picture. For myself at least, the ah-ha! moment didn't come until I zoomed out and looked at iterators and generators holistically. 4 | 5 | This tutorial follows the same approach. First we lay out the case for iterators, then we move on to generators in order to unify our understanding. 6 | 7 | ---------------- 8 | 9 | {{ next }} 10 | 11 | ---------------- 12 | 13 | {{ toc }} 14 | -------------------------------------------------------------------------------- /dist/assets/styles.css: -------------------------------------------------------------------------------- 1 | body { 2 | font-family: 'Roboto Slab'; 3 | font-weight: 300; 4 | margin: 0; 5 | padding: 0; 6 | line-height: 1.7; 7 | font-size: 2.8em; 8 | background:#ccc; 9 | } 10 | ul, ol { 11 | padding: 0; 12 | margin: 1em 5em 1em 3em; 13 | } 14 | h2 { 15 | margin-top: 1.5em; 16 | } 17 | h1,h2,h3,h4,h5,h6{ 18 | color: #c60; 19 | } 20 | hr { 21 | height:0; 22 | border: 1px solid #ccc; 23 | border-width: 1px 0 0 0; 24 | margin: 2.5em 0; 25 | } 26 | #wrapper { 27 | max-width: 45em; 28 | margin: -2em auto; 29 | border:.3em solid #c60; 30 | border-width: 0 .3em; 31 | padding: 3em 3.5em 5em; 32 | background:#f6f6f6; 33 | box-shadow: 0 0 .5em rgba(0,0,0,0.2); 34 | } 35 | pre { 36 | font-family: Inconsolata, monospace; 37 | line-height: 1.3; 38 | } 39 | code { 40 | color: #c60; 41 | } 42 | .hljs { 43 | padding: 1em; 44 | border-left: .5em solid rgba(0,0,0,0.1); 45 | } 46 | a[href] { 47 | text-decoration: none; 48 | } 49 | #bottom-note{ 50 | font-style:italic; 51 | font-weight:100; 52 | } 53 | img { 54 | max-width: 100%; 55 | } 56 | -------------------------------------------------------------------------------- /src/chapters/02-for-of.md: -------------------------------------------------------------------------------- 1 | # Chapter II: The for/of loop 2 | 3 | {{ toc }} 4 | 5 | ## A new kind of loop 6 | 7 | In the first chapter, we learned about tradeoffs between *pull* and *push* mode for consuming collections. However, ES6 introduces a new way of looping that we failed to take into account: the for/of loop. 8 | 9 | ## Solving the pull/push conundrum 10 | 11 | Compare these two ways of looping an array: 12 | 13 | ```js 14 | for (var i=0; i 2 | 3 | 4 | 5 | 6 | {{heading}} 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 | {{{innerHtml}}} 24 |
25 |

26 | Copyright © 2016 by Greg Reimer (github, twitter). 27 | Submit issues to the GitHub issues page. 28 |

29 |

30 | 31 | 32 |

33 |
34 | 35 | 36 | 37 | 38 | -------------------------------------------------------------------------------- /src/chapters/06-iteration-everywhere.md: -------------------------------------------------------------------------------- 1 | # Chapter VI: Beyond the for/of loop 2 | 3 | {{ toc }} 4 | 5 | ## What else can iterators do? 6 | 7 | This chapter is a quick survey of various other places iterables are used, besides the for/of loop. 8 | 9 | ### Things that are iterables 10 | 11 | * Arrays 12 | * Strings 13 | * Maps 14 | * Sets 15 | * Function `arguments` objects 16 | * DOM collections 17 | 18 | ### Things that accept iterables 19 | 20 | #### Spread operators 21 | 22 | ```js 23 | function* foo() { yield 'a', yield 'b', yield 'c'; } 24 | function bar() { console.log(arguments); } 25 | bar(...foo()); // => { 0: 'a', 1: 'b', 2: 'c' } 26 | ``` 27 | 28 | ```js 29 | function* foo() { yield 1, yield 2, yield 3; } 30 | console.log([...foo()]); // => [ 1, 2, 3 ] 31 | ``` 32 | 33 | #### Destructuring 34 | 35 | ```js 36 | function* foo() { yield 1, yield 2, yield 3; } 37 | const [ x, y, z ] = foo(); 38 | console.log(x); // => 1 39 | console.log(y); // => 2 40 | console.log(z); // => 3 41 | ``` 42 | 43 | #### Construction of maps and sets 44 | 45 | ```js 46 | function* foo() { yield 1, yield 2, yield 3; } 47 | const set = new Set(foo()); 48 | console.log(set.has(1)); // => true 49 | console.log(set.has(2)); // => true 50 | console.log(set.has(3)); // => true 51 | ``` 52 | 53 | ```js 54 | function* foo() { yield ['a', 1], yield ['b', 2]; } 55 | const map = new Map(foo()); 56 | console.log(map.get('a')); // => 1 57 | console.log(map.get('b')); // => 2 58 | ``` 59 | 60 | #### Array.from() 61 | 62 | ```js 63 | function* foo() { yield 1, yield 2, yield 3; } 64 | console.log(Array.from(foo())); // => [ 1, 2, 3 ] 65 | ``` 66 | 67 | #### Promise.all() 68 | 69 | ```js 70 | function* foo() { 71 | yield Promise.resolve(1); 72 | yield Promise.resolve(2); 73 | yield Promise.resolve(3); 74 | } 75 | Promise.all(foo()).then(arr => { 76 | console.log(arr); // => [ 1, 2, 3 ] 77 | }); 78 | ``` 79 | 80 | #### Generator delegation 81 | 82 | ```js 83 | function* foo() { yield 1, yield 2, yield 3; } 84 | function* bar() { yield* foo(); } 85 | const arr = Array.from(bar()); 86 | console.log(arr); // => [ 1, 2, 3 ] 87 | ``` 88 | 89 | ---------------- 90 | 91 | {{ next }} 92 | 93 | ---------------- 94 | 95 | {{ toc }} 96 | -------------------------------------------------------------------------------- /src/chapters/01-pull-push.md: -------------------------------------------------------------------------------- 1 | # Chapter I: The pull/push conundrum 2 | 3 | {{ toc }} 4 | 5 | ## Looping collections 6 | 7 | It all starts with looping. Let's consider two kinds of data structures you can loop through: trees and arrays. In an array, everything exists sequentially in a line, so looping is easy. A tree has a non-linear branching structure, so looping it isn't so straightforward. Also, terminology note: when you loop through a collection, we'll say that you're the **consumer**, while the collection is the **producer**. 8 | 9 | ## Pull mode 10 | 11 | As data flows from producer to consumer, "pull mode" means that the consumer initiates the transfer, while the producer remains passive: 12 | 13 | ```js 14 | for (var i=0; i 0) { 24 | var node = queue.shift(); 25 | visit(node.value); 26 | if (node.left) { queue.push(node.left); } 27 | if (node.right) { queue.push(node.right); } 28 | } 29 | ``` 30 | 31 | That's quite a pile of code to have to memorize and type out just to loop a collection, but since I'm initiating the transfer, the task falls to me. 32 | 33 | ## Push mode 34 | 35 | Which raises the question: why should I initiate the transfer? Why not let producer do it? Hence push mode, in which the collection has a `forEach()` method that accepts a callback, which it uses to *push* values to us, while we sit back and passively receive the values. 36 | 37 | ```js 38 | array.forEach(elmt => visit(elmt)); 39 | tree.forEach(elmt => visit(elmt)); 40 | ``` 41 | 42 | Much better. The various state tracking and algorithmic details are encapsulated away from us inside the `forEach()` method, which is a great separation of concerns. 43 | 44 | ## Unfortunate tradeoffs 45 | 46 | So push mode wins, right? Sadly, by switching from pull to push, we lost some power and flexibility: 47 | 48 | * We can't `return` the outer function from inside the callback. 49 | * We can't `break` or `continue` from inside the callback. 50 | * We can't `yield` from within the callback. 51 | 52 | Those behaviors could be simulated using some sort of pre-agreed-upon signaling mechanism between the callback and its runner, but by then we've begun to re-invent the wheel, since the language gives us those capabilities for free with loops. 53 | 54 | ---------------- 55 | 56 | {{ next }} 57 | 58 | ---------------- 59 | 60 | {{ toc }} 61 | -------------------------------------------------------------------------------- /src/chapters/04-generators.md: -------------------------------------------------------------------------------- 1 | # Chapter IV: Generators 2 | 3 | {{ toc }} 4 | 5 | ## The problem 6 | 7 | In the last chapter we discussed how to make anything for/of-able by using iterators. But I got stuck trying to do it for a binary search tree. Recall where we left off: 8 | 9 | ```js 10 | class Tree { 11 | 12 | // Assume various BST methods already exist 13 | // here like add() and remove() 14 | 15 | [Symbol.iterator]() { 16 | return { 17 | next() { 18 | // Put the algorithm here, maybe? 19 | } 20 | }; 21 | } 22 | } 23 | 24 | // Our tree-traversal algorithm. 25 | // Need to drop this in above somewhere... 26 | var queue = this.root ? [this.root] : []; 27 | while (queue.length > 0) { 28 | var node = queue.shift(); 29 | // do something with node.value 30 | if (node.left) { queue.push(node.left); } 31 | if (node.right) { queue.push(node.right); } 32 | } 33 | ``` 34 | 35 | Down in the guts of this tree-traversal algorithm, there's a point where we have the value in-hand. But we can't just let the loop run to completion. We have to hand off a value, then *suspend* the loop mid-flight and wait... somehow. 36 | 37 | ## The solution 38 | 39 | It turns out that *suspending and waiting* is exactly what generators do. The mechanics of this are discussed in more detail below, but let's just jump right to the solution. 40 | 41 | First, we declare `[Symbol.iterator]` as a generator by adding an asterisk `*`. Then we drop our algorithm in wholesale. Finally, we hand off the value using a `yield` expression. 42 | 43 | ```js 44 | class Tree { 45 | // ... 46 | *[Symbol.iterator]() { 47 | var queue = this.root ? [this.root] : []; 48 | while (queue.length > 0) { 49 | var node = queue.shift(); 50 | yield node.value; // <-- hand off the value 51 | if (node.left) { queue.push(node.left); } 52 | if (node.right) { queue.push(node.right); } 53 | } 54 | } 55 | } 56 | ``` 57 | 58 | As far as the outside world is concerned, a generator is merely a function that returns an iterator. Because of this, we've satisfied the iterable protocol and we can consume our tree: 59 | 60 | ```js 61 | for (const value of tree) { ... } 62 | ``` 63 | 64 | ## Okay, but *how* do generators work? 65 | 66 | Essentially, what happens is that when the `*[Symbol.iterator]` function is called, instead of running our algorithm to completion, JavaScript puts it into a *paused* state, without running it. 67 | 68 | Meanwhile, JavaScript returns an iterator "`itr`" to the caller. When `itr.next()` is called, JavaScript presses *play* on the algorithm, which runs right up until the point it encounters a `yield`, then pauses again. The yielded value is subsequently returned from `itr.next()` along with `done: false`. 69 | 70 | ```js 71 | { value: /* whatever was yielded */, done: false } 72 | ``` 73 | 74 | This happens anywhere between zero and infinity times. If/when the generator algorithm hits a `return` instead of a `yield` (the return can be either implicit or explicit) the returned value is returned from `itr.next()` along with `done: true`. 75 | 76 | ```js 77 | { value: /* whatever was returned */, done: true } 78 | ``` 79 | 80 | ---------------- 81 | 82 | {{ next }} 83 | 84 | ---------------- 85 | 86 | {{ toc }} 87 | -------------------------------------------------------------------------------- /dist/00-intro.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Generators and Iterators 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Generators and Iterators

24 |

Generators are an under-appreciated feature of modern JavaScript. I believe this is because generators are just one half of a two-part whole comprising both generators and iterators, and that learning one without the other paints an incomplete picture. For myself at least, the ah-ha! moment didn't come until I zoomed out and looked at iterators and generators holistically.

25 |

This tutorial follows the same approach. First we lay out the case for iterators, then we move on to generators in order to unify our understanding.

26 |
27 |

Next: Chapter I: The pull/push conundrum →

28 |
29 | 40 | 41 |
42 |

43 | Copyright © 2016 by Greg Reimer (github, twitter). 44 | Submit issues to the GitHub issues page. 45 |

46 |

47 | 48 | 49 |

50 |
51 | 52 | 53 | 54 | 55 | -------------------------------------------------------------------------------- /src/chapters/05-polymorphic-iterators.md: -------------------------------------------------------------------------------- 1 | # Chapter V: Polymorphic iterators 2 | 3 | {{ toc }} 4 | 5 | ## A plot twist 6 | 7 | Having covered the basics of iteration and generators, we now take a break in order to go down a small rabbit-hole. 8 | 9 | Recall that iterators and iterables are separate things. Well, plot twist: Pretty much every *native* JavaScript iterator object implements the iterable protocol by having a `[Symbol.iterator]` method which returns itself. In effect, most iterators are both iterators and iterables, simultaneously! 10 | 11 | ## Possibly confusing, but useful 12 | 13 | If you're like me, this will hurt your brain... at first. But there's a good reason for it, since it allows passing any iterator where an iterable is expected. Why would I want to do that, you ask? Because it allows doing this: 14 | 15 | ```js 16 | for (const key of map.keys()) { ... } 17 | ``` 18 | 19 | As the `Map#keys()` method demonstrates, `object[Symbol.iterator]()` isn't the only thing that can return iterators. Some methods return iterators directly. We definitely want the ability to use these in places where iterables are expected, such as for/of loops. 20 | 21 | ## What does this mean for you? 22 | 23 | This shakes out in a number of ways in practice. 24 | 25 | ### Always deal in iterables 26 | 27 | It's generally a good idea to structure your programs to always deal in iterables, not iterators. Along those lines, you should almost never be explicitly calling `[Symbol.iterator]()` and `iterator.next()`. Let for/of and other language-level constructs handle that for you. That way, no matter whether an iterable or iterator is passed to your program, as long as you treat it as an iterable, you're covered. 28 | 29 | ### Don't bother manually implementing the iterator protocol 30 | 31 | While it's great to understand the iterator protocol, in the vast majority of cases it's better to just let generators create them for you, because then you get this behavior for free, and your iterators can be used wherever iterables are expected. Plus, generators are *sooo* much easier to work with. For example, here's that `range()` function from two chapters ago, re-implemented as a generator: 32 | 33 | ```js 34 | function* range(from, to) { 35 | for (let i=from; i /^\d+\-/.test(name)) 26 | .map(name => ({ name })); 27 | }); 28 | 29 | const addNoExt = co.wrap(function*(data) { 30 | for (const d of data) d.noExt = d.name.replace(/\.[a-z]*$/, ''); 31 | }); 32 | 33 | const addDestBase = co.wrap(function*(data) { 34 | for (const d of data) d.destBase = d.noExt + '.html'; 35 | }); 36 | 37 | const addSource = co.wrap(function*(data) { 38 | for (const d of data) d.source = path.join(srcDir, d.name); 39 | }); 40 | 41 | const addRaw = co.wrap(function*(data) { 42 | for (const d of data) d.raw = yield read(d.source, 'utf8'); 43 | }); 44 | 45 | const addCompiled = co.wrap(function*(data) { 46 | for (const d of data) d.compiled = hbs.compile(d.raw); 47 | }); 48 | 49 | const addDest = co.wrap(function*(data) { 50 | for (const d of data) d.dest = path.join(destDir, d.destBase); 51 | }); 52 | 53 | const addHeading = co.wrap(function*(data) { 54 | for (const d of data) d.heading = getheading(d.raw); 55 | }); 56 | 57 | const addToc = co.wrap(function*(data) { 58 | data.forEach((d, i) => { 59 | d.toc = data.map((d2, j) => { 60 | return i == j 61 | ? ` * **${d2.heading}**` 62 | : ` * [${d2.heading}](./${d2.destBase})` 63 | }).join('\n'); 64 | }); 65 | }); 66 | 67 | const addNext = co.wrap(function*(data) { 68 | data.forEach((d, i) => { 69 | if (i < data.length - 1) { 70 | const next = data[i + 1]; 71 | d.next = `**[Next: ${next.heading} \u2192](./${next.destBase})**`; 72 | } else { 73 | d.next = '**Fin**'; 74 | } 75 | }); 76 | }); 77 | 78 | const addInnerMd = co.wrap(function*(data) { 79 | for (const d of data) d.innerMd = d.compiled(d); 80 | }); 81 | 82 | const addInnerHtml = co.wrap(function*(data) { 83 | for (const d of data) d.innerHtml = md(d.innerMd); 84 | }); 85 | 86 | const addWrapped = co.wrap(function*(data, wrapper) { 87 | for (const d of data) d.wrapped = wrapper(d); 88 | }); 89 | 90 | const writeAll = co.wrap(function*(data) { 91 | for (const d of data) { 92 | console.log(`${d.source} => ${d.dest}`); 93 | yield write(d.dest, d.wrapped, 'utf8'); 94 | } 95 | }); 96 | 97 | function getheading(content) { 98 | const matches = content.match(/# (.*)/); 99 | if (!matches) { 100 | const err = new Error('could not find heading in contents'); 101 | throw err; 102 | } else { 103 | return matches[1]; 104 | } 105 | } 106 | 107 | co(function*() { 108 | const wrapper = yield getWrapper(); 109 | const data = yield getInitialData(); 110 | yield addNoExt(data); 111 | yield addDestBase(data); 112 | yield addSource(data); 113 | yield addRaw(data); 114 | yield addCompiled(data); 115 | yield addDest(data); 116 | yield addHeading(data); 117 | yield addToc(data); 118 | yield addNext(data); 119 | yield addInnerMd(data); 120 | yield addInnerHtml(data); 121 | yield addWrapped(data, wrapper); 122 | yield writeAll(data); 123 | }).then( 124 | () => console.log('All done!'), 125 | (err) => console.error(err.stack) 126 | ); 127 | -------------------------------------------------------------------------------- /src/chapters/07-recursion.md: -------------------------------------------------------------------------------- 1 | # Chapter VII: Delegation and recursion 2 | 3 | {{ toc }} 4 | 5 | ## Revisiting our tree traversal 6 | 7 | In a previous chapter, we outfitted our binary search tree with generator-based iterability. Venturing a bit further down that rabbit hole, we note that we used [breadth-first iteration](https://en.wikipedia.org/wiki/Breadth-first_search), meaning that we visited every value at the child level before moving to the grandchild level, etc. 8 | 9 | ```js 10 | // a breadth-first tree traversal algorithm 11 | var queue = this.root ? [this.root] : []; 12 | while (queue.length > 0) { 13 | var node = queue.shift(); 14 | yield node.value; 15 | if (node.left) { queue.push(node.left); } 16 | if (node.right) { queue.push(node.right); } 17 | } 18 | ``` 19 | 20 | This works, but one of the cool things about BSTs is that they keep their values in sorted order. To take advantage of that, we need to switch to [in-order iteration](https://en.wikipedia.org/wiki/Tree_traversal#In-order). An easy way to do that is to use recursion, which means our generator will have to call itself. 21 | 22 | ## Tree structure 23 | 24 | The way our tree works is that we have a `Tree` class and an inner `Node` class. Instances of `Tree` have a root which is null if the tree is empty. Otherwise, it's an instance of `Node`, which in turn can have left and right children, themselves instances of `Node`, and so on. 25 | 26 | ``` 27 | tree 28 | | 29 | node:9 30 | / \ 31 | node:4 node:11 32 | / \ / \ 33 | null node:6 null null 34 | ``` 35 | 36 | ## Adding recursion to a generator: attempt one 37 | 38 | We'll make both `Tree` and `Node` iterable, and have `Tree` delegate to its root `Node`, if it exists. Then the root will recursively delegate to its left and right children, if they exist, etc. 39 | 40 | ```js 41 | class Tree { 42 | // ... 43 | *[Symbol.iterator]() { 44 | if (this.root) { 45 | // delegate to root 46 | for (const val of this.root) { 47 | yield val; 48 | } 49 | } 50 | } 51 | } 52 | 53 | class Node { 54 | // ... 55 | *[Symbol.iterator]() { 56 | if (this.left) { 57 | // delegate to this.left 58 | for (const val of this.left) { 59 | yield val; 60 | } 61 | } 62 | yield this.value; 63 | if (this.right) { 64 | // delegate to this.right 65 | for (const val of this.right) { 66 | yield val; 67 | } 68 | } 69 | } 70 | } 71 | ``` 72 | 73 | This works, but we've written too much code! It turns out that generators have a much easier way to do delegation. 74 | 75 | ## Generator delegation 76 | 77 | Generators have a special syntax to delegate to any iterable. Whenever we `yield* obj`, where `obj` is iterable, we inline that sequence into the sequence we're currently generating. Read `yield* foo` as *yield all the things in foo*. Even with arbitrarily-deeply nested delegation, the consumer always sees a flat stream. 78 | 79 | ```js 80 | function* a() { 81 | yield 1; 82 | yield* arr; 83 | yield 2; 84 | } 85 | 86 | var arr = [ 3, 4 ]; 87 | 88 | for (let n of a()) { 89 | console.log(n); 90 | } 91 | // => 1 92 | // => 3 93 | // => 4 94 | // => 2 95 | ``` 96 | 97 | ## Adding recursion to a generator: attempt two 98 | 99 | Going back and applying this to our tree, we get this: 100 | 101 | ```js 102 | class Tree { 103 | // ... 104 | *[Symbol.iterator]() { 105 | if (this.root) yield* this.root; 106 | } 107 | } 108 | 109 | class Node { 110 | // ... 111 | *[Symbol.iterator]() { 112 | if (this.left) yield* this.left; 113 | yield this.value; 114 | if (this.right) yield* this.right; 115 | } 116 | } 117 | ``` 118 | 119 | And voila! We have an in-order iteration over every value in the tree. There's also a gist containing a fuller example of an [iterable binary search tree](https://gist.github.com/greim/17ccec50e8ac45a35edf08efec5fe059). 120 | 121 | ---------------- 122 | 123 | {{ next }} 124 | 125 | ---------------- 126 | 127 | {{ toc }} 128 | -------------------------------------------------------------------------------- /dist/02-for-of.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter II: The for/of loop 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter II: The for/of loop

24 | 35 |

A new kind of loop

36 |

In the first chapter, we learned about tradeoffs between pull and push mode for consuming collections. However, ES6 introduces a new way of looping that we failed to take into account: the for/of loop.

37 |

Solving the pull/push conundrum

38 |

Compare these two ways of looping an array:

39 | 
for (var i=0; i<arr.length; i++) {
40 |   var value = arr[i];
41 |   visit(value);
42 | }
43 | 
44 | for (var value of arr) {
45 |   visit(value);
46 | }
47 |
48 |

Unlike push mode, for/of retains all the language-level powers of a loop:

49 |
    50 |
  • You can return the outer function from inside the loop body.
    • 51 |
  • You can break or continue iteration from inside the loop body.
    • 52 |
  • You can yield from inside the loop body.
    • 53 |
54 |

Unlike pull mode, for/of avoids problems with other kinds of loops:

55 |
    56 |
  • We don't have to track any state (e.g. an "i" variable).
    • 57 |
  • We don't need to worry about when to end the loop.
    • 58 |
  • We don't need to know any specifics about the data structure.
    • 59 |
60 |

In other words, for/of retains the benefits of pull mode, while also gaining the benefits of push mode.

61 |

What makes something for/of-able?

62 |

Above, we used for/of to loop an array, but it raises the question: what makes something for/of-able? Do only arrays have this ability, or could our binary search tree from the last chapter also be for/of'd? That's where ES6 iterators come in, and it's the topic of the next chapter.

63 |
64 |

Next: Chapter III: Iterators →

65 |
66 | 77 | 78 |
79 |

80 | Copyright © 2016 by Greg Reimer (github, twitter). 81 | Submit issues to the GitHub issues page. 82 |

83 |

84 | 85 | 86 |

87 |
88 | 89 | 90 | 91 | 92 | -------------------------------------------------------------------------------- /dist/06-iteration-everywhere.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter VI: Beyond the for/of loop 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter VI: Beyond the for/of loop

24 | 35 |

What else can iterators do?

36 |

This chapter is a quick survey of various other places iterables are used, besides the for/of loop.

37 |

Things that are iterables

38 |
    39 |
  • Arrays
    • 40 |
  • Strings
    • 41 |
  • Maps
    • 42 |
  • Sets
    • 43 |
  • Function arguments objects
    • 44 |
  • DOM collections
    • 45 |
46 |

Things that accept iterables

47 |

Spread operators

48 | 
function* foo() { yield 'a', yield 'b', yield 'c'; }
 49 | function bar() { console.log(arguments); }
 50 | bar(...foo()); // => { 0: 'a', 1: 'b', 2: 'c' }
 51 |
52 | 
function* foo() { yield 1, yield 2, yield 3; }
 53 | console.log([...foo()]); // => [ 1, 2, 3 ]
 54 |
55 |

Destructuring

56 | 
function* foo() { yield 1, yield 2, yield 3; }
 57 | const [ x, y, z ] = foo();
 58 | console.log(x); // => 1
 59 | console.log(y); // => 2
 60 | console.log(z); // => 3
 61 |
62 |

Construction of maps and sets

63 | 
function* foo() { yield 1, yield 2, yield 3; }
 64 | const set = new Set(foo());
 65 | console.log(set.has(1)); // => true
 66 | console.log(set.has(2)); // => true
 67 | console.log(set.has(3)); // => true
 68 |
69 | 
function* foo() { yield ['a', 1], yield ['b', 2]; }
 70 | const map = new Map(foo());
 71 | console.log(map.get('a')); // => 1
 72 | console.log(map.get('b')); // => 2
 73 |
74 |

Array.from()

75 | 
function* foo() { yield 1, yield 2, yield 3; }
 76 | console.log(Array.from(foo())); // => [ 1, 2, 3 ]
 77 |
78 |

Promise.all()

79 | 
function* foo() {
 80 |   yield Promise.resolve(1);
 81 |   yield Promise.resolve(2);
 82 |   yield Promise.resolve(3);
 83 | }
 84 | Promise.all(foo()).then(arr => {
 85 |   console.log(arr); // => [ 1, 2, 3 ]
 86 | });
 87 |
88 |

Generator delegation

89 | 
function* foo() { yield 1, yield 2, yield 3; }
 90 | function* bar() { yield* foo(); }
 91 | const arr = Array.from(bar());
 92 | console.log(arr); // => [ 1, 2, 3 ]
 93 |
94 |
95 |

Next: Chapter VII: Delegation and recursion →

96 |
97 | 108 | 109 |
110 |

111 | Copyright © 2016 by Greg Reimer (github, twitter). 112 | Submit issues to the GitHub issues page. 113 |

114 |

115 | 116 | 117 |

118 |
119 | 120 | 121 | 122 | 123 | -------------------------------------------------------------------------------- /dist/01-pull-push.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter I: The pull/push conundrum 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter I: The pull/push conundrum

24 | 35 |

Looping collections

36 |

It all starts with looping. Let's consider two kinds of data structures you can loop through: trees and arrays. In an array, everything exists sequentially in a line, so looping is easy. A tree has a non-linear branching structure, so looping it isn't so straightforward. Also, terminology note: when you loop through a collection, we'll say that you're the consumer, while the collection is the producer.

37 |

Pull mode

38 |

As data flows from producer to consumer, "pull mode" means that the consumer initiates the transfer, while the producer remains passive:

39 | 
for (var i=0; i<array.length; i++) {
40 |   visit(array[i]);
41 | }
42 |
43 |

Looping the array was fairly easy. Looping a tree is more complicated. Let's assume it's a binary search tree. A bit of googling turns up an algorithm we can use:

44 | 
var queue = tree.root ? [tree.root] : [];
45 | while (queue.length > 0) {
46 |   var node = queue.shift();
47 |   visit(node.value);
48 |   if (node.left) { queue.push(node.left); }
49 |   if (node.right) { queue.push(node.right); }
50 | }
51 |
52 |

That's quite a pile of code to have to memorize and type out just to loop a collection, but since I'm initiating the transfer, the task falls to me.

53 |

Push mode

54 |

Which raises the question: why should I initiate the transfer? Why not let producer do it? Hence push mode, in which the collection has a forEach() method that accepts a callback, which it uses to push values to us, while we sit back and passively receive the values.

55 | 
array.forEach(elmt => visit(elmt));
56 | tree.forEach(elmt => visit(elmt));
57 |
58 |

Much better. The various state tracking and algorithmic details are encapsulated away from us inside the forEach() method, which is a great separation of concerns.

59 |

Unfortunate tradeoffs

60 |

So push mode wins, right? Sadly, by switching from pull to push, we lost some power and flexibility:

61 |
    62 |
  • We can't return the outer function from inside the callback.
    • 63 |
  • We can't break or continue from inside the callback.
    • 64 |
  • We can't yield from within the callback.
    • 65 |
66 |

Those behaviors could be simulated using some sort of pre-agreed-upon signaling mechanism between the callback and its runner, but by then we've begun to re-invent the wheel, since the language gives us those capabilities for free with loops.

67 |
68 |

Next: Chapter II: The for/of loop →

69 |
70 | 81 | 82 |
83 |

84 | Copyright © 2016 by Greg Reimer (github, twitter). 85 | Submit issues to the GitHub issues page. 86 |

87 |

88 | 89 | 90 |

91 |
92 | 93 | 94 | 95 | 96 | -------------------------------------------------------------------------------- /src/chapters/03-iterators.md: -------------------------------------------------------------------------------- 1 | # Chapter III: Iterators 2 | 3 | {{ toc }} 4 | 5 | ## Introducing iterators 6 | 7 | In the previous chapter, we learned how the for/of loop retains the powers of pull mode, while gaining powers of push mode. But to understand why that's the case, we have to look at iterators. 8 | 9 | ## First, a bit of theory 10 | 11 | Fundamentally, iterators are an abstract way to represent a sequence. Before ES6, it was common to use arrays for this, but that's unworkable in at least two cases: 12 | 13 | 1. **Open-ended sequences**: Sometimes it's useful to model *infinite or ridiculously long sequences*. For example, the set of all positive integers. 14 | 2. **Lazy sequences**: Sometimes it's useful to model lazy sequences, which don't have a value until the moment the consumer asks for it. This can save both memory and CPU cycles. 15 | 16 | Arrays come loaded with a lot of capabilities and assumptions, which makes them powerful but also means they can't do these things. Iterators have no such limitations, because they define a protocol comprising only the *minimum* operations for sequence traversal: A) what's the next thing? and B) are we done yet? 17 | 18 | By establishing a set of minimal rules everyone agrees to follow, the protocol establishes separation of concerns. As long as you—the producer—implement the protocol, you're free to model a sequence however you want. As long as you—the consumer—adhere to this protocol, you're free to decide when to iterate and whether to bail out of the iteration. 19 | 20 | Finally, because it's defined in the language, language-level hooks exist that make working with iterators ultra-simple. On the consumer side, this is the for/of loop. On the producer side, it's generators. But we're getting ahead of ourselves! First let's look at how these protocols work. There are actually two concepts in play—iterables and iterators—each with its own protocol. 21 | 22 | ## Concept: Iterables 23 | 24 | Any *iterable* can (among other things) be for/of'd. Technically, an iterable is an object that implements the *iterable protocol*. 25 | 26 | **The iterable protocol:** To implement the iterable protocol, an object must have a [[Symbol.iterator]](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol) property which is a function that receives no arguments and returns an *iterator*. 27 | 28 | ## Concept: Iterators 29 | 30 | An *iterator* is any object that implements the *iterator protocol*. It's transient in that every time you loop an iterable, a new iterator is created and then discarded. It's stateful in that it remembers its current position in the sequence at any given time. 31 | 32 | **The iterator protocol:** To implement the iterator protocol, an object must have a `next` method that can be called over and over until iteration is done, at which point the iterator is depleted. Every call to `next()` returns a `{done,value}` object. While the iterator isn't depleted, `done` will be false. After it's depleted, `done` will be true. 33 | 34 | ## Iteration protocols in action 35 | 36 | Okay then, enough theory. Let's actually create an iterator from an array and then deplete it. (If you're using a modern browser, feel free to paste this code in your console and try it out.) 37 | 38 | ```js 39 | // arrays are iterables, so let's create one 40 | var array = [ 2, 4, 6 ]; 41 | 42 | // now we'll create an iterator 43 | var itr = array[Symbol.iterator](); 44 | 45 | // deplete the iterator 46 | console.log(itr.next()); // { done: false, value: 2 } 47 | console.log(itr.next()); // { done: false, value: 4 } 48 | console.log(itr.next()); // { done: false, value: 6 } 49 | console.log(itr.next()); // { done: true, value: undefined } 50 | ``` 51 | 52 | Obviously it would be better to consume the iterator using a loop: 53 | 54 | ```js 55 | var itr = array[Symbol.iterator](); 56 | while (true) { 57 | var next = itr.next(); 58 | if (!next.done) { 59 | visit(next.value); 60 | } else { 61 | break; 62 | } 63 | } 64 | ``` 65 | 66 | Finally, the above is just a manual way of doing what for/of loops do automatically: 67 | 68 | ```js 69 | for (var n of array) { 70 | visit(n); 71 | } 72 | ``` 73 | 74 | ## Let's make our own iterable 75 | 76 | In the above, we used an array, which is a native object that's iterable. Next, let's try making our own objects iterable. We'll have a `range()` function that returns an iterable representing a finite sequence of numbers. Our goal is to be able to do this: 77 | 78 | ```js 79 | for (var n of range(3, 6)) { ... } // visits 3, 4, 5 80 | ``` 81 | 82 | Here's the code: 83 | 84 | ```js 85 | function range(from, to) { 86 | var iterable = {}; 87 | // implement iterable protocol 88 | iterable[Symbol.iterator] = function() { 89 | var i = from; 90 | var iterator = {}; 91 | // implement iterator protocol 92 | iterator.next = function() { 93 | var value = i++; 94 | var done = value >= to; 95 | if (done) value = undefined; 96 | return { value, done }; 97 | }; 98 | return iterator; 99 | }; 100 | return iterable; 101 | } 102 | ``` 103 | 104 | This is a bit ugly, but never mind that for now. Feel free to try it out in your browser console. 105 | 106 | ## Making our own iterable, round two 107 | 108 | Flushed with success, let's try making our binary search tree from the first chapter iterable. Our end goal is to be able to do this: 109 | 110 | ```js 111 | for (var val of tree) { ... } 112 | ``` 113 | 114 | Here's all the ingredients laid out for us, we merely need to assemble them together: 115 | 116 | ```js 117 | class Tree { 118 | 119 | // Assume various BST methods already exist 120 | // here like add() and remove() 121 | 122 | [Symbol.iterator]() { 123 | return { 124 | next() { 125 | // Put the algorithm here, maybe? 126 | } 127 | }; 128 | } 129 | } 130 | 131 | // Our tree-iteration algorithm. 132 | // Need to drop this in above somewhere... 133 | var queue = this.root ? [this.root] : []; 134 | while (queue.length > 0) { 135 | var node = queue.shift(); 136 | // do something with node.value 137 | if (node.left) { queue.push(node.left); } 138 | if (node.right) { queue.push(node.right); } 139 | } 140 | ``` 141 | 142 | If you're like me, this is where you get stuck. The tree-iteration algorithm *runs to completion*, which isn't what we want. Rather, we want it to run bit-by-bit as a result of calls to the `next()` method. Maybe I could instantiate the `queue` array at the top of the `[Symbol.iterator]` function, then get rid of the `while` loop and replace it with... 143 | 144 | But wait. Stop. It turns out there's a better way. Enter *generators*. 145 | 146 | ---------------- 147 | 148 | {{ next }} 149 | 150 | ---------------- 151 | 152 | {{ toc }} 153 | -------------------------------------------------------------------------------- /dist/04-generators.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter IV: Generators 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter IV: Generators

24 | 35 |

The problem

36 |

In the last chapter we discussed how to make anything for/of-able by using iterators. But I got stuck trying to do it for a binary search tree. Recall where we left off:

37 | 
class Tree {
 38 | 
 39 |   // Assume various BST methods already exist
 40 |   // here like add() and remove()
 41 | 
 42 |   [Symbol.iterator]() {
 43 |     return {
 44 |       next() {
 45 |         // Put the algorithm here, maybe?
 46 |       }
 47 |     };
 48 |   }
 49 | }
 50 | 
 51 | // Our tree-traversal algorithm.
 52 | // Need to drop this in above somewhere...
 53 | var queue = this.root ? [this.root] : [];
 54 | while (queue.length > 0) {
 55 |   var node = queue.shift();
 56 |   // do something with node.value
 57 |   if (node.left) { queue.push(node.left); }
 58 |   if (node.right) { queue.push(node.right); }
 59 | }
 60 |
61 |

Down in the guts of this tree-traversal algorithm, there's a point where we have the value in-hand. But we can't just let the loop run to completion. We have to hand off a value, then suspend the loop mid-flight and wait... somehow.

62 |

The solution

63 |

It turns out that suspending and waiting is exactly what generators do. The mechanics of this are discussed in more detail below, but let's just jump right to the solution.

64 |

First, we declare [Symbol.iterator] as a generator by adding an asterisk *. Then we drop our algorithm in wholesale. Finally, we hand off the value using a yield expression.

65 | 
class Tree {
 66 |   // ...
 67 |   *[Symbol.iterator]() {
 68 |     var queue = this.root ? [this.root] : [];
 69 |     while (queue.length > 0) {
 70 |       var node = queue.shift();
 71 |       yield node.value; // <-- hand off the value
 72 |       if (node.left) { queue.push(node.left); }
 73 |       if (node.right) { queue.push(node.right); }
 74 |     }
 75 |   }
 76 | }
 77 |
78 |

As far as the outside world is concerned, a generator is merely a function that returns an iterator. Because of this, we've satisfied the iterable protocol and we can consume our tree:

79 | 
for (const value of tree) { ... }
 80 |
81 |

Okay, but how do generators work?

82 |

Essentially, what happens is that when the *[Symbol.iterator] function is called, instead of running our algorithm to completion, JavaScript puts it into a paused state, without running it.

83 |

Meanwhile, JavaScript returns an iterator "itr" to the caller. When itr.next() is called, JavaScript presses play on the algorithm, which runs right up until the point it encounters a yield, then pauses again. The yielded value is subsequently returned from itr.next() along with done: false.

84 | 
{ value: /* whatever was yielded */, done: false }
 85 |
86 |

This happens anywhere between zero and infinity times. If/when the generator algorithm hits a return instead of a yield (the return can be either implicit or explicit) the returned value is returned from itr.next() along with done: true.

87 | 
{ value: /* whatever was returned */, done: true }
 88 |
89 |
90 |

Next: Chapter V: Polymorphic iterators →

91 |
92 | 103 | 104 |
105 |

106 | Copyright © 2016 by Greg Reimer (github, twitter). 107 | Submit issues to the GitHub issues page. 108 |

109 |

110 | 111 | 112 |

113 |
114 | 115 | 116 | 117 | 118 | -------------------------------------------------------------------------------- /dist/05-polymorphic-iterators.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter V: Polymorphic iterators 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter V: Polymorphic iterators

24 | 35 |

A plot twist

36 |

Having covered the basics of iteration and generators, we now take a break in order to go down a small rabbit-hole.

37 |

Recall that iterators and iterables are separate things. Well, plot twist: Pretty much every native JavaScript iterator object implements the iterable protocol by having a [Symbol.iterator] method which returns itself. In effect, most iterators are both iterators and iterables, simultaneously!

38 |

Possibly confusing, but useful

39 |

If you're like me, this will hurt your brain... at first. But there's a good reason for it, since it allows passing any iterator where an iterable is expected. Why would I want to do that, you ask? Because it allows doing this:

40 | 
for (const key of map.keys()) { ... }
 41 |
42 |

As the Map#keys() method demonstrates, object[Symbol.iterator]() isn't the only thing that can return iterators. Some methods return iterators directly. We definitely want the ability to use these in places where iterables are expected, such as for/of loops.

43 |

What does this mean for you?

44 |

This shakes out in a number of ways in practice.

45 |

Always deal in iterables

46 |

It's generally a good idea to structure your programs to always deal in iterables, not iterators. Along those lines, you should almost never be explicitly calling [Symbol.iterator]() and iterator.next(). Let for/of and other language-level constructs handle that for you. That way, no matter whether an iterable or iterator is passed to your program, as long as you treat it as an iterable, you're covered.

47 |

Don't bother manually implementing the iterator protocol

48 |

While it's great to understand the iterator protocol, in the vast majority of cases it's better to just let generators create them for you, because then you get this behavior for free, and your iterators can be used wherever iterables are expected. Plus, generators are sooo much easier to work with. For example, here's that range() function from two chapters ago, re-implemented as a generator:

49 | 
function* range(from, to) {
 50 |   for (let i=from; i<to; i++) yield i;
 51 | }
 52 |
53 |

Avoid double-consuming an iterator

54 |

Iterators are unicast by design, meaning that when one consumer pulls out a value, no other consumer will ever see it. Thus, even though you've structured your code to deal exclusively in iterables, you're still sort of on the hook to know which things are actually iterators.

55 | 
var keys1 = Object.keys(obj); // iterable
 56 | var keys2 = map.keys(); // iterator
 57 | 
 58 | for (var x of keys1) { ... } // can do this over and over again
 59 | for (var x of keys2) { ... } // can NEVER do this again
 60 |
61 |

Personally, I prefer to sidestep the issue by not storing references to iterators in variables. For example, by calling map.keys() directly inside the for/of loop, I avoid the temptation to try to re-iterate that variable later on:

62 | 
for (var x of map.keys()) { ... }
 63 |
64 |

When in doubt, don't double-consume

65 |

Sometimes you'll just be accepting an object from somewhere else. In that case you have no way of knowing whether it's an iterable or an iterator, and it's best to avoid double-consumption.

66 | 
function doStuff(iterable) {
 67 |   for (let x of iterable) { ... }
 68 |   // Now consider `iterable` to have
 69 |   // been consumed and don't try to
 70 |   // re-consume it.
 71 | }
 72 |
73 |
74 |

Next: Chapter VI: Beyond the for/of loop →

75 |
76 | 87 | 88 |
89 |

90 | Copyright © 2016 by Greg Reimer (github, twitter). 91 | Submit issues to the GitHub issues page. 92 |

93 |

94 | 95 | 96 |

97 |
98 | 99 | 100 | 101 | 102 | -------------------------------------------------------------------------------- /dist/07-recursion.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter VII: Delegation and recursion 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter VII: Delegation and recursion

24 | 35 |

Revisiting our tree traversal

36 |

In a previous chapter, we outfitted our binary search tree with generator-based iterability. Venturing a bit further down that rabbit hole, we note that we used breadth-first iteration, meaning that we visited every value at the child level before moving to the grandchild level, etc.

37 | 
// a breadth-first tree traversal algorithm
 38 | var queue = this.root ? [this.root] : [];
 39 | while (queue.length > 0) {
 40 |   var node = queue.shift();
 41 |   yield node.value;
 42 |   if (node.left) { queue.push(node.left); }
 43 |   if (node.right) { queue.push(node.right); }
 44 | }
 45 |
46 |

This works, but one of the cool things about BSTs is that they keep their values in sorted order. To take advantage of that, we need to switch to in-order iteration. An easy way to do that is to use recursion, which means our generator will have to call itself.

47 |

Tree structure

48 |

The way our tree works is that we have a Tree class and an inner Node class. Instances of Tree have a root which is null if the tree is empty. Otherwise, it's an instance of Node, which in turn can have left and right children, themselves instances of Node, and so on.

49 | 
           tree
 50 |             |
 51 |           node:9
 52 |          /      \
 53 |     node:4      node:11
 54 |     /   \        /    \
 55 | null   node:6  null   null
 56 |

Adding recursion to a generator: attempt one

57 |

We'll make both Tree and Node iterable, and have Tree delegate to its root Node, if it exists. Then the root will recursively delegate to its left and right children, if they exist, etc.

58 | 
class Tree {
 59 |   // ...
 60 |   *[Symbol.iterator]() {
 61 |     if (this.root) {
 62 |       // delegate to root
 63 |       for (const val of this.root) {
 64 |         yield val;
 65 |       }
 66 |     }
 67 |   }
 68 | }
 69 | 
 70 | class Node {
 71 |   // ...
 72 |   *[Symbol.iterator]() {
 73 |     if (this.left) {
 74 |       // delegate to this.left
 75 |       for (const val of this.left) {
 76 |         yield val;
 77 |       }
 78 |     }
 79 |     yield this.value;
 80 |     if (this.right) {
 81 |       // delegate to this.right
 82 |       for (const val of this.right) {
 83 |         yield val;
 84 |       }
 85 |     }
 86 |   }
 87 | }
 88 |
89 |

This works, but we've written too much code! It turns out that generators have a much easier way to do delegation.

90 |

Generator delegation

91 |

Generators have a special syntax to delegate to any iterable. Whenever we yield* obj, where obj is iterable, we inline that sequence into the sequence we're currently generating. Read yield* foo as yield all the things in foo. Even with arbitrarily-deeply nested delegation, the consumer always sees a flat stream.

92 | 
function* a() {
 93 |   yield 1;
 94 |   yield* arr;
 95 |   yield 2;
 96 | }
 97 | 
 98 | var arr = [ 3, 4 ];
 99 | 
100 | for (let n of a()) {
101 |   console.log(n);
102 | }
103 | // => 1
104 | // => 3
105 | // => 4
106 | // => 2
107 |
108 |

Adding recursion to a generator: attempt two

109 |

Going back and applying this to our tree, we get this:

110 | 
class Tree {
111 |   // ...
112 |   *[Symbol.iterator]() {
113 |     if (this.root) yield* this.root;
114 |   }
115 | }
116 | 
117 | class Node {
118 |   // ...
119 |   *[Symbol.iterator]() {
120 |     if (this.left) yield* this.left;
121 |     yield this.value;
122 |     if (this.right) yield* this.right;
123 |   }
124 | }
125 |
126 |

And voila! We have an in-order iteration over every value in the tree. There's also a gist containing a fuller example of an iterable binary search tree.

127 |
128 |

Next: Chapter VIII: Advanced topics →

129 |
130 | 141 | 142 |
143 |

144 | Copyright © 2016 by Greg Reimer (github, twitter). 145 | Submit issues to the GitHub issues page. 146 |

147 |

148 | 149 | 150 |

151 |
152 | 153 | 154 | 155 | 156 | -------------------------------------------------------------------------------- /src/chapters/08-advanced-topics.md: -------------------------------------------------------------------------------- 1 | # Chapter VIII: Advanced topics 2 | 3 | {{ toc }} 4 | 5 | ## It's dangerous to go alone... 6 | 7 | The goal of this tutorial was to introduce generators as the centerpiece of iteration in ES6. With that done, we'll briefly look at a few advanced use cases. This isn't meant to be an exhaustive tour of everything about generators, just a few items to take with you as you leave and explore the world of generators on your own. 8 | 9 | ## A word of warning 10 | 11 | Recall that iterators are stateful objects. When an iterator comes from a collection, it "remembers" its place in that collection at each step. Hence the rule: *Never modify a collection while iterating it.* Otherwise, things can go haywire since the iterator's state could be invalidated. 12 | 13 | ```js 14 | // BAD! 15 | for (const val of collection) { 16 | collection.remove(val); 17 | } 18 | ``` 19 | 20 | ## POJOs aren't iterable 21 | 22 | Given that you can for/in loop over keys in a plain old JavaScript object (POJO), you could be forgiven for thinking POJOs are iterable. Well, [they're not](http://exploringjs.com/es6/ch_iteration.html#sec_plain-objects-not-iterable). You can use a Map instead, or a write yourself a utility generator function. 23 | 24 | ```js 25 | function* iterObj(ob) { 26 | for (let key in ob) { 27 | if (ob.hasOwnProperty(key)) { 28 | yield [ key, ob[key] ]; 29 | } 30 | } 31 | } 32 | 33 | for (const [ key, val ] of iterObj(obj)) { ... } 34 | ``` 35 | 36 | ## The generator-detection antipattern 37 | 38 | If you work with generators much, eventually you'll find yourself thinking, "it sure would be handy to detect whether this function is a generator." Don't do it! 39 | 40 | While it may be possible to exploit various quirks of your local JavaScript engine to sniff out a generator, the intent is that, to the outside world, generators should be indistinguishable from normal functions that return iterators. For example, suppose you have a function that accepts a callback: 41 | 42 | ```js 43 | function runGenerator(generator) { 44 | if (!isGenerator(generator)) { 45 | throw new Error('requires a generator!'); 46 | } else { 47 | for (const x of generator()) { ... } 48 | } 49 | } 50 | 51 | // intended usage 52 | runGenerator(function*() { ... }); 53 | ``` 54 | 55 | That would break for this perfectly legitimate use case, since `Function#bind()` returns a new function: 56 | 57 | ```js 58 | runGenerator(function*() { ... }.bind(this)); 59 | ``` 60 | 61 | If you need to do some sort of validation, consider instead just inspecting the return value of the function: 62 | 63 | ```js 64 | function runGenerator(generator) { 65 | var itr = generator(); 66 | if (!isIterator(itr)) { 67 | throw new Error('must return an iterator!'); 68 | } else { 69 | for (const x of itr) { ... } 70 | } 71 | } 72 | ``` 73 | 74 | ## Functional programming over sequences 75 | 76 | Considering that iteration introduces a conceptual shift from *collections* to *abstract sequences*, collection-oriented libraries like lodash start to seem incomplete. Operations like map, filter, and reduce can just as easily operate on infinite or lazy sequences, for example. The [wu library](https://fitzgen.github.io/wu.js/) offers these kinds of capabilities, and can be thought of as "lodash for iterators": 77 | 78 | ```js 79 | const evenSquares = wu(range(0, Infinity)) 80 | .map(n => n * n) 81 | .filter(n => n % 2 === 0); 82 | 83 | for (let n of evenSquares) { 84 | console.log(n); 85 | } 86 | // => 0 87 | // => 4 88 | // => 16 89 | // => 36 90 | // => ... 91 | ``` 92 | 93 | ## Two-way communication 94 | 95 | It's syntactically valid for `yield` or `yield x` to appear anywhere an expression is expected. For example: 96 | 97 | ```js 98 | var myString = `Hello ${yield 4}`; 99 | ``` 100 | 101 | This raises the question: what end up being the contents of `myString`? 102 | 103 | So far we've only looked at *consuming* information from generators. But here's yet another twist: if you have an iterator "`itr`" gotten from a generator, you can pass in an argument: `itr.next(someValue)`. Inside the generator, what this looks like is that a `yield` expression evaluates to `someValue`. 104 | 105 | This is significant, because now instead of a one-way consumer/producer relationship, we have a two-way, "ping-pong" sort of relationship, with the generator sending things to its runner, and the runner sending things back to the generator. 106 | 107 | ```js 108 | // generator 109 | function* foo() { 110 | console.log(`Hello ${yield 1}!`); // 'Hello world!' 111 | } 112 | 113 | // runner 114 | var itr = foo(); 115 | console.log(itr.next().value); // => 1 116 | itr.next('world'); 117 | ``` 118 | 119 | Not only can information be sent into a generator via `next()`, but so can errors via `throw()`, which inside the generator you handle like any other error, treating a `yield` as something that might throw an exception. 120 | 121 | ```js 122 | function* foo() { 123 | try { yield 1; } 124 | catch(ex) { console.log(ex); } // 'oops' 125 | } 126 | 127 | var itr = foo(); 128 | console.log(itr.next().value); // 1 129 | itr.throw('oops'); 130 | ``` 131 | 132 | Note that these capabilities aren't part of the iterator protocol. Rather, they're extra powers that generator-created iterators have. [Read more about it here](http://www.2ality.com/2015/03/es6-generators.html#return%28%29-and-throw%28%29). 133 | 134 | ## Async generators 135 | 136 | This two-way communication between a generator and its runner opens up new vistas in async flow control. A generator can yield promises out to a special runner, which resolves it and then "bounces" the resolved value right back into the generator, asynchronously. What this looks like inside a generator is that a `yield promise` expression evaluates to the resolved value of that promise, or else throws an exception. 137 | 138 | Here's an example using the [fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API), which is promise-based: 139 | 140 | ```js 141 | // in the generator 142 | var resp = yield fetch('/users/123'); 143 | console.log(resp.status); 144 | var user = yield resp.json(); 145 | console.log(user.name); 146 | ``` 147 | 148 | This "trampoline" technique gives us callback-free async flow control. All we need is a runner function, which we'll call `coroutine()`, and a generator function to pass to it: 149 | 150 | ```js 151 | const getUsername = coroutine(function*(id) { 152 | let resp = yield fetch(`/users/${id}`); 153 | var obj = yield resp.json(); 154 | if (resp.status === 200) return obj.username; 155 | else throw new Error(obj.errorMessage); 156 | }); 157 | 158 | const getGreeting = coroutine(function*() { 159 | const myId = localStorage.getItem('my_id'); 160 | if (!myId) throw new Error('not logged in'); 161 | else return `Hello, ${yield getUsername(myId)}`; 162 | }); 163 | 164 | getGreeting().then( 165 | greeting => alert(greeting), 166 | error => alert(error.message) 167 | ); 168 | ``` 169 | 170 | Observations: 171 | 172 | * `coroutine()` accepts a generator and returns a function, both of which have the same signature and `this` context. 173 | * `yield somePromise` expressions inside the generator either evaluate to a value or throw, depending on whether `somePromise` resolves or rejects. 174 | * Calls to the returned function return a promise which resolves to the return value of the generator. 175 | * Thrown exceptions inside the generator reject the returned promise. 176 | 177 | The upshot is that we retain the power of asynchronous programming, while gaining the power of imperative, synchronous-style programming. No more callback hell, or thenable heck! 178 | 179 | All of this of course depends on having a `coroutine()` function that does the right thing. Fortunately, there are multiple libraries to choose from here, including [co](https://www.npmjs.com/package/co), [Bluebird.coroutine](https://www.npmjs.com/package/bluebird), and [Q.spawn](https://github.com/kriskowal/q). 180 | 181 | Finally, it's worth noting that this approach is so powerful that it inspired the [async functions](https://jakearchibald.com/2014/es7-async-functions/) proposal. As of early 2016, it's a stage 3 EcmaScript proposal. 182 | 183 | ---------------- 184 | 185 | {{ next }} 186 | 187 | ---------------- 188 | 189 | {{ toc }} 190 | -------------------------------------------------------------------------------- /dist/03-iterators.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter III: Iterators 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter III: Iterators

24 | 35 |

Introducing iterators

36 |

In the previous chapter, we learned how the for/of loop retains the powers of pull mode, while gaining powers of push mode. But to understand why that's the case, we have to look at iterators.

37 |

First, a bit of theory

38 |

Fundamentally, iterators are an abstract way to represent a sequence. Before ES6, it was common to use arrays for this, but that's unworkable in at least two cases:

39 |
    40 |
  1. Open-ended sequences: Sometimes it's useful to model infinite or ridiculously long sequences. For example, the set of all positive integers.
    2. 41 |
  2. Lazy sequences: Sometimes it's useful to model lazy sequences, which don't have a value until the moment the consumer asks for it. This can save both memory and CPU cycles.
    3. 42 |
43 |

Arrays come loaded with a lot of capabilities and assumptions, which makes them powerful but also means they can't do these things. Iterators have no such limitations, because they define a protocol comprising only the minimum operations for sequence traversal: A) what's the next thing? and B) are we done yet?

44 |

By establishing a set of minimal rules everyone agrees to follow, the protocol establishes separation of concerns. As long as you—the producer—implement the protocol, you're free to model a sequence however you want. As long as you—the consumer—adhere to this protocol, you're free to decide when to iterate and whether to bail out of the iteration.

45 |

Finally, because it's defined in the language, language-level hooks exist that make working with iterators ultra-simple. On the consumer side, this is the for/of loop. On the producer side, it's generators. But we're getting ahead of ourselves! First let's look at how these protocols work. There are actually two concepts in play—iterables and iterators—each with its own protocol.

46 |

Concept: Iterables

47 |

Any iterable can (among other things) be for/of'd. Technically, an iterable is an object that implements the iterable protocol.

48 |

The iterable protocol: To implement the iterable protocol, an object must have a [Symbol.iterator] property which is a function that receives no arguments and returns an iterator.

49 |

Concept: Iterators

50 |

An iterator is any object that implements the iterator protocol. It's transient in that every time you loop an iterable, a new iterator is created and then discarded. It's stateful in that it remembers its current position in the sequence at any given time.

51 |

The iterator protocol: To implement the iterator protocol, an object must have a next method that can be called over and over until iteration is done, at which point the iterator is depleted. Every call to next() returns a {done,value} object. While the iterator isn't depleted, done will be false. After it's depleted, done will be true.

52 |

Iteration protocols in action

53 |

Okay then, enough theory. Let's actually create an iterator from an array and then deplete it. (If you're using a modern browser, feel free to paste this code in your console and try it out.)

54 | 
// arrays are iterables, so let's create one
 55 | var array = [ 2, 4, 6 ];
 56 | 
 57 | // now we'll create an iterator
 58 | var itr = array[Symbol.iterator]();
 59 | 
 60 | // deplete the iterator
 61 | console.log(itr.next()); // { done: false, value: 2 }
 62 | console.log(itr.next()); // { done: false, value: 4 }
 63 | console.log(itr.next()); // { done: false, value: 6 }
 64 | console.log(itr.next()); // { done: true, value: undefined }
 65 |
66 |

Obviously it would be better to consume the iterator using a loop:

67 | 
var itr = array[Symbol.iterator]();
 68 | while (true) {
 69 |   var next = itr.next();
 70 |   if (!next.done) {
 71 |     visit(next.value);
 72 |   } else {
 73 |     break;
 74 |   }
 75 | }
 76 |
77 |

Finally, the above is just a manual way of doing what for/of loops do automatically:

78 | 
for (var n of array) {
 79 |   visit(n);
 80 | }
 81 |
82 |

Let's make our own iterable

83 |

In the above, we used an array, which is a native object that's iterable. Next, let's try making our own objects iterable. We'll have a range() function that returns an iterable representing a finite sequence of numbers. Our goal is to be able to do this:

84 | 
for (var n of range(3, 6)) { ... } // visits 3, 4, 5
 85 |
86 |

Here's the code:

87 | 
function range(from, to) {
 88 |   var iterable = {};
 89 |   // implement iterable protocol
 90 |   iterable[Symbol.iterator] = function() {
 91 |     var i = from;
 92 |     var iterator = {};
 93 |     // implement iterator protocol
 94 |     iterator.next = function() {
 95 |       var value = i++;
 96 |       var done = value >= to;
 97 |       if (done) value = undefined;
 98 |       return { value, done };
 99 |     };
100 |     return iterator;
101 |   };
102 |   return iterable;
103 | }
104 |
105 |

This is a bit ugly, but never mind that for now. Feel free to try it out in your browser console.

106 |

Making our own iterable, round two

107 |

Flushed with success, let's try making our binary search tree from the first chapter iterable. Our end goal is to be able to do this:

108 | 
for (var val of tree) { ... }
109 |
110 |

Here's all the ingredients laid out for us, we merely need to assemble them together:

111 | 
class Tree {
112 | 
113 |   // Assume various BST methods already exist
114 |   // here like add() and remove()
115 | 
116 |   [Symbol.iterator]() {
117 |     return {
118 |       next() {
119 |         // Put the algorithm here, maybe?
120 |       }
121 |     };
122 |   }
123 | }
124 | 
125 | // Our tree-iteration algorithm.
126 | // Need to drop this in above somewhere...
127 | var queue = this.root ? [this.root] : [];
128 | while (queue.length > 0) {
129 |   var node = queue.shift();
130 |   // do something with node.value
131 |   if (node.left) { queue.push(node.left); }
132 |   if (node.right) { queue.push(node.right); }
133 | }
134 |
135 |

If you're like me, this is where you get stuck. The tree-iteration algorithm runs to completion, which isn't what we want. Rather, we want it to run bit-by-bit as a result of calls to the next() method. Maybe I could instantiate the queue array at the top of the [Symbol.iterator] function, then get rid of the while loop and replace it with...

136 |

But wait. Stop. It turns out there's a better way. Enter generators.

137 |
138 |

Next: Chapter IV: Generators →

139 |
140 | 151 | 152 |
153 |

154 | Copyright © 2016 by Greg Reimer (github, twitter). 155 | Submit issues to the GitHub issues page. 156 |

157 |

158 | 159 | 160 |

161 |
162 | 163 | 164 | 165 | 166 | -------------------------------------------------------------------------------- /dist/08-advanced-topics.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Chapter VIII: Advanced topics 7 | 8 | 9 | 10 | 11 | 20 | 21 | 22 |
23 |

Chapter VIII: Advanced topics

24 | 35 |

It's dangerous to go alone...

36 |

The goal of this tutorial was to introduce generators as the centerpiece of iteration in ES6. With that done, we'll briefly look at a few advanced use cases. This isn't meant to be an exhaustive tour of everything about generators, just a few items to take with you as you leave and explore the world of generators on your own.

37 |

A word of warning

38 |

Recall that iterators are stateful objects. When an iterator comes from a collection, it "remembers" its place in that collection at each step. Hence the rule: Never modify a collection while iterating it. Otherwise, things can go haywire since the iterator's state could be invalidated.

39 | 
// BAD!
 40 | for (const val of collection) {
 41 |   collection.remove(val);
 42 | }
 43 |
44 |

POJOs aren't iterable

45 |

Given that you can for/in loop over keys in a plain old JavaScript object (POJO), you could be forgiven for thinking POJOs are iterable. Well, they're not. You can use a Map instead, or a write yourself a utility generator function.

46 | 
function* iterObj(ob) {
 47 |   for (let key in ob) {
 48 |     if (ob.hasOwnProperty(key)) {
 49 |       yield [ key, ob[key] ];
 50 |     }
 51 |   }
 52 | }
 53 | 
 54 | for (const [ key, val ] of iterObj(obj)) { ... }
 55 |
56 |

The generator-detection antipattern

57 |

If you work with generators much, eventually you'll find yourself thinking, "it sure would be handy to detect whether this function is a generator." Don't do it!

58 |

While it may be possible to exploit various quirks of your local JavaScript engine to sniff out a generator, the intent is that, to the outside world, generators should be indistinguishable from normal functions that return iterators. For example, suppose you have a function that accepts a callback:

59 | 
function runGenerator(generator) {
 60 |   if (!isGenerator(generator)) {
 61 |     throw new Error('requires a generator!');
 62 |   } else {
 63 |     for (const x of generator()) { ... }
 64 |   }
 65 | }
 66 | 
 67 | // intended usage
 68 | runGenerator(function*() { ... });
 69 |
70 |

That would break for this perfectly legitimate use case, since Function#bind() returns a new function:

71 | 
runGenerator(function*() { ... }.bind(this));
 72 |
73 |

If you need to do some sort of validation, consider instead just inspecting the return value of the function:

74 | 
function runGenerator(generator) {
 75 |   var itr = generator();
 76 |   if (!isIterator(itr)) {
 77 |     throw new Error('must return an iterator!');
 78 |   } else {
 79 |     for (const x of itr) { ... }
 80 |   }
 81 | }
 82 |
83 |

Functional programming over sequences

84 |

Considering that iteration introduces a conceptual shift from collections to abstract sequences, collection-oriented libraries like lodash start to seem incomplete. Operations like map, filter, and reduce can just as easily operate on infinite or lazy sequences, for example. The wu library offers these kinds of capabilities, and can be thought of as "lodash for iterators":

85 | 
const evenSquares = wu(range(0, Infinity))
 86 | .map(n => n * n)
 87 | .filter(n => n % 2 === 0);
 88 | 
 89 | for (let n of evenSquares) {
 90 |   console.log(n);
 91 | }
 92 | // => 0
 93 | // => 4
 94 | // => 16
 95 | // => 36
 96 | // => ...
 97 |
98 |

Two-way communication

99 |

It's syntactically valid for yield or yield x to appear anywhere an expression is expected. For example:

100 | 
var myString = `Hello ${yield 4}`;
101 |
102 |

This raises the question: what end up being the contents of myString?

103 |

So far we've only looked at consuming information from generators. But here's yet another twist: if you have an iterator "itr" gotten from a generator, you can pass in an argument: itr.next(someValue). Inside the generator, what this looks like is that a yield expression evaluates to someValue.

104 |

This is significant, because now instead of a one-way consumer/producer relationship, we have a two-way, "ping-pong" sort of relationship, with the generator sending things to its runner, and the runner sending things back to the generator.

105 | 
// generator
106 | function* foo() {
107 |   console.log(`Hello ${yield 1}!`); // 'Hello world!'
108 | }
109 | 
110 | // runner
111 | var itr = foo();
112 | console.log(itr.next().value); // => 1
113 | itr.next('world');
114 |
115 |

Not only can information be sent into a generator via next(), but so can errors via throw(), which inside the generator you handle like any other error, treating a yield as something that might throw an exception.

116 | 
function* foo() {
117 |   try { yield 1; }
118 |   catch(ex) { console.log(ex); } // 'oops'
119 | }
120 | 
121 | var itr = foo();
122 | console.log(itr.next().value); // 1
123 | itr.throw('oops');
124 |
125 |

Note that these capabilities aren't part of the iterator protocol. Rather, they're extra powers that generator-created iterators have. Read more about it here.

126 |

Async generators

127 |

This two-way communication between a generator and its runner opens up new vistas in async flow control. A generator can yield promises out to a special runner, which resolves it and then "bounces" the resolved value right back into the generator, asynchronously. What this looks like inside a generator is that a yield promise expression evaluates to the resolved value of that promise, or else throws an exception.

128 |

Here's an example using the fetch API, which is promise-based:

129 | 
// in the generator
130 | var resp = yield fetch('/users/123');
131 | console.log(resp.status);
132 | var user = yield resp.json();
133 | console.log(user.name);
134 |
135 |

This "trampoline" technique gives us callback-free async flow control. All we need is a runner function, which we'll call coroutine(), and a generator function to pass to it:

136 | 
const getUsername = coroutine(function*(id) {
137 |   let resp = yield fetch(`/users/${id}`);
138 |   var obj = yield resp.json();
139 |   if (resp.status === 200) return obj.username;
140 |   else throw new Error(obj.errorMessage);
141 | });
142 | 
143 | const getGreeting = coroutine(function*() {
144 |   const myId = localStorage.getItem('my_id');
145 |   if (!myId) throw new Error('not logged in');
146 |   else return `Hello, ${yield getUsername(myId)}`;
147 | });
148 | 
149 | getGreeting().then(
150 |   greeting => alert(greeting),
151 |   error => alert(error.message)
152 | );
153 |
154 |

Observations:

155 |
    156 |
  • coroutine() accepts a generator and returns a function, both of which have the same signature and this context.
    • 157 |
  • yield somePromise expressions inside the generator either evaluate to a value or throw, depending on whether somePromise resolves or rejects.
    • 158 |
  • Calls to the returned function return a promise which resolves to the return value of the generator.
    • 159 |
  • Thrown exceptions inside the generator reject the returned promise.
    • 160 |
161 |

The upshot is that we retain the power of asynchronous programming, while gaining the power of imperative, synchronous-style programming. No more callback hell, or thenable heck!

162 |

All of this of course depends on having a coroutine() function that does the right thing. Fortunately, there are multiple libraries to choose from here, including co, Bluebird.coroutine, and Q.spawn.

163 |

Finally, it's worth noting that this approach is so powerful that it inspired the async functions proposal. As of early 2016, it's a stage 3 EcmaScript proposal.

164 |
165 |

Fin

166 |
167 | 178 | 179 |
180 |

181 | Copyright © 2016 by Greg Reimer (github, twitter). 182 | Submit issues to the GitHub issues page. 183 |

184 |

185 | 186 | 187 |

188 |
189 | 190 | 191 | 192 | 193 | --------------------------------------------------------------------------------