├── .gitignore ├── LICENSE ├── README.md ├── SUMMARY.md ├── acknowledgements.md ├── async-buffers.md ├── async-functions.md ├── async-generator-functions.md ├── async-generators.md ├── async-iterators.md ├── async-queues.md ├── async-values.md ├── behavior.js ├── behaviors.md ├── byte-stream.js ├── cancelation.md ├── clock.js ├── concepts.md ├── conclusion.md ├── further.md ├── generator-functions.md ├── generators.md ├── glossary.md ├── intro.md ├── iteration.js ├── iterator.js ├── iterators.md ├── observable.js ├── observables.md ├── observer.js ├── operators.js ├── package.json ├── presentation ├── README.md ├── build.bash ├── duals-buffer.png ├── duals-deferred.png ├── duals-generator-function.png ├── duals-generator-observer.png ├── duals-generator.png ├── duals-iterator.png ├── duals-promise-queue.png ├── duals-value.png ├── everything-is-array.jpg ├── everything-is-stream.png ├── fork.gif ├── gtor-master.svg ├── map-reduce.gif ├── map.gif ├── perspective-collection.png ├── perspective-promise.png ├── perspective-stream.png ├── perspective-value.png ├── promise-order-independence.png ├── promise-queue-iteration-transport.png ├── promise-queue-order-independence.png ├── promise-queue-temporal-independence.png ├── reduce.gif ├── share.gif └── stream-using-queues.png ├── primitives.md ├── progress.md ├── promise-generator.js ├── promise-iterator.js ├── promise-machine.js ├── promise-queue.js ├── promise.js ├── remote-iterators.md ├── scripts ├── generate.js └── publish.js ├── semaphores.md ├── signals.md ├── stream.js ├── task-queue.js ├── task.js ├── test ├── behavior-test.js ├── byte-stream-test.js ├── observable-test.js ├── promise-test.js ├── stream-test.js └── task-test.js └── time-to-completion.md /.gitignore: -------------------------------------------------------------------------------- 1 | /node_modules 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Kris Kowal 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /SUMMARY.md: -------------------------------------------------------------------------------- 1 | 2 | # Summary 3 | 4 | * [A General Theory of Reactivity](intro.md) 5 | * [Concepts](concepts.md) 6 | * Idioms 7 | - [Iterators](iterators.md) 8 | - [Generator Functions](generator-functions.md) 9 | - [Generators](generators.md) 10 | - [Async Values](async-values.md) 11 | - [Async Functions](async-functions.md) 12 | - [Async Queues](async-queues.md) 13 | - [Semaphores](semaphores.md) 14 | - [Async Buffers](async-buffers.md) 15 | - [Async Iterators](async-iterators.md) 16 | - [Remote Iterators](remote-iterators.md) 17 | - [Async Generators](async-generators.md) 18 | - [Async Generator Functions](async-generator-functions.md) 19 | - [Observables](observables.md) 20 | - [Observables and Signals](signals.md) 21 | - [Behaviors](behaviors.md) 22 | * Concrete Cases 23 | - [Progress and ETA](progress.md) 24 | * [Conclusion](conclusion.md) 25 | * [Glossary](glossary.md) 26 | * [Acknowledgements](acknowledgements.md) 27 | 28 | -------------------------------------------------------------------------------- /acknowledgements.md: -------------------------------------------------------------------------------- 1 | 2 | # Acknowledgements 3 | 4 | I am grateful to Domenic Denicola, Ryan Paul, and Kevin Smith for reviewing and 5 | providing feedback on various drafts of this article. 6 | 7 | -------------------------------------------------------------------------------- /async-buffers.md: -------------------------------------------------------------------------------- 1 | 2 | # Promise Buffers 3 | 4 | Consider another application. 5 | You have a producer and a consumer, both doing work asynchronously, the producer 6 | periodically sending a value to the consumer. 7 | To ensure that the producer does not produce faster than the consumer can 8 | consume, we put an object between them that regulates their flow rate: a buffer. 9 | The buffer uses a promise queue to transport values from the producer to the 10 | consumer, and another promise queue to communicate that the consumer is ready 11 | for another value from the producer. 12 | The following is a sketch to that effect. 13 | 14 | ```js 15 | var outbound = new PromiseQueue(); 16 | var inbound = new PromiseQueue(); 17 | var buffer = { 18 | out: { 19 | next: function (value) { 20 | outbound.put({ 21 | value: value, 22 | done: false 23 | }); 24 | return inbound.get(); 25 | }, 26 | return: function (value) { 27 | outbound.put({ 28 | value: value, 29 | done: true 30 | }) 31 | return inbound.get(); 32 | }, 33 | throw: function (error) { 34 | outbound.put(Promise.throw(error)); 35 | return inbound.get(); 36 | } 37 | }, 38 | in: { 39 | yield: function (value) { 40 | inbound.put({ 41 | value: value, 42 | done: false 43 | }) 44 | return outbound.get(); 45 | }, 46 | return: function (value) { 47 | inbound.put({ 48 | value: value, 49 | done: true 50 | }) 51 | return outbound.get(); 52 | }, 53 | throw: function (error) { 54 | inbound.put(Promise.throw(error)); 55 | return outbound.get(); 56 | } 57 | } 58 | }; 59 | ``` 60 | 61 | This sketch uses the vernacular of iterators and generators, but each of these 62 | has equivalent nomenclature in the world of streams. 63 | 64 | - `in.yield` means “write”. 65 | - `in.return` means “close”. 66 | - `in.throw` means “terminate prematurely with an error”. 67 | - `out.next` means “read”. 68 | - `out.throw` means “abort or cancel with an error”. 69 | - `out.return` means “abort or cancel prematurely but without an error”. 70 | 71 | So a buffer fits in the realm of reactive interfaces. 72 | A buffer has an asynchronous iterator, which serves as the getter side. 73 | It also has an asynchronous generator, which serves as the setter dual. 74 | The buffer itself is akin to an asynchronous, plural value. 75 | In addition to satisfying the requirements needed just to satisfy the 76 | triangulation between synchronous iterables and asynchronous promises, 77 | it solves the very real world need for streams that support pressure 78 | to regulate the rate of flow and avoid over-commitment. 79 | An asynchronous iterator is a readable stream. 80 | An asynchronous generator is a writable stream. 81 | 82 | | Stream | | | | 83 | | ----------------- | ------- | -------- | ------------ | 84 | | Promise Buffer | Value | Plural | Temporal | 85 | | Promise Iterator | Getter | Plural | Temporal | 86 | | Promise Generator | Setter | Plural | Temporal | 87 | 88 | A buffer has a reader and writer, but there are implementations of reader and 89 | writer that interface with the outside world, mostly files and sockets. 90 | 91 | In the particular case of an object stream, if we treat `yield` and `next` as 92 | synonyms, the input and output implementations are perfectly symmetric. 93 | This allows a single constructor to serve as both reader and writer. 94 | Also, standard promises use the [Revealing Constructor] pattern, exposing the 95 | constructor for the getter side. 96 | The standard hides the `Promise.defer()` constructor method behind the scenes, 97 | only exposing the `resolver` as arguments to a setup function, and never 98 | revealing the `{promise, resolver}` deferred object at all. 99 | Similarly, we can hide the promise buffer constructor and reveal the input side 100 | of a stream only as arguments to the output stream constructor. 101 | 102 | ```js 103 | var reader = new Stream(function (write, close, abort) { 104 | // ... 105 | }); 106 | ``` 107 | 108 | The analogous method to `Promise.defer()` might be `Stream.buffer()`, which 109 | would return an `{in, out}` pair of entangled streams. 110 | 111 | [Revealing Constructor]: http://domenic.me/2014/02/13/the-revealing-constructor-pattern/ 112 | 113 | See the accompanying sketch of a [stream][] implementation. 114 | 115 | [stream]: http://kriskowal.github.io/gtor/docs/stream 116 | 117 | -------------------------------------------------------------------------------- /async-functions.md: -------------------------------------------------------------------------------- 1 | 2 | # Asynchronous Functions 3 | 4 | Generator functions have existed in other languages, like Python, for quite some 5 | time, so folks have made some clever uses of them. 6 | We can combine promises and generator functions to emulate asynchronous 7 | functions. 8 | The key insight is a single, concise method that decorates a generator, creating 9 | an internal "promise trampoline". 10 | An asynchronous function returns a promise for the eventual return value, or the 11 | eventual thrown error, of the generator function. 12 | However, the function may yield promises to wait for intermediate values on its 13 | way to completion. 14 | The trampoline takes advantage of the ability of an iterator to send a value 15 | from `next` to `yield`. 16 | 17 | ```js 18 | var authenticated = Promise.async(function *() { 19 | var username = yield getUsernameFromConsole(); 20 | var user = getUserFromDatabase(username); 21 | var password = getPasswordFromConsole(); 22 | [user, password] = yield Promise.all([user, password]); 23 | if (hash(password) !== user.passwordHash) { 24 | throw new Error("Can't authenticate because the password is invalid"); 25 | } 26 | }) 27 | ``` 28 | 29 | Mark Miller’s [implementation][Async] of the `async` decorator is succinct and 30 | insightful. 31 | We produce a wrapped function that will create a promise generator and proceed 32 | immediately. 33 | Each requested iteration has three possible outcomes: yield, return, or throw. 34 | Yield waits for the given promise and resumes the generator with the eventual 35 | value. 36 | Return stops the trampoline and returns the value, all the way out to the 37 | promise returned by the async function. 38 | If you yield a promise that eventually throws an error, the async function 39 | resumes the generator with that error, giving it a chance to recover. 40 | 41 | [Async]: http://wiki.ecmascript.org/doku.php?id=strawman:async_functions#reference_implementation 42 | 43 | ```js 44 | Promise.async = function async(generate) { 45 | return function () { 46 | function resume(verb, argument) { 47 | var result; 48 | try { 49 | result = generator[verb](argument); 50 | } catch (exception) { 51 | return Promise.throw(exception); 52 | } 53 | if (result.done) { 54 | return result.value; 55 | } else { 56 | return Promise.return(result.value).then(donext, dothrow); 57 | } 58 | } 59 | var generator = generate.apply(this, arguments); 60 | var donext = resume.bind(this, "next"); 61 | var dothrow = resume.bind(this, "throw"); 62 | return donext(); 63 | }; 64 | } 65 | ``` 66 | 67 | As much as JavaScript legitimizes the async promise generators by supporting 68 | returning and throwing, now that Promises are part of the standard, the powers 69 | that sit on the ECMAScript standards committee are contemplating special `async` 70 | and `await` syntax for this case. 71 | The syntax is inspired by the same feature of C#. 72 | 73 | ```js 74 | var authenticate = async function () { 75 | var username = await getUsernameFromConsole(); 76 | var user = getUserFromDatabase(username); 77 | var password = getPasswordFromConsole(); 78 | [user, password] = await Promise.all([user, password]); 79 | return hash(password) === user.passwordHash; 80 | }) 81 | ``` 82 | 83 | One compelling reason to support special syntax is that `await` may have higher 84 | precedence than the `yield` keyword. 85 | 86 | ```js 87 | async function addPromises(a, b) { 88 | return await a + await b; 89 | } 90 | ``` 91 | 92 | By decoupling **async functions** from **generator functions**, JavaScript opens 93 | the door for **async generator functions**, foreshadowing a **plural** and 94 | **temporal** getter, a standard form for readable streams. 95 | 96 | -------------------------------------------------------------------------------- /async-generator-functions.md: -------------------------------------------------------------------------------- 1 | 2 | # Asynchronous Generator Functions 3 | 4 | Jafar Husain recently [asked the ECMAScript committee][JH1], whether generator 5 | functions and async functions were composable, and if so, how they should 6 | compose. (His [proposal][JH2] continues to evolve.) 7 | 8 | [JH1]: https://docs.google.com/file/d/0B7zweKma2uL1bDBpcXV4OWd2cnc 9 | [JH2]: https://github.com/jhusain/asyncgenerator 10 | 11 | One key question is what type an async generator function would return. 12 | We look to precedent. 13 | A generator function returns an iterator. 14 | A asynchronous function returns a promise. 15 | Should the asynchronous generator return a promise for an iterator, an iterator 16 | for promises? 17 | 18 | If ``Iterator`` means that an iterator implements `next` such that it 19 | produces ``Iteration``, the `next` method of an ``Iterator>`` 20 | would return an ``Iteration>``, which is to say, iterations that 21 | carry promises for values. 22 | 23 | There is another possibility. 24 | An asynchronous iterator might implement `next` such that it produces 25 | ``Promise>`` rather than ``Iteration>``. 26 | That is to say, a promise that would eventually produce an iteration containing 27 | a value, rather than an iteration that contains a promise for a value. 28 | 29 | This is, an iterator of promises, yielding ``Iteration>``: 30 | 31 | ```js 32 | var iteration = iterator.next(); 33 | iteration.value.then(function (value) { 34 | return callback.call(thisp, value); 35 | }); 36 | ``` 37 | 38 | This is a promise iterator, yielding ``Promise>``: 39 | 40 | ```js 41 | promiseIterator.next() 42 | .then(function (iteration) { 43 | return callback.call(thisp, iteration.value); 44 | }) 45 | ``` 46 | 47 | Promises capture asynchronous results. 48 | That is, they capture both the value and error cases. 49 | If `next` returns a promise, the error case would model abnormal termination of 50 | a sequence. 51 | Iterations capture the normal continuation or termination of a sequence. 52 | If the value of an iteration were a promise, the error case would capture 53 | inability to transport a single value but would not imply termination of the 54 | sequence. 55 | 56 | In the context of this framework, the answer is clear. 57 | An asynchronous generator function uses both `await` and `yield`. 58 | The `await` term allows the function to idle until some asynchronous work has 59 | settled, and the `yield` allows the function to produce a value. 60 | An asynchronous generator returns a promise iterator, the output side of a 61 | stream. 62 | 63 | Recall that an iterator returns an iteration, but a promise iterator returns a 64 | promise for an iteration, and also a promise generator returns a similar promise 65 | for the acknowledgement from the iterator. 66 | 67 | ```js 68 | promiseIterator.next() 69 | .then(function (iteration) { 70 | console.log(iteration.value); 71 | if (iteration.done) { 72 | console.log("fin"); 73 | } 74 | }); 75 | 76 | promiseGenerator.yield("alpha") 77 | .then(function (iteration) { 78 | console.log("iterator has consumed alpha"); 79 | }); 80 | ``` 81 | 82 | The following example will fetch quotes from the works of Shakespeare, retrieve 83 | quotes from each work, and push those quotes out to the consumer. 84 | Note that the `yield` expression returns a promise for the value to flush, so 85 | awaiting on that promise allows the generator to pause until the consumer 86 | catches up. 87 | 88 | ```js 89 | async function *shakespeare(titles) { 90 | for (let title of titles) { 91 | var quotes = await getQuotes(title); 92 | for (let quote of quotes) { 93 | await yield quote; 94 | } 95 | } 96 | } 97 | 98 | var reader = shakespeare(["Hamlet", "Macbeth", "Othello"]); 99 | reader.reduce(function (length, quote) { 100 | return length + quote.length; 101 | }, 0, null, 100) 102 | .then(function (totalLength) { 103 | console.log(totalLength); 104 | }); 105 | ``` 106 | 107 | It is useful for `await` and `yield` to be completely orthogonal because there 108 | are cases where one will want to yield but ignore pressure from the consumer, 109 | forcing the iteration to buffer. 110 | 111 | Jafar also proposes the existence of an `on` operator. 112 | In the context of this framework, the `on` operator would be similar to the 113 | ECMAScript 6 `of` operator, which accepts an iterable, produces an iterator, and 114 | then walks the iterator. 115 | 116 | ```js 117 | for (let a of [1, 2, 3]) { 118 | console.log(a); 119 | } 120 | 121 | // is equivalent to: 122 | 123 | var anIterable = [1, 2, 3]; 124 | var anIterator = anIterable[Symbol.iterate](); 125 | while (true) { 126 | let anIteration = anIterator.next(); 127 | if (anIteration.done) { 128 | break; 129 | } else { 130 | var aValue = anIteration.value; 131 | console.log(aValue); 132 | } 133 | } 134 | ``` 135 | 136 | The `on` operator would operate on an asynchronous iterable, producing an 137 | asynchronous iterator, and await each promised iteration. 138 | Look for the `await` in the following example. 139 | 140 | ```js 141 | for (let a on anAsyncIterable) { 142 | console.log(a); 143 | } 144 | 145 | // is equivalent to: 146 | 147 | var anAsyncIterator = anAsyncIterable[Symbol.iterate](); 148 | while (true) { 149 | var anAsyncIteration = anAsyncIterator.next(); 150 | var anIteration = await anAsyncIteration; 151 | if (anIteration.done) { 152 | break; 153 | } else { 154 | var aValue = anIteration.value; 155 | console.log(aValue); 156 | } 157 | } 158 | ``` 159 | 160 | One point of interest is that the `on` operator would work for both asynchronous 161 | and synchronous iterators and iterables, since `await` accepts both values and 162 | promises. 163 | 164 | Jafar proposes that the asynchronous analogues of `iterate()` would be 165 | `observe(generator)`, from which it is trivial to derrive `forEach`, but I 166 | propose that the asynchronous analogues of `iterate()` would just be 167 | `iterate()` and differ only in the type of the returned iterator. 168 | What Jafar proposes as the `asyncIterator.observe(asyncGenerator)` method is 169 | effectively equivalent to synchronous `iterator.copy(generator)` or 170 | `stream.pipe(stream)`. 171 | In this framework, `copy` would be implemented in terms of `forEach`. 172 | 173 | ```js 174 | Stream.prototype.copy = function (stream) { 175 | return this.forEach(stream.next).then(stream.return, stream.throw); 176 | }; 177 | ``` 178 | 179 | And, `forEach` would be implemented in terms of `next`, just as it would be 180 | layered on a synchronous iterator. 181 | 182 | -------------------------------------------------------------------------------- /async-generators.md: -------------------------------------------------------------------------------- 1 | 2 | # Promise Generators 3 | 4 | A promise generator is analogous in all ways to a plain generator. 5 | Promise generators implement `yield`, `return`, and `throw`. 6 | The return and throw methods both terminate the stream. 7 | Yield accepts a value. 8 | They all return promises for an acknowledgement iteration from the consumer. 9 | Waiting for this promise to settle causes the producer to idle long enough for 10 | the consumer to process the data. 11 | 12 | One can increase the number of promises that can be held in flight by a promise 13 | buffer. 14 | The buffer constructor takes a `length` argument that primes the acknowledgement 15 | queue, allowing you to send that number of values before having to wait for the 16 | consumer to flush. 17 | 18 | ```js 19 | var buffer = new Buffer(1024); 20 | function fibStream(a, b) { 21 | return buffer.in.yield(a) 22 | .then(function () { 23 | return fibStream(b, a + b); 24 | }); 25 | } 26 | fibStream(1, 1).done(); 27 | return buffer.out; 28 | ``` 29 | 30 | If the consumer would like to terminate the producer prematurely, it calls the 31 | `throw` method on the corresponding promise iterator. 32 | This will eventually propagate back to the promise returned by the generator’s 33 | `yield`, `return`, or `throw`. 34 | 35 | ```js 36 | buffer.out.throw(new Error("That's enough, thanks")); 37 | ``` 38 | 39 | -------------------------------------------------------------------------------- /async-iterators.md: -------------------------------------------------------------------------------- 1 | 2 | # Promise Iterators 3 | 4 | One very important kind of promise iterator lifts a spatial iterator into the 5 | temporal dimension so it can be consumed on demand over time. 6 | In this sketch, we just convert a synchronous `next` method to a method that 7 | returns a promise for the next iteration instead. 8 | We use a mythical `iterate` function which would create iterators for all 9 | sensible JavaScript objects and delegate to the `iterate` method of anything 10 | else that implements it. 11 | There is talk of using symbols in ES7 to avoid recognizing accidental iterables 12 | as this new type of duck. 13 | 14 | ```js 15 | function PromiseIterator(iterable) { 16 | this.iterator = iterate(iterable); 17 | } 18 | PromiseIterator.prototype.next = function () { 19 | return Promise.return(this.iterator.next()); 20 | }; 21 | ``` 22 | 23 | The conversion may seem superfluous at first. 24 | However, consider that a synchronous iterator might, apart from implementing 25 | `next()`, also implement methods analogous to `Array`, like `forEach`, 26 | `map`, `filter`, and `reduce`. 27 | Likewise, an asynchronous iterator might provide analogues to these functions 28 | lifted into the asynchronous realm. 29 | 30 | The accompanying sketch of a stream constructor implements a method 31 | `Stream.from`, analogous to ECMAScript 6's own `Array.from`. 32 | This function coerces any iterable into a stream, consuming that iterator on 33 | demand. 34 | This allows us, for example, to run an indefinite sequence of jobs, counting 35 | from 1, doing four jobs at any time. 36 | 37 | ```js 38 | Stream.from(Iterator.range(1, Infinity)) 39 | .forEach(function (n) { 40 | return Promise.delay(1000).thenReturn(n); 41 | }, null, 4) 42 | .done(); 43 | ``` 44 | 45 | ## map 46 | 47 | For example, asynchronous `map` would consume iterations and run jobs on each of 48 | those iterations using the callback. 49 | However, unlike a synchronous `map`, the callback might return a promise for 50 | its eventual result. 51 | The results of each job would be pushed to an output reader, resulting in 52 | another promise that the result has been consumed. 53 | This promise not only serves to produce the corresponding output iteration, but 54 | also serves as a signal that the job has completed, that the output has been 55 | consumed, and that the `map` can schedule additional work. 56 | An asynchronous `map` would accept an additional argument that would limit the 57 | number of concurrent jobs. 58 | 59 | ```js 60 | promiseIterator.map(function (value) { 61 | return Promise.return(value + 1000).delay(1000); 62 | }); 63 | ``` 64 | 65 | ## forEach 66 | 67 | Synchronous `forEach` does not produce an output collection or iterator. 68 | However, it does return `undefined` *when it is done*. 69 | Of course synchronous functions are implicitly completed when they return, 70 | but asynchronous functions are done when the asynchronous value they return 71 | settles. 72 | `forEach` returns a promise for `undefined`. 73 | 74 | Since streams are **unicast**, asynchronous `forEach` would return a task. 75 | It stands to reason that the asynchronous result of `forEach` on a stream would 76 | be able to propagate a cancellation upstream, stopping the flow of data from the 77 | producer side. 78 | Of course, the task can be easily forked or coerced into a promise if it needs 79 | to be shared freely among multiple consumers. 80 | 81 | ```js 82 | var task = reader.forEach(function (n) { 83 | console.log("consumed", n); 84 | return Promise.delay(1000).then(function () { 85 | console.log("produced", n); 86 | }); 87 | }) 88 | var subtask = task.fork(); 89 | var promise = Promise.return(task); 90 | ``` 91 | 92 | Like `map` it would execute a job for each iteration, but by default it would 93 | perform these jobs in serial. 94 | Asynchronous `forEach` would also accept an additional argument that would 95 | *expand* the number of concurrent jobs. 96 | In this example, we would reach out to the database for 10 user records at any 97 | given time. 98 | 99 | ```js 100 | reader.forEach(function (username) { 101 | return database.getUser(username) 102 | .then(function (user) { 103 | console.log(user.lastModified); 104 | }) 105 | }, null, 10); 106 | ``` 107 | 108 | ## reduce 109 | 110 | Asynchronous `reduce` would aggregate values from the input reader, returning a 111 | promise for the composite value. 112 | The reducer would have an internal pool of aggregated values. 113 | When the input is exhausted and only one value remains in that pool, the reducer 114 | would resolve the result promise. 115 | If you provide a basis value as an argument, this would be used to "prime the 116 | pump". 117 | The reducer would then start some number of concurrent aggregator jobs, each 118 | consuming two values. 119 | The first value would preferably come from the pool, but if the pool is empty, 120 | would come from the input. 121 | The second value would come unconditionally from the input. 122 | Whenever a job completes, the result would be placed back in the pool. 123 | 124 | ## pipe 125 | 126 | An asynchronous iterator would have additional methods like `copy` or `pipe` 127 | that would send iterations from this reader to another writer. 128 | This method would be equivalent to using `forEach` to forward iterations and 129 | `then` to terminate the sequence. 130 | 131 | ```js 132 | iterator.copy(generator); 133 | // is equivalent to: 134 | iterator.forEach(generator.yield).then(generator.return, generator.throw); 135 | ``` 136 | 137 | Note that the promise returned by yield applies pressure on the `forEach` 138 | machine, pushing ultimately back on the iterator. 139 | 140 | ## buffer 141 | 142 | It would have `buffer` which would construct a buffer with some capacity. 143 | The buffer would try to always have a value on hand for its consumer by 144 | prefetching values from its producer. 145 | If the producer is faster than the consumer, this can help avoid round trip 146 | latency when the consumer needs a value from the producer. 147 | 148 | ## read 149 | 150 | Just as it is useful to transform a synchronous collection into an iterator and 151 | an iterator into a reader, it is also useful to go the other way. 152 | An asynchronous iterator would also have methods that would return a promise for 153 | a collection of all the values from the source, for example `all`, or in the 154 | case of readers that iterate collections of bytes or characters, `join` or 155 | `read`. 156 | 157 | -------------------------------------------------------------------------------- /async-queues.md: -------------------------------------------------------------------------------- 1 | 2 | # Promise Queues 3 | 4 | Consider an asynchronous queue. 5 | With a conventional queue, you must put a value in before you can take it out. 6 | That is not the case for a promise queue. 7 | Just as you can attach an observer to a promise before it is resolved, with a 8 | promise queue, you can get a promise for the next value in order before that 9 | value has been given. 10 | 11 | ```js 12 | var queue = new Queue(); 13 | queue.get().then(function (value) { 14 | console.log(value); 15 | }) 16 | queue.put("Hello, World!"); 17 | ``` 18 | 19 | Likewise of course you can add a value to the queue before observing it. 20 | 21 | ```js 22 | var queue = new Queue(); 23 | queue.put("Hello, World!"); 24 | queue.get().then(function (value) { 25 | console.log(value); 26 | }) 27 | ``` 28 | 29 | Although promises come out of the queue in the same order their corresponding 30 | resolutions enter, a promise obtained later may settle sooner than another 31 | promise. 32 | The values you put in the queue may themselves be promises. 33 | 34 | ```js 35 | var queue = new Queue(); 36 | queue.get().then(function () { 37 | console.log("Resolves later"); 38 | }); 39 | queue.get().then(function () { 40 | console.log("Resolves sooner"); 41 | }); 42 | queue.put(Promise.delay(100)); 43 | queue.put(); 44 | ``` 45 | 46 | A promise queue qualifies as an asynchronous collection, specifically a 47 | collection of results: values or thrown errors captured by promises. 48 | The queue is not particular about what those values mean and is a suitable 49 | primitive for many more interesting tools. 50 | 51 | | Interface | | | | 52 | | ------------- | ------- | ------ | -------- | 53 | | PromiseQueue | Value | Plural | Temporal | 54 | | queue.get | Getter | Plural | Temporal | 55 | | queue.put | Setter | Plural | Temporal | 56 | 57 | The implementation of a promise queue is sufficiently succinct that there’s no 58 | harm in embedding it here. 59 | This comes from Mark Miller's [Concurrency Strawman][] for ECMAScript and is a 60 | part of the Q library, exported by the `q/queue` module. 61 | Internally, a promise queue is an asynchronous linked list that tracks the 62 | `head` promise and `tail` deferred. 63 | `get` advances the `head` promise and `put` advances the `tail` deferred. 64 | 65 | ```js 66 | module.exports = PromiseQueue; 67 | function PromiseQueue() { 68 | if (!(this instanceof PromiseQueue)) { 69 | return new PromiseQueue(); 70 | } 71 | var ends = Promise.defer(); 72 | this.put = function (value) { 73 | var next = Promise.defer(); 74 | ends.resolve({ 75 | head: value, 76 | tail: next.promise 77 | }); 78 | ends.resolve = next.resolve; 79 | }; 80 | this.get = function () { 81 | var result = ends.promise.get("head"); 82 | ends.promise = ends.promise.get("tail"); 83 | return result; 84 | }; 85 | } 86 | ``` 87 | 88 | [Concurrency Strawman]: http://wiki.ecmascript.org/doku.php?id=strawman:concurrency 89 | 90 | The implementation uses methods defined in a closure. 91 | Regardless of how this is accomplished, it is important that it be possible to 92 | pass the free `get` function to a consumer and `put` to a producer to preserve 93 | the principle of least authority and the unidirectional flow of data from 94 | producer to consumer. 95 | 96 | See the accompanying sketch of a [promise queue][] implementation. 97 | 98 | [promise queue]: http://kriskowal.github.io/gtor/docs/promise-queue 99 | 100 | A promise queue does not have a notion of termination, graceful or otherwise. 101 | We will later use a pair of promise queues to transport iterations between 102 | **streams**. 103 | 104 | -------------------------------------------------------------------------------- /async-values.md: -------------------------------------------------------------------------------- 1 | # Asynchronous Values 2 | 3 | The asynchronous analogue of a getter is a promise. 4 | Each promise has a corresponding resolver as its asynchronous setter. 5 | Collectively the promise and resolver are a deferred value. 6 | 7 | The salient method of a promise is `then`, which creates a new promise for the 8 | result of a function that will eventually observe the value of the promise. 9 | If a promise were plural, the `then` method might be called `map`. 10 | If you care to beg an esoteric distinction, it might be called `map` if the 11 | observer returns a value and `flatMap` if the observer returns a promise. 12 | The `then` method of a promise allows either. 13 | 14 | ```js 15 | var promiseForThirty = promiseForTen.then(function (ten) { 16 | return ten + 20; 17 | }) 18 | ``` 19 | 20 | Promises can also have a `done` method that observes the value but does not 21 | return a promise nor captures the result of the observer. 22 | Again, if a promise were plural, the `done` method might be called `forEach`. 23 | 24 | ```js 25 | promiseForTen.done(function (ten) { 26 | }); 27 | ``` 28 | 29 | The `then` method also supports a second function that would observe whether the 30 | input promise radiates an exception, and there is a `catch` method to use as a 31 | shorthand if you are only interested in catching errors. 32 | 33 | ```js 34 | promise.then(onreturn, onthrow); 35 | promise.catch(onthrow); 36 | ``` 37 | 38 | At this point, the design described here begins to differ from the standard 39 | `Promise` proposed for ECMAScript 6, arriving in browsers at time of writing. 40 | The purpose of these differences is not to propose an alternative syntax, but to 41 | reinforce the relationship between a promise and its conceptual neighbors. 42 | 43 | A resolver is the singular analogue of a generator. 44 | Rather than yielding, returning, and throwing errors, the resolver can only 45 | return or throw. 46 | 47 | ```js 48 | resolver.return(10); 49 | resolver.throw(new Error("Sorry, please return during business hours.")); 50 | ``` 51 | 52 | With the standard promise, a free `resolve` function is sufficient and ergonomic 53 | for expressing both of these methods. 54 | `resolver.return(promise)` is equivalent to `resolve(promise)`. 55 | `resolver.return(10)` is equivalent to `resolve(10)` or 56 | `resolve(Promise.resolve(10))`since non-promise values are automatically boxed 57 | in an already-fulfilled promise. 58 | `resolver.throw(error)` is equivalent to `resolve(Promise.reject(error))`. 59 | In all positions, `resolve` is the temporal analogue of `return` and `reject` is 60 | the temporal analogue of `throw`. 61 | Since promises as we know them today bridged the migration gap from ECMAScript 3 62 | to ECMAScript 6, it was also necessary to use non-keywords for method names. 63 | 64 | A deferred value can be deferred further by resolving it with another promise. 65 | This can occur either expressly through the resolver, or implicitly by returning 66 | a promise as the result of a observer function. 67 | 68 | ```js 69 | var authenticated = getUsernameFromConsole() 70 | .then(function (username) { 71 | return Promise.all([ 72 | getUserFromDatabase(username), 73 | getPasswordFromConsole() 74 | ]) 75 | .then(function ([user, password]) { 76 | if (hash(password) !== user.passwordHash) { 77 | throw new Error("Can't authenticate because the password is invalid"); 78 | } 79 | }) 80 | }) 81 | ``` 82 | 83 | The `then` method internally creates a new deferred, returns the promise, and 84 | later forwards the return value of the observer to the resolver. 85 | This is a sketch of a `then` method that illustrates this adapter. 86 | Note that we create a deferred, use the resolver, and return the promise. 87 | The adapter is responsible for catching errors and giving the consumer an 88 | opportunity to do further work or to recover. 89 | 90 | ```js 91 | Promise.prototype.then = function Promise_then(onreturn, onthrow) { 92 | var self = this; 93 | var deferred = Promise.defer(); 94 | var resolver = deferred.resolver; 95 | this.done(function (value) { 96 | if (onreturn) { 97 | try { 98 | resolver.return(onreturn(value)); 99 | } catch (error) { 100 | resolver.throw(error); 101 | } 102 | } else { 103 | resolver.return(value); 104 | } 105 | }, function (error) { 106 | if (onthrow) { 107 | try { 108 | resolver.return(onthrow(value)); 109 | } catch (error) { 110 | resolver.throw(error); 111 | } 112 | } else { 113 | resolver.throw(error); 114 | } 115 | }); 116 | return deferred.promise; 117 | ``` 118 | 119 | The standard `Promise` does not reveal `Promise.defer()`. 120 | Instead, it is hidden by `then` and by the `Promise` constructor, which elects 121 | to hide the deferred object and the resolver object, instead "revealing" the 122 | `resolve` and `reject` methods as free arguments to a setup function, furthering 123 | the need to give these functions names that are not keywords. 124 | 125 | ```js 126 | var promise = new Promise(function (resolve, reject) { 127 | // ... 128 | }); 129 | ``` 130 | 131 | With a promise, information flows only from the first call to a resolver method 132 | to all promise observers, whether they are registered before or after the 133 | resolution. 134 | 135 | With a task, information flows from the first call to a resolver method to the 136 | first call to an observer method, regardless of their relative order, but one 137 | kind of information can flow upstream. 138 | The observer may unsubscribe with an error. 139 | This is conceptually similar to throwing an error back into a generator from an 140 | iterator and warrants the same interface. 141 | 142 | ```js 143 | task.throw(new Error("Never mind")); 144 | ``` 145 | 146 | This interface foreshadows its plural analogue: streams. 147 | 148 | -------------------------------------------------------------------------------- /behavior.js: -------------------------------------------------------------------------------- 1 | 2 | // A behavior is a **pull** or **poll** representation of a value that varies 3 | // **continuously** over time. 4 | // Behaviors only model the **getter** side of the *getter, setter, value* 5 | // triad because they produce the value for any given time on demand. 6 | 7 | "use strict"; 8 | 9 | var WeakMap = require("collections/weak-map"); 10 | var Operators = require("./operators"); 11 | var Iteration = require("./iteration"); 12 | 13 | // ### BehaviorHandler 14 | 15 | // The private handler for a behavior captures its internal state. 16 | 17 | function BehaviorHandler(callback, thisp) { 18 | this.callback = callback; 19 | this.thisp = thisp; 20 | } 21 | 22 | // We have to set this up before constructing singleton behaviors. 23 | var handlers = new WeakMap(); 24 | 25 | // ## Behavior 26 | // 27 | // The behavior constructor accepts a method and optional context object for 28 | // that method. 29 | // This method will be polled at each time the behavior is called upon to 30 | // produce a new value. 31 | 32 | module.exports = Behavior; 33 | function Behavior(callback, thisp) { 34 | // Behaviors use a weak map of hidden records for private state. 35 | var handler = new BehaviorHandler(callback, thisp); 36 | handlers.set(this, handler); 37 | } 38 | 39 | // The `yield` static method creates a behavior that provides the given 40 | // constant value. 41 | Behavior.yield = function (value) { 42 | return new Behavior(function () { 43 | return value; 44 | }); 45 | }; 46 | 47 | // The `index` singleton behavior provides the time or "index" any time it is 48 | // polled. 49 | Behavior.index = new Behavior(function (index) { 50 | return index; 51 | }); 52 | 53 | // The `next` method of a behavior returns an iteration that captures both the 54 | // value and index time, and gives a behavior the same shape as an iterator. 55 | // Unlike a stream, the `next` method does not return a promise for an iteration. 56 | Behavior.prototype.next = function (index) { 57 | var handler = handlers.get(this); 58 | var value = handler.callback.call(handler.thisp, index); 59 | return new Iteration(value, false, index); 60 | }; 61 | 62 | // The static `lift` method accepts an operator and its context object and 63 | // produces the analogous operator on behaviors. 64 | // The resulting operator accepts and returns behaviors instead of values. 65 | // Each time the user polls the returned behavior they will get the result of 66 | // applying the current value of each behavior argument to the operator. 67 | // 68 | // For example, `Beahavior.lift(add)` will produce a `behaviorAdd` operator. 69 | // `behaviorAdd(Behavior.return(10), Behavior.return(20))` will produce a 70 | // behavior that will always yield `30`. 71 | Behavior.lift = function (operator, operators) { 72 | return function operatorBehavior() { 73 | var operands = Array.prototype.slice.call(arguments); /* TODO unroll */ 74 | return new Behavior(function (index) { 75 | var values = operands.map(function (operand) { 76 | return operand.next(index).value; 77 | }); 78 | if (values.every(Operators.defined)) { 79 | return operator.apply(operators, values); 80 | } 81 | }); 82 | }; 83 | }; 84 | 85 | // The `tupleLift` static method is the same as `lift` accept that the returned 86 | // behavior operator accepts an array of behaviors instead of variadic behavior 87 | // arguments. 88 | Behavior.tupleLift = function (operator, operators) { 89 | return function operatorBehavior(operands) { 90 | return new Behavior(function (index) { 91 | var values = operands.map(function (operand) { 92 | return operand.next(index).value; 93 | }); 94 | if (values.every(Operators.defined)) { 95 | return operator.apply(operators, values); 96 | } 97 | }); 98 | }; 99 | }; 100 | 101 | // Using `lift` and `tupleLift`, we generate both a method for both the 102 | // behavior constructor and prototype for each of the operators defined in the 103 | // `Operators` module. 104 | for (var name in Operators) { 105 | (function (operator, name) { 106 | Behavior[name] = Behavior.lift(operator, Operators); 107 | var tupleLift = Behavior.tupleLift(operator, Operators); 108 | Behavior.prototype[name] = function () { 109 | var operands = [this]; 110 | for (var index = 0; index < arguments.length; index++) { 111 | operands[index + 1] = arguments[index]; 112 | } 113 | return tupleLift(operands); 114 | }; 115 | })(Operators[name], name); 116 | } 117 | 118 | -------------------------------------------------------------------------------- /behaviors.md: -------------------------------------------------------------------------------- 1 | 2 | # Behaviors 3 | 4 | A behavior represents a time series value. 5 | A behavior may produce a different value for every moment in time. 6 | As such, they must be **polled** at an interval meaningful to the consumer, 7 | since the behavior itself has no inherent resolution. 8 | 9 | Behaviors are analogous to Observables, but there is no corresponding setter, 10 | since it produces values on demand. 11 | The behavior constructor accepts a function that returns the value for a given 12 | time. 13 | An asynchronous behavior returns promises instead of values. 14 | 15 | See the accompanying sketch of a [behavior][] implementation. 16 | 17 | [behavior]: http://kriskowal.github.io/gtor/docs/behavior 18 | 19 | -------------------------------------------------------------------------------- /byte-stream.js: -------------------------------------------------------------------------------- 1 | 2 | var WeakMap = require("weak-map"); 3 | var Promise = require("./promise"); 4 | var Iteration = require("./iteration"); 5 | var Stream = require("./stream"); 6 | 7 | var handlers = new WeakMap(); 8 | 9 | // ## ByteStream 10 | 11 | module.exports = ByteStream; 12 | function ByteStream(setup, length) { 13 | var buffer = this.constructor.buffer(length); 14 | handlers.set(this, handlers.get(buffer.out)); 15 | ByteStream_bind(this); 16 | setup(buffer.in.yield, buffer.in.return, buffer.in.throw); 17 | } 18 | 19 | ByteStream.buffer = function (length) { 20 | var handler = new ByteHandler(length); 21 | var input = Object.create(ByteSource.prototype); 22 | ByteSource_bind(input); 23 | handlers.set(input, handler); 24 | var output = Object.create(ByteStream.prototype); 25 | ByteStream_bind(output); 26 | handlers.set(output, handler); 27 | return {in: input, out: output}; 28 | }; 29 | 30 | ByteStream_bind = function (stream) { 31 | stream.next = stream.next.bind(stream); 32 | } 33 | 34 | ByteStream.of = function (buffer, length) { 35 | return new this(function (_yield, _return, _throw) { 36 | _yield(buffer).then(function (iteration) { 37 | return _return(iteration.value); 38 | }, _throw).done(); 39 | }, length); 40 | }; 41 | 42 | ByteStream.prototype = Object.create(Stream.prototype); 43 | ByteStream.prototype.constructor = ByteStream; 44 | 45 | ByteStream.prototype.next = function () { 46 | var handler = handlers.get(this); 47 | return handler.next(); 48 | }; 49 | 50 | // ## ByteSource 51 | 52 | function ByteSource() { 53 | } 54 | 55 | ByteSource_bind = function (source) { 56 | source.yield = source.yield.bind(source); 57 | source.return = source.return.bind(source); 58 | source.throw = source.throw.bind(source); 59 | } 60 | 61 | ByteSource.prototype.yield = function (chunk) { 62 | if (!(chunk instanceof Buffer)) { 63 | throw new Error("Can't write non-buffer to stream"); 64 | } 65 | var handler = handlers.get(this); 66 | return handler.yield(new Window(chunk)); 67 | }; 68 | 69 | ByteSource.prototype.return = function (value) { 70 | var handler = handlers.get(this); 71 | return handler.return(value); 72 | }; 73 | 74 | ByteSource.prototype.throw = function (error) { 75 | var handler = handlers.get(this); 76 | return handler.throw(error); 77 | }; 78 | 79 | // ## ByteHandler 80 | 81 | function ByteHandler(length) { 82 | length = length || 4096; 83 | this.buffer = new Buffer(length); 84 | this.buffer.fill(255); // XXX debug 85 | this.head = 0; // first index of stored bytes 86 | this.tail = 0; // last index of stored bytes 87 | /* if head < tail, contiguous 88 | * if tail < head, broken into two regions 89 | * [head, length) and [0, tail) */ 90 | this.length = 0; // the quantity of stored bytes 91 | /* head === tail may mean empty or full */ 92 | this.index = 0; 93 | this.read = 0; // [head, read) is being read 94 | this.write = 0; // [tail, write) is being written 95 | this.closed = false; 96 | this.returnValue = null; 97 | this.exhausted = Promise.defer(); 98 | this.canRead = Promise.defer(); 99 | this.canWrite = Promise.defer(); 100 | this.canWrite.resolver.return(); 101 | } 102 | 103 | ByteHandler.prototype.next = function () { 104 | if (this.read !== this.head) { 105 | // Releases the last window that was checked out for reading. 106 | this.length -= this.read - this.head; 107 | this.head = this.read % this.buffer.length; 108 | this.canWrite.resolver.return(); 109 | } 110 | // Returns a promise for a chunk of bytes that can be read. 111 | return this.canRead.promise.then(function () { 112 | this.canRead = Promise.defer(); 113 | var more; 114 | /* trying to check out the next contiguous RRRR region */ 115 | if (!this.length) { 116 | /* WWWWW HT WWWW head === tail && !length, means empty*/ 117 | this.exhausted.resolver.return(); 118 | return new Iteration(this.returnValue, true); 119 | } else if (this.head < this.tail) { 120 | /* head !== length is implied by remainder operator above. */ 121 | /* RRRRR HT RRRR head === tail && length, means full */ 122 | /* WWW H RRRR T WWW */ 123 | this.read = this.tail; 124 | more = false; 125 | } else { 126 | /* RRR T WWWW H RRR */ 127 | this.read = this.length; 128 | more = this.tail !== 0; 129 | } 130 | if (more || this.closed) { 131 | // Causes canRead to open even though length is 0 132 | this.canRead.resolver.return(); 133 | } 134 | var index = this.index; 135 | this.index += this.read - this.head; 136 | return new Iteration(this.buffer.slice(this.head, this.read), false, index); 137 | }, this); 138 | }; 139 | 140 | ByteHandler.prototype.yield = function (source) { 141 | if (this.closed) { 142 | throw new Error("Can't write to closed stream"); 143 | } 144 | // Writing, closing, and aborting all imply that the last slice from any of 145 | // these methods is now free space for reading. 146 | return this.canWrite.promise.then(function () { 147 | this.canWrite = Promise.defer(); 148 | var length; 149 | var more; 150 | if (this.length === this.buffer.length) { 151 | throw new Error("Can't write until previous flush completes (wait for returned promise)"); 152 | } else if (this.head <= this.tail) { 153 | /* WWW H RRRR T WWW */ 154 | length = Math.min(source.length, this.buffer.length - this.tail); 155 | more = this.head !== 0; 156 | } else { 157 | /* RRR T WWWW H RRR */ 158 | length = Math.min(source.length, this.head - this.tail); 159 | more = false; 160 | } 161 | if (length) { 162 | source.flush(this.buffer.slice(this.tail, this.tail + length)); 163 | this.tail = (this.tail + length) % this.buffer.length; 164 | this.length += length; 165 | this.canRead.resolver.return(); 166 | } 167 | if (more) { 168 | this.canWrite.resolver.return(); 169 | } 170 | // If the flush did not exhaust the source, there was not enough room 171 | // in the target buffer window and we filled it to the brim. 172 | // We must wait for another opportunity to write. 173 | if (source.length) { 174 | return this.yield(source); 175 | } else { 176 | return new Iteration(); 177 | } 178 | }, this); 179 | }; 180 | 181 | ByteHandler.prototype.return = function (value) { 182 | if (this.closed) { 183 | throw new Error("Can't write to closed stream"); 184 | } 185 | this.closed = true; 186 | this.returnValue = value; 187 | this.canRead.resolver.return(); 188 | return this.exhausted.promise; 189 | }; 190 | 191 | ByteHandler.prototype.throw = function (error) { 192 | if (this.closed) { 193 | throw new Error("Can't write to closed stream"); 194 | } 195 | this.closed = true; 196 | this.canRead.resolver.throw(error); 197 | return Promise.return(Iteration.done); 198 | }; 199 | 200 | // ## Window 201 | 202 | function Window(buffer, tail, head) { 203 | this.buffer = buffer; 204 | this.tail = tail || 0; 205 | this.head = head || buffer.length; 206 | this.length = this.head - this.tail; 207 | } 208 | 209 | Window.prototype.flush = function (target) { 210 | var length = Math.min(this.length); 211 | this.buffer.copy(target, 0, this.tail); 212 | this.tail += target.length; 213 | this.length -= target.length; 214 | }; 215 | 216 | Window.prototype.capture = function () { 217 | return this.buffer.slice(this.tail, this.head); 218 | }; 219 | 220 | -------------------------------------------------------------------------------- /cancelation.md: -------------------------------------------------------------------------------- 1 | 2 | - TODO 3 | 4 | - From MarkM: 5 | The requestor saying it is no longer interested should not be a source of 6 | possible requestee inconsistent state that the requestee needs to recover from, 7 | unless the requestee goes out of its way to take on that burden in order to stop 8 | early. 9 | The key thing is that cancellation is the communication of lack of interest or 10 | impatience (see recent paper again) from requestor to requestee, and so is 11 | handled well by a promise going in the direction opposite to results. 12 | 13 | - KrisK 14 | the other open issue in my mind is whether cancel() (in whatever form it comes) 15 | should return a promise, and whether that promise should settled before 16 | attempting another request, 17 | yes, and GC does not communicate impatience effectively. 18 | and furthermore, the possibility that cancel() (in whatever form it takes) might 19 | be able to provide a useful “all clear” promise, though the producer should by 20 | no means rely upon the consumer to observe that promise before making another 21 | request. 22 | a signal that internal consistency has been restored. 23 | a signal that it would internally wait for before accepting another request 24 | anyway, so that is moot. 25 | 26 | # Canceling asynchronous tasks 27 | 28 | Promises are a tool for managing asynchronous code. 29 | A promise represents the eventual result of a function call that can't be 30 | completed without blocking. 31 | It is simple to create a new promise that depends on the result of one or more 32 | other promises, and more than one promise can depend on another. 33 | 34 | MontageJS makes a great deal of use of promises, [using Q][Q]. 35 | Although you can use MontageJS modules without ever encountering a promise, 36 | the module system uses them extensively, and you can use promises to coordinate 37 | Undo and Redo operations. 38 | 39 | [Q]: https://github.com/kriskowal/q 40 | 41 | Whenever you set out to do some work asynchronously, you return a promise. 42 | This is an example of creating a promise for some value after a delay. 43 | 44 | ```js 45 | var Promise = require("q").Promise; 46 | function delay(value, ms) { 47 | return new Promise(function (resolve, reject) { 48 | setTimeout(function () { 49 | resolve(value); 50 | }, delay); 51 | }); 52 | } 53 | ``` 54 | 55 | Promises seem like an obvious object to support a method for canceling work. 56 | If you are using `setTimeout` directly, you would use `clearTimeout` to cancel 57 | the delay. 58 | However, this conflicts with the requirement that a single promise object can 59 | have multiple consumers. 60 | A good promise implementation ensures that information flows only from the 61 | producer to consumer. 62 | If one consumer can cancel the promise, it would be able to use this capability 63 | to interfere with any other consumer of that promise. 64 | 65 | That being said, the desire for promises to support cancellation is 66 | well-grounded. 67 | If a promise represents a large quantity of work, as it often does, it does not 68 | make sense to continue doing that work when there remain no consumers interested 69 | in the result. 70 | It makes even less sense if the consumer goes on to ask a similar question that 71 | will require the same resources, like a database lock or even just time on the 72 | CPU. 73 | The issue compounds further when the user is fickle and frequently changing its 74 | query, as in search-as-you-type interfaces. 75 | 76 | ## Proof by contradiction 77 | 78 | To support cancellation, we would need a new reactive primitive. 79 | Provisionally, let us call such an object a Task because it is foremost a 80 | representation of work in progress, not merely a proxy for the result. 81 | The interface and terminology for a Task would have much in common with a 82 | Promise, but would differ subtly in usage. 83 | 84 | There would need to be a single producer to single consumer relationship. 85 | Branches in the chain from producer to consumer would still be possible, 86 | but would need to be explicit. 87 | Upon construction, a task would be in a detached state. 88 | It has a producer but it does not have a consumer. 89 | A consumer would be attached with a method like `then`. 90 | 91 | In this example, we implement `task.delay(ms)`, which will cause the task's result 92 | to be delayed by a certain duration after its result has been fulfilled. 93 | If the task is cancelled, the timer will be cancelled, and the returned task will 94 | be implicitly rejected with a cancellation error. 95 | In the `Task` constructor, we have an opportunity to return a cancellation hook. 96 | 97 | ```js 98 | Task.prototype.delay = function Task_delay(ms) { 99 | return this.then(function (value) { 100 | return new Task(function (resolve, reject, estimate) { 101 | var handle = setTimeout(function () { 102 | resolve(value); 103 | }, delay); 104 | return function cancel() { 105 | clearTimeout(handle); 106 | }; 107 | }); 108 | }); 109 | } 110 | ``` 111 | 112 | Once a consumer has been attached, any subsequent attempt to attach another 113 | consumer would throw an error. 114 | 115 | ```js 116 | var plusTenTask = task.then(function (value) { return value + 10 }); 117 | // The next line throws an error. 118 | var plusTwentyTask = task.then(function (value) { return value + 20 }); 119 | ``` 120 | 121 | The second consumer must throw an error to preserve the one-producer to 122 | one-consumer relationship. 123 | If we cancel `plusTenTask`, it implies that we can cancel `task`. 124 | This would interfere with the resolution of `plusTwentyTask`. 125 | 126 | ## Explicitly branching 127 | 128 | Although a task can only have one consumer by default, there does need to be a 129 | mechanism by which a result can be retained for multiple consumers explicitly. 130 | A call to `fork` would retain the result for a subsequent consumer. 131 | This could be accomplished internally in a variety of ways, but ultimately 132 | counting interested consumers. 133 | 134 | ```js 135 | var plusTenTask = task.fork().then(function (value) { return value + 10 }); 136 | var plusTwentyTask = task.then(function (value) { return value + 20 }); 137 | ``` 138 | 139 | Now, if we cancel `plusTenTask`, we know that one out of the two parties 140 | interested in the result of `task` have been cancelled, and thus, we cannot 141 | cancel `task`. If we then cancel `plusTwentyTask`, we know that there are no 142 | consumers interested in the `task` and the cancellation can propagate. 143 | 144 | A common pattern for promises is memoization. 145 | We can keep a promise around just in case at some point in the future, another 146 | consumer becomes interested in the result. 147 | 148 | ```js 149 | var memo = new Dict(); 150 | function getUser(userName) { 151 | if (!memo.has(userName)) { 152 | memo.set(reallyGetUser(userName)); 153 | } 154 | return memo.has(userName); 155 | } 156 | ``` 157 | 158 | The first call to `getUser("alice")` will create a promise for her user record. 159 | However, if that promise is cancelled, it provides no indication that the 160 | underlying task may be cancelled, assuming of course that it has not already 161 | completed. 162 | At some point, another consumer might call `getUser("alice")` and will expect 163 | *not* to receive promise for a cancellation error. 164 | 165 | With tasks, the story is different. 166 | We want the memo to strictly grow and we *never* intend to cancel any task once 167 | a user record has been requested. 168 | To ensure this, we must explicitly call `fork` for each new consumer. 169 | 170 | ```js 171 | var memo = new Dict(); 172 | function getUser(userName) { 173 | if (!memo.has(userName)) { 174 | memo.set(reallyGetUser(userName)); 175 | } 176 | return memo.has(userName).fork(); 177 | } 178 | ``` 179 | 180 | This illustrates the necessity that a task *enforce* its single consumer 181 | mandate. 182 | If the task implicitly forked, there would be no indication that the explicit 183 | `fork` were missing. 184 | The following example would throw an exception if the author of `getUser` 185 | neglected to explicitly call `fork`. 186 | 187 | ```js 188 | getUser("alice").then(getPassword); 189 | getUser("alice").then(getPassword); 190 | ``` 191 | 192 | ## Distributed garbage collection 193 | 194 | Also, consider the case of remote tasks. 195 | If we have a connection between two processes and one process requests a promise 196 | for the result of a task on the other, the far side must retain a promise for the 197 | result indefinitely, since the near side may continue communicating with it 198 | indefinitely. 199 | There are bodies of research in distributed garbage collection addressing this 200 | topic. 201 | 202 | With tasks, once a result has been communicated to the other side of a 203 | connection, assuming that it has not been consumed by a `fork`, the far side no 204 | longer needs to retain a reference to it. 205 | If the consumer does *fork* the far reference, the reference can be retained 206 | until all forks are either consumed or cancelled. 207 | 208 | ## *Quod est absurdum* 209 | 210 | A task is a form of a promise with explicit reference management, a concept that 211 | we largely left behind in garbage collected languages like JavaScript. 212 | This is a hint as to how and why promises are the better bet in the long term. 213 | 214 | In due course, we can expect [Weak references][WeakRef] and post-mortem 215 | finalization to be introduced to JavaScript. 216 | This would be a new language primitive that would allow us to receive a message 217 | by way of callback *after* a weakly-held reference has been garbage collected. 218 | Dropping a promise is the surest sign that a consumer is no longer interested in 219 | the result of a promise, both locally and remotely. 220 | When we can observe that a promise has been dropped, we may be able to 221 | implicitly cancel the associated work. 222 | 223 | [WeakRef]: http://wiki.ecmascript.org/doku.php?id=strawman:weak_references 224 | 225 | Until then however, tasks remain an interesting idea. 226 | 227 | -------------------------------------------------------------------------------- /clock.js: -------------------------------------------------------------------------------- 1 | // TODO rebase on observable 2 | 3 | // A clock is an observable signal that emits a pulse with the current time 4 | // with a given period and offset. 5 | // The period is a number of miliseconds and the pulse will be emitted at times 6 | // past the epoch that are multiples of that period, plus the offset 7 | // miliseconds. 8 | 9 | "use strict"; 10 | 11 | var Signal = require("./signal"); 12 | 13 | // ## Clock 14 | // 15 | // The clock constructor accepts the period and offsets in miliseconds, and 16 | // allows you to optionally override the `now` function, which should provide 17 | // the current time in miliseconds. 18 | 19 | module.exports = Clock; 20 | function Clock(options) { 21 | options = options || {}; 22 | var handler = new this.Handler(options.period, options.offset, options.now); 23 | Signal.prototype.Observable.call(this, handler); 24 | } 25 | 26 | // Clock inherits exclusively from the observable side of a signal and 27 | // generates values internally. 28 | Clock.prototype = Object.create(Signal.prototype.Observable.prototype); 29 | Clock.prototype.constructor = Clock; 30 | Clock.prototype.Handler = ClockHandler; 31 | 32 | // ## ClockHandler 33 | // 34 | // The clock handler inherits the utilities of a normal signal handler, but 35 | // also does some book keeping to schedule timers, activate, and deactivate. 36 | 37 | function ClockHandler(period, offset, now) { 38 | Signal.prototype.Handler.call(this); 39 | this.next = null; 40 | this.period = period || 1000; 41 | this.offset = offset || 0; 42 | this.now = now || this.now; 43 | this.value = this.now(); 44 | this.timeout = null; 45 | // We use the tock method as a timeout handler, so it has to be bound. 46 | this.tock = this.tock.bind(this); 47 | } 48 | 49 | ClockHandler.prototype = Object.create(Signal.prototype.Handler.prototype); 50 | ClockHandler.prototype.constructor = ClockHandler; 51 | ClockHandler.prototype.now = Date.now; 52 | 53 | // The `tick` method arranges for a `tock` method call at the next time that 54 | // aligns with the period and offset. 55 | // Note that a clock may miss beats if the process spends too much time between 56 | // events. 57 | ClockHandler.prototype.tick = function (now) { 58 | if (this.next === null) { 59 | this.next = Math.ceil( 60 | Math.floor( 61 | (now - this.offset) / this.period 62 | ) + .5 63 | ) * this.period + this.offset; 64 | this.timeout = setTimeout(this.tock, this.next - now); 65 | } 66 | }; 67 | 68 | // The `tock` method responds to timers and propagates the current time, both 69 | // as the pulse value and time index. 70 | ClockHandler.prototype.tock = function () { 71 | this.next = null; 72 | var now = this.now(); 73 | this.yield(now, now); 74 | if (this.active) { 75 | this.tick(now); 76 | } 77 | }; 78 | 79 | ClockHandler.prototype.timer = true; 80 | 81 | // The signal handler arranges for active and inactive state changes by calling 82 | // either `onstart` or `onstop` depending on whether there are any observers 83 | // registered. 84 | 85 | // When the clock becomes active, we kick off an initial tick. 86 | // Each subsequent tick is scheduled by the `tock` method if the clock remains 87 | // active. 88 | ClockHandler.prototype.onstart = function () { 89 | this.tick(this.value); 90 | }; 91 | 92 | // The `onstop` method cancels the next `tick`, breaking the chain of scheduled 93 | // pulses. 94 | ClockHandler.prototype.onstop = function () { 95 | if (this.timeout) { 96 | clearTimeout(this.timeout); 97 | } 98 | }; 99 | 100 | -------------------------------------------------------------------------------- /concepts.md: -------------------------------------------------------------------------------- 1 | 2 | # Concepts 3 | 4 | For the purpose of discussion, we must establish a vocabulary. 5 | Some of these names have a long tradition, or at least some precedent in 6 | JavaScript. 7 | Some are disputed, borrowed, or fabricated. 8 | 9 | A **value** is **singular** and **spatial**. 10 | It can be accessed or modified. 11 | If we break this atom, it will fall into two parts: the **getter** and the 12 | **setter**. 13 | Data flows in one direction, from the setter to the getter. 14 | 15 | The duality of a getter and a setter, a producer and a consumer, or a writer and 16 | a reader, exists in every reactive primitive. 17 | Erik Meijer shows us the parallelism and reflection that exists between various 18 | types of reactive duals in his [keynote for Lang.NEXT, 2014][EM]. 19 | 20 | [EM]: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Keynote-Duality 21 | 22 | Singular is as opposed to **plural** or multiple. 23 | An array, or generally any collection, contains multiple values. 24 | An **iterator** is a plural getter. 25 | A **generator** and iterator form the plural dual for values in space. 26 | 27 | Spatial is as opposed to **temporal**. 28 | Reactivity is about time. 29 | 30 | A **promise** is a getter for a single value from the past or the future. 31 | In JavaScript, and in the language E from which we borrowed the concept, the 32 | corresponding setter is a **resolver**. 33 | Collectively, an asynchronous value is a **deferred**. 34 | 35 | If a promise is the temporal analogue of a value, a **stream** is the temporal 36 | analogue of an array. 37 | The producer side of a stream is a writer and the consumer side is a reader. 38 | A **reader** is an asynchronous iterator and a **writer** is an asynchronous 39 | generator. 40 | 41 | 42 | | Interface | | | | 43 | | ---------- | ------------- | -------- | -------- | 44 | | Value | Value | Singular | Spatial | 45 | | Getter | Getter | Singular | Spatial | 46 | | Setter | Setter | Singular | Spatial | 47 | | Array | Value | Plural | Spatial | 48 | | Iterator | Getter | Plural | Spatial | 49 | | Generator | Setter | Plural | Spatial | 50 | | Deferred | Value | Singular | Temporal | 51 | | Promise | Getter | Singular | Temporal | 52 | | Resolver | Setter | Singular | Temporal | 53 | | Stream | Value | Plural | Temporal | 54 | | Reader | Getter | Plural | Temporal | 55 | | Writer | Setter | Plural | Temporal | 56 | 57 | 58 | ## Singular and plural 59 | 60 | An observer can subscribe to eventually see the value of a promise. 61 | They can do this before or after the promise has a value. 62 | Any number of observers can subscribe multiple times and any single observer can 63 | subscribe to the same promise multiple times. 64 | 65 | As such, promises model dependency. 66 | Promises and resolvers can be safely distributed to any number of producers and 67 | consumers. 68 | If multiple producers race to resolve a promise, the experience of each producer 69 | is indistinguishable regardless of whether they won or lost the race. 70 | Likewise, if multiple consumers subscribe to a promise, the experience of each 71 | consumer is indistinguishable. 72 | One consumer cannot prevent another consumer from making progress. 73 | Information flows in one direction. 74 | Promises make reactive programs more robust and composable. 75 | 76 | Promises are **broadcast**. 77 | 78 | The law that no consumer can interfere with another consumer makes it impossible 79 | for promises to abort work in progress. 80 | A promise represents a result, not the work leading to that result. 81 | 82 | A **task** has mostly the same form and features as a promise, but is unicast by 83 | default and can be cancelled. 84 | A task can have only one subscriber, but can be explicitly forked to create a 85 | new task that depends on the same result. 86 | Each subscriber can unsubscribe, and if all subscribers have unsubscribed and no 87 | further subscribers can be introduced, a task can abort its work. 88 | 89 | Tasks are **unicast** and therefore cancelable. 90 | 91 | See the accompanying sketch of a [task][] implementation. 92 | 93 | [task]: http://kriskowal.github.io/gtor/docs/task 94 | 95 | There is also an esoteric difference between a promise and a future. 96 | Promise resolvers accept either a value or a promise and will recursively unwrap 97 | transitive promises for promises. 98 | In most if not all strongly typed languages, this behavior makes it hard if not 99 | impossible to infer the type of a promise. 100 | A **future** is a promise’s strongly typed alter ego, which can take advantage 101 | of type inference to avoid having to explicitly cast the type of a promise. 102 | 103 | Promises, tasks, and futures are three variations of a singular reactive value. 104 | They vary by being either broadcast or unicast, or by being suitable for strong 105 | or weak type systems. 106 | 107 | 108 | ## Plural and temporal 109 | 110 | There are many plural reactive value types. 111 | Each has a different adaptation for dealing with limited resources. 112 | 113 | A **stream** has many of the same constraints as an array. 114 | Imagine a plane with space and time. 115 | If you rotate an array from the space axis to the time axis, it would become a 116 | stream. 117 | The order is important, and every value is significant. 118 | 119 | Consumers and producers are unlikely to process values at the same rate. 120 | If the consumer is faster than the producer, it must idle between receiving 121 | values. 122 | If a producer is faster than their corresponding consumer, it must idle between 123 | sending values. 124 | 125 | This pushing and pulling is captured by the concept of **pressure**. 126 | On the producer side, a vacuum stalls the consumer and a pressure sends values 127 | forward. 128 | On the consumer side, a vacuum draws values forward and pressure, often called 129 | **back pressure**, stalls the producer. 130 | Pressure exists to ensure that every value transits the setter to the getter. 131 | 132 | Since the consumer of a stream expects to see every value, streams are **unicast** 133 | like tasks. 134 | As they are unicast they are also cancelable. 135 | Streams are a cooperation between the reader and the writer and information 136 | flows both ways. 137 | Data flows forward, acknowledgements flow backward, and either the consumer or 138 | producer can terminate the flow. 139 | 140 | Although a stream is unicast, it is certainly possible to branch a stream into 141 | multiple streams in a variety of ways. 142 | A fork in a stream is an operator that ensures that every value gets sent to 143 | each of an array of consumers. 144 | The slowest of the forks determines the pressure, so the pressure of a fork can 145 | only be higher than that of a single consumer. 146 | The simpler strategy of providing a stream to multiple consumers produces a 147 | “round robin” load balancing effect, where each consumer receives an exclusive, 148 | possibly random, portion of the stream. 149 | The pressure of a shared stream can only be lower than that of a single 150 | consumer. 151 | 152 | In the following example, the `map` operator creates two new streams from a 153 | single input stream. 154 | The slow map will see half as many values as the fast map. 155 | The slow map will consume and produce five values per second, and the fast map 156 | will consume and produce ten, sustaining a maximum throughput of fifteen values 157 | per second if the original stream can produce values that quickly. 158 | If the original stream can only produce ten or less values per second, the 159 | values will be distributed fairly between both consumers. 160 | 161 | ```js 162 | var slow = stream.map(function (n) { 163 | return Promise.return(n).delay(200); 164 | }); 165 | var fast = stream.map(function (n) { 166 | return Promise.return(n).delay(100); 167 | }); 168 | ``` 169 | 170 | In contrast, **publishers** and **subscribers** are **broadcast**. 171 | Information flows only one direction, from the publishers to the subscribers. 172 | Also, there is no guarantee of continuity. 173 | The publisher does not wait for a subscriber and the subscriber receives 174 | whatever values were published during the period of their subscription. 175 | A stream would buffer all values produced until the consumer arrives. 176 | 177 | With *time series data*, values that change over time but belie 178 | the same meaning, order and integrity may not be important. 179 | For example, if you were bombarded with weather forecasts, you could discard 180 | every report except the one you most recently received. 181 | Alternately, consider a value that represents the current time. 182 | Since the current time is always changing, it would not be meaningful, much less 183 | possible, to respond every moment it changes. 184 | 185 | Time series data comes in two varieties: **discrete** and **continuous**. 186 | Discrete values should be **pushed** whereas continuous values should be 187 | **pulled** or **polled**. 188 | (If a homophone is a disaster, what are synonymous homophones?) 189 | 190 | The current time or temperature are examples of **continuous behaviors**. 191 | Animation frames and morse code are examples of **discrete signals**. 192 | 193 | Let us consider each primitive in detail. 194 | Since the temporal primitives have spatial analogies, and since some of these 195 | spatial primitives are relatively new to JavaScript, we will review these first. 196 | 197 | -------------------------------------------------------------------------------- /conclusion.md: -------------------------------------------------------------------------------- 1 | 2 | # Conclusion 3 | 4 | Reactive primitives can be categorized in multiple dimensions. 5 | The interfaces of analogous non-reactive constructs including getters, setters, 6 | and generators are insightful in the design of their asynchronous counterparts. 7 | Identifying whether a primitive is singular or plural also greatly informs the 8 | design. 9 | 10 | We can use pressure to deal with resource contention while guaranteeing 11 | consistency. 12 | We can alternately use push or poll strategies to skip irrelevant states for 13 | either continuous or discrete time series data with behaviors or signals. 14 | 15 | There is a tension between cancelability and robustness, but we have primitives 16 | that are useful for both cases. 17 | Streams and tasks are inherently cooperative, cancelable, and allow 18 | bidirectional information flow. 19 | Promises guarantee that consumers and producers cannot interfere. 20 | 21 | All of these concepts are related and their implementations benefit from mutual 22 | availability. 23 | Promises and tasks are great for single result data, but can provide a 24 | convenient channel for plural signals and behaviors. 25 | 26 | Bringing all of these reactive concepts into a single framework gives us an 27 | opportunity to tell a coherent story about reactive programming, promotes a 28 | better understanding about what tool is right for the job, and obviates the 29 | debate over whether any single primitive is a silver bullet. 30 | 31 | -------------------------------------------------------------------------------- /further.md: -------------------------------------------------------------------------------- 1 | ## Further Work 2 | 3 | There are many more topics that warrant discussion and I will expand upon these 4 | here. 5 | 6 | Reservoir sampling can be modeled as a behavior that watches a stream or signal 7 | and produces a representative sample on demand. 8 | 9 | A clock user interface is a good study in the interplay between behaviors, 10 | signals, time, and animation scheduling. 11 | 12 | Drawing from my experience at FastSoft, we exposed variables from the kernel's 13 | networking stack so we could monitor the bandwidth running through our TCP 14 | acceleration appliance. 15 | Some of those variables modeled the number of packets transmitted and the number 16 | of bytes transmitted. 17 | These counters would frequently overflow. 18 | There are several interesting ways to architect a solution that would provide 19 | historical data in multiple resolutions, accounting for the variable overflow, 20 | involving a combination of streams, behaviors, and signals. 21 | I should draw your attention to design aspects of RRDTool. 22 | 23 | An advantage of having a unified framework for reactive primitives is to create 24 | simple stories for passing one kind of primitive to another. 25 | Promises can be coerced to tasks, tasks to promises. 26 | A signal can be used as a behavior, and a behavior can be captured by a signal. 27 | Signals can be channeled into streams, and streams can be channeled into 28 | signals. 29 | 30 | It is worth exploring in detail how operators can be lifted in each of these 31 | value spaces. 32 | 33 | Implementing distributed sort using streams is also a worthy exercise. 34 | 35 | Asynchronous behaviors would benefit from an operator that solves the thundering 36 | herd problem, the inverse of throttling. 37 | 38 | How to implement type ahead suggestion is a great case to explore cancelable 39 | streams and tasks. 40 | 41 | I also need to discuss how these reactive concepts can propagate operational 42 | transforms through queries, using both push and pull styles, and how this 43 | relates to bindings, both synchronous and asynchronous. 44 | 45 | I also need to compare and contrast publishers and subscribers to the related 46 | concepts of signals and streams. 47 | In short, publishers and subscribers are broadcast, unlike unicast streams, 48 | but a single subscription could be modeled as a stream. 49 | However, a subscriber can typically not push back on a publisher, so how 50 | resource contention is alleviated is an open question. 51 | 52 | Related to publishing and subscribing, streams can certainly be forked, in which 53 | case both branches would put pressure back on the source. 54 | 55 | Streams also have methods that return tasks. 56 | All of these could propagate estimated time to completion. 57 | Each of the cases for `all`, `any`, `race`, and `read` are worth exploring. 58 | 59 | High performance buffers for bytewise data with the promise buffer interface 60 | require further exploration. 61 | 62 | Implementing a retry loop with promises and tasks is illustrative. 63 | 64 | Reactive Extensions (Rx) beg a distinction between [hot and cold][] observables, 65 | which is worth exploring. 66 | The clock reference implementation shows one way to implement a signal that can 67 | be active or inactive based on whether anyone is looking. 68 | 69 | [hot and cold]: https://github.com/Reactive-Extensions/RxJS/blob/master/doc/gettingstarted/creating.md#cold-vs-hot-observables 70 | 71 | The research into continuous behaviors and the original idea of Functional 72 | Reactive Programming by [Conal Elliott][] deserves attention. 73 | 74 | [Conal Elliott]: http://conal.net/ 75 | 76 | The interplay between promises and tasks with their underlying progress behavior 77 | and estimated time to completion and status signals require further explanation. 78 | These ideas need to be incorporated into the sketches of promise and task 79 | implementations. 80 | -------------------------------------------------------------------------------- /generator-functions.md: -------------------------------------------------------------------------------- 1 | # Generator Functions 2 | 3 | Consider the eager and lazy `range` function implementations. 4 | We lose a certain clarity when we convert the array range maker into an iterator 5 | range maker. 6 | Generator functions alleviate this problem by offering a way to express 7 | iterations procedurally, with a lazy behavior. 8 | 9 | A JavaScript engine near you may already support generator functions. 10 | The syntax consists of adding an asterisk to the function declaration and using 11 | `yield` to produce iterations. 12 | Calling a generator function does not execute the function, but instead sets up 13 | a state machine to track where we are in the function and returns an iterator. 14 | Whenever we ask the iterator for an iteration, the state machine will resume the 15 | execution of the function until it produces an iteration or terminates. 16 | 17 | ```js 18 | function *range(start, stop, step) { 19 | while (start < stop) { 20 | yield start; 21 | start += step; 22 | } 23 | } 24 | 25 | var iterator = range(0, Infinity, 1); 26 | expect(iterator.next().value).toBe(0); 27 | expect(iterator.next().value).toBe(1); 28 | expect(iterator.next().value).toBe(2); 29 | // ... 30 | ``` 31 | 32 | Notice that the range generator function restores and perhaps even exceeds the 33 | clarity of the range array maker. 34 | 35 | Calling `next` has three possible outcomes. 36 | If the iterator encounters a `yield`, the iteration will have a `value`. 37 | If the iterator runs the function to either an express or implied `return`, the 38 | iteration will have a `value` and `done` will be true. 39 | If the iterator runs to an explicit return, this terminal iteration carries the 40 | return value. 41 | If the generator function throws an error, this will propagate out of `next()`. 42 | 43 | Generators and iterators are **unicast**. 44 | The consumer expects to see every value from the producer. 45 | Since generators and iterators cooperate, information flows both forward as 46 | values, and backward as requests for more values. 47 | 48 | However, the consumer can send other information back to the producer. 49 | The `next` method, familiar from basic iterators, gains the ability to determine 50 | the value of the `yield` expression from which the generator resumes. 51 | As a trivial example, consider a generator that echoes whatever the consumer 52 | requests. 53 | 54 | ```js 55 | function *echo() { 56 | var message; 57 | while (true) { 58 | message = yield message; 59 | } 60 | } 61 | 62 | var iterator = echo(); 63 | expect(iterator.next().value).toBe(undefined); 64 | expect(iterator.next("Hello").value).toBe(undefined); 65 | expect(iterator.next("Goodbye").value).toBe("Hello"); 66 | expect(iterator.next().value).toBe("Goodbye"); 67 | expect(iterator.next().value).toBe(undefined); 68 | ``` 69 | 70 | We must prime the generator because it does not begin with a `yield`. 71 | We advance the state machine to the first `yield` and allow it to produce the 72 | initial, undefined message. 73 | We then populate the message variable with a value, receiving its former 74 | undefined content again. 75 | Then we begin to see the fruit of our labor as the values we previously sent 76 | backward come forward again. 77 | This foreshadows the ability of stream readers to push back on stream writers. 78 | 79 | Additionally, the iterator gains a `throw` method that allows the iterator to 80 | terminate the generator by causing the `yield` expression to raise the given 81 | error. 82 | The error will unravel the stack inside the generator. 83 | If the error unravels a try-catch-finally block, the catch block may handle the 84 | error, leaving the generator in a resumable state if the returned iteration is 85 | not `done`. 86 | If the error unravels all the way out of the generator, it will pass into the 87 | stack of the `throw` caller. 88 | 89 | The iterator also gains a `return` method that causes the generator to resume as 90 | if from a `return` statement, regardless of whether it actually paused at a 91 | `yield` expression. 92 | Like a thrown error, this unravels the stack, executing finally blocks, but not 93 | catch blocks. 94 | 95 | As such, like `next`, the `throw` and `return` methods may either return an 96 | iteration, done or not, or throw an error. 97 | This foreshadows the ability of a stream reader to prematurely stop a stream 98 | writer. 99 | 100 | ```js 101 | iterator.throw(new Error("Do not want!")); 102 | ``` 103 | 104 | Note that in Java, [iterators][Java Iterator] have a `hasNext()` method. 105 | This is not implementable for generators owing to the [Halting Problem][]. 106 | The iterator must try to get a value from the generator before the generator can 107 | conclude that it cannot produce another value. 108 | 109 | [Java Iterator]: http://docs.oracle.com/javase/7/docs/api/java/util/Iterator.html 110 | [Halting Problem]: http://en.wikipedia.org/wiki/Halting_problem 111 | 112 | -------------------------------------------------------------------------------- /generators.md: -------------------------------------------------------------------------------- 1 | # Generators 2 | 3 | There is no proposal for a standard generator, but for the sake of completeness, 4 | if an array iterator consumes an array, an array generator would lazily produce 5 | one. 6 | An array generator object would implement `yield` as a method with behavior 7 | analogous to the same keyword within a generator function. 8 | The `yield` method would add a value to the array. 9 | 10 | ```js 11 | var array = []; 12 | var generator = generate(array); 13 | generator.yield(10); 14 | generator.yield(20); 15 | generator.yield(30); 16 | expect(array).toEqual([10, 20, 30]); 17 | ``` 18 | 19 | 20 | Since ECMAScript 5, at Doug Crockford’s behest, JavaScript allows keywords to be 21 | used for property names, making this parallel between keywords and methods 22 | possible. 23 | A generator might also implement `return` and `throw` methods, but a meaningful 24 | implementation for an array generator is a stretch of the imagination. 25 | Although an array generator is of dubious utility, it foreshadows the interface 26 | of asynchronous generators, for which meaningful implementations of `return` and 27 | `throw` methods are easier to obtain, and go on to inform a sensible design for 28 | asynchronous generator functions. 29 | 30 | -------------------------------------------------------------------------------- /glossary.md: -------------------------------------------------------------------------------- 1 | 2 | # Glossary 3 | 4 | - accumulator 5 | - array 6 | - asynchronous 7 | - await 8 | - behavior 9 | - blocking 10 | - broadcast 11 | - buffer 12 | - cancelable 13 | - continuous 14 | - control 15 | - counter 16 | - deferred 17 | - discrete 18 | - flow gauge 19 | - future 20 | - gauge 21 | - getter 22 | - getter getter 23 | - iterable 24 | - iterator 25 | - multiple 26 | - non-blocking 27 | - observable 28 | - observer 29 | - operation 30 | - poll 31 | - pressure 32 | - probe 33 | - promise 34 | - publisher 35 | - pull 36 | - pulse 37 | - push 38 | - readable 39 | - result 40 | - retriable 41 | - sensor 42 | - setter 43 | - setter setter 44 | - signal 45 | - single 46 | - sink 47 | - spatial 48 | - stream 49 | - strobe 50 | - subscriber 51 | - synchronous 52 | - task 53 | - temporal 54 | - throttle 55 | - unicast 56 | - value 57 | - writable 58 | - yield 59 | 60 | -------------------------------------------------------------------------------- /intro.md: -------------------------------------------------------------------------------- 1 | 2 | # A General Theory of Reactivity 3 | 4 | *A work in progress.* 5 | 6 | In the context of a computer program, reactivity is the process of receiving 7 | external stimuli and propagating events. 8 | This is a rather broad definition that covers a wide variety of topics. 9 | The term is usually reserved for systems that respond in turns to sensors, 10 | schedules, and above all, problems that exist between the chair and keyboard. 11 | 12 | The field of reactivity is carved into plots ranging from "reactive programming" 13 | to the subtly distinct "*functional* reactive programming", with acrage set 14 | aside for "self adjusting computation" and with neighbors like "bindings" and 15 | "operational transforms". 16 | Adherents favor everything from "continuation passing style" to "promises", or 17 | the related concepts of "deferreds" and "futures". 18 | Other problems lend themselves to "observables", "signals", or "behaviors", and 19 | everyone agrees that "streams" are a good idea, but "publishers" and 20 | "subscribers" are distinct. 21 | 22 | In 1905, Einstein created a theory of special relativity that unified the 23 | concepts of space and time, and went on to incorporate gravity, to bring the 24 | three fundamentals of physical law into a single model. 25 | To a similar end, [various][Rx] minds in the field of reactivity have been 26 | converging on a model that unifies at least promises and observables. 27 | 28 | [Rx]: https://github.com/Reactive-Extensions/RxJS/blob/aaebfe8962cfa06a6c80908d079928ba5b800c66/doc/readme.md 29 | 30 | | | **Singular** | **Plural** | 31 | | :----------: | :------------------: | :---------------------: | 32 | | **Spatial** | Value | Iterable<Value> | 33 | | **Temporal** | Promise<Value> | Observable<Value> | 34 | 35 | However, this description fails to capture all of the varigated concepts of 36 | reactivity. 37 | Rather, Rx conflates all reactive primitives into a single Observable type that 38 | can perform any role. 39 | Just as an array is an exemplar of an entire taxonomy of collections, promises, 40 | streams, and observables are merely representatives of their class of reactive 41 | primitives. 42 | As the common paraphrase of Einstein goes, everything should be made as simple 43 | as possible, but no simpler. 44 | 45 | -------------------------------------------------------------------------------- /iteration.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | // ## Iteration 4 | // 5 | // Various methods of both synchronous and asynchronous iterators and 6 | // generators transport iterations, which represent either a yielded value of 7 | // an ongoing sequence or the return value of a sequence that has terminated. 8 | // While object literals are sufficient to capture iterations, the iteration 9 | // constructor is handy for readability and allows V8 at least to use a hidden 10 | // class for all instances. 11 | 12 | module.exports = Iteration; 13 | function Iteration(value, done, index) { 14 | this.value = value; 15 | this.done = done; 16 | this.index = index; 17 | } 18 | 19 | // The Collections library operators, and therefore the Jasminum expectation 20 | // operators, defer to the `equals` method of any object that implements it. 21 | Iteration.prototype.equals = function (that, equals, memo) { 22 | if (!that) return false; 23 | return ( 24 | equals(this.value, that.value, equals, memo) && 25 | this.index === that.index && 26 | this.done === that.done 27 | ); 28 | 29 | }; 30 | 31 | // The `done` iteration singleton suffices for many cases where a terminal 32 | // iteration does not need to carry a return value. 33 | // This singleton exists only to avoid unnecessarily allocating a new iteration 34 | // for each of these cases. 35 | Iteration.done = new Iteration(undefined, true, undefined); 36 | 37 | -------------------------------------------------------------------------------- /iterator.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | module.exports = Iterator; 4 | 5 | var WeakMap = require("collections/weak-map"); 6 | var GenericCollection = require("collections/generic-collection"); 7 | var Iteration = require("./iteration"); 8 | 9 | // upgrades an iterable to a Iterator 10 | function Iterator(iterable, start, stop, step) { 11 | if (!iterable) { 12 | return Iterator.empty; 13 | } else if (iterable instanceof Iterator) { 14 | return iterable; 15 | } else if (!(this instanceof Iterator)) { 16 | return new Iterator(iterable, start, stop, step); 17 | } else if (Array.isArray(iterable) || typeof iterable === "string") { 18 | handlers.set(this, new IndexIterator(iterable, start, stop, step)); 19 | return; 20 | } 21 | iterable = Object(iterable); 22 | if (iterable.next) { 23 | handlers.set(this, iterable); 24 | } else if (iterable.iterate) { 25 | handlers.set(this, iterable.iterate(start, stop, step)); 26 | } else if (Object.prototype.toString.call(iterable) === "[object Function]") { 27 | this.next = iterable; 28 | } else if (Object.getPrototypeOf(iterable) === Object.prototype) { 29 | handlers.set(this, new ObjectIterator(iterable)); 30 | } else { 31 | throw new TypeError("Can't iterate " + iterable); 32 | } 33 | } 34 | 35 | Iterator.probe = function (callback, thisp) { 36 | return new Iterator(new Probe(callback, thisp)); 37 | }; 38 | 39 | function Probe(callback, thisp) { 40 | this.callback = callback; 41 | this.thisp = thisp; 42 | } 43 | 44 | Probe.prototype.next = function (value, index) { 45 | return this.callback.call(this.thisp, value, index); 46 | }; 47 | 48 | // Using handlers as a hidden table associating a full-fledged Iterator with 49 | // an underlying, usually merely "nextable", iterator. 50 | var handlers = new WeakMap(); 51 | 52 | // Selectively apply generic methods of GenericCollection 53 | Iterator.prototype.forEach = GenericCollection.prototype.forEach; 54 | Iterator.prototype.map = GenericCollection.prototype.map; 55 | Iterator.prototype.filter = GenericCollection.prototype.filter; 56 | Iterator.prototype.every = GenericCollection.prototype.every; 57 | Iterator.prototype.some = GenericCollection.prototype.some; 58 | Iterator.prototype.min = GenericCollection.prototype.min; 59 | Iterator.prototype.max = GenericCollection.prototype.max; 60 | Iterator.prototype.sum = GenericCollection.prototype.sum; 61 | Iterator.prototype.average = GenericCollection.prototype.average; 62 | Iterator.prototype.flatten = GenericCollection.prototype.flatten; 63 | Iterator.prototype.zip = GenericCollection.prototype.zip; 64 | Iterator.prototype.enumerate = GenericCollection.prototype.enumerate; 65 | Iterator.prototype.sorted = GenericCollection.prototype.sorted; 66 | Iterator.prototype.group = GenericCollection.prototype.group; 67 | Iterator.prototype.reversed = GenericCollection.prototype.reversed; 68 | Iterator.prototype.toArray = GenericCollection.prototype.toArray; 69 | Iterator.prototype.toObject = GenericCollection.prototype.toObject; 70 | 71 | // This is a bit of a cheat so flatten and such work with the generic reducible 72 | Iterator.prototype.constructClone = function (values) { 73 | var clone = []; 74 | clone.addEach(values); 75 | return clone; 76 | }; 77 | 78 | // A level of indirection so a full-interface iterator can proxy for a simple 79 | // nextable iterator. 80 | Iterator.prototype.next = function (value, index) { 81 | var nextable = handlers.get(this); 82 | if (nextable) { 83 | return nextable.next(value, index); 84 | } else { 85 | return Iteration.done; 86 | } 87 | }; 88 | 89 | Iterator.prototype.iterateMap = function (callback /*, thisp*/) { 90 | var self = Iterator(this), 91 | thisp = arguments[1]; 92 | return new MapIterator(self, callback, thisp); 93 | }; 94 | 95 | function MapIterator(iterator, callback, thisp) { 96 | this.iterator = iterator; 97 | this.callback = callback; 98 | this.thisp = thisp; 99 | } 100 | 101 | MapIterator.prototype = Object.create(Iterator.prototype); 102 | MapIterator.prototype.constructor = MapIterator; 103 | 104 | MapIterator.prototype.next = function (value, next) { 105 | var iteration = this.iterator.next(value, next); 106 | if (iteration.done) { 107 | return iteration; 108 | } else { 109 | return new Iteration( 110 | this.callback.call( 111 | this.thisp, 112 | iteration.value, 113 | iteration.index, 114 | this.iteration 115 | ), 116 | false, 117 | iteration.index 118 | ); 119 | } 120 | }; 121 | 122 | Iterator.prototype.iterateFilter = function (callback /*, thisp*/) { 123 | var self = Iterator(this), 124 | thisp = arguments[1], 125 | index = 0; 126 | 127 | return new FilterIterator(self, callback, thisp); 128 | }; 129 | 130 | function FilterIterator(iterator, callback, thisp) { 131 | this.iterator = iterator; 132 | this.callback = callback; 133 | this.thisp = thisp; 134 | } 135 | 136 | FilterIterator.prototype = Object.create(Iterator.prototype); 137 | FilterIterator.prototype.constructor = FilterIterator; 138 | 139 | FilterIterator.prototype.next = function () { 140 | var iteration; 141 | while (true) { 142 | iteration = this.iterator.next(); 143 | if (iteration.done || this.callback.call( 144 | this.thisp, 145 | iteration.value, 146 | iteration.index, 147 | this.iteration 148 | )) { 149 | return iteration; 150 | } 151 | } 152 | }; 153 | 154 | Iterator.prototype.reduce = function (callback /*, initial, thisp*/) { 155 | var self = Iterator(this), 156 | result = arguments[1], 157 | thisp = arguments[2], 158 | iteration; 159 | 160 | // First iteration unrolled 161 | iteration = self.next(); 162 | if (iteration.done) { 163 | if (arguments.length > 1) { 164 | return arguments[1]; 165 | } else { 166 | throw TypeError("Reduce of empty iterator with no initial value"); 167 | } 168 | } else if (arguments.length > 1) { 169 | result = callback.call( 170 | thisp, 171 | result, 172 | iteration.value, 173 | iteration.index, 174 | self 175 | ); 176 | } else { 177 | result = iteration.value; 178 | } 179 | 180 | // Remaining entries 181 | while (true) { 182 | iteration = self.next(); 183 | if (iteration.done) { 184 | return result; 185 | } else { 186 | result = callback.call( 187 | thisp, 188 | result, 189 | iteration.value, 190 | iteration.index, 191 | self 192 | ); 193 | } 194 | } 195 | }; 196 | 197 | Iterator.prototype.dropWhile = function (callback /*, thisp */) { 198 | var self = Iterator(this), 199 | thisp = arguments[1], 200 | iteration; 201 | 202 | while (true) { 203 | iteration = self.next(); 204 | if (iteration.done) { 205 | return Iterator.empty; 206 | } else if (!callback.call(thisp, iteration.value, iteration.index, self)) { 207 | return new DropWhileIterator(iteration, self); 208 | } 209 | } 210 | }; 211 | 212 | function DropWhileIterator(iteration, iterator) { 213 | this.iteration = iteration; 214 | this.iterator = iterator; 215 | this.parent = null; 216 | } 217 | 218 | DropWhileIterator.prototype = Object.create(Iterator.prototype); 219 | DropWhileIterator.prototype.constructor = DropWhileIterator; 220 | 221 | DropWhileIterator.prototype.next = function () { 222 | var result = this.iteration; 223 | if (result) { 224 | this.iteration = null; 225 | return result; 226 | } else { 227 | return this.iterator.next(); 228 | } 229 | }; 230 | 231 | Iterator.prototype.takeWhile = function (callback /*, thisp*/) { 232 | var self = Iterator(this), 233 | thisp = arguments[1]; 234 | return new TakeWhileIterator(self, callback, thisp); 235 | }; 236 | 237 | function TakeWhileIterator(iterator, callback, thisp) { 238 | this.iterator = iterator; 239 | this.callback = callback; 240 | this.thisp = thisp; 241 | } 242 | 243 | TakeWhileIterator.prototype = Object.create(Iterator.prototype); 244 | TakeWhileIterator.prototype.constructor = TakeWhileIterator; 245 | 246 | TakeWhileIterator.prototype.next = function () { 247 | var iteration = this.iterator.next(); 248 | if (iteration.done) { 249 | return iteration; 250 | } else if (this.callback.call( 251 | this.thisp, 252 | iteration.value, 253 | iteration.index, 254 | this.iterator 255 | )) { 256 | return iteration; 257 | } else { 258 | return Iteration.done; 259 | } 260 | }; 261 | 262 | Iterator.prototype.iterateZip = function () { 263 | return Iterator.unzip(Array.prototype.concat.apply(this, arguments)); 264 | }; 265 | 266 | Iterator.prototype.iterateUnzip = function () { 267 | return Iterator.unzip(this); 268 | }; 269 | 270 | Iterator.prototype.iterateEnumerate = function (start) { 271 | return Iterator.count(start).iterateZip(this); 272 | }; 273 | 274 | Iterator.prototype.iterateConcat = function () { 275 | return Iterator.flatten(Array.prototype.concat.apply(this, arguments)); 276 | }; 277 | 278 | Iterator.prototype.iterateFlatten = function () { 279 | return Iterator.flatten(this); 280 | }; 281 | 282 | Iterator.prototype.recount = function (start) { 283 | return new RecountIterator(this, start); 284 | }; 285 | 286 | function RecountIterator(iterator, start) { 287 | this.iterator = iterator; 288 | this.index = start || 0; 289 | } 290 | 291 | RecountIterator.prototype = Object.create(Iterator.prototype); 292 | RecountIterator.prototype.constructor = RecountIterator; 293 | 294 | RecountIterator.prototype.next = function () { 295 | var iteration = this.iterator.next(); 296 | if (iteration.done) { 297 | return iteration; 298 | } else { 299 | return new Iteration( 300 | iteration.value, 301 | false, 302 | this.index++ 303 | ); 304 | } 305 | }; 306 | 307 | // creates an iterator for Array and String 308 | function IndexIterator(iterable, start, stop, step) { 309 | if (step == null) { 310 | step = 1; 311 | } 312 | if (stop == null) { 313 | stop = start; 314 | start = 0; 315 | } 316 | if (start == null) { 317 | start = 0; 318 | } 319 | if (step == null) { 320 | step = 1; 321 | } 322 | if (stop == null) { 323 | stop = iterable.length; 324 | } 325 | this.iterable = iterable; 326 | this.start = start; 327 | this.stop = stop; 328 | this.step = step; 329 | } 330 | 331 | IndexIterator.prototype.next = function () { 332 | // Advance to next owned entry 333 | if (typeof this.iterable === "object") { // as opposed to string 334 | while (!(this.start in this.iterable)) { 335 | if (this.start >= this.stop) { 336 | return Iteration.done; 337 | } else { 338 | this.start += this.step; 339 | } 340 | } 341 | } 342 | if (this.start >= this.stop) { // end of string 343 | return Iteration.done; 344 | } 345 | var iteration = new Iteration( 346 | this.iterable[this.start], 347 | false, 348 | this.start 349 | ); 350 | this.start += this.step; 351 | return iteration; 352 | }; 353 | 354 | function ObjectIterator(object) { 355 | this.object = object; 356 | this.iterator = new Iterator(Object.keys(object)); 357 | } 358 | 359 | ObjectIterator.prototype.next = function () { 360 | var iteration = this.iterator.next(); 361 | if (iteration.done) { 362 | return iteration; 363 | } else { 364 | var key = iteration.value; 365 | return new Iteration(this.object[key], false, key); 366 | } 367 | }; 368 | 369 | Iterator.cycle = function (cycle, times) { 370 | if (arguments.length < 2) { 371 | times = Infinity; 372 | } 373 | return new CycleIterator(cycle, times); 374 | }; 375 | 376 | function CycleIterator(cycle, times) { 377 | this.cycle = cycle; 378 | this.times = times; 379 | this.iterator = Iterator.empty; 380 | } 381 | 382 | CycleIterator.prototype = Object.create(Iterator.prototype); 383 | CycleIterator.prototype.constructor = CycleIterator; 384 | 385 | CycleIterator.prototype.next = function () { 386 | var iteration = this.iterator.next(); 387 | if (iteration.done) { 388 | if (this.times > 0) { 389 | this.times--; 390 | this.iterator = new Iterator(this.cycle); 391 | return this.iterator.next(); 392 | } else { 393 | return iteration; 394 | } 395 | } else { 396 | return iteration; 397 | } 398 | }; 399 | 400 | Iterator.concat = function (/* ...iterators */) { 401 | return Iterator.flatten(Array.prototype.slice.call(arguments)); 402 | }; 403 | 404 | Iterator.flatten = function (iterators) { 405 | iterators = Iterator(iterators); 406 | return new ChainIterator(iterators); 407 | }; 408 | 409 | function ChainIterator(iterators) { 410 | this.iterators = iterators; 411 | this.iterator = Iterator.empty; 412 | } 413 | 414 | ChainIterator.prototype = Object.create(Iterator.prototype); 415 | ChainIterator.prototype.constructor = ChainIterator; 416 | 417 | ChainIterator.prototype.next = function () { 418 | var iteration = this.iterator.next(); 419 | if (iteration.done) { 420 | var iteratorIteration = this.iterators.next(); 421 | if (iteratorIteration.done) { 422 | return Iteration.done; 423 | } else { 424 | this.iterator = new Iterator(iteratorIteration.value); 425 | return this.iterator.next(); 426 | } 427 | } else { 428 | return iteration; 429 | } 430 | }; 431 | 432 | Iterator.unzip = function (iterators) { 433 | iterators = Iterator(iterators).map(Iterator); 434 | if (iterators.length === 0) 435 | return new Iterator.empty; 436 | return new UnzipIterator(iterators); 437 | }; 438 | 439 | function UnzipIterator(iterators) { 440 | this.iterators = iterators; 441 | this.index = 0; 442 | } 443 | 444 | UnzipIterator.prototype = Object.create(Iterator.prototype); 445 | UnzipIterator.prototype.constructor = UnzipIterator; 446 | 447 | UnzipIterator.prototype.next = function () { 448 | var done = false 449 | var result = this.iterators.map(function (iterator) { 450 | var iteration = iterator.next(); 451 | if (iteration.done) { 452 | done = true; 453 | } else { 454 | return iteration.value; 455 | } 456 | }); 457 | if (done) { 458 | return Iteration.done; 459 | } else { 460 | return new Iteration(result, false, this.index++); 461 | } 462 | }; 463 | 464 | Iterator.zip = function () { 465 | return Iterator.unzip(Array.prototype.slice.call(arguments)); 466 | }; 467 | 468 | Iterator.range = function (start, stop, step) { 469 | if (arguments.length < 3) { 470 | step = 1; 471 | } 472 | if (arguments.length < 2) { 473 | stop = start; 474 | start = 0; 475 | } 476 | start = start || 0; 477 | step = step || 1; 478 | return new RangeIterator(start, stop, step); 479 | }; 480 | 481 | Iterator.count = function (start, step) { 482 | return Iterator.range(start, Infinity, step); 483 | }; 484 | 485 | function RangeIterator(start, stop, step) { 486 | this.start = start; 487 | this.stop = stop; 488 | this.step = step; 489 | this.index = 0; 490 | } 491 | 492 | RangeIterator.prototype = Object.create(Iterator.prototype); 493 | RangeIterator.prototype.constructor = RangeIterator; 494 | 495 | RangeIterator.prototype.next = function () { 496 | if (this.start >= this.stop) { 497 | return Iteration.done; 498 | } else { 499 | var result = this.start; 500 | this.start += this.step; 501 | return new Iteration(result, false, this.index++); 502 | } 503 | }; 504 | 505 | Iterator.repeat = function (value, times) { 506 | if (times == null) { 507 | times = Infinity; 508 | } 509 | return new RepeatIterator(value, times); 510 | }; 511 | 512 | function RepeatIterator(value, times) { 513 | this.value = value; 514 | this.times = times; 515 | this.index = 0; 516 | } 517 | 518 | RepeatIterator.prototype = Object.create(Iterator.prototype); 519 | RepeatIterator.prototype.constructor = RepeatIterator; 520 | 521 | RepeatIterator.prototype.next = function () { 522 | if (this.index < this.times) { 523 | return new Iteration(this.value, false, this.index++); 524 | } else { 525 | return Iteration.done; 526 | } 527 | }; 528 | 529 | Iterator.enumerate = function (values, start) { 530 | return Iterator.count(start).iterateZip(new Iterator(values)); 531 | }; 532 | 533 | function EmptyIterator() {} 534 | 535 | EmptyIterator.prototype = Object.create(Iterator.prototype); 536 | EmptyIterator.prototype.constructor = EmptyIterator; 537 | 538 | EmptyIterator.prototype.next = function () { 539 | return Iteration.done; 540 | }; 541 | 542 | Iterator.empty = new EmptyIterator(); 543 | 544 | -------------------------------------------------------------------------------- /iterators.md: -------------------------------------------------------------------------------- 1 | 2 | # Iterators 3 | 4 | An iterator is an object that allows us to lazily but synchronously consume 5 | multiple values. 6 | Iterators are not new to JavaScript, but there is a new standard forming at time 7 | of writing. 8 | 9 | Iterators implement a `next()` method that returns an object that may have a 10 | `value` property, and may have a `done` property. 11 | Although the standard does not give this object a name, we will call it an 12 | **iteration**. 13 | If the iterator has produced the entirety of a sequence, the `done` property of 14 | the iteration will be `true`. 15 | Generator functions return iterators that expand on this basic definition. 16 | The `value` of a non-final iteration corresponds to a `yield` expression and the 17 | `value` of a `done` iteration corresponds to a `return` expression. 18 | 19 | Iterators are an interface with many implementations. 20 | The canonical iterator yields the values from an array. 21 | 22 | ```js 23 | var iterator = iterate([1, 2, 3]); 24 | var iteration = iterator.next(); 25 | expect(iteration.value).toBe(1); 26 | iteration = iterator.next(); 27 | expect(iteration.value).toBe(2); 28 | iteration = iterator.next(); 29 | expect(iteration.value).toBe(3); 30 | iteration = iterator.next(); 31 | expect(iteration.done).toBe(true); 32 | ``` 33 | 34 | What distinguishes an iterator from an array is that it is **lazy**. 35 | An iterator does not necessarily end. 36 | We can have iterators for non-terminating sequences, like counting or the 37 | fibonacci sequence. 38 | The `range` function produces a sequence of values within an interval and 39 | separated by a stride. 40 | 41 | ```js 42 | function range(start, stop, step) { 43 | return {next: function () { 44 | var iteration; 45 | if (start < stop) { 46 | iteration = {value: start}; 47 | start += step; 48 | } else { 49 | iteration = {done: true}; 50 | } 51 | return iteration; 52 | }}; 53 | } 54 | ``` 55 | 56 | If the `stop` value of the range is `Infinity`, the iterator will have no end, 57 | and will never produce a `done` iteration. 58 | Unlike an array, an indefinite iterator consumes no more memory than an empty 59 | one. 60 | 61 | ```js 62 | var iterator = range(0, Infinity, 1); 63 | expect(iterator.next().value).toBe(0); 64 | expect(iterator.next().value).toBe(1); 65 | expect(iterator.next().value).toBe(2); 66 | // ... 67 | ``` 68 | 69 | The **eager** equivalent would produce an array, but would only work for bounded 70 | intervals since it must create an exhaustive collection in memory before 71 | returning. 72 | 73 | ```js 74 | function range(start, stop, step) { 75 | var result = []; 76 | while (start < stop) { 77 | result.push(start); 78 | start += step; 79 | } 80 | return result; 81 | } 82 | 83 | expect(range(0, 6, 2)).toEqual([0, 2, 4]); 84 | ``` 85 | 86 | Iterators may have alternate implementations of some methods familiar from 87 | arrays. 88 | For example, `forEach` would walk the iterator until it is exhausted. 89 | `map` would produce a new iterator of values passed through some transform, 90 | while `filter` would produce a new iterator of the values that pass a test. 91 | An iterator can support `reduce`, which would exhaust the iteration, but 92 | `reduceRight` would be less sensible since iterators only walk forward. 93 | Iterators may also have some methods that are unique to their character, like 94 | `dropWhile` and `takeWhile`. 95 | 96 | We can save time and space by implementing pipelines with iterators instead of 97 | arrays. 98 | The following example can be interpreted as either eager or lazy, depending on 99 | whether `range` returns an array or an iterator. 100 | If we start with an array, `map` will create another array of 1000 values and 101 | `filter` will create another large array. 102 | If we start with an iterator, we will never construct an array of any size, 103 | instead percolating one value at a time as the reducer pulls them from the 104 | filter, as the filter pulls them from the mapping, and as the mapping pulls them 105 | from the range. 106 | 107 | ```js 108 | range(0, 1000, 1) 109 | .map(function (n) { 110 | return n * 2; 111 | }) 112 | .filter(function (n) { 113 | return n % 3 !== 0; 114 | }) 115 | .reduce(function (a, b) { 116 | return a + b; 117 | }) 118 | ``` 119 | 120 | -------------------------------------------------------------------------------- /observable.js: -------------------------------------------------------------------------------- 1 | 2 | // An observable is a **lossy** **push** representation of a value that varies 3 | // at **discrete** and observable moments in time. 4 | 5 | "use strict"; 6 | 7 | var asap = require("asap"); 8 | var WeakMap = require("weak-map"); 9 | require("collections/shim-array"); 10 | var Observer = require("./observer"); 11 | var Operators = require("./operators"); 12 | var Iteration = require("./iteration"); 13 | 14 | // ## Observable 15 | 16 | // Like promises, observables use the [revealing constructor pattern][Revealing 17 | // Constructor]. 18 | // 19 | // [Revealing Constructor]: http://domenic.me/2014/02/13/the-revealing-constructor-pattern/ 20 | // 21 | // An observable has a corresponding emitter with a `yield` method. 22 | // The constructor reveals the `yield` method as an argument to the setup function. 23 | 24 | module.exports = Observable; 25 | function Observable(setup) { 26 | var signal = Observable.signal(); 27 | setup(signal.in.yield); 28 | return signal.out; 29 | } 30 | 31 | // The `signal` constructor method is analogous to the `Promise.defer()` method 32 | // and returns an `{in, out}` pair consisting of a tangled emitter and 33 | // observable. 34 | Observable.signal = function (value, index) { 35 | var handler = new SignalHandler(value, index); 36 | var emitter = new Emitter(); 37 | var observable = Object.create(Observable.prototype); 38 | handlers.set(emitter, handler); 39 | handlers.set(observable, handler); 40 | return {in: emitter, out: observable}; 41 | }; 42 | 43 | // The `yield` constructor method returns an observable that will forever yield 44 | // the given value. 45 | Observable.yield = function (value, index) { 46 | return new Observable(function (_yield) { 47 | _yield(value, index); 48 | }); 49 | }; 50 | 51 | // The `next` method provides the portion of the interface necessary to mimick 52 | // an `Iterator`, and will always produce the last yielded iteration. 53 | // Unlike a stream, the `next` method does not return a promise for an iteration. 54 | Observable.prototype.next = function () { 55 | var handler = handlers.get(this); 56 | return new handler.Iteration(handler.value, false, handler.index); 57 | }; 58 | 59 | // `forEach` registers an observer for the signal and returns the observer. 60 | // An observer can be cancelled. 61 | Observable.prototype.forEach = function (callback, thisp) { 62 | var handler = handlers.get(this); 63 | var observers = handler.observers; 64 | var observer = new Observer(callback, thisp, handler); 65 | handler.addObserver(observer); 66 | return observer; 67 | }; 68 | 69 | // `map` produces a new signal that yields the return value of the given 70 | // callback for each value in from this signal. 71 | Observable.prototype.map = function (callback, thisp) { 72 | var signal = Observable.signal(); 73 | this.forEach(function (value, index) { 74 | signal.in.yield(callback.call(thisp, value, index, this), index); 75 | }, this); 76 | return signal.out; 77 | }; 78 | 79 | // `filter` produces a signal that yields the values from this signal if they 80 | // pass a test. 81 | Observable.prototype.filter = function (callback, thisp) { 82 | var signal = Observable.signal(); 83 | this.forEach(function (value, index) { 84 | if (callback.call(thisp, value, index, this)) { 85 | signal.in.yield(value, index); 86 | } 87 | }, this); 88 | return signal.out; 89 | }; 90 | 91 | // `reduce` produces a signal that yields the most recently accumulated value 92 | // by combining each of this signals values with the aggregate of all previous. 93 | // Note that unlike the array reducer, the basis is mandatory. 94 | Observable.prototype.reduce = function (callback, basis, thisp) { 95 | var signal = Observable.signal(); 96 | this.forEach(function (value, index) { 97 | basis = callback.call(thisp, basis, value, index, this); 98 | signal.in.yield(basis, index); 99 | }, this); 100 | return signal.out; 101 | }; 102 | 103 | // The `thenYield` method ransforms this signal into a pulse. 104 | // Each time this signal produces a value, the returned signal will yield the 105 | // given value. 106 | // The name is intended to parallel the `thenReturn` and `thenThrow` methods of 107 | // tasks and promises. 108 | Observable.prototype.thenYield = function (value) { 109 | return this.map(function () { 110 | return value; 111 | }); 112 | }; 113 | 114 | // The `count` method transforms this signal into a pulse counter. 115 | // For each value that this signal produces, the returned signal will produce 116 | // the count of values seen so far. 117 | Observable.prototype.count = function (count, increment) { 118 | var signal = Observable.signal(); 119 | count = count || 0; 120 | this.forEach(function (_, index) { 121 | count = (increment ? increment(count) : count + 1); 122 | signal.in.yield(count, index); 123 | }); 124 | return signal.out; 125 | }; 126 | 127 | // The `lift` constructor method lifts an operator from value space into signal 128 | // space, such that instead of accepting and returning values, it instead 129 | // accepts and returns signals. 130 | /* TODO alter this method so that it can accept a mix of behaviors and signals */ 131 | Observable.lift = function (operator, thisp) { 132 | return function signalOperator() { 133 | var operandSignals = Array.prototype.slice.call(arguments); 134 | var operands = new Array(operandSignals.length); 135 | var defined = new Array(operandSignals.length); 136 | var pending = operandSignals.length; 137 | var signal = Observable.signal(); 138 | operandSignals.forEach(function (operandSignal, index) { 139 | operandSignal.forEach(function (operand, time) { 140 | if (operand == null || operand !== operand) { 141 | if (defined[index]) { 142 | defined[index] = false; 143 | pending++; 144 | } 145 | operands[index] = operand; 146 | } else { 147 | operands[index] = operand; 148 | if (!defined[index]) { 149 | defined[index] = true; 150 | pending--; 151 | } 152 | if (!pending) { 153 | signal.in.yield(operator.apply(thisp, operands), time); 154 | } 155 | } 156 | }); 157 | }); 158 | return signal.out; 159 | }; 160 | } 161 | 162 | // For each operato in the `Operators` module, we produce both a constructor 163 | // and a prototype method with the corresponding operator or method in signal space. 164 | for (var name in Operators) { 165 | (function (operator, name) { 166 | Observable[name] = Observable.lift(operator, Operators); 167 | Observable.prototype[name] = function (that) { 168 | return Observable[name](this, that); 169 | }; 170 | })(Operators[name], name); 171 | } 172 | 173 | // ## SignalHandler 174 | // 175 | // The observable and generator sides of a signal share private state on a 176 | // signal handler hidden record. 177 | // We use a weak map to track the corresponding handler for each generator and 178 | // observable. 179 | 180 | var handlers = new WeakMap(); 181 | 182 | function SignalHandler(value, index) { 183 | this.observers = []; 184 | this.value = value; 185 | this.index = index; 186 | this.active = false; 187 | } 188 | 189 | SignalHandler.prototype.Iteration = Iteration; 190 | 191 | // The generator side uses the `yield` method to set the current value of the 192 | // signal for a given time index and to arrange for an update to all observers. 193 | // Note that we track observers in reverse order to take advantage of a small 194 | // optimization afforded by countdown loops. 195 | SignalHandler.prototype.yield = function (value, index) { 196 | this.value = value; 197 | this.index = index; 198 | if (!this.active) { 199 | return; 200 | } 201 | var observers = this.observers; 202 | var length = observers.length; 203 | var observerIndex = observers.length; 204 | while (observerIndex--) { 205 | observers[observerIndex].yield(value, index); 206 | } 207 | }; 208 | 209 | /* TODO yieldEach to mirror yield* syntax of generators, possibly using handler 210 | * trickery. */ 211 | 212 | // The observable side of the signal uses `addObserver` and `cancelObserver`. 213 | 214 | // The `addObserver` method will implicitly dispatch an initial value if the signal 215 | // has been initialized and has already captured a meaningful value. 216 | SignalHandler.prototype.addObserver = function (observer) { 217 | this.observers.unshift(observer); 218 | if (this.active && Operators.defined(this.value)) { 219 | observer.yield(this.value, this.index); 220 | } 221 | // If this is the first observer, we may need to activate the signal. 222 | asap(this); 223 | }; 224 | 225 | SignalHandler.prototype.cancelObserver = function (observer) { 226 | var index = this.observers.indexOf(observer); 227 | if (index < 0) { 228 | return; 229 | } 230 | this.observers.swap(index, 1); 231 | // If this was the last remaining observer, we may need to deactivate the 232 | // signal. 233 | asap(this); 234 | }; 235 | 236 | // The add and cancel observer methods both use asap to arrange for a possible 237 | // signal state change, between active and inactive, in a separate event. 238 | // Derrived signal handlers, for example the `ClockHandler`, may implement 239 | // `onstart` and `onstop` event handlers. 240 | SignalHandler.prototype.call = function () { 241 | if (!this.active) { 242 | if (this.observers.length) { 243 | if (this.onstart) { 244 | this.onstart(); 245 | } 246 | this.active = true; 247 | if (Operators.defined(this.value)) { 248 | this.yield(this.value, this.index); 249 | } 250 | } 251 | } else { 252 | if (!this.observers.length) { 253 | if (this.onstop) { 254 | this.onstop(); 255 | } 256 | this.active = false; 257 | } 258 | } 259 | }; 260 | 261 | // ## Emitter 262 | // 263 | // A producer should receive a reference to the generator side of a signal. 264 | // It hosts the methods needed to change the value captured by a signal and 265 | // propagate change notifications. 266 | 267 | function Emitter() { 268 | this.yield = this.yield.bind(this); 269 | } 270 | 271 | // The `yield` method updates the value for a given time index and radiates a 272 | // change notification to any registered observers. 273 | Emitter.prototype.yield = function (value, index) { 274 | var handler = handlers.get(this); 275 | handler.yield(value, index); 276 | }; 277 | 278 | // The `inc` method assumes that the signal captures an integer and increments 279 | // that value by one. 280 | Emitter.prototype.inc = function (index) { 281 | var handler = handlers.get(this); 282 | this.yield(handler.value + 1, index); 283 | }; 284 | 285 | // The `dec` method assumes that the signal captures an integer and decrements 286 | // that value by one. 287 | Emitter.prototype.dec = function (index) { 288 | var handler = handlers.get(this); 289 | this.yield(handler.value - 1, index); 290 | }; 291 | 292 | -------------------------------------------------------------------------------- /observables.md: -------------------------------------------------------------------------------- 1 | 2 | # Observables 3 | 4 | There is more than one way to solve the problem of processor contention or 5 | process over-scheduling. 6 | Streams have a very specific contract that makes pressurization necessary. 7 | Specifically, a stream is intended to transport the entirety of a collection and 8 | strongly resembles a spatial collection that has been rotated 90 degrees onto 9 | the temporal axis. 10 | However, there are other contracts that lead us to very different strategies to 11 | avoid over-commitment and they depend entirely on the meaning of the data in 12 | transit. 13 | The appropriate transport is domain specific. 14 | 15 | Consider a sensor, like a thermometer or thermocouple. 16 | At any given time, the subject will have a particular temperature. 17 | The temperature may change continuously in response to events that are not 18 | systematically observable. 19 | Suppose that you poll the thermocouple at one second intervals and place that on 20 | some plural, asynchronous setter. 21 | Suppose that this ultimately gets consumed by a visualization that polls the 22 | corresponding plural, asynchronous getter sixty times per second. 23 | The visualization is only interested in the most recently sampled value from the 24 | sensor. 25 | 26 | Consider a variable like the position of a scrollbar. 27 | The value is discrete. 28 | It does not change continuously. 29 | Rather, it changes only in response to an observable event. 30 | Each time one of these scroll events occurs, we place the position on the 31 | setter side of some temporal collection. 32 | Any number of consumers can subscribe to the getter side and it will push a 33 | notification their way. 34 | 35 | However, if we infer a smooth animation from the discrete scroll positions and 36 | their times, we can sample the scroll position *function* on each animation 37 | frame. 38 | 39 | These cases are distinct from streams and have interesting relationships with 40 | each other. 41 | With the temperature sensor, changes are **continuous**, whereas with the scroll 42 | position observer, the changes are **discrete**. 43 | With the temperature sensor, we sample the data at a much lower frequency than 44 | the display, in which case it is sufficient to remember the last sensed 45 | temperature and redisplay it. 46 | If we were to sample the data at a higher frequency than the display, it would 47 | be sufficient for the transport to forget old values each time it receives a new 48 | one. 49 | Also, unlike a stream, these cases are both well adapted for multiple-producer 50 | and multiple-consumer scenarios. 51 | 52 | Also unlike streams, one of these concepts pushes data and the other polls or 53 | pulls data. 54 | A stream has pressure, which is a kind of combined pushing and pulling. 55 | Data is pulled toward the consumer by a vacuum. 56 | Producers are pushed back by pressure when the vacuum is filled, thus the term: 57 | back-pressure. 58 | 59 | The discrete event pusher is a Signal. 60 | The continuous, pollable is a Behavior. 61 | 62 | 63 | | Interface | | | 64 | | ------------------ | --------------| ---- | 65 | | Signal Observable | Get | Push | 66 | | Signal Generator | Set | Push | 67 | | Signal | Value | Push | 68 | | Behavior Iterator | Get | Poll | 69 | | Behavior Generator | Set | Poll | 70 | | Behavior | Value | Poll | 71 | 72 | 73 | - TODO make sure this is a summary of the topics in the end: 74 | 75 | Yet even behaviors have variations like probes, gauges, counters, 76 | flow gauges, accumulators, flushable accumulators, and rotating counters. 77 | 78 | -------------------------------------------------------------------------------- /observer.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var asap = require("asap"); 4 | 5 | module.exports = Observer; 6 | function Observer(callback, thisp, signal) { 7 | this.callback = callback; 8 | this.thisp = thisp; 9 | this.signal = signal; 10 | this.value = null; 11 | this.index = null; 12 | this.pending = false; 13 | } 14 | 15 | Observer.prototype.yield = function (value, index) { 16 | this.value = value; 17 | this.index = index; 18 | this.done = false; 19 | if (!this.pending) { 20 | this.pending = true; 21 | asap(this); 22 | } 23 | }; 24 | 25 | Observer.prototype.call = function () { 26 | if (this.pending && !this.cancelled) { 27 | this.pending = false; 28 | this.callback.call(this.thisp, this.value, this.index, this.signal); 29 | } 30 | }; 31 | 32 | Observer.prototype.cancel = function () { 33 | this.signal.cancelObserver(this); 34 | this.cancelled = true; 35 | }; 36 | 37 | -------------------------------------------------------------------------------- /operators.js: -------------------------------------------------------------------------------- 1 | // The operators module provides named function objects corresponding to 2 | // language operators. 3 | 4 | "use strict"; 5 | 6 | // The equals and compare operators provided by the Collections package allow 7 | // deep comparison of arbitrary values and delegate to the eponymous methods of 8 | // instances if they are defined. 9 | 10 | require("collections/shim-object"); 11 | 12 | exports.equals = Object.equals; 13 | 14 | exports.compare = Object.compare; 15 | 16 | exports.not = function (x) { return !x }; 17 | 18 | exports.and = function (x, y) { return x && y }; 19 | 20 | exports.or = function (x, y) { return x || y }; 21 | 22 | exports.add = function (x, y) { 23 | return x + y; 24 | }; 25 | 26 | exports.sub = function (x, y) { 27 | return x - y; 28 | }; 29 | 30 | exports.div = function (x, y) { 31 | return x / y; 32 | }; 33 | 34 | exports.mul = function (x, y) { 35 | return x * y; 36 | }; 37 | 38 | exports.tuple = function () { 39 | return Array.prototype.slice.call(arguments); 40 | }; 41 | 42 | // Behaviors and signals will propagate undefined if any operand is not 43 | // defined. 44 | 45 | exports.defined = function (value) { 46 | // !NaN && !null && !undefined 47 | return value === value && value != null; 48 | }; 49 | 50 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "gtor", 3 | "version": "0.0.1", 4 | "description": "A General Theory of Reactivity", 5 | "author": "Kris Kowal", 6 | "license": "MIT", 7 | "dependencies": { 8 | "asap": "^1.0.0", 9 | "collections": "^2.0.1", 10 | "jasminum": "^2.0.5", 11 | "weak-map": "^1.0.5" 12 | }, 13 | "devDependencies": { 14 | "docco": "^0.6.3", 15 | "git-node-fs": "^0.1.0", 16 | "js-git": "creationix/js-git", 17 | "q-git": "^0.0.2", 18 | "q-io": "^2.0.6" 19 | }, 20 | "scripts": { 21 | "test": "jasminum test", 22 | "publish": "rm -rf docs; npm run docco; node scripts/publish.js", 23 | "docco": "docco promise.js promise-queue.js task.js stream.js behavior.js observable.js iteration.js operators.js clock.js byte-stream.js" 24 | } 25 | } 26 | -------------------------------------------------------------------------------- /presentation/build.bash: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | WW=$(inkscape -W "$PWD/gtor-master.svg") 4 | HH=$(inkscape -H "$PWD/gtor-master.svg") 5 | 6 | for NAME in perspective-value duals-value perspective-collection duals-iterator duals-deferred duals-generator-function duals-generator duals-generator-observer perspective-promise promise-order-independence promise-queue-order-independence promise-queue-temporal-independence promise-queue-iteration-transport duals-buffer duals-promsie-queue stream-using-queues perspective-stream 7 | do 8 | X=$(inkscape -X -I $NAME "$PWD/gtor-master.svg") 9 | Y=$(inkscape -Y -I $NAME "$PWD/gtor-master.svg") 10 | W=$(inkscape -W -I $NAME "$PWD/gtor-master.svg") 11 | H=$(inkscape -H -I $NAME "$PWD/gtor-master.svg") 12 | X0=$(echo "$X" | bc -l) 13 | Y0=$(echo "$HH - $Y" | bc -l) 14 | X1=$(echo "$X + $W" | bc -l) 15 | Y1=$(echo "$HH - $Y - $H" | bc -l) 16 | inkscape -z -e $PWD/$NAME.png -a "$X0:$Y0:$X1:$Y1" -w 600 $PWD/gtor-master.svg 2>/dev/null 17 | done 18 | -------------------------------------------------------------------------------- /presentation/duals-buffer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-buffer.png -------------------------------------------------------------------------------- /presentation/duals-deferred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-deferred.png -------------------------------------------------------------------------------- /presentation/duals-generator-function.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-generator-function.png -------------------------------------------------------------------------------- /presentation/duals-generator-observer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-generator-observer.png -------------------------------------------------------------------------------- /presentation/duals-generator.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-generator.png -------------------------------------------------------------------------------- /presentation/duals-iterator.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-iterator.png -------------------------------------------------------------------------------- /presentation/duals-promise-queue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-promise-queue.png -------------------------------------------------------------------------------- /presentation/duals-value.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/duals-value.png -------------------------------------------------------------------------------- /presentation/everything-is-array.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/everything-is-array.jpg -------------------------------------------------------------------------------- /presentation/everything-is-stream.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/everything-is-stream.png -------------------------------------------------------------------------------- /presentation/fork.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/fork.gif -------------------------------------------------------------------------------- /presentation/map-reduce.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/map-reduce.gif -------------------------------------------------------------------------------- /presentation/map.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/map.gif -------------------------------------------------------------------------------- /presentation/perspective-collection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/perspective-collection.png -------------------------------------------------------------------------------- /presentation/perspective-promise.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/perspective-promise.png -------------------------------------------------------------------------------- /presentation/perspective-stream.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/perspective-stream.png -------------------------------------------------------------------------------- /presentation/perspective-value.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/perspective-value.png -------------------------------------------------------------------------------- /presentation/promise-order-independence.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/promise-order-independence.png -------------------------------------------------------------------------------- /presentation/promise-queue-iteration-transport.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/promise-queue-iteration-transport.png -------------------------------------------------------------------------------- /presentation/promise-queue-order-independence.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/promise-queue-order-independence.png -------------------------------------------------------------------------------- /presentation/promise-queue-temporal-independence.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/promise-queue-temporal-independence.png -------------------------------------------------------------------------------- /presentation/reduce.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/reduce.gif -------------------------------------------------------------------------------- /presentation/share.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/share.gif -------------------------------------------------------------------------------- /presentation/stream-using-queues.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kriskowal/gtor/af3fb6880c12153c43811c1f75e67bc4e284b7c7/presentation/stream-using-queues.png -------------------------------------------------------------------------------- /primitives.md: -------------------------------------------------------------------------------- 1 | 2 | # Primitives 3 | 4 | Let us consider each primitive in detail. 5 | Since the temporal primitives have spatial analogies, and since some of these 6 | spatial primitives are relatively new to JavaScript, we will review these first. 7 | 8 | -------------------------------------------------------------------------------- /progress.md: -------------------------------------------------------------------------------- 1 | 2 | # Progress and estimated time to completion 3 | 4 | Imagine you are copying the values from a stream into an array. 5 | You know how long the array will be and when you started reading. 6 | Knowing these variables and assuming that the rate of flow is steady, you can 7 | infer the amount of **progress** that has been made up to the current time. 8 | This is a simple matter of dividing the number of values you have so far 9 | received, by the total number of values you expect to receive. 10 | 11 | ```js 12 | var progress = index / length; 13 | ``` 14 | 15 | This is a discrete measurement that you can push each time you receive another 16 | value. 17 | It is discrete because it does not change between events. 18 | 19 | You can also infer the average **throughput** of the stream, also a discrete 20 | time series. 21 | 22 | ```js 23 | var elapsed = now - start; 24 | var throughput = index / elapsed; 25 | ``` 26 | 27 | From progress you can divine an **estimated time of completion**. 28 | This will be the time you started plus the time you expect the whole stream to 29 | take. 30 | 31 | ```js 32 | var stop = start + elapsed / progress; 33 | var stop = start + elapsed / (index / length); 34 | var stop = start + elapsed * length / index; 35 | ``` 36 | 37 | We could update a progress bar whenever we receive a new value, but frequently 38 | we would want to display a smooth animation continuously changing. 39 | Ideally, progress would proceed linearly from 0 at the start time to 1 at the 40 | stop time. 41 | You could sample progress at any moment in time and receive a different value. 42 | Values that lack an inherent resolution are *continuous*. 43 | It becomes the responsibility of the consumer to determine when to sample, 44 | **pull** or **poll** the value. 45 | 46 | For the purposes of a smooth animation of a continuous behavior, the frame rate 47 | is a sensible polling frequency. 48 | We can infer a continuous progress time series from the last known estimated time 49 | of completion. 50 | 51 | ```js 52 | var progress = (now - start) / (estimate - start); 53 | ``` 54 | 55 | -------------------------------------------------------------------------------- /promise-generator.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var WeakMap = require("weak-map"); 4 | var Promise = require("./promise"); 5 | 6 | var handlers = new WeakMap(); 7 | "use strict"; 8 | 9 | module.exports = require("./promise-buffer").prototype.PromiseBuffer; 10 | 11 | -------------------------------------------------------------------------------- /promise-iterator.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | module.exports = require("./promise-buffer").prototype.PromiseIterator; 4 | 5 | -------------------------------------------------------------------------------- /promise-machine.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var PromiseBuffer = require("./promise-buffer"); 4 | 5 | module.exports = PromiseMachine; 6 | function PromiseMachine(callback, thisp, limit) { 7 | var input = new PromiseBuffer(); 8 | var output = new PromiseBuffer(); 9 | this.in = input.in; 10 | this.out = output.out; 11 | this.done = input.out.map(callback, thisp).forEach(output.in.yield, output.in, limit); 12 | } 13 | 14 | -------------------------------------------------------------------------------- /promise-queue.js: -------------------------------------------------------------------------------- 1 | 2 | // A promise queue is an asynchronous linked list, representing a sequence of 3 | // values over time. 4 | // Consuming and producing that sequence are temporaly independent. 5 | // For each respective promise and resolution, the promise may be gotten first 6 | // and put later, or put first and gotten later. 7 | 8 | // This implementation comes from Mark Miller's [Concurrency Strawman][] for 9 | // ECMAScript. 10 | // 11 | // [Concurrency Strawman]: http://wiki.ecmascript.org/doku.php?id=strawman:concurrency 12 | 13 | "use strict"; 14 | 15 | var Promise = require("./promise"); 16 | 17 | // ## PromiseQueue 18 | 19 | // The promise queue constructor returns an entangled `get` and `put` pair. 20 | // These methods may be passed as functions, granting either the capability to 21 | // give or take but not necessarily both. 22 | 23 | // Internally, a promise queue is an asynchronous linked list of deferreds. 24 | // The `ends` variable is a `promise` and `resolver` pair. 25 | // The `promise` is a promise for the next `ends` pair after this promise is 26 | // taken. 27 | // The `resolver` advances the `ends` pair after a resolution is given. 28 | // The `promise` and `resolver` are independent properties, not necessarily 29 | // corresponding to the same deferred. 30 | 31 | module.exports = PromiseQueue; 32 | function PromiseQueue(values) { 33 | if (!(this instanceof PromiseQueue)) { 34 | return new PromiseQueue(); 35 | } 36 | var self = this; 37 | 38 | var ends = Promise.defer(); 39 | 40 | // The `resolver` side of a promise queue adds a `{head, tail}` node to the 41 | // asynchronous linked list. 42 | // The `put` method creates a new link to the resolver side with the given 43 | // `head` value, and advances the `resolver` side of the list. 44 | this.put = function (value) { 45 | var next = Promise.defer(); 46 | ends.resolver.return({ 47 | head: value, 48 | tail: next.promise 49 | }); 50 | ends.resolver = next.resolver; 51 | }; 52 | 53 | // The `promise` end of a promise queue is a promise for a `{head, tail}` 54 | // pair. 55 | // The `head` will be the next value, and the `tail` will be a promise for 56 | // the remaining nodes of the list. 57 | // The `get` method obtains and returns a promise for the `head` value and 58 | // advances the `promise` to become the `tail`. 59 | this.get = function () { 60 | var result = ends.promise.get("head"); 61 | ends.promise = ends.promise.get("tail"); 62 | return result; 63 | }; 64 | 65 | // The promise queue constructor allows the queue to be initialized with 66 | // a given population of values. 67 | if (values) { 68 | values.forEach(this.put, this); 69 | } 70 | } 71 | 72 | -------------------------------------------------------------------------------- /remote-iterators.md: -------------------------------------------------------------------------------- 1 | 2 | # Remote iterators 3 | 4 | Consider also that a reader may be a proxy for a remote reader. 5 | A promise iterator be easily backed by a promise for a remote object. 6 | 7 | ```js 8 | function RemotePromiseIterator(promise) { 9 | this.remoteIteratorPromise = promise.invoke("iterate"); 10 | } 11 | RemotePromiseIterator.prototype.next = function (value) { 12 | return this.remoteIteratorPromise.invoke("next"); 13 | }; 14 | 15 | var remoteReader = remoteFilesystem.invoke("open", "primes.txt"); 16 | var reader = new RemotePromiseIterator(remoteReader); 17 | reader.forEach(console.log); 18 | ``` 19 | 20 | Apart from `then` and `done`, promises provide methods like `get`, `call`, and 21 | `invoke` to allow promises to be created from promises and for messages to be 22 | pipelined to remote objects. 23 | An `iterate` method should be a part of that protocol to allow values to be 24 | streamed on demand over any message channel. 25 | 26 | -------------------------------------------------------------------------------- /scripts/generate.js: -------------------------------------------------------------------------------- 1 | 2 | var fs = require("q-io/fs"); 3 | 4 | module.exports = generate; 5 | function generate(gitFs) { 6 | return gitFs.makeTree("docs") 7 | .then(function () { 8 | return fs.copyTree(fs.join(__dirname, "..", "docs"), "docs", gitFs); 9 | }); 10 | } 11 | 12 | -------------------------------------------------------------------------------- /scripts/publish.js: -------------------------------------------------------------------------------- 1 | 2 | var path = require("path"); 3 | var GitFs = require("q-git/fs"); 4 | var generate = require("./generate"); 5 | var fs = require("q-io/fs"); 6 | 7 | var repo = {}; 8 | repo.rootPath = path.join(__dirname, "..", ".git"); 9 | require("git-node-fs/mixins/fs-db")(repo, repo.rootPath); 10 | require('js-git/mixins/create-tree')(repo); 11 | require('js-git/mixins/pack-ops')(repo); 12 | require('js-git/mixins/walkers')(repo); 13 | require('js-git/mixins/read-combiner')(repo); 14 | require('js-git/mixins/formats')(repo); 15 | 16 | var gitFs = new GitFs(repo); 17 | 18 | gitFs.load("refs/remotes/origin/gh-pages") 19 | .then(function () { 20 | return gitFs.removeTree("/"); 21 | }) 22 | .then(function () { 23 | return generate(gitFs); 24 | }) 25 | .then(function () { 26 | return gitFs.commit({ 27 | author: {name: "Git Bot", email: "kris@cixar.com"}, 28 | message: "Build" 29 | }) 30 | }) 31 | .then(function () { 32 | return gitFs.saveAs("refs/heads/gh-pages"); 33 | }) 34 | .done(); 35 | 36 | -------------------------------------------------------------------------------- /semaphores.md: -------------------------------------------------------------------------------- 1 | 2 | # Semaphores 3 | 4 | Semaphores are flags or signs used for communication and were precursors to 5 | telegraphs and traffic lights. 6 | In programming, semaphores are usually used to synchronize programs that share 7 | resources, where only one process can use a resource at one time. 8 | For example, if a process has a pool of four database connections, it would use 9 | a semaphore to manage that pool. 10 | 11 | Typically, semaphores are used to block a thread or process from continuing 12 | until a resource becomes available. 13 | The process will "down" the semaphore whenever it enters a region where it needs 14 | a resource, and will "up" the semaphore whenever it exits that region. 15 | The terminology goes back to raising and lowering flags. 16 | You can imagine your process as being a train and a semaphore as guarding the 17 | entrance to a particular length of track. 18 | Your process stops at the gate until the semaphore goes up. 19 | 20 | Of course, in a reactive program, we don’t block. 21 | Instead of blocking, we return promises and continue when a promise resolves. 22 | We can use a promise as a non-blocking mutex for a single resource, and a 23 | promise queue as a non-blocking semaphore for multiple resources. 24 | 25 | In this example, we establish three database connections that are shared by a 26 | function that can be called to do some work with the first available connection. 27 | We get the resource, do our work, and regardless of whether we succeed or fail, 28 | we put the resource back in the pool. 29 | 30 | ```js 31 | var connections = new Queue(); 32 | connections.put(connectToDb()); 33 | connections.put(connectToDb()); 34 | connections.put(connectToDb()); 35 | 36 | function work() { 37 | return connections.get() 38 | .then(function (db) { 39 | return workWithDb(db) 40 | .finally(function () { 41 | connections.put(db); 42 | }) 43 | }); 44 | } 45 | ``` 46 | 47 | -------------------------------------------------------------------------------- /signals.md: -------------------------------------------------------------------------------- 1 | 2 | # Observables and Signals 3 | 4 | A **signal** represents a value that changes over time. 5 | The signal is asynchronous and plural, like a stream. 6 | Unlike a stream, a signal can have multiple producers and consumers. 7 | The output side of a signal is an **observable**. 8 | 9 | A signal has a getter side and a setter side. 10 | The asynchronous getter for a signal is an observable instead of a reader. 11 | The observable implements `forEach`, which subscribes an observer to receive 12 | push notifications whenever the signal value changes. 13 | 14 | ```js 15 | signal.out.forEach(function (value, time, signal) { 16 | console.log(value); 17 | }) 18 | ``` 19 | 20 | The signal generator is the asynchronous setter. 21 | Like a stream writer, it implements `yield`. 22 | However, unlike a stream writer, `yield` does not return a promise. 23 | 24 | ```js 25 | signal.in.yield(10); 26 | ``` 27 | 28 | Signals do not support pressure. 29 | Just as `yield` does not return a promise, the callback you give to `forEach` 30 | does not accept a promise. 31 | A signal can only push. 32 | The consumer (or consumers) cannot push back. 33 | 34 | Observables *also* implement `next`, which returns an iteration that captures 35 | the most recently dispatched value. 36 | This allows us to poll a signal as if it were a behavior. 37 | 38 | See the accompanying sketch of a [observable][] implementation. 39 | 40 | [observable]: http://kriskowal.github.io/gtor/docs/observable 41 | 42 | Just as streams relate to buffers, not every observable must be paired with a 43 | signal generator. 44 | A noteworthy example of an external observable is a clock. 45 | A clock emits a signal with the current time at a regular period and offset. 46 | 47 | ```js 48 | var tick = new Clock({period: 1000}); 49 | var tock = new Clock({period: 1000, offset: 500}); 50 | tick.forEach(function (time) { 51 | console.log("tick", time); 52 | }) 53 | tock.forEach(function (time) { 54 | console.log("tock", time); 55 | }); 56 | ``` 57 | 58 | See the accompanying sketch of a [clock][] implementation. 59 | 60 | [clock]: http://kriskowal.github.io/gtor/docs/clock 61 | 62 | Signals may correspond to system or platform signals like keyboard or mouse 63 | input or other external sensors. 64 | Furthermore, a signal generator might dispatch a system level signal to another 65 | process, for example SIGHUP, which typically asks a daemon to reload its 66 | configuration. 67 | 68 | ```js 69 | daemon.signals.yield("SIGHUP"); 70 | ``` 71 | 72 | -------------------------------------------------------------------------------- /stream.js: -------------------------------------------------------------------------------- 1 | 2 | // A stream represents either end of a buffer that transports values 3 | // asynchronously in either direction. 4 | // By convention, values are transported in one direction, and acknowledgements 5 | // are returned. 6 | // 7 | // A stream is a promise iterator and a promise generator. 8 | // All of the kernel methods, `yield` or `next`, `return`, and `throw`, 9 | // both send and receive promises for iterations. 10 | // 11 | // Promise streams borrow the jargon of iterators and generators but each 12 | // method is equivalent to a conventional stream method name. 13 | // 14 | // - `yield` is akin to `write`. 15 | // - `next` is akin to `read`. 16 | // - `yield` and `next` are interchangeable. The argument is written and the 17 | // return value is a promise for what will be read. 18 | // - `return` is akin to `close`. 19 | // - `throw` is akin to `abort`, `cancel`, or `destroy`. 20 | // 21 | // A stream is **unicast**, in the sense that it is a cooperation between a 22 | // single producer and consumer, mediated by the buffer to control the 23 | // throughput of both sides. 24 | // 25 | // Since a stream is unicast, it is also **cancelable**. 26 | // Either side of a connection can terminate the other. 27 | 28 | "use strict"; 29 | 30 | var Task = require("./task"); 31 | var Promise = require("./promise"); 32 | var Observable = require("./observable"); 33 | var PromiseQueue = require("./promise-queue"); 34 | var Iterator = require("./iterator"); 35 | var Iteration = require("./iteration"); 36 | var WeakMap = require("weak-map"); 37 | 38 | // Every stream has a private dual, the opposite end of the stream. 39 | // For the input, there is the output; for the output, there is the input. 40 | var duals = new WeakMap(); 41 | // Every stream has a private promise queue for transporting iterations. 42 | // The stream uses its own queue to receive iterations from its dual, and uses 43 | // the dual's queue to send iterations to its dual. 44 | var queues = new WeakMap(); 45 | 46 | // ## Stream 47 | // 48 | // Like promises, streams use the [revealing constructor pattern][Revealing 49 | // Constructor]. 50 | // 51 | // [Revealing Constructor]: http://domenic.me/2014/02/13/the-revealing-constructor-pattern/ 52 | // 53 | // However, unlike promises, streams are symmetric and support bidirectional 54 | // communication. 55 | // By convention, the stream constructor creates an output stream and reveals 56 | // the methods of the input stream as arguments to a setup function. 57 | 58 | module.exports = Stream; 59 | function Stream(setup, length) { 60 | var buffer = Stream.buffer(length); 61 | setup(buffer.in.yield, buffer.in.return, buffer.in.throw); 62 | return buffer.out; 63 | } 64 | 65 | // The `buffer` constructor method of a stream creates a tangled pair of 66 | // streams, dubbed `in` and `out`. 67 | // 68 | // The `buffer` method is analogous to `Promise.defer`. 69 | 70 | Stream.buffer = function (length) { 71 | var outgoing = new PromiseQueue(); // syn 72 | var incoming = new PromiseQueue(); // ack 73 | var input = Object.create(Stream.prototype); 74 | var output = Object.create(Stream.prototype); 75 | duals.set(input, output); 76 | duals.set(output, input); 77 | queues.set(input, incoming); 78 | queues.set(output, outgoing); 79 | Stream_bind(input); 80 | Stream_bind(output); 81 | // If the user provides a buffer length, we prime the incoming message 82 | // queue (pre-acknowledgements) with that many iterations. 83 | // This allows the producer to stay this far ahead of the consumer. 84 | for (; length > 0; length--) { 85 | incoming.put(new Iteration()); 86 | } 87 | // By contrast, if the buffer has a negative length, we prime the outgoing 88 | // message queue (data) with that many undefined iterations. 89 | // This gives some undefined values to the consumer, allowing it to proceed 90 | // before the producer has provided any iterations. 91 | for (; length < 0; length++) { 92 | outgoing.put(new Iteration()); 93 | } 94 | return {in: input, out: output}; 95 | }; 96 | 97 | // The `from` method creates a stream from an iterable or a promise iterable. 98 | Stream.from = function (iterable) { 99 | var stream = Object.create(Stream.prototype); 100 | var iterator = new Iterator(iterable); 101 | stream.yield = function (value, index) { 102 | return Promise.return(iterator.next(value, index)); 103 | }; 104 | Stream_bind(stream); 105 | return stream; 106 | }; 107 | 108 | // The kernel methods of a stream are bound to the stream so they can be passed 109 | // as free variables. 110 | // Particularly, the methods of an input stream are revealed to the setup 111 | // function of an output stream's constructor. 112 | function Stream_bind(stream) { 113 | stream.next = stream.next.bind(stream); 114 | stream.yield = stream.yield.bind(stream); 115 | stream.return = stream.return.bind(stream); 116 | stream.throw = stream.throw.bind(stream); 117 | } 118 | 119 | Stream.prototype.Iteration = Iteration; 120 | 121 | // ### Kernel Methods 122 | 123 | // The `next` and `yield` methods are equivalent. 124 | // By convention, `next` is used to consume, and `yield` to produce, 125 | // but both methods have the same signature and behavior. 126 | // They return a promise for the next iteration from the other side of the 127 | // connection, and send an iteration with the given value to the other. 128 | 129 | Stream.prototype.next = function (value, index) { 130 | return this.yield(value, index); 131 | }; 132 | 133 | Stream.prototype.yield = function (value, index) { 134 | var dual = duals.get(this); 135 | var incoming = queues.get(this); 136 | var outgoing = queues.get(dual); 137 | outgoing.put(new this.Iteration(value, false, index)); 138 | return incoming.get(); 139 | }; 140 | 141 | // The `return` method sends a final iteration to the other side of a stream, 142 | // which by convention terminates communication in this direction normally. 143 | 144 | Stream.prototype.return = function (value) { 145 | var dual = duals.get(this); 146 | var incoming = queues.get(this); 147 | var outgoing = queues.get(dual); 148 | outgoing.put(new this.Iteration(value, true)); 149 | return incoming.get(); 150 | }; 151 | 152 | // The `throw` method sends an error to the other side of the stream, 153 | // in an attempt to break communication in this direction, and, unless the 154 | // other side handles the exception, the error should bounce back. 155 | 156 | Stream.prototype.throw = function (error) { 157 | var dual = duals.get(this); 158 | var incoming = queues.get(this); 159 | var outgoing = queues.get(dual); 160 | outgoing.put(Promise.throw(error)); 161 | return incoming.get(); 162 | }; 163 | 164 | // ### do 165 | 166 | // The `do` method is a utility for `forEach` and `map`, responsible for 167 | // setting up an appropriate semaphore for the concurrency limit. 168 | 169 | Stream.prototype.do = function (callback, errback, limit) { 170 | var next; 171 | // If there is no concurrency limit, we are free to batch up as many jobs 172 | // as the producer can create. 173 | if (limit == null) { 174 | next = function () { 175 | return this.next() 176 | .then(function (iteration) { 177 | // Before even beginning the job, we start waiting for another 178 | // value. 179 | if (!iteration.done) { 180 | next.call(this); 181 | } 182 | return callback(iteration); 183 | }, null, this) 184 | }; 185 | } else { 186 | // If there is a concurrency limit, we will use a promise queue as a 187 | // semaphore. We will enqueue a value representing a resource 188 | // (undefined) for each concurrent task. 189 | var semaphore = new PromiseQueue(); 190 | while (limit--) { 191 | semaphore.put(); 192 | } 193 | next = function () { 194 | // Whenever a resource is available from the queue, we will start 195 | // another job. 196 | return semaphore.get() 197 | .then(function (resource) { 198 | // Each job begins with waiting for a value from the iterator. 199 | return this.next() 200 | .then(function (iteration) { 201 | // Once we have begun a job, we can begin waiting for 202 | // another job. 203 | // A resource may already be available on the queue. 204 | if (!iteration.done) { 205 | next.call(this); 206 | } 207 | // We pass the iteration forward to the callback, as 208 | // defined by either `forEach` or `map`, to handle the 209 | // iteration appropriately. 210 | return Promise.try(callback, null, iteration) 211 | .finally(function () { 212 | // And when the job is complete, we will put a resource 213 | // back on the semaphore queue, allowing another job to 214 | // start. 215 | semaphore.put(resource); 216 | }) 217 | }, null, this); 218 | }, null, this) 219 | .done(null, errback); 220 | } 221 | } 222 | next.call(this); 223 | }; 224 | 225 | // ### pipe 226 | 227 | // Copies all output of this stream to the input of the given stream, including 228 | // the completion or any thrown errors. 229 | // Some might call this `subscribe`. 230 | 231 | Stream.prototype.pipe = function (stream) { 232 | // The default concurrency for `forEach` limit is 1, making it execute 233 | // serially. 234 | // We will use signals to track the number of outstanding jobs and whether 235 | // we have seen the last iteration. 236 | var count = Observable.signal(0); 237 | var done = Observable.signal(false); 238 | // We will capture the return value in scope. 239 | var returnValue; 240 | // Using the do utility function to limit concurrency and give us 241 | // iterations, or prematurely terminate, in which case we forward the error 242 | // to the result task. 243 | this.do(function (iteration) { 244 | // If this was the last iteration, capture the return value and 245 | // dispatch the done signal. 246 | if (iteration.done) { 247 | returnValue = iteration.value; 248 | done.in.yield(true); 249 | } else { 250 | // Otherwise, we start a new job. 251 | // Incrementing the number of outstanding jobs. 252 | count.in.inc(); 253 | // Kick off the job, passing the callback argument pattern familiar 254 | // to users of arrays, but allowing the task to return a promise to 255 | // push back on the producer. 256 | return Promise.try(callback, thisp, iteration.value, iteration.index) 257 | .then(function (value) { 258 | return stream.yield(value); 259 | }) 260 | .finally(function () { 261 | // And then decrementing the outstanding job counter, 262 | // regardless of whether the job succeeded. 263 | count.in.dec(); 264 | }) 265 | } 266 | }, stream.throw, limit); 267 | // We have not completed the task until all outstanding jobs have completed 268 | // and no more iterations are available. 269 | count.out.equals(Observable.yield(0)).and(done.out).forEach(function (done) { 270 | if (done) { 271 | stream.return(returnValue); 272 | } 273 | }); 274 | return stream; 275 | }; 276 | 277 | 278 | // ### forEach 279 | 280 | // The `forEach` method will execute jobs, typically in serial, and returns a 281 | // cancelable promise (`Task`) for the completion of all jobs. 282 | // The default concurrency limit is 1, making `forEach` as serial as it is for 283 | // arrays, but can be expanded by passing a number in the third argument 284 | // position. 285 | 286 | Stream.prototype.forEach = function (callback, thisp, limit) { 287 | // We create a task for the result. 288 | var result = Task.defer(function (error) { 289 | // If the task is canceled, we will propagate the error back to the 290 | // generator. 291 | this.throw(error); 292 | }, this); 293 | // The default concurrency for `forEach` limit is 1, making it execute 294 | // serially. 295 | // For other operators, `map` and `filter`, there is no inherent 296 | // parallelism limit. 297 | if (limit == null) { limit = 1; } 298 | // We will use signals to track the number of outstanding jobs and whether 299 | // we have seen the last iteration. 300 | var count = Observable.signal(0); 301 | var done = Observable.signal(false); 302 | // We will capture the return value in scope. 303 | var returnValue; 304 | // Using the do utility function to limit concurrency and give us 305 | // iterations, or prematurely terminate, in which case we forward the error 306 | // to the result task. 307 | this.do(function (iteration) { 308 | // If this was the last iteration, capture the return value and 309 | // dispatch the done signal. 310 | if (iteration.done) { 311 | returnValue = iteration.value; 312 | done.in.yield(true); 313 | } else { 314 | // Otherwise, we start a new job. 315 | // Incrementing the number of outstanding jobs. 316 | count.in.inc(); 317 | // Kick off the job, passing the callback argument pattern familiar 318 | // to users of arrays, but allowing the task to return a promise to 319 | // push back on the producer. 320 | return Promise.try(callback, thisp, iteration.value, iteration.index) 321 | .finally(function () { 322 | // And then decrementing the outstanding job counter, 323 | // regardless of whether the job succeeded. 324 | count.in.dec(); 325 | }) 326 | } 327 | }, result.in.throw, limit); 328 | // We have not completed the task until all outstanding jobs have completed 329 | // and no more iterations are available. 330 | count.out.equals(Observable.yield(0)).and(done.out).forEach(function (done) { 331 | if (done) { 332 | result.in.return(returnValue); 333 | } 334 | }); 335 | return result.out; 336 | }; 337 | 338 | // ### map 339 | 340 | // The `map` method runs jobs in parallel, taking values from this iterator and 341 | // sending them to the returned promise iterator. 342 | // There is no default limit to concurrency, but you can pass a number. 343 | // Also, the order in which values pass from the input to the output is 344 | // determined by how quickly the jobs are processed. 345 | // However, the index of the input iterations propagates to the output 346 | // iterations. 347 | // A concurrency limit of 1 will ensure that order is preserved. 348 | 349 | Stream.prototype.map = function (callback, thisp, limit) { 350 | // We use our own constructor so subtypes can alter behavior. 351 | var result = new this.constructor.buffer(); 352 | // As with `forEach`, we track the number of outstanding jobs and whether 353 | // we have seen the last iteration. 354 | var count = Observable.signal(0); 355 | var done = Observable.signal(false); 356 | // And we will capture the return value here to pass it along to the result 357 | // stream. 358 | var returnValue; 359 | this.do(function (iteration) { 360 | // If this is the last iteration, track the return value and dispatch 361 | // the done signal. 362 | if (iteration.done) { 363 | returnValue = iteration.value; 364 | done.in.yield(true); 365 | } else { 366 | // Otherwise, start another job, first incrementing the outstanding 367 | // job counter so the result stream can't terminate until we are 368 | // done. 369 | count.in.inc(); 370 | // Then pass the familiar argument pattern for map callbacks, 371 | // except allowing the job to return a promise for the result. 372 | return Promise.try(callback, thisp, iteration.value, iteration.index) 373 | .then(function (value) { 374 | // We forward the result to the output iterator, preserving its 375 | // index if not its order. 376 | return result.in.yield(value, iteration.index); 377 | }) 378 | .finally(function () { 379 | // Regardless of whether the job succeeds or fails, we drop the 380 | // outstanding job count so the stream has an opportunity to 381 | // terminate if no more iterations are available. 382 | count.in.dec(); 383 | }); 384 | } 385 | }, result.in.throw, limit); 386 | // If no more iterations are available and all jobs are done, we can close 387 | // the output stream with the same return value as the input stream. 388 | count.out.equals(Observable.yield(0)).and(done.out).forEach(function (done) { 389 | if (done) { 390 | result.in.return(returnValue); 391 | } 392 | }); 393 | return result.out; 394 | }; 395 | 396 | // ### filter 397 | 398 | // The filter method runs concurrent tests to determine whether to include an 399 | // iteration from the input stream on the output stream. 400 | // The regularity of the duration of the test will determine whether iterations 401 | // are likely to be processed in order, but a concurrency limit of 1 guarantees 402 | // that the input and output order will be the same. 403 | 404 | Stream.prototype.filter = function (callback, thisp, limit) { 405 | var result = new this.constructor.buffer(); 406 | // As with map and forEach, we use signals to track the termination 407 | // condition. 408 | var count = Observable.signal(0); 409 | var done = Observable.signal(false); 410 | var returnValue; 411 | this.do(function (iteration) { 412 | // If this is the last iteration, we track the return value to later 413 | // forward to the output stream and note that no more iterations are 414 | // available, pending any outstanding jobs. 415 | if (iteration.done) { 416 | returnValue = iteration.value; 417 | done.in.yield(true); 418 | } else { 419 | // Otherwise we start another job, incrementing the outstanding job 420 | // counter and using the usual filter argument pattern. 421 | count.in.inc(); 422 | return Promise.try(callback, thisp, iteration.value, iteration.index) 423 | .then(function (value) { 424 | // Only if the test passes do we forward the value, and its 425 | // original index, to the output stream. 426 | if (value) { 427 | return result.in.yield(iteration.value, iteration.index); 428 | } 429 | }) 430 | .finally(function () { 431 | // Regardless of whether the test ran without error, we note 432 | // that the job is done. 433 | count.in.dec(); 434 | }); 435 | } 436 | }, result.in.throw, limit); 437 | /* when (count == 0 && done) */ 438 | count.out.equals(Observable.yield(0)).and(done.out).forEach(function (done) { 439 | // When there are no more outstanding jobs and the input has been 440 | // exhausted, we forward the input return value to the output stream. 441 | if (done) { 442 | result.in.return(returnValue); 443 | } 444 | }); 445 | return result.out; 446 | }; 447 | 448 | // ### reduce 449 | 450 | // The `reduce` method runs concurrent jobs to acrete values from the input 451 | // stream until only one value remains, returning a cancelable promise (task) 452 | // for that last value. 453 | // 454 | // Yet to be ported. 455 | 456 | Stream.prototype.reduce = function (callback, limit) { 457 | var self = this; 458 | var result = Task.defer(); 459 | var pool = Stream.buffer(); 460 | 461 | var sempahore = new PromiseQueue(); 462 | sempahore.put(); 463 | 464 | var done = false; 465 | var size = 0; 466 | for (var index = 0; index < limit; index++) { 467 | next(); 468 | } 469 | 470 | this.forEach(function (value) { 471 | return pool.in.yield(value); 472 | }).then(function (value) { 473 | return pool.in.return(value); 474 | }, function (error) { 475 | return pool.in.throw(error); 476 | }); 477 | 478 | var active = 0; 479 | 480 | function next() { 481 | return sempahore.get() 482 | .then(function () { 483 | return pool.out.yield().then(function (left) { 484 | if (left.done) { 485 | done = true; 486 | sempahore.put(); 487 | next(); 488 | return; 489 | } 490 | if (done && active === 0) { 491 | result.in.return(left.value); 492 | return; 493 | } 494 | return pool.out.yield().then(function (right) { 495 | sempahore.put(); 496 | if (right.done) { 497 | next(); 498 | return pool.in.yield(left.value); 499 | } 500 | active++; 501 | return Task.return() 502 | .then(function () { 503 | return callback(left.value, right.value); 504 | }) 505 | .then(function (value) { 506 | active--; 507 | next(); 508 | return pool.in.yield(value); 509 | }); 510 | }); 511 | }); 512 | }) 513 | .done(null, function (error) { 514 | result.in.throw(error); 515 | }) 516 | } 517 | 518 | return result.out; 519 | }; 520 | 521 | /* TODO some, every, takeWhile, dropWhile, concat */ 522 | 523 | // ### fork 524 | 525 | // The fork method creates an array of streams that will all see every value 526 | // from this stream. 527 | // All of the returned streams put back pressure on this stream. 528 | // This stream can only advance when all of the output streams have advanced. 529 | 530 | Stream.prototype.fork = function (length) { 531 | length = length || 2; 532 | var ins = []; 533 | var outs = []; 534 | for (var index = 0; index < length; index++) { 535 | var buffer = this.constructor.buffer(); 536 | ins.push(buffer.in); 537 | outs.push(buffer.out); 538 | } 539 | this.forEach(function (value, index) { 540 | return Promise.all(ins.map(function (input) { 541 | return input.yield(value, index); 542 | })); 543 | }).then(function (value) { 544 | return Promise.all(ins.map(function (input) { 545 | return input.return(value); 546 | })); 547 | }, function (error) { 548 | return Promise.all(ins.map(function (input) { 549 | return input.throw(value); 550 | })); 551 | }).done(); 552 | return outs; 553 | }; 554 | 555 | // ### relieve 556 | 557 | // If we consume this stream more slowly than it produces iterations, pressure 558 | // will accumulate between the consumer and the producer, slowing the producer. 559 | // The `relieve` method alleviates this pressure. 560 | // This stream will be allowed to produce values as quickly as it can, 561 | // and the returned stream will lose intermediate values if it can not keep up. 562 | // The consumer will only see the most recent value from the producer upon 563 | // request. 564 | // However, if the consumer is faster than the producer, the relief will have no 565 | // effect. 566 | 567 | Stream.prototype.relieve = function () { 568 | var current = Promise.defer(); 569 | this.forEach(function (value, index) { 570 | current.resolver.return(new Iteration(value, false)); 571 | current = Promise.defer(); 572 | }) 573 | .done(function (value) { 574 | current.resolver.return(new Iteration(value, true)); 575 | }, current.resolver.throw); 576 | return Stream.from(function () { 577 | return current.promise; 578 | }); 579 | }; 580 | -------------------------------------------------------------------------------- /task-queue.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var Task = require("./task"); 4 | 5 | module.exports = TaskQueue; 6 | function TaskQueue(values) { 7 | if (!(this instanceof TaskQueue)) { 8 | return new TaskQueue(); 9 | } 10 | var self = this; 11 | 12 | var ends = new this.Task; 13 | 14 | this.put = function (value) { 15 | var next = self.Task.defer(); 16 | ends.resolve({ 17 | head: value, 18 | tail: next.out 19 | }); 20 | ends.resolve = next.resolve; 21 | }; 22 | 23 | this.get = function () { 24 | var result = ends.out.get("head"); 25 | ends.out = ends.out.get("tail"); 26 | return result; 27 | }; 28 | 29 | if (values) { 30 | values.forEach(this.put, this); 31 | } 32 | } 33 | 34 | TaskQueue.prototype.Task = Task; 35 | 36 | TaskQueue.prototype.forEach = function (callback, thisp) { 37 | var self = this; 38 | function next() { 39 | return self.get() 40 | .then(function (value) { 41 | return callback.call(thisp, value); 42 | }) 43 | .then(next); 44 | } 45 | return next(); 46 | }; 47 | 48 | -------------------------------------------------------------------------------- /task.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | // A Task is a cancelable variant of a promise. 4 | // Like a promise, the observable is a proxy for the result of some work. 5 | // The interface is largely the same, but a observable can only have one 6 | // observer. 7 | // For example, calling `then` a second time will throw an error. 8 | // Instead, if a task has multiple observers, you can sacrifice cancelability 9 | // by coercing it to a promise, or use `fork` before observing it. 10 | // If every fork is cancelled, the cancelation will propagate back to the 11 | // original job. 12 | // 13 | // The price of cancelability is a less robust system and more book keeping. 14 | // A system that makes a great deal of use of tasks allows information to flow 15 | // from any observable to any related task, even if distantly related. 16 | // The cancelation of one task can propagate throughout an entire system of 17 | // tasks, both forward and backward between consumers and producers. 18 | // In exchange the system gains the ability to either free or avoid consuming 19 | // resources proactively. 20 | 21 | var asap = require("asap"); 22 | var WeakMap = require("weak-map"); 23 | 24 | // ## Task 25 | // 26 | // The consumer side of a task should receive the task's observable. 27 | // This object provides the ability to register exactly one observer for the 28 | // result of the task, and the ability to cancel the task with an error. 29 | 30 | function Task(setup, thisp) { 31 | var deferred = Task.defer(); 32 | var handler = handlers.get(deferred.out); 33 | handler.cancel = setup.call(thisp, deferred.in.return, deferred.in.throw); 34 | return deferred.out; 35 | } 36 | 37 | /* 38 | TODO Task.prototype = Object.create(Observable); 39 | Such that it is possible to create parallel signaling for status and estimated 40 | time to completion, or other arbitrary signals from the resolver to the 41 | observable. 42 | */ 43 | 44 | // The `done` method registers an observer for any combination of completion or 45 | // failure with the given methods and optional context object. 46 | // The `done` method does not return a new task and does not capture errors 47 | // thrown by the observer methods. 48 | Task.prototype.done = function (onreturn, onthrow, thisp) { 49 | var self = this; 50 | var handler = Task_getHandler(self); 51 | handler.done(onreturn, onthrow, thisp); 52 | }; 53 | 54 | // The `then` method registers an observer for any combination of completion or 55 | // failure, and creates a new task that will be completed with the result of 56 | // either the completion or failure handler. 57 | Task.prototype.then = function (onreturn, onthrow, thisp) { 58 | // TODO accept status and estimated time to completion arguments in 59 | // arbitrary order. 60 | var handler = Task_getHandler(this); 61 | var task = Task.defer(this.cancel, this); 62 | var _onreturn, _onthrow; 63 | if (typeof onreturn === "function") { 64 | _onreturn = function (value) { 65 | try { 66 | task.in.return(onreturn.call(thisp, value)); 67 | } catch (error) { 68 | task.in.throw(error); 69 | } 70 | }; 71 | } 72 | if (typeof onthrow === "function") { 73 | _onthrow = function (error) { 74 | try { 75 | task.in.return(onthrow.call(thisp, error)); 76 | } catch (error) { 77 | task.in.throw(error); 78 | } 79 | }; 80 | } 81 | this.done(_onreturn, _onthrow); 82 | return task.out; 83 | }; 84 | 85 | // The `spread` method fills a temporary need to be able to spread an array 86 | // into the arguments of the completion handler of a `then` observer. 87 | // ECMAScript 6 introduces the ability to spread arguments into an array in the 88 | // signature of the method. 89 | Task.prototype.spread = function (onreturn, onthrow, thisp) { 90 | return this.then(function (args) { 91 | return onreturn.apply(thisp, args); 92 | }, onthrow, thisp); 93 | }; 94 | 95 | // The `catch` method registers an error observer on a task and returns a new 96 | // task to be completed with the result of the observer. 97 | // The observer may return another task or thenable to transfer responsibility 98 | // to complete this task to another stage of the process. 99 | Task.prototype.catch = function (onthrow, thisp) { 100 | return this.then(null, onthrow, thisp); 101 | }; 102 | 103 | // The `finally` method registers an observer for when the task either 104 | // completes or fails and returns a new task to perform some further work but 105 | // forward the original value or error otherwise. 106 | Task.prototype.finally = function (onsettle, thisp) { 107 | return this.then(function (value) { 108 | return onsettle.call(thisp).then(function Task_finally_value() { 109 | return value; 110 | }) 111 | }, function (error) { 112 | return onsettle.call(thisp).then(function Task_finally_error() { 113 | throw error; 114 | }); 115 | }); 116 | }; 117 | 118 | // The `get` method creates a task that will get a property of the completion 119 | // object for this task. 120 | Task.prototype.get = function (key) { 121 | return task.then(function (object) { 122 | return object[key]; 123 | }); 124 | }; 125 | 126 | // The `call` method creates a task that will call the function that is the 127 | // completion value of this task with the given spread arguments. 128 | Task.prototype.call = function (thisp /*, ...args*/) { 129 | var args = []; 130 | for (var index = 1; index < arguments.length; index++) { 131 | args[index - 1] = arguments[index]; 132 | } 133 | return task.then(function (callable) { 134 | return callable.apply(thisp, args); 135 | }); 136 | }; 137 | 138 | // The `invoke` method creates a task that will invoke a property of the 139 | // completion object for this task. 140 | Task.prototype.invoke = function (name /*, ...args*/) { 141 | var args = []; 142 | for (var index = 1; index < arguments.length; index++) { 143 | args[index - 1] = arguments[index]; 144 | } 145 | return task.then(function (object) { 146 | return object[name].apply(object, args); 147 | }); 148 | }; 149 | 150 | // The `thenReturn` method registers an observer for the completion of this 151 | // task and returns a task that will be completed with the given value when 152 | // this task is completed. 153 | Task.prototype.thenReturn = function (value) { 154 | return this.then(function () { 155 | return value; 156 | }); 157 | }; 158 | 159 | // The `thenReturn` method registers an observer for the completion of this 160 | // task and returns a task that will fail with the given error when this task 161 | // is completed. 162 | Task.prototype.thenThrow = function (error) { 163 | return this.then(function () { 164 | return error; 165 | }); 166 | }; 167 | 168 | // Effects cancelation from the consumer side. 169 | Task.prototype.throw = function (error) { 170 | var handler = Task_getHandler(this); 171 | if (handler.cancel) { 172 | handler.throw(error); 173 | } 174 | }; 175 | 176 | // A task can only be observed once, but it can be forked. 177 | // The `fork` method returns a new task that will observe the same completion 178 | // or failure of this task. 179 | // Hereafter, this task and all forked tasks must *all* be cancelled for this 180 | // task's canceller to propagate. 181 | Task.prototype.fork = function () { 182 | // The fork method works by fiddling with the handler of this task. 183 | // First, we extract this task's handler and make it the new parent for two 184 | // child tasks. 185 | var parentHandler = Task_getHandler(this); 186 | parentHandler.done(function (value) { 187 | left.in.return(value); 188 | right.in.return(value); 189 | }, function (error) { 190 | left.in.throw(error); 191 | right.in.throw(error); 192 | }); 193 | /* TODO estimated time to completion forwarding */ 194 | /* TODO use a signal operator to propagate cancellation */ 195 | var leftCanceled = false, rightCanceled = false; 196 | var left = Task.defer(function (error) { 197 | if (leftCanceled) { 198 | return; 199 | } 200 | leftCanceled = true; 201 | if (rightCanceled) { 202 | parentHandler.throw(error); 203 | } 204 | }); 205 | var right = Task.defer(function (error) { 206 | if (rightCanceled) { 207 | return; 208 | } 209 | rightCanceled = true; 210 | if (leftCanceled) { 211 | parentHandler.throw(error); 212 | } 213 | }); 214 | // We replace our own handler with the left child 215 | handlers.set(this, Task_getHandler(left.out)); 216 | // And return the task with the right child handler 217 | return right.out; 218 | }; 219 | 220 | // The `delay` method of a task adds a delay of some miliseconds after the task 221 | // *completes*. 222 | // Cancelling the delayed task will cancel either the delay or the delayed 223 | // task. 224 | Task.prototype.delay = function (ms) { 225 | var self = this; 226 | var task = Task.defer(function cancelDelayedTask() { 227 | self.throw(); 228 | clearTimeout(handle); 229 | }); 230 | var result = Task.defer(); 231 | var handle = setTimeout(function taskDelayed() { 232 | task.in.return(result.out); 233 | }, ms); 234 | this.done(function (value) { 235 | result.in.return(value); 236 | }, function (error) { 237 | task.in.throw(error); 238 | }); 239 | return task.out; 240 | }; 241 | 242 | // The `timeout` method will automatically cancel a task if it takes longer 243 | // than a given delay in miliseconds. 244 | Task.prototype.timeout = function (ms, message) { 245 | var self = this; 246 | var task = Task.defer(function cancelTimeoutTask() { 247 | this.throw(); 248 | clearTimeout(handle); 249 | }, this); 250 | var handle = setTimeout(function Task_timeout() { 251 | self.throw(); 252 | task.in.throw(new Error(message || "Timed out after " + ms + "ms")); 253 | }, ms); 254 | this.done(function Task_timeoutValue(value) { 255 | clearTimeout(handle); 256 | task.in.return(value); 257 | }, function Task_timeoutError(error) { 258 | clearTimeout(handle); 259 | task.in.throw(error); 260 | }); 261 | return task.out; 262 | }; 263 | 264 | 265 | // ## Completer 266 | // 267 | // The producer side of a task should get a reference to a task's resolver. 268 | // The object provides the capability to settle the task with a completion 269 | // value or a failure error. 270 | 271 | function Completer(handler) { 272 | // The task resolver implicitly binds its return and throw methods so these 273 | // can be passed as free functions. 274 | this.return = this.return.bind(this); 275 | this.throw = this.throw.bind(this); 276 | } 277 | 278 | // The `return` method sets the tasks state to "fulfilled" (in the words of 279 | // promises) or "completed" (in the vernacular of tasks), with a given value. 280 | // If the corresponding observer was registered already, this will inform 281 | // the observer as soon as possible. 282 | // If the corresponding observer gets registered later, it will receive the 283 | // result as soon as possible thereafter. 284 | Completer.prototype.return = function (value) { 285 | var handler = Task_getHandler(this); 286 | handler.become(Task.return(value)); 287 | }; 288 | 289 | // The `throw` method sets the tasks state to "rejected" (a term borrowed from 290 | // promises) or "failed" (the corresponding task jargon), with the given error. 291 | // Again, if the corresponding observer was registered already, this will 292 | // inform the observer as soon as possible. 293 | // If the corresponding observer gets registered later, it will receive the 294 | // result as soon as possible thereafter. 295 | Completer.prototype.throw = function (error) { 296 | var handler = Task_getHandler(this); 297 | handler.become(Task.throw(error)); 298 | }; 299 | 300 | 301 | // ## Task 302 | // 303 | // The task constructor creates a resolver "in" and an observer "out" pair 304 | // with some shared internal state. 305 | // Particularly, since tasks can be canceled, the task constructor accepts a 306 | // reference to the cancellation method and optionally the instance that hosts 307 | // it. 308 | 309 | module.exports = Task; 310 | Task.defer = function (cancel, thisp) { // TODO estimate, label 311 | var handler = new TaskHandler(); // TODO polymorph constructors 312 | var input = Object.create(Completer.prototype); 313 | var output = Object.create(Task.prototype); 314 | Completer_bind(input); 315 | handlers.set(input, handler); 316 | handlers.set(output, handler); 317 | handler.cancel = cancel; 318 | handler.cancelThisp = thisp; 319 | return {in: input, out: output}; 320 | } 321 | 322 | function Completer_bind(completer) { 323 | completer.return = completer.return.bind(completer); 324 | completer.throw = completer.throw.bind(completer); 325 | }; 326 | 327 | // The `isTask` utility method allows us to identify a task that was 328 | // constructed by this library. 329 | // This library does not attempt to make it provably impossible to trick the 330 | // Task. 331 | Task.isTask = isTask; 332 | function isTask(object) { 333 | return ( 334 | Object(object) === object && 335 | !!handlers.get(object) && 336 | object instanceof Task 337 | ); 338 | }; 339 | 340 | // The `isThenable` method is used internally to identify other singular 341 | // asynchronous duck types, including promises, which can be coerced into 342 | // tasks. 343 | function isThenable(object) { 344 | return object && typeof object === "object" && typeof object.then === "function"; 345 | } 346 | 347 | // The `return` function lifts a value into a task that has already completed 348 | // with a value. 349 | Task.return = function (value) { 350 | if (isTask(value)) { 351 | return value; 352 | } else if (isThenable(value)) { 353 | // FIXME sloppy with closures and should use weakmap of value to task 354 | var deferred = Task.defer(); 355 | asap(function () { 356 | value.then(function (value) { 357 | deferred.in.return(value); 358 | }, function (error) { 359 | deferred.in.throw(value); 360 | }); 361 | }); 362 | return deferred.out; 363 | } else { 364 | var handler = new TaskHandler(); 365 | handler.state = "fulfilled"; 366 | handler.value = value; 367 | var task = Object.create(Task.prototype); 368 | handlers.set(task, handler); 369 | return task; 370 | } 371 | }; 372 | 373 | // The `throw` function lifts an error into a task that has already failed with 374 | // that error. 375 | Task.throw = function (error) { 376 | var handler = new TaskHandler(); 377 | handler.state = "rejected"; 378 | handler.error = error; 379 | var task = Object.create(Task.prototype); 380 | handlers.set(task, handler); 381 | return task; 382 | }; 383 | 384 | // The `all` function accepts an array of tasks, or values that can be coerced 385 | // into tasks, and produces a task that when completed will produce an array of 386 | // the individual completion values. 387 | Task.all = function Task_all(tasks) { 388 | // If the task is cancelled, or if any individual task fails, all of the 389 | // outstanding individual tasks will be cancelled. 390 | function cancelAll(error) { 391 | for (var otherIndex = 0; otherIndex < tasks.length; otherIndex++) { 392 | // Note that throwing an error upstream consitutes talking back to the producer. 393 | // This is a reminder that tasks are a cooperation between a single 394 | // consumer and a single producer and that information flows both 395 | // ways and in fact allows information to propagate laterally by 396 | // passing up one stream and down another. 397 | tasks[otherIndex].throw(error); 398 | } 399 | result.in.throw(error); 400 | } 401 | // The number of outstanding tasks, tracked to determine when all tasks are 402 | // completed. 403 | var remaining = tasks.length; 404 | var result = Task.defer(cancelAll); 405 | var results = Array(tasks.length); 406 | /* TODO estimated time to completion, label signals */ 407 | var estimates = []; 408 | var estimate = -Infinity; 409 | var setEstimate; 410 | var estimates = tasks.map(function Task_all_each(task, index) { 411 | task = tasks[index] = Task.return(task); // Coerce values to tasks 412 | task.done(function Task_all_anyReturn(value) { 413 | results[index] = value; 414 | if (--remaining === 0) { 415 | result.in.return(results); 416 | } 417 | }, cancelAll); 418 | }); 419 | return result.out; 420 | }; 421 | 422 | // The `any` method accepts an array of tasks, or value coercable to tasks, and 423 | // returns a task that will receive the value from the first task that 424 | // completes with a value. 425 | // After one succeeds, all remaining tasks will be cancelled. 426 | // If one of the tasks fails, it will be ignored. 427 | // If all tasks fail, this task will fail with the last error. 428 | Task.any = function (tasks) { 429 | /* TODO */ 430 | }; 431 | 432 | // The `any` method accepts an array of tasks, or value coercable to tasks, and 433 | // returns a task that will receive the value or error of the first task that 434 | // either completes or fails. 435 | // Afterward, all remaining tasks will be cancelled. 436 | Task.race = function (tasks) { 437 | /* TODO */ 438 | }; 439 | 440 | // The `delay` method accepts a duration of time in miliseconds and returns a 441 | // task that will complete with the given value after that amount of time has 442 | // elapsed. 443 | Task.delay = function (ms, value) { 444 | return Task.return(value).delay(ms); 445 | }; 446 | 447 | // ## TaskHandler 448 | 449 | // The resolver and observable side of a task share a hidden internal record 450 | // with their shared state. 451 | // Handlers are an alternative to using closures. 452 | var handlers = new WeakMap(); 453 | 454 | function TaskHandler() { 455 | // When a task is resolved, it "becomes" a different task and its 456 | // observable, if any, must be forwarded to the new task handler. 457 | // In the `become` method, we also adjust the "handlers" table so any 458 | // subsequent request for this handler jumps to the end of the "became" 459 | // chain. 460 | this.became = null; 461 | // Tasks may be created with a corresponding canceler. 462 | this.cancel = null; 463 | this.cancelThisp = null; 464 | // Tasks may be "pending", "fulfilled" with a value, or "rejected" with an 465 | // error 466 | this.state = "pending"; 467 | this.value = null; 468 | this.error = null; 469 | // A task may only be observed once. 470 | // Any future attempt to observe a task will throw an error. 471 | this.observed = false; 472 | // Since a task can only be observed once, we only need to track one 473 | // handler for fulfillment with a value or rejection with an error. 474 | // A promise keeps an array of handlers to forward messages to. 475 | // These handlers can be forgotten once a task settles since thereafter 476 | // the observer would be informed immediately. 477 | this.onreturn = null; 478 | this.onthrow = null; 479 | // The object to use as `this` in the context of `onreturn` and `onthrow`. 480 | this.thisp = null; 481 | } 482 | 483 | // Since a task handler can become another task handler, this utility method 484 | // will look up the end of the chain of "became" properties and rewrite the 485 | // handler look up table so we never have to walk the same length of chain 486 | // again. 487 | function Task_getHandler(task) { 488 | var handler = handlers.get(task); 489 | while (handler && handler.became) { 490 | handler = handler.became; 491 | } 492 | handlers.set(task, handler); 493 | return handler; 494 | } 495 | 496 | // The `done` method is kernel for subscribing to a task observer. 497 | // If the task has already completed or failed, this will also arrange for the 498 | // observer to be notified as soon as possible. 499 | TaskHandler.prototype.done = function (onreturn, onthrow, thisp) { 500 | if (this.observed) { 501 | throw new Error("Can't observe a task multiple times. Use fork"); 502 | } 503 | this.observed = true; 504 | this.onreturn = onreturn; 505 | this.onthrow = onthrow; 506 | this.thisp = thisp; 507 | // If we are observing a task after it completed or failed, we dispatch the 508 | // result immediately. 509 | if (this.state !== "pending") { 510 | // Instead of passing a callable closure, we pass ourself to avoid 511 | // allocating another object. 512 | // The task handler serves as a psuedo-function by implementing "call". 513 | asap(this); 514 | } 515 | // We handle the case of observing *before* completion or failure in the 516 | // `become` method. 517 | }; 518 | 519 | // Above, we pass the task handler to `asap`. 520 | // The event dispatcher treats functions and callable objects alike. 521 | // This method will get called if this task has settled into a "fulfilled" or 522 | // "rejected" state so we can call the appropriate handler. 523 | TaskHandler.prototype.call = function () { 524 | if (this.state === "fulfilled") { 525 | if (this.onreturn) { 526 | this.onreturn.call(this.thisp, this.value); 527 | } 528 | } else if (this.state === "rejected") { 529 | if (this.onthrow) { 530 | this.onthrow.call(this.thisp, this.error); 531 | } else { 532 | throw this.error; 533 | } 534 | } 535 | // We release the handlers so they can be potentially garbage collected. 536 | this.onreturn = null; 537 | this.onthrow = null; 538 | this.thisp = null; 539 | }; 540 | 541 | // The `become` method is the kernel of the task resolver. 542 | TaskHandler.prototype.become = function (task) { 543 | var handler = Task_getHandler(task); 544 | // A task can only be resolved once. 545 | // Subsequent resolutions are ignored. 546 | // Ignoring, rather than throwing an error, greatly simplifies a great 547 | // number of cases, like racing tasks and cancelling tasks, where handling 548 | // an error would be unnecessary and inconvenient. 549 | if (this.state !== "pending") { 550 | return; 551 | } 552 | // The `became` property gets used by the internal handler getter to 553 | // rewrite the handler table and shorten chains. 554 | this.became = handler; 555 | // Once a task completes or fails, we no longer need to retain the 556 | // canceler. 557 | this.cancel = null; 558 | this.cancelThisp = null; 559 | // If an observer subscribed before it completed or failed, we forward the 560 | // resolution. 561 | // If an observer subscribes later, we take care of that case in `done`. 562 | if (this.observed) { 563 | handler.done(this.onreturn, this.onthrow, this.thisp); 564 | } 565 | }; 566 | 567 | // The `throw` method is used by the promise observer to cancel the task from 568 | // the consumer side. 569 | TaskHandler.prototype.throw = function (error) { 570 | if (this.cancel) { 571 | this.cancel.call(this.cancelThisp); 572 | } 573 | this.become(Task.throw(error || new Error("Consumer canceled task"))); 574 | }; 575 | 576 | -------------------------------------------------------------------------------- /test/behavior-test.js: -------------------------------------------------------------------------------- 1 | 2 | var Behavior = require("../behavior"); 3 | 4 | it("", function () { 5 | expect(Behavior.yield(10).next().value).toBe(10); 6 | }); 7 | 8 | it("", function () { 9 | expect(new Behavior(function () { 10 | return 10; 11 | }).next().value).toBe(10); 12 | }); 13 | 14 | it("", function () { 15 | expect(Behavior.yield(20).add(Behavior.yield(10)).next().value).toBe(30); 16 | }); 17 | 18 | describe("index behavior", function () { 19 | it("produces the given index", function () { 20 | expect(Behavior.index.add(Behavior.yield(20)).next(10).value).toBe(30); 21 | }); 22 | }); 23 | 24 | -------------------------------------------------------------------------------- /test/byte-stream-test.js: -------------------------------------------------------------------------------- 1 | 2 | var ByteStream = require("../byte-stream"); 3 | 4 | describe("of", function () { 5 | it("pumps large byte array into small stream buffer", function () { 6 | var string = "Hello, World! Good bye!" 7 | var stream = ByteStream.of(new Buffer(string, "utf-8"), 3); 8 | return stream.forEach(function (chunk, index) { 9 | expect(chunk.toString()).toBe(string.slice(index, index + 3)); 10 | }) 11 | .then(function (value) { 12 | expect(value).toBe(undefined); 13 | }); 14 | }); 15 | }); 16 | 17 | it("can write then read", function () { 18 | var buffer = ByteStream.buffer(10); 19 | var flushed = buffer.in.yield(new Buffer("a")); 20 | flushed.done(); 21 | return buffer.out.next() 22 | .then(function (iteration) { 23 | expect(iteration.value.toString()).toBe("a"); 24 | }); 25 | }); 26 | 27 | it("can read then write", function () { 28 | var buffer = ByteStream.buffer(10); 29 | var done = buffer.out.next() 30 | .then(function (iteration) { 31 | expect(iteration.value.toString()).toBe("a"); 32 | }); 33 | buffer.in.yield(new Buffer("a")).done(); 34 | return done; 35 | }); 36 | 37 | it("small writes to a buffer should accumulate", function () { 38 | var buffer = ByteStream.buffer(2); 39 | buffer.in.yield(new Buffer("a")).done(); 40 | buffer.in.yield(new Buffer("b")).done(); 41 | return buffer.out.next() 42 | .then(function (iteration) { 43 | expect(iteration.value.toString()).toBe("ab"); 44 | }); 45 | }); 46 | 47 | it("successive writes to small buffer must wait for successive reads", function () { 48 | var buffer = ByteStream.buffer(1); 49 | buffer.in.yield(new Buffer("a")) 50 | .then(function () { 51 | return buffer.in.yield(new Buffer("b")); 52 | }) 53 | .done(); 54 | return buffer.out.next() 55 | .then(function (iteration) { 56 | expect(iteration.value.toString()).toBe("a"); 57 | return buffer.out.next(); 58 | }) 59 | .then(function (iteration) { 60 | expect(iteration.value.toString()).toBe("b"); 61 | }); 62 | }); 63 | 64 | -------------------------------------------------------------------------------- /test/observable-test.js: -------------------------------------------------------------------------------- 1 | 2 | "use strict"; 3 | 4 | var Observable = require("../observable"); 5 | 6 | it("can observe after emit", function (done) { 7 | var s = Observable.signal(); 8 | s.in.yield(10); 9 | s.out.forEach(function (x) { 10 | expect(x).toBe(10); 11 | done(); 12 | }); 13 | }); 14 | 15 | it("can observe before emit", function (done) { 16 | var s = Observable.signal(); 17 | s.out.forEach(function (x) { 18 | expect(x).toBe(10); 19 | done(); 20 | }); 21 | s.in.yield(10); 22 | }); 23 | 24 | it("can be cancelled", function () { 25 | var setUp = false; 26 | var observer = new Observable(function (_yield) { 27 | setUp = true; 28 | _yield(10); 29 | }).forEach(function () { 30 | expect(true).toBe(false); 31 | }); 32 | expect(setUp).toBe(true); 33 | observer.cancel(); 34 | }); 35 | 36 | it("can get a value with revealing constructor", function (done) { 37 | new Observable(function (_yield) { 38 | _yield(10); 39 | }).forEach(function (x) { 40 | expect(x).toBe(10); 41 | done(); 42 | }); 43 | }); 44 | 45 | it("can count", function (done) { 46 | var signal = Observable.signal(); 47 | var i = 0; 48 | signal.in.yield(0); 49 | signal.out.forEach(function (n) { 50 | if (n < 3) { 51 | expect(n).toBe(i++); 52 | signal.in.yield(i); 53 | } else { 54 | done(); 55 | } 56 | }); 57 | }); 58 | 59 | -------------------------------------------------------------------------------- /test/promise-test.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var Promise = require("../promise"); 4 | 5 | describe("then", function () { 6 | 7 | it("lifts a value", function () { 8 | return Promise.return(10) 9 | .then(function (ten) { 10 | expect(ten).toBe(10); 11 | }) 12 | }); 13 | 14 | describe("defer", function () { 15 | it("makes a promise resolver pair", function () { 16 | var d = Promise.defer(); 17 | d.resolver.return(10); 18 | return d.promise.then(function (v) { 19 | expect(v).toBe(10); 20 | }); 21 | }); 22 | }); 23 | 24 | it("forwards this", function () { 25 | var delegate = {beep: 10, boop: function () { 26 | expect(this.beep).toBe(10); 27 | }}; 28 | return Promise.return().then(delegate.boop, delegate); 29 | }); 30 | 31 | }); 32 | 33 | -------------------------------------------------------------------------------- /test/stream-test.js: -------------------------------------------------------------------------------- 1 | 2 | "use strict"; 3 | 4 | var Stream = require("../stream"); 5 | var Promise = require("../promise"); 6 | 7 | describe("revealing constructor", function () { 8 | it("works", function () { 9 | var at = 0; 10 | return new Stream(function (_yield, _return) { 11 | _yield(10).then(function () { 12 | return _yield(20); 13 | }).then(function () { 14 | return _return(30); 15 | }) 16 | .done(); 17 | }).forEach(function (value) { 18 | at++; 19 | expect(value).toBe(at * 10); 20 | }).then(function (value) { 21 | at++; 22 | expect(value).toBe(at * 10); 23 | }); 24 | }); 25 | }) 26 | 27 | it("single iteration produced then consumed", function () { 28 | var buffer = Stream.buffer(); 29 | var produced = buffer.in.yield(10) 30 | .then(function (iteration) { 31 | expect(iteration.value).toBe(undefined); 32 | expect(iteration.done).toBe(false); 33 | expect(consumed.isFulfilled()).toBe(false); 34 | }); 35 | var consumed = buffer.out.next() 36 | .then(function (iteration) { 37 | expect(iteration.value).toBe(10); 38 | expect(iteration.done).toBe(false); 39 | expect(produced.isFulfilled()).toBe(true); 40 | }); 41 | return Promise.all([produced, consumed]).thenReturn(); 42 | }); 43 | 44 | it("synchronize and acknowledge cycle", function () { 45 | var buffer = Stream.buffer(); 46 | var index = 0; 47 | var produced = buffer.in.yield(10) 48 | .then(function (iteration) { 49 | expect(iteration.value).toBe("A"); 50 | expect(iteration.done).toBe(false); 51 | expect(index++).toBe(0); 52 | return buffer.in.return(20); 53 | }).then(function (iteration) { 54 | expect(iteration.value).toBe("B"); 55 | expect(iteration.done).toBe(false); 56 | expect(index++).toBe(2); 57 | }); 58 | var consumed = buffer.out.next("A") 59 | .then(function (iteration) { 60 | expect(iteration.value).toBe(10); 61 | expect(iteration.done).toBe(false); 62 | expect(index++).toBe(1); 63 | return buffer.out.next("B"); 64 | }).then(function (iteration) { 65 | expect(iteration.value).toBe(20); 66 | expect(iteration.done).toBe(true); 67 | expect(index++).toBe(3); 68 | }); 69 | return Promise.all([produced, consumed]).thenReturn(); 70 | }); 71 | 72 | describe("forEach", function () { 73 | 74 | it("works", function () { 75 | var stream = Stream.from([0, 1, 2]); 76 | var index = 0; 77 | return stream.forEach(function (value) { 78 | expect(value).toBe(index++); 79 | }).then(function (value) { 80 | expect(value).toBe(undefined); 81 | }); 82 | }); 83 | 84 | }); 85 | 86 | describe("map", function () { 87 | 88 | it("works", function () { 89 | var index = 0; 90 | return Stream.from([1, 2, 3]) 91 | .map(function (n, i) { 92 | expect(i).toBe(n - 1); 93 | return n + 1; 94 | }).forEach(function (n, i) { 95 | expect(i).toBe(index++); 96 | expect(n).toBe(i + 2) 97 | }).then(function (value) { 98 | expect(index).toBe(3); 99 | expect(value).toBe(undefined); 100 | }); 101 | }); 102 | 103 | it("executes two jobs in parallel", function () { 104 | var index = 0; 105 | var limit = 2; 106 | var delay = 100; 107 | var tolerance = 40; 108 | var start; 109 | return Stream.from([0, 1, 2, 3, 4, 5]).map(function (n) { 110 | return Promise.return(n).delay(delay); 111 | }, null, limit).forEach(function (n, i) { 112 | if (i === 0) { 113 | start = Date.now(); 114 | } else { 115 | expect(Date.now() - start).toBeNear(delay * Math.floor(index / limit), tolerance); 116 | } 117 | expect(i).toBe(index); 118 | expect(n).toBe(index); 119 | index++; 120 | }).then(function (value) { 121 | expect(index).toBe(6); 122 | expect(value).toBe(undefined); 123 | }) 124 | .timeout(1000); 125 | }); 126 | 127 | it("weird feedback loop", function () { 128 | var buffer = Stream.buffer(); 129 | buffer.in.yield(0); // kick this off 130 | var index = 0; 131 | return buffer.out.forEach(function (value) { 132 | expect(value).toBe(index++); 133 | if (value < 10) { 134 | return buffer.in.yield(value + 1); 135 | } else { 136 | return buffer.in.return("fin"); 137 | } 138 | }, null, 2) 139 | .then(function (fin) { 140 | expect(fin).toBe("fin"); 141 | }); 142 | }); 143 | 144 | }); 145 | 146 | -------------------------------------------------------------------------------- /test/task-test.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | var Task = require("../task"); 4 | 5 | it("observe immediately completed task", function () { 6 | return Task.return(10).done(function (value) { 7 | expect(value).toBe(10); 8 | }) 9 | }); 10 | 11 | it("observe immediately failed task", function () { 12 | return Task.throw(new Error("Can't finish")) 13 | .catch(function (error) { 14 | }) 15 | }); 16 | 17 | it("coerces return values to returned tasks and forwards", function () { 18 | return Task.return(10).then(function (a) { 19 | return a + 20; 20 | }).then(function (b) { 21 | expect(b).toBe(30); 22 | }); 23 | }); 24 | 25 | it("recover from failed task", function () { 26 | return Task.throw(new Error("Can't finish")) 27 | .catch(function (error) { 28 | expect(error.message).toBe("Can't finish"); 29 | }) 30 | .then(function () { 31 | expect(true).toBe(true); 32 | }) 33 | }); 34 | 35 | it("can't observe multiple times", function () { 36 | var task = Task.return(); 37 | task.done(); 38 | expect(function () { 39 | task.done(); 40 | }).toThrow(); 41 | }); 42 | 43 | it("multiple resolution is ignored", function () { 44 | // TODO consider carefully whether this makes sense outside the cancellation case 45 | var task = Task.defer(); 46 | task.in.return(10); 47 | task.in.return(20); 48 | return task.out.then(function (value) { 49 | expect(value).toBe(10); 50 | }); 51 | }); 52 | 53 | it("cancelling before settling has effect", function () { 54 | var canceled = false; 55 | var thisp = {}; 56 | var task = Task.defer(function () { 57 | canceled = true; 58 | expect(this).toBe(thisp); 59 | }, thisp); 60 | task.out.throw(); // TODO evaluate whether this should indeed be synchronous 61 | expect(canceled).toBe(true); 62 | task.in.return(); // should be ignored 63 | return task.out.catch(function (error) { 64 | expect(error && error.message).toBe("Consumer canceled task"); 65 | }); 66 | }); 67 | 68 | it("cancelling after settling has no effect", function () { 69 | var canceled = false; 70 | var task = Task.defer(function () { 71 | canceled = true; 72 | }); 73 | task.in.return(); 74 | return task.out.then(function () { 75 | task.out.throw(); 76 | expect(canceled).toBe(false); 77 | }); 78 | }); 79 | 80 | describe("timeout", function () { 81 | 82 | it("works", function () { 83 | var canceled = false; 84 | return Task.defer(function () { 85 | canceled = true; 86 | }).out.delay(1000).timeout(0) 87 | .catch(function (error) { 88 | expect(canceled).toBe(true); 89 | expect(error && error.message).toBe("Timed out after 0ms"); 90 | }); 91 | }); 92 | 93 | }); 94 | 95 | describe("delay", function () { 96 | 97 | it("works", function () { 98 | var now = Date.now(); 99 | var task = Task.return(10) 100 | .delay(100) 101 | .then(function (n) { 102 | expect(Date.now()).toBeNear(now + 100, 40); 103 | expect(n).toBe(10); 104 | }); 105 | return task; 106 | }); 107 | 108 | it("can be canceled", function () { 109 | var now = Date.now(); 110 | var canceled = false; 111 | var task = Task.defer(function () { 112 | canceled = true; 113 | }).out; 114 | var delayed = task.delay(100); 115 | delayed.throw(); 116 | expect(canceled).toBe(true); 117 | return delayed.catch(function (error) { 118 | expect(Date.now()).toBeNear(now, 10); // as opposed to + 100 119 | expect(error && error.message).toBe("Consumer canceled task"); 120 | }); 121 | }); 122 | 123 | }); 124 | 125 | describe("all", function () { 126 | 127 | it("works", function () { 128 | return Task.all([1, 2, 3]) 129 | .spread(function (a, b, c) { 130 | expect(a).toBe(1); 131 | expect(b).toBe(2); 132 | expect(c).toBe(3); 133 | }) 134 | }); 135 | 136 | it("can be canceled in aggregate", function () { 137 | var canceled = {}; 138 | var all = Task.all([1, 2, 3].map(function (n) { 139 | return Task.defer(function () { 140 | canceled[n] = true; 141 | }).out.delay(1000); 142 | })); 143 | all.throw(); 144 | var now = Date.now(); 145 | return all.catch(function (error) { 146 | expect(Date.now()).toBeNear(now, 10); // immediate 147 | expect(canceled).toEqual({1: true, 2: true, 3: true}); // all children 148 | expect(error && error.message).toBe("Consumer canceled task"); 149 | }); 150 | }); 151 | 152 | it("can be canceled by individual cancelation", function () { 153 | var canceled = {}; 154 | var tasks = [1, 2, 3].map(function (n) { 155 | return Task.defer(function () { 156 | canceled[n] = true; 157 | }).out.delay(1000); 158 | }); 159 | tasks[1].throw(); 160 | var now = Date.now(); 161 | return Task.all(tasks).catch(function (error) { 162 | expect(Date.now()).toBeNear(now, 10); // immediate 163 | expect(canceled).toEqual({1: true, 2: true, 3: true}); // all children 164 | expect(error && error.message).toBe("Consumer canceled task"); 165 | }); 166 | }); 167 | 168 | it("can be canceled by individual failure", function () { 169 | var canceled = {}; 170 | var tasks = [1, 2, 3].map(function (n) { 171 | return Task.defer(function () { 172 | canceled[n] = true; 173 | }).out.delay(1000); 174 | }); 175 | var task = Task.defer(); 176 | tasks.push(task.out); 177 | task.in.throw(new Error("Failed")); 178 | var now = Date.now(); 179 | return Task.all(tasks).catch(function (error) { 180 | expect(Date.now()).toBeNear(now, 10); // immediate 181 | expect(canceled).toEqual({1: true, 2: true, 3: true}); // all children 182 | expect(error && error.message).toBe("Failed"); 183 | }); 184 | }); 185 | 186 | }); 187 | 188 | describe("fork", function () { 189 | 190 | it("works", function () { 191 | var parent = Task.return(10); 192 | var child = parent.fork(); 193 | return Task.all([parent, child]).spread(function (a, b) { 194 | expect(a).toBe(10); 195 | expect(b).toBe(10); 196 | }); 197 | }); 198 | 199 | it("can be canceled by child", function () { 200 | var canceled = false; 201 | var root = Task.defer(function () { 202 | canceled = true; 203 | }); 204 | var child = root.out.fork(); 205 | root.out.throw(); 206 | expect(canceled).toBe(false); 207 | child.throw(); 208 | expect(canceled).toBe(true); 209 | }); 210 | 211 | it("can be canceled by parent", function () { 212 | var canceled = false; 213 | var root = Task.defer(function () { 214 | canceled = true; 215 | }); 216 | var child = root.out.fork(); 217 | child.throw(); 218 | expect(canceled).toBe(false); 219 | root.out.throw(); 220 | expect(canceled).toBe(true); 221 | }); 222 | 223 | }); 224 | 225 | -------------------------------------------------------------------------------- /time-to-completion.md: -------------------------------------------------------------------------------- 1 | 2 | # Estimated Time to Completion 3 | 4 | *Kris Kowal, published August 8, 2014.* 5 | 6 | A promise is an object that represents a period of asynchronous work. 7 | This is a simplified example of issuing an HTTP request in a browser, returning 8 | a promise for the response when it has been received. 9 | 10 | ```js 11 | function request(location) { 12 | return new Promise(function (resolve, reject) { 13 | var request = new XMLHttpRequest(); 14 | request.open(GET, location, true); 15 | request.onload = request.load = function () { 16 | resolve(request); 17 | }; 18 | request.onerror = request.error = reject; 19 | request.send(); 20 | }); 21 | } 22 | ``` 23 | 24 | Between calling `request` and when the request calls `onload`, the HTTP request 25 | may call another method, `onprogress`, multiple times as chunks of the response 26 | body arrive asynchronously. 27 | Although the promise represents the eventual result, the whole response, it can 28 | also make these progress events observable. 29 | 30 | Various promise implementations have loose support for progress 31 | notifications. 32 | Usually, you must call a `notify` function with an arbitrary progress value, and 33 | the corresponding `progress` handler passed to the `then` method will receive 34 | that value. 35 | 36 | ```js 37 | function request(location) { 38 | var deferred = Promise.defer(); 39 | var request = new XMLHttpRequest(); 40 | request.open(GET, location, true); 41 | request.onload = request.load = function () { 42 | deferred.resolve(request.responseText); 43 | }; 44 | request.onprogress = function () { 45 | var length = request.getResponseHeader("length"); 46 | if (length !== null) { 47 | deferred.notify(request.response.length / length); 48 | } 49 | }; 50 | request.onerror = request.error = deferred.reject; 51 | request.send(); 52 | return deferred.promise; 53 | } 54 | 55 | request(url) 56 | .then(function onFulfilled(response) { 57 | }, function onRejected(error) { 58 | }, function onProgressed(progress) { 59 | // <------------ 60 | }) 61 | ``` 62 | 63 | The progress value is presumably a number in the interval from 0 to 1 64 | representing how much progress has been made since the request was issued, 65 | but the progress notification has been used to stream all manner of updates 66 | including status messages. 67 | Otherwise, the pattern is under-specified and conceptually flawed. 68 | There is no meaningful way for the various operators that accept promises to 69 | infer the progress of composite promises. 70 | 71 | 72 | ## Making progress 73 | 74 | The next major version of Q will support an alternative to arbitrary progress 75 | messages: estimated time to completion. 76 | This feature is available for review in the [v2][] branch of Q but will likely 77 | changed as the general theory of reactivity develops. 78 | 79 | [v2]: https://github.com/kriskowal/q/tree/v2 80 | 81 | An estimated time to completion is an *observable measurement*. 82 | These measurements compose well based on a little knowledge of the work being 83 | estimated. 84 | 85 | In the most trivial case, one might have a promise for a result at an exact 86 | time. 87 | Such a promise can simply initialize its estimated time to completion. 88 | The estimate will never change. 89 | 90 | ```js 91 | function at(time) { 92 | return new Promise(function (resolve, reject, setEstimate) { 93 | setTimeout(function () { 94 | resolve(); 95 | }, time - Date.now()); 96 | setEstimate(time); 97 | }); 98 | } 99 | ``` 100 | 101 | ## All 102 | 103 | The Promise.all method accepts an array of promises and returns a promise for an 104 | array with the respective values. 105 | The estimated time to completion for the aggregate promise is, by definition, 106 | the last of the individual estimates. 107 | The resulting array will be available when all of the individual values are 108 | available. 109 | Whenever the estimate for one of these promises changes, the composite promise 110 | should emit a new estimate. 111 | 112 | [figure] 113 | 114 | There are two ways to track incremental changes to the estimated time to 115 | completion. 116 | One is to use a heap data structure, a priority queue backed by a nearly 117 | complete binary tree, using an array. 118 | Such a data structure will always keep the last estimated time to completion on 119 | top, requiring time proportional to the logarithm of the number of tracked 120 | promises to update an estimate, `O(log n)`. 121 | 122 | However, if instead one uses a simple array, they can find the last estimate 123 | in time proportional to the number of input promises, `O(n)`. 124 | Typically, these arrays are small, and this is sufficient. 125 | But even if the array is very large, there are few cases where we need to 126 | perform this linear search. 127 | If an individual estimate changes, the change is only relevant if it applies to 128 | the promise that once held the current aggregate estimate and it decreased. 129 | If the individual promise exceeds the current estimate, it becomes the new 130 | estimate. 131 | In all other cases, the individual estimate changes have no affect on the 132 | aggregate. 133 | Typically, the probability that we will need to perform a linear search for the 134 | new estimate is inversely proportional to the number of promises. Thus, the 135 | average time to update the aggregate estimate is amortized constant time, 136 | `O(n / n)`. 137 | 138 | I credit this epiphany to a conversation I had with Erik Bryn when we met after 139 | Fluent 2014. 140 | 141 | ## Then 142 | 143 | To be a useful measurement, the estimated time to completion must be composable 144 | through the `then` method of a promise. 145 | The then method takes an input promise, returns an output promise, and 146 | in the relevant case, calls the fulfillment handler. 147 | The estimated time to complete the output promise depends on the estimated time 148 | to fulfill the input promise, plus the estimated duration of the fulfillment 149 | handler. 150 | Unfortunately, there are not many cases where we can divine a meaningful 151 | estimate, but when we can, passing that duration as an argument to `then` 152 | provides enough information to compose the output estimate. 153 | 154 | For example, if the fulfillment handler is known to simply delay by one second, 155 | we can pass that as an argument to `then`. 156 | 157 | ```js 158 | return promise.then(function (value) { 159 | return Promise.return(value).delay(1000); 160 | }, null, 1000) 161 | ``` 162 | 163 | In another case, the fulfillment handler might consistently require the same 164 | amount of time, regardless of the inputs, in which case it is a simple matter of 165 | measuring. 166 | In other cases, we might need more sophisticated heuristics. 167 | 168 | ## Progress 169 | 170 | One can derrive a “progress” value from the estimated time to completion, the 171 | time that one began observing the promise, and the current time. 172 | 173 | ``` 174 | progress = (now - start) / (estimate - start) 175 | ``` 176 | 177 | However, the current time is always changing. 178 | It is not useful for a producer to push a new measurement at an arbitrary 179 | interval. 180 | It is however useful for a consumer to pull (or poll) a new value on demand, 181 | perhaps in tandem with the composition of animation frames. 182 | 183 | If a task makes progress smoothly, the promise can produce its estimated time to 184 | completion if it knows how much progress it has made until now from the time it 185 | began working. 186 | 187 | ``` 188 | esimtate = start + progress * (now - start) 189 | ``` 190 | 191 | ## Streams 192 | 193 | The above formula is good for estimating the time until the termination of a 194 | *stream* of known length. 195 | At each time an object or chunk is produced or consumed, one can emit a new 196 | estimated time to completion. 197 | 198 | In the following example, we assume that `response` is an HTTP response with 199 | `status`, `headers`, and `body`. The headers are normalized to lower-case key 200 | names, and the body is a *promise stream*, in this case, of strings. 201 | 202 | ```js 203 | function read(response) { 204 | var length = +response.headers["content-length"]; 205 | var start = Date.now(); 206 | var chunks = []; 207 | var chunksLength = 0; 208 | return new Promise(function (resolve, reject, setEstimate) { 209 | response.body 210 | .forEach(function (chunk) { 211 | chunks.push(chunk); 212 | chunksLength += chunk.length; 213 | var progress = chunksLength / length; 214 | var duration = Date.now() - start; 215 | setEstimate(start + progress * duration); 216 | }) 217 | .then(function () { 218 | resolve(Buffer.concat(chunks)); 219 | }, reject) 220 | .done(); 221 | }); 222 | } 223 | ``` 224 | 225 | ## Observing Estimates 226 | 227 | A measurement that has a current value that can change over time is best modeled 228 | as an *observable measurement*. 229 | This is distinct from a promise itself which may not have a current result, 230 | which may be a value or error, and can only resolve once. 231 | It would be strange to use promise idioms to expose a promise’s own estimated 232 | time to completion, just as it would be awkward to use an observable for the 233 | result of an asynchronous function call. 234 | 235 | For the experimental branch of Q, we can observe the estimate for a promise 236 | using `observeEstimate`. The given callback will be informed of the current 237 | estimate [as soon as possible][asap], and ASAP after each change. 238 | 239 | [asap]: https://github.com/kriskowal/asap 240 | 241 | ```js 242 | promise.observeEstimate(function (estimate) { 243 | component.estimate = estimate; 244 | component.needsDraw = true; 245 | }); 246 | ``` 247 | 248 | A pending promise can be informed of a new estimate using `setEstimate`. 249 | As with all things promise, information flows only from the creator of the 250 | promise to all its observers. 251 | As such, the `setEstimate` method is the third argument of the `Promise` 252 | initializer, and also exists on “deferred” objects. 253 | 254 | ```js 255 | new Promise(function (resolve, reject, setEstimate) { 256 | setEstimate(Date.now() + 1000); 257 | }); 258 | ``` 259 | 260 | ```js 261 | var deferred = Promise.defer(); 262 | deferred.setEstimate(Date.now() + 1000); 263 | return deferred.promise; 264 | ``` 265 | 266 | As consistent with most JavaScript interfaces, times are measured in miliseconds 267 | since the Epoch. 268 | An estimate of Infinity means that there is no meaningful estimate. 269 | `observeEstimate` will always produce a number and defaults to Infinity. 270 | 271 | The interface and behavior are subject to change as we shake-down this future 272 | release of Q, and all [feedback][] is welcome for anyone willing to brave the 273 | instability. 274 | 275 | [feedback]: https://github.com/kriskowal/q 276 | 277 | --------------------------------------------------------------------------------