├── README.md
├── design-rules.md
├── fs.md
├── stream.md
├── tcp.md
└── udp.md


/README.md:
--------------------------------------------------------------------------------
 1 | ## LL-API Interface
 2 | 
 3 | The purpose of these documents is to provide a uniform API across Node. It is
 4 | meant to be more low-level than the current API, has been tuned for performance
 5 | and flexibility. This API has the following goals:
 6 | 
 7 | 1) Be so light weight that it offers no observable impact on users' code.
 8 | 
 9 | 2) Be flexible enough that any other desired interface could be written using
10 |    it.
11 | 
12 | 3) Be resilient and allow the developer to recover.
13 | 
14 | 
15 | ### Documents
16 | 
17 | Most of the documents address API, but there are a couple that explain
18 | generalities.
19 | 
20 | * `README.md` - You're reading it.
21 | * `design-rules.md` - List of general rules for creating new APIs. It is to
22 |   help guide the design process so all APIs can feel as uniform as possible.
23 | * `fs.md` - Some revised API for the file system module.
24 | * `tcp.md` - A TCP API, instead of unified `'net'` module.
25 | * `udp.md` - A UDP API
26 | * `stream.md` - Some streams examples that will be used to write the spec.
27 | 


--------------------------------------------------------------------------------
/design-rules.md:
--------------------------------------------------------------------------------
 1 | ## Design Rules
 2 | 
 3 | This document outlines the rules of the overarching design behind the API. Any
 4 | API decision anywhere else should first consult this document. This document
 5 | should be expanded as the other APIs do.
 6 | 
 7 | All APIs should have a similar feel. Basically it should be intuitive to pick
 8 | up an API of a subsystem not used if another is already known.
 9 | 
10 | 
11 | ------
12 | 
13 | ### Class Structure
14 | 
15 | **Class functions** don't require an instance to run. Shouldn't have special
16 | cases where the instance is passed as an argument. Also meant to be a
17 | generalized call that can address all instances (e.g. `FS.rx()`).
18 | 
19 | **Instance properties** should be used with values that are set set once (e.g.
20 | retrieving the port a server is running on) or when user operation explicitly
21 | changes the property (e.g. popping an item off an array).
22 | 
23 | **Instance methods** also used as "getters" for values that can be changed
24 | internally (e.g. the "state" of a handle).
25 | 
26 | It is also possible to set a default callback for new instances. Reason for
27 | this is mainly for performance. For example, when an incoming connection is
28 | handled the callbacks set on the object either have to be made `Persistent`, or
29 | have to go through a property lookup from C++. Both those are slow. By setting
30 | a default this can be bypassed.
31 | 
32 | 
33 | ------
34 | 
35 | ### Callback Usage
36 | 
37 | Callbacks use the "error back" style of callback. As many errors are passed to
38 | the callback as possible.
39 | 
40 | All callbacks are called asynchronously. If an error is immediately detected
41 | then the error is immediately created, stored and set to be called
42 | asynchronously.
43 | 
44 | The calling context (e.g. `this`) of all callbacks should be the same as the
45 | handle making the call.
46 | 
47 | Depending on variable scope in outer function closures is prohibited. If a
48 | variable needs to reach a callback then it should be passed as an argument.
49 | 
50 | 
51 | ------
52 | 
53 | ### Events
54 | 
55 | There is no fancy event emitter. Instead all methods use similar `.on*(` syntax
56 | to which a single callback can be passed. An abstraction layer on top of that
57 | can be made to handle multiple callbacks for a single event.
58 | 
59 | The event callback can receive any number of arguments. This is greatly
60 | simplified by there being no multi-event handler. For example:
61 | ```javascript
62 | var e = new Event();
63 | e.onevent(function(err) {
64 |   // handle the event
65 | });
66 | 
67 | //// internally ////
68 | // can pass N arguments to the callback
69 | this.event_cb(err);
70 | ```
71 | 
72 | 
73 | ------
74 | 
75 | ### Exceptions
76 | 
77 | Throwing should be considered last resort. It should truly mean that there's an
78 | unrecoverable issue and the application needs to be brought down. When the API
79 | has no idea what to do with user's code. For example, when expecting a callback
80 | but none is passed there is then no place for any other errors to be passed.
81 | But say another argument is incorrect, that error should be created immediately
82 | then passed to the callback.
83 | 
84 | 
85 | ------
86 | 
87 | ### Misc
88 | 
89 | Items that don't yet have any particular place to go.
90 | 


--------------------------------------------------------------------------------
/fs.md:
--------------------------------------------------------------------------------
  1 | ## Class `File`
  2 | 
  3 | The `File` class allows operations on files by creating and returning a handle
  4 | to the given file. The handle only serves as a way to perform operations.
  5 | Creating the handle doesn't have any affect on the passed `path`.
  6 | 
  7 | **NOTE:** This spec does not contain the entire API that currently exists in
  8 | Node. Instead only those things that are new/critical/different.
  9 | 
 10 | #### `new File(path)`
 11 | 
 12 | * Returns `Object` file handle instance
 13 | 
 14 | Use the returned handle to operate on a file.
 15 | 
 16 | **Usage:**
 17 | ```javascript
 18 | var handle = new File('/some/path/to/file');
 19 | ```
 20 | 
 21 | **Reason:**
 22 | The reason the file path is passed to the `File()` constructor is because many
 23 | operations can then be taken from the returned handle. e.g. `stat()`,
 24 | `access()`, `open()`.
 25 | 
 26 | 
 27 | ------
 28 | 
 29 | ### Class Functions
 30 | 
 31 | 
 32 | #### `File.lsof()`
 33 | 
 34 | * Returns `Array` of file handles opened by the `File` module in this
 35 |   application instance
 36 | 
 37 | Useful for iterating over all file handles that have been opened by the `File`
 38 | module.
 39 | 
 40 | **Usage:**
 41 | ```javascript
 42 | var openHandles = File.lsof();
 43 | // Loop through and print the status of each file handle.
 44 | for (var handle of openHandles)
 45 |   console.log(handle.path, handle.status());
 46 | ```
 47 | 
 48 | 
 49 | #### `File.rx()`
 50 | 
 51 | * Returns `Number` of bytes read from the `File` module by the application
 52 |   since the start of the application
 53 | 
 54 | **Usage:**
 55 | ```javascript
 56 | // Print number of MB read to disk by File
 57 | console.log(File.rx() / 0x100000);
 58 | ```
 59 | 
 60 | 
 61 | #### `File.tx()`
 62 | 
 63 | * Returns `Number` of bytes written by the `File` module by the application
 64 |   since the start of the application
 65 | 
 66 | **Usage:**
 67 | ```javascript
 68 | // Print number of MB written from disk by File
 69 | console.log(File.tx() / 0x100000);
 70 | ```
 71 | 
 72 | 
 73 | ------
 74 | 
 75 | ### Instance Properties
 76 | 
 77 | #### `fh.path`
 78 | 
 79 | * Returns `String` of the path for the file handle
 80 | 
 81 | 
 82 | ------
 83 | 
 84 | ### Instance Methods
 85 | 
 86 | #### `fh.status()`
 87 | 
 88 | * Returns `String` of current file handle status
 89 | 
 90 | **NOTE:** The basic statuses would be `'open'`/`'closed'`, but others like
 91 | `'writing'`/`'reading'` could be helpful. Looking for feedback.
 92 | 
 93 | 
 94 | #### `fh.rx()`
 95 | 
 96 | * Returns `Number` of bytes read by the file handle.
 97 | 
 98 | 
 99 | #### `fh.tx()`
100 | 
101 | * Returns `Number` of bytes written to the file handle.
102 | 
103 | 
104 | #### `fh.open(flags[, mode], callback[, ... vargs])`
105 | 
106 | * `flags` - `String`
107 | * `mode` - `Number`, Default `0o666`
108 | * `callback` - `Function`
109 | * `vargs` - Variable number of `Value`s passed that will reach the callback
110 | * Returns `Object` file handle instance
111 | 
112 | Asynchronous file open. See open(2).
113 | 
114 | `flags` can be:
115 | 
116 | * `'r'` - Open file for reading. An `Error` is passed to `callback` if file
117 |   does not exist.
118 | * `'r+'` - Open file for reading and writing. An `Error` is passed to
119 |   `callback` if file does not exist.
120 | * `'rs'` - Open file for reading in synchronous mode. Instructs the operating
121 |   system to bypass the local file system cache. An `Error` is passed to
122 |   `callback` if file does not exist.
123 |   This is mainly useful for opening files on NFS mounts as it allows you to
124 |   skip the potentially stale local cache. It has a very real impact on I/O
125 |   performance.
126 | * `'rs+'` - Open file synchronously for reading and writing. An `Error` is
127 |   passed to `callback` if file does not exist. See notes on `'rs'` for usage.
128 | * `'w'` - Open file for writing. The file is created if it does not exist or is
129 |   truncated otherwise.
130 | * `'w+'` - Open file for reading and writing. The file is created if it does
131 |   not exist or is truncated otherwise.
132 | * `'a'` - Open file for appending. File is created if it does not exist.
133 | * `'ax'` - Open file for appending. An `Error` is passed to `callback` if file
134 |   does not exist.
135 | * `'a+'` - Open file for reading and appending. File is created if it does not
136 |   exist.
137 | * `'ax+'` - Open file for reading and appending. An `Error` is passed to
138 |   `callback` if file does not exist.
139 | 
140 | `mode` sets the file mode if the file was created.
141 | 
142 | `callback` receives one argument. `this` context is the file handle `Object`
143 | returned by `fh.open()`
144 | 
145 | **Usage:**
146 | ```javascript
147 | var handle = new File('/tmp/test.out');
148 | handle.open('r', function(err) {
149 |   if (err) { /* handle error */ }
150 | 
151 |   // use now file handle
152 | });
153 | ```
154 | 
155 | 
156 | #### `fh.close(callback[, ... vargs])`
157 | 
158 | * `callback` - `Function`
159 | * `vargs` - Variable number of `Value`s passed that will reach the callback
160 | * Returns `Object` file handle instance
161 | 
162 | Asynchronous close(2).
163 | 
164 | `callback` receives one argument. `this` context is the file handle that has
165 | been closed.
166 | 
167 | **Usage:**
168 | ```javascript
169 | var handle = new File('/tmp/test.out');
170 | handle.open('r', function(err) {
171 |   if (err) { /* handle error */ }
172 | 
173 |   this.close(handleClosed);
174 | });
175 | 
176 | function handleClosed(err) {
177 |   if (err) { /* handle error */ }
178 | 
179 |   // file handle has now been closed
180 | }
181 | ```
182 | 
183 | 
184 | #### `fh.read(callback[, ... vargs])`
185 | 
186 | * `callback` - `Function`
187 | * `vargs` - Variable number of `Value`s passed that will reach the callback
188 | * Returns `Object file handle instance
189 | 
190 | Read the entire contents of file into a new `Buffer`.
191 | 
192 | **Usage:**
193 | ```javascript
194 | var handle = new File('/tmp/test.out');
195 | handle.open('r', function(err) {
196 |   if (err) { /* handle error */ }
197 |   this.read(doneReading);
198 | });
199 | 
200 | function doneReading(err, bytesRead, buffer) {
201 |   if (err) { /* handle error */ }
202 |   this.close(handleClosed);
203 | }
204 | 
205 | function handleClosed(err) {
206 |   if (err) { /* handle closed */ }
207 | }
208 | ```
209 | 
210 | 
211 | #### `fh.read(buffer, offset, length[, position], callback[, ... vargs])`
212 | 
213 | * `buffer` - `Buffer`
214 | * `offset` - `Number` from beginning of buffer to write into
215 | * `length` - `Number` of bytes to read from file
216 | * `position` - `Number` of bytes offset from beginning of file
217 | * `callback` - `Function`
218 | * `vargs` - Variable number of `Value`s passed that will reach the callback
219 | * Returns `Object file handle instance
220 | 
221 | Read contents of file into supplied `buffer`.
222 | 
223 | **Usage:**
224 | ```javascript
225 | var handle = new File('/tmp/test.out');
226 | handle.open('r', function(err) {
227 |   if (err) { /* handle error */ }
228 |   var b = new Buffer(1024);
229 |   this.read(b, 0, 1024, doneReading);
230 | });
231 | 
232 | function doneReading(err, bytesRead, buffer) {
233 |   if (err) { /* handle error */ }
234 |   this.close(handleClosed);
235 | }
236 | 
237 | function handleClosed(err) {
238 |   if (err) { /* handle error */ }
239 | }
240 | ```
241 | 
242 | 
243 | #### `fh.write(buffer, offset, length[, position], callback[, ... vargs])`
244 | 
245 | * `buffer` - `Buffer` containing data to be written to handle
246 | * `offset` - `Number` offset from beginning of buffer
247 | * `length` - `Number` how many bytes to write
248 | * `position` - `Number` where in file to write, Default: `0`
249 | * `callback` - `Function` called when write is complete
250 | * `vargs` - Variable number of `Value`s passed that will reach the callback
251 | * Returns `Object` file handle instance
252 | 
253 | `callback` receives 3 arguments `(err, written, buffer)` where `written`
254 | specifies how many bytes from `buffer` were written.
255 | 
256 | Note that it is unsafe to use `fh.write()` multiple times on the same file
257 | without waiting for the callback.
258 | 
259 | On Linux, positional writes don't work when the file is opened in append mode.
260 | The kernel ignores the position argument and always appends the data to the end
261 | of the file.
262 | 
263 | **Usage:**
264 | ```javascript
265 | var handle = new File('/tmp/test.out');
266 | handle.open('r', function(err) {
267 |   if (err) { /* handle error */ }
268 |   var b = new Buffer(1024).fill('abc');
269 |   this.write(b, 0, b.length, 0, doneWriting);
270 | });
271 | 
272 | function doneWriting(err, written, buffer) {
273 |   if (err) { /* handle error */ }
274 |   this.close(handleClosed);
275 | }
276 | 
277 | function handleClosed(err) {
278 |   if (err) { /* handle error */ }
279 | }
280 | ```
281 | 
282 | 
283 | #### `fh.write(data[, position][, encoding], callback[, ... vargs])`
284 | 
285 | * `data` - `Value`
286 | * `position` - `Number`
287 | * `encoding` - `String` expected string encoding
288 | * `callback` - `Function`
289 | * `vargs` - Variable number of `Value`s passed that will reach the callback
290 | * Returns `Object` file handle instance
291 | 
292 | If `data` is not a `Buffer` then it will be coerced to a `String`.
293 | 
294 | `position` refers to the offset from the beginning of the file where this data
295 | should be written.
296 | 
297 | `callback` receives 3 arguments `(err, written, buffer)` where `written`
298 | specifies how many bytes from `buffer` were written.
299 | 
300 | If only a partial string needs to be written then please use `substr()` to grab
301 | the needed characters. Reason this is not handled in the API call is because
302 | the substring and the number of bytes it takes may not be equal.
303 | 
304 | Note that it is unsafe to use `fh.write()` multiple times on the same file
305 | without waiting for the callback.
306 | 
307 | On Linux, positional writes don't work when the file is opened in append mode.
308 | The kernel ignores the position argument and always appends the data to the end
309 | of the file.
310 | 
311 | **Usage:**
312 | ```javascript
313 | var handle = new File('/tmp/test.out');
314 | handle.open('r', function(err) {
315 |   if (err) { /* handle error */ }
316 |   this.write('hello world', 'latin1', doneWriting);
317 | });
318 | 
319 | function doneWriting(err, written, buffer) {
320 |   if (err) { /* handle error */ }
321 |   this.close(handleClosed);
322 | }
323 | 
324 | function handleClosed(err) {
325 |   if (err) { /* handle error */ }
326 | }
327 | ```
328 | 
329 | **Reason:**
330 | The reason for explicitly having an API that would allow writing strings
331 | instead of requiring all strings be first converted to a `Buffer` is for
332 | performance.  When the `String` is written into memory resources can be saved
333 | by not needing to attach that memory to an `Object`, then place the GC under
334 | extra strain to clean it up afterwards.
335 | 
336 | 
337 | 
338 | 
339 | ------
340 | 
341 | ### Misc Examples:
342 | 
343 | The following examples are to help bring new APIs under consideration, or to
344 | help hammer out existing APIs.
345 | 
346 | 
347 | Attempt to write a large file to disk, then cancel the write in progress if
348 | it's taking more than a given time.
349 | ```javascript
350 | var handle = new File('file');
351 | handle.open('w', function(err) {
352 |   if (err) { /* handle error */ }
353 | 
354 |   var b = new Buffer(1024 * 1024 * 500).fill('abc');  // 500 MB file
355 |   // Queue up the buffer to be written to disk
356 |   this.write(b, 0, b.length, finishedWriting);
357 | });
358 | 
359 | // Close the file handle after 100ms regardless of whether the file
360 | // has completed writing or not.
361 | setTimeout(function(h) {
362 |   if (!this.closed())
363 |     h.close(fdClosed);
364 | }, 100, handle);
365 | 
366 | function finishedWriting(err, length, buffer) {
367 |   if (err) {
368 |     // check "length" for amount of data written before error occurred.
369 |     // close handle if still open
370 |     if (!this.closed())
371 |       this.close(fdClosed);
372 |   }
373 | }
374 | 
375 | function fdClosed(err) {
376 |   if (err) { /* handle error */ }
377 | }
378 | ```
379 | 
380 | 
381 | Check the progress of writing a large file to disk.
382 | ```javascript
383 | var handle = new File('file');
384 | handle.open('w', function(err) {
385 |   if (err) { /* handle error */ }
386 | 
387 |   var b = new Buffer(1024 * 1024 * 500).fill('abc');
388 |   this.write(b, 0, b.length, finishedWriting);
389 | });
390 | 
391 | setInterval(function(h) {
392 |   // print progress of queue currently set to be written
393 |   console.log(h.progress());
394 | }, 1000, handle);
395 | 
396 | function finishedWriting(err, length, buffer) {
397 |   if (err) { /* handle error */ }
398 | 
399 |   this.close(fdClosed);
400 | }
401 | 
402 | function fdClosed(err) {
403 |   if (err) { /* handle error */ }
404 | }
405 | ```
406 | 
407 | 
408 | Alternative to `.progress()` would be to simply return the number of bytes
409 | remaining the queue to be written.
410 | ```javascript
411 | var handle = new File('file');
412 | handle.open('w', function(err) {
413 |   if (err) { /* handle error */
414 | 
415 |   var b = new Buffer(1024 * 1024 * 500).fill('abc');
416 |   this.write(b, 0, b.length, finishedWriting);
417 | });
418 | 
419 | setInterval(function(h) {
420 |   // print number of bytes remaining to be written
421 |   console.log(h.pending());
422 | }, 1000, handle);
423 | 
424 | function finishedWriting(err, length, buffer) {
425 |   if (err) { /* handle error */ }
426 | 
427 |   this.close(fdClosed);
428 | }
429 | 
430 | function fdClosed(err) {
431 |   if (err) { /* handle error */ }
432 | }
433 | ```
434 | 


--------------------------------------------------------------------------------
/stream.md:
--------------------------------------------------------------------------------
  1 | ## Streams
  2 | 
  3 | Currently this is in a phase for creating usage examples. The actual spec will
  4 | be created after enough examples have been accumulated. Everything here is up
  5 | for debate.
  6 | 
  7 | 
  8 | ------
  9 | 
 10 | ### Examples
 11 | 
 12 | Create a TCP server, pipe the data to a file and once the data has been queued
 13 | to be written to the file system echo it back out to the client. The stream
 14 | will automatically terminate after 10 MB have been written.
 15 | ```javascript
 16 | let server = new TCP();
 17 | 
 18 | server.onconnection(function onConnection(c) {
 19 |   // Generate random id for log file for each new connection.
 20 |   let id = Math.random().toString(36).substr(2);
 21 |   let fd = new File(`/tmp/${id}.log`);
 22 | 
 23 |   // Open log file for the new connection. Pass connection as argument
 24 |   // to fdOpen() callback.
 25 |   fd.open('r+', fdOpen, c);
 26 | });
 27 | 
 28 | 
 29 | function fdOpen(err, c) {
 30 |   if (err) { /* handle error */ }
 31 | 
 32 |   // Now that the log file is open, proceed reading from connection.
 33 |   c.onreadable(cOnReadable);
 34 | 
 35 |   // First pipe daata to the file descriptor, then pipe data back to the
 36 |   // connection to act as an echo server. Reason for chaining the pipes is so
 37 |   // each will be alerted to issues with the other.  Also setup close event
 38 |   // handlers for the pipes specifically.
 39 |   c.pipe(this).onclose(fdOnPClose).onwrite(fdOnPWrite)
 40 |     .pipe(c).onclose(cOnPClose).onwrite(cOnPWrite);
 41 | }
 42 | 
 43 | 
 44 | function cOnReadable() {
 45 |   // Echo data back out
 46 |   while (this.status() === 'readable' &&
 47 |          this.tx() <= 0xa00000)
 48 |     this.write(this.read());
 49 | 
 50 |   // If more than 1MB has been written then close the connection. This will
 51 |   // automatically run the pipe's onend callback so the fd will know it's time
 52 |   // to cleanup, if desired.
 53 |   if (this.tx() > 0xa00000)
 54 |     this.close();
 55 | }
 56 | 
 57 | 
 58 | // Called automatically when the connection calls .end().
 59 | function fdOnPClose(err) {
 60 |   if (err) { /* handle error */ }
 61 |   this.close();
 62 | }
 63 | 
 64 | 
 65 | function fdOnPWrite(err, data) {
 66 |   if (err) { /* handle error */ }
 67 | }
 68 | 
 69 | 
 70 | function cOnPClose(err) {
 71 |   if (err) { /* handle error */ }
 72 | }
 73 | 
 74 | 
 75 | function cOnPWrite(err, data) {
 76 |   if (err) { /* handle error */ }
 77 | }
 78 | ```
 79 | 
 80 | 
 81 | In this example `queued()` will be used to check how much data is waiting to be
 82 | written to a connection. Note, this means there is no need for a high water
 83 | mark for reading data. It simply depends on the user to check that the amount
 84 | of queued data is within their okay parameters before writing:
 85 | ```javascript
 86 | let server = new TCP();
 87 | 
 88 | server.onconnection(function onConnection(c) {
 89 |   let fh = new File('/some/random/path');
 90 |   fh.onreadable(fhOnReadable, c);
 91 |   fn.onend(fhOnEnd, c);
 92 |   fn.open();
 93 | }).listen(8001);
 94 | 
 95 | function fhOnReadable(c) {
 96 |   while (this.status() === 'readable') {
 97 |     // Make sure no more than 10 MB is queued to be written to the
 98 |     // connection. If there is then wait until all the data has flushed
 99 |     // before writing more.
100 |     if (c.queued() > 0xa00000)
101 |       return c.onflush(cOnFlush, this);
102 | 
103 |     c.write(this.read());
104 |   }
105 | }
106 | 
107 | function cOnFlush(fh) {
108 |   // Clear the flush callback. Only need this called if writing out
109 |   // can't keep up with the data being read.
110 |   this.onflush(null);
111 |   // For simplicity, hand logical control back over to the other cb.
112 |   fhOnReadable.call(fh, this);
113 | }
114 | 
115 | // File has been completely read, so close the connection.
116 | function fhOnEnd(c) {
117 |   this.close();
118 |   c.close();
119 | }
120 | ```
121 | **Note:** This does have repercussions for `pipe()`. If there is no high water
122 | mark then pipe will just push data in as fast as it can, or will pipe wait for
123 | a write to complete before writing more?
124 | 
125 | 
126 | This example shows using the write request object returned by `write()` to set
127 | a timeout on how long that write can be pending before it's canceled.
128 | ```javascript
129 | let server = new TCP();
130 | 
131 | server.onconnection(function onConnection(c) {
132 |   let writeReq = c.write('bye!');
133 |   let timeout = setTimeout(cancelWrite, 1000, writeReq);
134 |   // Set oncomplete after so timeout can be passed as an argument.
135 |   writeReq.oncomplete(writeComplete, timeout);
136 | }).listen(8001);
137 | 
138 | // The write hasn't completed in 1 sec, so cancel the write.
139 | function cancelWrite(writeReq) {
140 |   writeReq.cancel();
141 | }
142 | 
143 | // Write has completed so we can clear the timeout.
144 | function writeComplete(timeout) {
145 |   clearTimeout(timeout);
146 | }
147 | ```
148 | 
149 | 
150 | In this example the bubbled `server.queued()` is used to check the amount of
151 | data queued to be sent to all connections. If the amount queued is higher than
152 | a specific value then pause the connections until things are sorted out.
153 | Possibly `abort()` connections that have too many bytes pending.
154 | ```javascript
155 | let server = new TCP();
156 | 
157 | // Setup the default onreadable() callback for all connections.
158 | server.onreadable(cOnReadable);
159 | 
160 | // Now start listening for new connections.
161 | server.listen(8001);
162 | 
163 | function cOnReadable() {
164 |   // Shorthand the reference to the connection server.
165 |   let server = this.server;
166 | 
167 |   while (this.status() === 'readable') {
168 |     // If more than 100 MB of data are pending, fix it.
169 |     if (server.queued() > 0x6400000) {
170 |       // Loop through and abort any connections that are pending more than
171 |       // 1 MB to be written.
172 |       for (let handle of server.lsoc()) {
173 |         if (handle.queued() > 0x100000)
174 |           handle.abort();
175 |       }
176 |       // Check the amount of queued data again. If still high then simply
177 |       // pause all connections until all connections have flushed or are
178 |       // aborted. So the longest time lapse before the flush callback will
179 |       // run is 1 sec (based on timeout set below).
180 |       if (server.queued() > 0x6400000) {
181 |         server.pause();
182 |         server.onflush(serverOnFlush);
183 |       }
184 |     }
185 | 
186 |     let writeReq = this.write(this.read());
187 |     // Make sure data is written within 1 sec.
188 |     let timer = setTimeout(abortConnection, 1000, this);
189 |     // Clear the abort timer if the write completes in time.
190 |     writeReq.oncomplete(clearTimeout, timer);
191 |   }
192 | }
193 | 
194 | // All data to all connections has been flushed.
195 | function serverOnFlush() {
196 |   // Clear the flush callback.
197 |   this.onflush(null);
198 |   // Resume all connections.
199 |   this.resume();
200 | }
201 | 
202 | // Write took too long, so abort the connection.
203 | function abortConnection(c) {
204 |   c.abort(new Error('write request exceeded timeout'));
205 | }
206 | ```
207 | **Note:** The `queued()` callback must be defined by the implementing stream.
208 | Because for non byte streams it's up to the implementor to determine the value
209 | that `queued()` should return.
210 | 
211 | 
212 | This example shows usage of the `onclose()` error argument to determine what's
213 | happening with the stream, and recovery of the pipeline.
214 | ```javascript
215 | // Open log file to which all streams will be aggregated and logged.
216 | let fh = new File('random_log_file');
217 | // A custom stream that is meant to aggregate many different streams into a
218 | // single stream. This way many new TCP connections can all pipe to the same
219 | // stream and have their contents logged correctly.
220 | let sag = new StreamAggregation();
221 | let server = new TCP();
222 | 
223 | 
224 | // Passing as arguments for portability sake. Don't like depending on variable
225 | // scoping for asynchronous calls.
226 | fh.open('r+', function fhOnOpen(err, sag, server) {
227 |   // There was an error opening the log file. Cannot continue from here.
228 |   if (err)
229 |     throw err;
230 | 
231 |   // Pipe the stream aggregator to the file.
232 |   sag.pipe(this);
233 | 
234 |   // Now start accepting connections to the server.
235 |   server.listen(8001);
236 | }, sag, server);
237 | 
238 | 
239 | // The "onclose()" event lets the user know that the stream is able to be
240 | // closed. The input sources from piping results in ref counting to the piped
241 | // stream. When all connections have closed and ref count == 0 this callback is
242 | // called. Though it does _not_ mean the stream _will_ be closed, simply that
243 | // the stream is preparing to be closed. It is possible to keep this stream
244 | // alive. The only time it is not possible to keep the stream alive is if an
245 | // Error is passed to the stream. In that case something critically bad
246 | // happened and the stream must close so it can cleanup all its internal
247 | // resources.
248 | //
249 | // Since this callback is only suggestive, there's an internal close() callback
250 | // that needs to be set by the implementor that's run as the final step. This
251 | // callback cleans up all resources, and it's function should essentially be
252 | // invisible to the user.
253 | // TODO(trevnorris): What exactly are the semantics for keeping the stream
254 | // alive? There should be a call that simply states "keep this stream alive and
255 | // don't call the onclose callback again until the ref counter increases then
256 | // reaches zero again". Right now using .holdOpen(), but would love a better
257 | // name.
258 | sag.onclose(function sagOnClose(err, server) {
259 |   // Critical error on the stream itself. Nothing to do with the incoming
260 |   // streams. In this case we can pause the attached streams and attempt to
261 |   // resurrect the stream.
262 |   if (err) {
263 |     // Prevent the server from receiving any new connections until the stream
264 |     // error has been handled.
265 |     server.pause();
266 | 
267 |     // Now Walk the list of handles that have been attached to this stream via
268 |     // .pipe() and pause them.
269 |     // TODO(trevnorris): Not married to .handles(). Was thinking it would
270 |     // return .values() from the internal Set() of handles.
271 |     for (let handle of this.handles())
272 |       handle.readStop();
273 | 
274 |     // Recover the stream. This is implementation defined, and nothing
275 |     // specifically to do with streams.
276 |     this.recover(sagOnRecover, server);
277 | 
278 |     return;
279 |   }
280 | 
281 |   // There was no error. All the existing connections simply closed, so keep
282 |   // this stream alive and don't run the internal close cleanup.
283 |   this.holdOpen();
284 | }, server);
285 | 
286 | 
287 | function sagOnRecover(err, server) {
288 |   // If there was an error recovering the stream then bring down the server.
289 |   // Something is seriously wrong.
290 |   if (err)
291 |     throw err;
292 | 
293 |   // Stream has recovered. Resume the server and resume reading from all the
294 |   // connections.
295 |   for (let handle of this.handles())
296 |     handle.readStart();
297 |   server.resume();
298 | }
299 | 
300 | 
301 | // Setup the default onclose callback for all new connections
302 | server.onclose(function cOnClose(err) {
303 |   if (err) {
304 |     // Server had a critical error. Should probably bring down the process.
305 |   }
306 | });
307 | 
308 | 
309 | // TODO(trevnorris): Should a default pipe for all new connections be able to
310 | // be set via server.pipe(dest)?
311 | server.onconnection(function onConnection(c, sag) {
312 |   // Pipe the stream aggregator to the log file.
313 |   c.pipe(sag);
314 | }, sag);
315 | ```
316 | 


--------------------------------------------------------------------------------
/tcp.md:
--------------------------------------------------------------------------------
  1 | ## Class `TCP`
  2 | 
  3 | The `TCP` class creates a TCP handle.
  4 | 
  5 | **NOTE:** This spec does not contain the entire API that currently exists in
  6 | Node. Instead only those things that are new/critical/different.
  7 | 
  8 | #### `new TCP()`
  9 | 
 10 | * Returns `Object` TCP handle instance
 11 | 
 12 | **Usage:**
 13 | ```javascript
 14 | var server = new TCP();
 15 | ```
 16 | 
 17 | Is also used to create a TCP connection out to a remote service.
 18 | ```javascript
 19 | var client = new TCP();
 20 | client.connect(8080);
 21 | ```
 22 | 
 23 | 
 24 | ------
 25 | 
 26 | ### Class Functions
 27 | 
 28 | #### `TCP.rx()`
 29 | 
 30 | * Returns `Number` of bytes received over all TCP connections since the start
 31 |   of the application.
 32 | 
 33 | **Usage:**
 34 | ```javascript
 35 | // Print number of MB received from all connections
 36 | console.log(TCP.rx() / 0x100000);
 37 | ```
 38 | 
 39 | 
 40 | #### `TCP.tx()`
 41 | 
 42 | * Returns `Number` of bytes written to all TCP connections since the start of
 43 |   the application.
 44 | 
 45 | **Usage:**
 46 | ```javascript
 47 | // Print number of MB written to all connections
 48 | console.log(TCP.tx() / 0x100000);
 49 | ```
 50 | 
 51 | 
 52 | #### `TCP.stat()`
 53 | 
 54 | * Returns `Array` of all open server handles.
 55 | 
 56 | 
 57 | ------
 58 | 
 59 | ### Instance Properties
 60 | 
 61 | #### `server.port`
 62 | 
 63 | * Returns port `Number` the server is running on, or null if not running.
 64 | 
 65 | 
 66 | #### `server.hostname`
 67 | 
 68 | * Returns `String` of host name.
 69 | 
 70 | 
 71 | #### `server.type`
 72 | 
 73 | * Returns `String` of type of server (e.g. `'ipv4'`)
 74 | 
 75 | 
 76 | ------
 77 | 
 78 | ### Instance Methods
 79 | 
 80 | #### `server.rx()`
 81 | 
 82 | * Returns `Number` of bytes read from all connections.
 83 | 
 84 | 
 85 | #### `server.tx()`
 86 | 
 87 | * Returns `Number` of bytes written to all connections.
 88 | 
 89 | 
 90 | #### `server.lsoc()`
 91 | 
 92 | * Returns `Array` of all open connections to the server.
 93 | 
 94 | 
 95 | #### `server.listen(port[, hostname][, backlog])`
 96 | 
 97 | * `port` - `Number`
 98 | * `hostname` - `String`, Default `'localhost'`
 99 | * `backlog` - `Number`, Default `511`
100 | * Returns `Object` server instance
101 | 
102 | Begin listening on `port`. Consider this call immediate.
103 | 
104 | 
105 | #### `server.close([callback[, ... vargs]])`
106 | 
107 | * `callback` - `Function`
108 | * Returns `Object` server instance
109 | 
110 | Passing `callback` is a shorthand for setting `.onclose()`.
111 | 
112 | 
113 | #### `server.onconnection(callback[, ... vargs])`
114 | 
115 | * `callback` - `Function`
116 | * Returns `Object` server instance
117 | 
118 | Call `callback` when a new connection is received by the server. The connection
119 | is the first argument of `callback`.
120 | 
121 | 
122 | #### `server.onclose(callback[, ... vargs])`
123 | 
124 | * `callback` - `Function`
125 | * Returns `Object` server instance
126 | 
127 | Call `callback` when the server has closed. If an error caused the server to
128 | close will be passed as first argument to `callback`.
129 | 
130 | 
131 | #### `server.onreadable(callback[, ... vargs])`
132 | 
133 | * `callback` - `Function`
134 | * Returns `undefined`
135 | 
136 | Sets the default `onreadable()` callback for all new connections.
137 | 
138 | **Usage:**
139 | ```javascript
140 | var server = new TCP();
141 | 
142 | server.onreadable(function onReadable() {
143 |   // Echo data back to client
144 |   while (this.status() === 'readable') {
145 |     this.write(this.read());
146 |   }
147 | });
148 | 
149 | server.onconnection(function onConnection(c) {
150 |   // Nothing much to do here.
151 | });
152 | 
153 | server.listen(8080);
154 | ```
155 | 
156 | **Reason:**
157 | By setting a default callback for new instances of a connection construction
158 | can be made more performant. Specifically if the `callback` is made to be a
159 | `Persistent<Function>` on the class instance of the connection then a trip
160 | to C++ and a call to `Persistent::New()` can be saved in the process.
161 | 
162 | 
163 | #### `server.onend(callback[, ... vargs])`
164 | 
165 | * `callback` - `Function`
166 | * Returns `undefined`
167 | 
168 | Set the default `onend()` callback for all new connections.
169 | 
170 | **Usage:**
171 | ```javascript
172 | var server = new TCP();
173 | 
174 | server.onend(function onEnd(err) {
175 |   if (err) { /* handle error */ }
176 | });
177 | 
178 | server.onconnection(function onConnection(c) {
179 |   c.end('bye!');
180 | });
181 | 
182 | server.listen(8080);
183 | ```
184 | 
185 | **Reason:**
186 | By setting a default callback for new instances of a connection construction
187 | can be made more performant. Specifically if the `callback` is made to be a
188 | `Persistent<Function>` on the class instance of the connection then a trip
189 | to C++ and a call to `Persistent::New()` can be saved in the process.
190 | 
191 | 
192 | ------
193 | 
194 | TCP Client
195 | 
196 | #### `client.flush()`
197 | 
198 | * Returns client `Object` handle instance
199 | 
200 | All `write()` are automatically queued internally and then written at the end
201 | of the synchronous executable block. This takes advantage of `writev` where
202 | available, and can at least help minimize the number of syscalls where it's
203 | not. If you want the data to be written immediately call `flush()` on the
204 | connection.
205 | 
206 | **Usage:**
207 | ```javascript
208 | server.onconnection(function onConnection(c) {
209 |   c.write('hello ');
210 |   // Now write it out immediately
211 |   c.flush();
212 |   // And continue writing more
213 |   c.write('world!');
214 | });
215 | ```
216 | 
217 | 
218 | #### `client.write(data[, callback[, ... vargs]])`
219 | 
220 | * `data` - TBD
221 | * `callback` - `Function`
222 | * Returns `Object` write request
223 | 
224 | **Usage:**
225 | ```javascript
226 | server.onconnection(function(c) {
227 |   let req = c.write('hi!', onWritten);
228 |   req.time = process.hrtime();
229 | });
230 | 
231 | function onWritten(err) {
232 |   if (err) { /* handle error */ }
233 |   let t = process.hrtime(this.time);
234 |   console.log(`Write req took ${t[0] * 1e3 + t[1] / 1e6} ms`);
235 | }
236 | ```
237 | 
238 | **Reason:**
239 | By returning the actual write request created for the write information about
240 | the write request can be tracked during the lifetime of the request.
241 | 
242 | **Issues:**
243 | One issue is that under the hood `trywrite` will attempted to be used
244 | automatically.  Which means the write may have succeeded immediately. Another
245 | issue is that `writev` is used under the hood which means while the write is
246 | queued to be written, it will not actually be sent until the data is flushed.
247 | 


--------------------------------------------------------------------------------
/udp.md:
--------------------------------------------------------------------------------
  1 | ## Class `UDP`
  2 | 
  3 | The `UDP` class creates a UDP handle.
  4 | 
  5 | #### `new UDP([type])`
  6 | 
  7 | * `type` - `String` of handle type, Default `'udp4'`
  8 | * Returns new `Object` UDP handle instance
  9 | 
 10 | **Usage:**
 11 | ```javascript
 12 | let handle = new UDP();
 13 | ```
 14 | 
 15 | 
 16 | ------
 17 | 
 18 | ### Class Functions
 19 | 
 20 | #### `UDP.rx()`
 21 | 
 22 | * Returns `Number` of bytes received over all UDP connections since the start
 23 |   of the application.
 24 | 
 25 | **Usage:**
 26 | ```javascript
 27 | // Print number of MB received from all connections
 28 | console.log(UDP.rx() / 0x100000);
 29 | ```
 30 | 
 31 | 
 32 | #### `UDP.tx()`
 33 | 
 34 | * Returns `Number` of bytes written to all UDP connections since the start of
 35 |   the application.
 36 | 
 37 | **Usage:**
 38 | ```javascript
 39 | // Print number of MB written to all connections
 40 | console.log(UDP.tx() / 0x100000);
 41 | ```
 42 | 
 43 | 
 44 | #### `UDP.stat()`
 45 | 
 46 | * Returns `Array` of all open handles.
 47 | 
 48 | **Usage:**
 49 | ```javascript
 50 | let list = UDP.stat();
 51 | 
 52 | for (let handle of list)
 53 |   // Print how many bytes each connection has received
 54 |   console.log(handle.rx());
 55 | ```
 56 | 
 57 | 
 58 | ------
 59 | 
 60 | ### Instance Properties
 61 | 
 62 | #### `handle.port`
 63 | 
 64 | * Returns port `Number`.
 65 | 
 66 | 
 67 | #### `handle.address`
 68 | 
 69 | * Returns `String` of `address` passed to `.bind()`.
 70 | 
 71 | 
 72 | #### `handle.type`
 73 | 
 74 | * Returns `String` of handle type (e.g. `'udp4'`)
 75 | 
 76 | 
 77 | ------
 78 | 
 79 | ### Instance Methods
 80 | 
 81 | #### `handle.bind(port[, address][, exclusive][, callback[, ... vargs]])`
 82 | 
 83 | * `port` - `Number`
 84 | * `address` - `String`, Default `'localhost'`
 85 | * `exclusive` - `Boolean`, Default `false`
 86 | * `callback` - `Function`
 87 | * Returns UDP handle `Object`
 88 | 
 89 | Bind the handle to a `port` and optional `address`. If `address` is not
 90 | specified the OS will try to listen on all addresses. `callback` is used to
 91 | receive any errors that may have occurred while binding.
 92 | 
 93 | **Usage:**
 94 | ```javascript
 95 | let handle = new UDP('udp4');
 96 | 
 97 | handle.bind(8080, function onBind(err) {
 98 |   if (err) { /* handle error */ }
 99 |   console.log(`Listening on ${this.address}:${this.port}`);
100 | });
101 | ```
102 | 
103 | 
104 | #### `handle.close([callback[, ... vargs]])`
105 | 
106 | * `callback` - `Function`
107 | * Returns UDP handle `Object` instance
108 | 
109 | Stop listening for new data and close the underlying socket. Passing a
110 | `callback` is just short hand for setting the `onclose()` callback and will
111 | override any callback passed to `onclose()`.
112 | 
113 | 
114 | #### `handle.onclose(callback[, ... vargs])`
115 | 
116 | * `callback` - `Function`
117 | * Returns UDP handle `Object` instance
118 | 
119 | `callback` will receive an `Error` as the first argument if something went
120 | awry.
121 | 
122 | **Usage:**
123 | ```javascript
124 | let handle = new UDP('udp6');
125 | 
126 | handle.onclose(function onClose(err) {
127 |   if (err) { /* handle error */ }
128 | });
129 | 
130 | handle.bind(8080);
131 | ```
132 | 
133 | 
134 | #### `handle.write(data, offset, length, port[, address][, callback[, ... vargs]])`
135 | 
136 | * `data` - `Buffer` or `String`
137 | * `offset` - `Number`
138 | * `length` - `Number`
139 | * `port` - `Number`
140 | * `address` - `String`
141 | * `callback` - `Function`
142 | * Returns handle `Object` request
143 | 
144 | For UDP sockets, the destination `port` and `address` must be specified. A
145 | hostname may be supplied for the `address` parameter, and it will be resolved
146 | with DNS.
147 | 
148 | If the address is omitted or is an empty string, `'0.0.0.0'` or `'::0'` is used
149 | instead. Depending on the network configuration, those defaults may or may not
150 | work; it's best to be explicit about the destination address.
151 | 
152 | If the socket has not been previously bound with a call to bind, it gets
153 | assigned a random port number and is bound to the "all interfaces" address
154 | (`'0.0.0.0'` for udp4 sockets, `'::0'` for udp6 sockets.)
155 | 
156 | An optional callback may be specified to detect DNS errors or for determining
157 | when it's safe to reuse the buf object. Note that DNS lookups delay the time to
158 | send for at least one tick. The only way to know for sure that the datagram has
159 | been sent is by using a callback.
160 | 
161 | With consideration for multi-byte characters, `offset` and `length` will be
162 | calculated with respect to byte length and not the character position.
163 | 
164 | **Usage:**
165 | ```javascript
166 | let handle = new UDP('udp4');
167 | let msg = new Buffer('hello world!');
168 | handle.write(msg, 0, msg.length, 41234, function onWrite(err) {
169 |   if (err) { /* handle error */ }
170 |   this.close();
171 | });
172 | ```
173 | 
174 | **Notes:**
175 | The maximum size of an IPv4/v6 datagram depends on the MTU (_Maximum
176 | Transmission Unit_) and on the Payload Length field size.
177 | 
178 | * The Payload Length field is `16 bits` wide, which means that a normal payload
179 |   cannot be larger than 64K octets including internet header and data (65,507
180 |   bytes = 65,535 − 8 bytes UDP header − 20 bytes IP header); this is generally
181 |   true for loopback interfaces, but such long datagrams are impractical for
182 |   most hosts and networks.
183 | 
184 | * The `MTU` is the largest size a given link layer technology can support for
185 |   datagrams. For any link, `IPv4` mandates a minimum `MTU` of 68 octets, while
186 |   the recommended `MTU` for `IPv4` is 576 (typically recommended as the `MTU`
187 |   for dial-up type applications), whether they arrive whole or in fragments.
188 | 
189 |   For IPv6, the minimum `MTU` is 1280 octets, however, the mandatory minimum
190 |   fragment reassembly buffer size is 1500 octets. The value of 68 octets is
191 |   very small, since most current link layer technologies have a minimum `MTU`
192 |   of 1500 (like Ethernet).
193 | 
194 | Note that it's impossible to know in advance the `MTU` of each link through
195 | which a packet might travel, and that generally sending a datagram greater than
196 | the (receiver) `MTU` won't work (the packet gets silently dropped, without
197 | informing the source that the data did not reach its intended recipient).
198 | 
199 | 
200 | #### `handle.setBroadcast(flag)`
201 | 
202 | * `flag` - `Boolean`
203 | * Returns UDP handle `Object` instance
204 | 
205 | Sets or clears the `SO_BROADCAST` socket option. When this option is set, UDP
206 | packets may be sent to a local interface's broadcast address.
207 | 
208 | 
209 | #### `handle.setTTL(ttl)`
210 | 
211 | * `ttl` - `Number`, between `1` and `255`
212 | * Returns UDP handle `Object` instance
213 | 
214 | Sets the `IP_TTL` socket option. TTL stands for "Time to Live," but in this
215 | context it specifies the number of IP hops that a packet is allowed to go
216 | through. Each router or gateway that forwards a packet decrements the TTL. If
217 | the TTL is decremented to 0 by a router, it will not be forwarded. Changing TTL
218 | values is typically done for network probes or when multicasting.
219 | 
220 | The argument to `setTTL()` is a number of hops between 1 and 255. The default
221 | on most systems is 64.
222 | 
223 | 
224 | #### `handle.setMulticastTTL(ttl)`
225 | 
226 | * `ttl` - `Number`, between `1` and `255`
227 | * Returns UDP handle `Object` instance
228 | 
229 | Sets the `IP_MULTICAST_TTL` socket option. TTL stands for "Time to Live," but
230 | in this context it specifies the number of IP hops that a packet is allowed to
231 | go through, specifically for multicast traffic. Each router or gateway that
232 | forwards a packet decrements the TTL. If the TTL is decremented to 0 by a
233 | router, it will not be forwarded.
234 | 
235 | The argument to `setMulticastTTL()` is a number of hops between 0 and 255. The
236 | default on most systems is 1.
237 | 
238 | 
239 | #### `handle.setMulticastLoopback(flag)`
240 | 
241 | * `flag` - `Boolean`
242 | * Returns UDP handle `Object` instance
243 | 
244 | Sets or clears the `IP_MULTICAST_LOOP` socket option. When this option is set,
245 | multicast packets will also be received on the local interface.
246 | 
247 | 
248 | #### `handle.addMembership(multicastAddress[, multicastInterface])`
249 | 
250 | * `multicastAddress` - `String`
251 | * `multicastInterface` - `String`
252 | * Returns UDP handle `Object` instance
253 | 
254 | Tells the kernel to join a multicast group with `IP_ADD_MEMBERSHIP` socket
255 | option.
256 | 
257 | If `multicastInterface` is not specified, the OS will try to add membership to
258 | all valid interfaces.
259 | 
260 | 
261 | #### `handle.dropMembership(multicastAddress[, multicastInterface])`
262 | 
263 | * `multicastAddress` - `String`
264 | * `multicastInterface` - `String`
265 | * Returns UDP handle `Object` instance
266 | 
267 | Tells the kernel to leave a multicast group with `IP_DROP_MEMBERSHIP` socket
268 | option. This is automatically called by the kernel when the socket is closed or
269 | process terminates, so most apps will never need to call this.
270 | 
271 | If `multicastInterface` is not specified, the OS will try to drop membership to
272 | all valid interfaces.
273 | 
274 | 
275 | #### `handle.unref()`
276 | 
277 | * Returns UDP handle `Object` instance
278 | 
279 | Calling `unref()` on a socket will allow the program to exit if this is the
280 | only active socket in the event system. If the socket is already unrefd calling
281 | `unref()` again will have no effect.
282 | 
283 | 
284 | #### `handle.ref()`
285 | 
286 | * Returns UDP handle `Object` instance
287 | 
288 | Calling ref on a previously unrefd socket will not let the program exit if it's
289 | the only socket left (the default behavior). If the socket is refd calling ref
290 | again will have no effect.
291 | 


--------------------------------------------------------------------------------