├── .gitignore ├── .travis.yml ├── design.md ├── gulpfile.js ├── package.json ├── readme.md ├── spec.md ├── src ├── externs.js ├── index.js ├── index.ls └── index.min.js ├── tests ├── index.js └── index.ls └── why.md /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: node_js 2 | node_js: 3 | - node 4 | - lts/* 5 | script: travis_retry npm test 6 | -------------------------------------------------------------------------------- /design.md: -------------------------------------------------------------------------------- 1 | # Detox design 2 | 3 | Complements specification version: 0.5.1 4 | 5 | Author: Nazar Mokrynskyi 6 | 7 | License: Detox design (this document) is hereby placed in the public domain 8 | 9 | ### Introduction 10 | This document is a high level design overview of the Detox. 11 | The goal of this document is to give general understanding what Detox is, how it works and why it is designed the way it is. 12 | 13 | Refer to the specification if you intend to create alternative implementation. 14 | 15 | The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in IETF [RFC 2119](http://www.ietf.org/rfc/rfc2119.txt). 16 | 17 | #### Glossary 18 | * Initiator: the node that initiates communication 19 | * Responder: the node with which initiator wants to communicate 20 | * Routing path: a sequence of nodes that form a path through which initiator and responder are connected and can anonymously send encrypted data 21 | * Friend: a node that have established friendship with another node, so that these 2 nodes are friends with each other 22 | 23 | ### What Detox is 24 | Detox is an overlay network that is intended to offer security, strong anonymity, robustness and scalability, while having relatively low latency and capable of running in modern browsers without having any additional software installed. 25 | 26 | On the flip side it is only suitable for transferring small amounts of data and has large overhead because of constant flow of cover traffic. 27 | 28 | ### Foundation 29 | Detox is based on DHT and anonymous (onion) routing that share the same encrypted data channel. 30 | 31 | Transport layer in Detox is primarily based on WebRTC Data Channel, which is really the only P2P transport available in moder web browsers. 32 | There is only one direct connection between any 2 nodes and various types of traffic (DHT, Routing, internal commands) are multiplexed through this single connection. 33 | Constant bandwidth is used by each connection regardless of amount of useful data to be transferred. Data transfer rate is defined during node startup and doesn't change, cover traffic is sent when there is no useful data. 34 | 35 | Transport layer is designed in a way that protects against analysis of the traffic between 2 nodes in terms of shape and rate of transferred data. 36 | 37 | DHT is based on [ES-DHT](https://github.com/nazar-pc/es-dht) framework, make yourself familiar with ES-DHT first as this document will not cover it. 38 | 39 | DHT implementation in Detox makes following choices on top of ES-DHT framework: 40 | * ID space is 256 bits 41 | * uses ed25519 public key as node ID in DHT 42 | * uses ed25519 signatures for mutable values stored in DHT and Blake2b truncated to 256 bits for immutable values 43 | * plugs into shared transport layer based on WebRTC 44 | * bootstrap nodes additionally run HTTP server alongside regular DHT operations, so that other nodes can connect to them directly on startup 45 | 46 | DHT is designed in a way that doesn't allow to choose IDs deliberately and facilitates lookups over fixed snapshot of DHT so that adversary can't generate fake nodes on the fly as lookup progresses 47 | 48 | Anonymous routing is based on [Ronion](https://github.com/nazar-pc/ronion) framework, make yourself familiar with Ronion first as this document will not cover it. 49 | 50 | Following choices were made for this particular implementation of Ronion: 51 | * `Noise_NK_25519_ChaChaPoly_BLAKE2b` from [Noise Protocol Framework](https://noiseprotocol.org/) is used for encryption 52 | * [AEZ block cipher](http://web.cs.ucdavis.edu/%7Erogaway/aez/) is used for re-wrapping 53 | * after routing path construction data are only transferred from initiator to responder and back, all messages from other nodes on routing path to initiator are ignored 54 | * plugs into shared transport layer based on WebRTC 55 | 56 | Anonymous routing is designed in a way that doesn't reveal any information to nodes that provide routing tasks about initiator, responder or length of routing path, this way nodes that do routing tasks have only bare minimum of information they need to do what they are supposed to be doing. 57 | 58 | ### Types of key pairs 59 | There are 3 types of independent key pairs in Detox: bootstrap node's key pair, DHT key pair and long-term key pair (zero or multiple). 60 | 61 | Bootstrap node's key pair is only used by bootstrap nodes to sign their responses. It is fixed and never changes unless bootstrap node was compromised. 62 | 63 | DHT key pair is used for DHT operation, it is typically temporary and is used for anything besides DHT. New DHT key pair is typically re-generated on each startup. 64 | 65 | Long-term key pair identifies the user across sessions, public key of this key pair is used by friends to find each other. Bootstrap node will have no long-term key pairs and normal user can have any number of such key pairs for different purposes. 66 | 67 | System is designed in a way that all key pairs are independent and while exchanging data with friends, user's location is hidden and no one can easily link DHT and long-term key pairs together. 68 | 69 | ### Announcement to the network and discovery/connection 70 | In order for friends to find each other they need to know public key from long-term key pair of a friend and a secret. 71 | 72 | Public key acts as ID and secret is a random unique string used for authentication. 73 | 74 | Secret might be one of 2 kinds. 75 | The first is used for friendship requests and is not unique to a friend (MAY be changed at any time to prevent unwanted friendship requests). 76 | Once friendship is established friends generate a unique secret that they use during discovery for authentication between them, typically this secret is larger as user doesn't need to type it and is not needed to be rotated. If friendship is revoked by one friend the other one will not be able to connect using old secret and will need to request friendship again. A friend MAY notify the other one about revocation. 77 | 78 | When the node wants to announce itself to the network it will: 79 | * constructs one or more routing paths of desired length to nodes that will act as introduction nodes 80 | * generates announcement message that lists all of the introduction nodes, which is regular mutable data in DHT with long-term public key as its key 81 | * introduction nodes publish this announcement message to DHT 82 | 83 | When a friend wants to connect to other friend it will: 84 | * construct routing path of desired length to the node which will act as rendezvous node 85 | * gets introduction nodes through rendezvous node 86 | * generate invitation message that contains information about friend's long-term public key, rendezvous token, introduction node ID, encrypted and signed application name and secret for a friend together with rendezvous node ID so that friend can connect back if needed 87 | 88 | Rendezvous node will connect to introduction node and will ask to forward introduction, after which a friend will have to choices: 89 | * accept introduction, build another routing path of desired length to rendezvous node and ask to confirm connection with a friend 90 | * do nothing 91 | 92 | Routing paths lengths are selected depending on anonymity and performance requirements. Nodes MAY have no routing paths if they don't care about anonymity and might announce themselves as well as act as rendezvous nodes for themselves. 93 | 94 | ### Acknowledgements 95 | Detox is heavily inspired by [Tor](https://www.torproject.org/) and [Tox](https://tox.chat/). 96 | -------------------------------------------------------------------------------- /gulpfile.js: -------------------------------------------------------------------------------- 1 | /** 2 | * @package Detox core 3 | * @author Nazar Mokrynskyi 4 | * @license 0BSD 5 | */ 6 | require('build-gc').default('src/index.js', 'src/index.min.js', 'src/externs.js'); 7 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name" : "@detox/core", 3 | "description" : "Core library of Detox project that provides high-level APIs used when building end-user applications", 4 | "keywords" : [ 5 | "detox", 6 | "anonymous", 7 | "distributed", 8 | "p2p" 9 | ], 10 | "version" : "0.20.0", 11 | "homepage" : "https://github.com/Detox/core", 12 | "author" : "Nazar Mokrynskyi ", 13 | "repository" : { 14 | "type" : "git", 15 | "url" : "git://github.com/Detox/core.git" 16 | }, 17 | "license" : "0BSD", 18 | "main" : "src/index.js", 19 | "files" : [ 20 | "src" 21 | ], 22 | "scripts" : { 23 | "test" : "tape tests/**/*.js" 24 | }, 25 | "dependencies" : { 26 | "@detox/crypto" : "^0.7.1", 27 | "@detox/dht" : "^0.16.9", 28 | "@detox/nodes-manager" : "^0.1.3", 29 | "@detox/routing" : "^0.3.2", 30 | "@detox/transport" : "^0.38.2", 31 | "@detox/utils" : "^1.10.0", 32 | "async-eventer" : "^1.1.7", 33 | "fixed-size-multiplexer" : "^1.0.2", 34 | "node-fetch" : "^2.1.2" 35 | }, 36 | "devDependencies" : { 37 | "@detox/simple-peer-mock" : "github:Detox/simple-peer-mock", 38 | "build-gc" : "^0.2.0", 39 | "tape" : "^4.9.1" 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # Detox core [![Travis CI](https://img.shields.io/travis/Detox/core/master.svg?label=Travis%20CI)](https://travis-ci.org/Detox/core) 2 | Core library of Detox project that provides high-level APIs used when building end-user applications. 3 | 4 | This repository contains high level design overview ([design.md](https://github.com/Detox/core/blob/master/design.md)), specification for implementors ([spec.md](https://github.com/Detox/core/blob/master/spec.md)) and reference implementation. 5 | 6 | Essentially glues `@detox/crypto`, `@detox/transport`, `@detox/dht` and `@detox/routing` together and provides very simple API for connecting to and interacting with Detox network. 7 | 8 | WARNING: INSECURE UNTIL PROVEN THE OPPOSITE!!! 9 | 10 | This protocol and reference implementation are intended to be secure, but until proper independent audit is conducted you shouldn't consider it to actually be secure and shouldn't use in production applications. 11 | 12 | ## Why? 13 | Rough explanation and short comparison to networks such as Tor and Loopix is given in [why.md](https://github.com/Detox/core/blob/master/why.md). 14 | 15 | ## Key features 16 | Detox network is an overlay network that uses HTTP and WebRTC technologies under the hood and is capable of running in modern web browser (with caveat that some HTTP bootstrap nodes are still needed). 17 | 18 | Here are a few key features that Detox network aims to offer: 19 | * security 20 | * strong anonymity 21 | * robustness 22 | * scalability 23 | 24 | For features mentioned above we sacrifice maximum throughput, latency (to a degree) and efficiency (there is a lot of cover traffic), which makes it only suitable for low-bandwidth and relatively low-latency (few seconds) data transfers. 25 | 26 | ### Security 27 | Relies on [The Noise Protocol Framework](https://noiseprotocol.org/), more specifically, on `Noise_NK_25519_ChaChaPoly_BLAKE2b` for end-to-end encryption. 28 | 29 | Should be secure already (WARNING: not proven by independent cryptographer yet, so don't rely on it being actually secure!). 30 | 31 | ### Strong anonymity 32 | Data transfer is always at constant rate, regardless of data presence and its size. 33 | 34 | [Ronion](https://github.com/nazar-pc/ronion) anonymous routing framework with AEZ block cipher (not secure, but functionally working implementation, see [aez.wasm](https://github.com/nazar-pc/aez.wasm)) and `Noise_NK_25519_ChaChaPoly_BLAKE2b` used for anonymous routing (WARNING: not proven by independent cryptographer yet, so don't rely on it being actually secure!). 35 | Nodes for anonymous routing will need to be somehow received from DHT, which is a challenging and currently unsolved issue. 36 | 37 | Higher level glue is used to select introduction nodes, announce them to DHT, then on other node select rendezvous node and introduce itself using rendezvous node through introduction node to a friend. 38 | 39 | Anonymity is implemented on architecture level, but implementation is not anonymous yet. 40 | 41 | ### Robustness 42 | Code and its dependencies are a still bit fragile. Partially robustness is responsibility of the higher level consumer (for instance, there is no confirmation that data were received). 43 | 44 | Robustness is implemented on architecture level, but more work is needed to make it much more robust. 45 | 46 | ### Scalability 47 | Scalability is based on scalability of DHT implementation (based on [ES-DHT](https://github.com/nazar-pc/es-dht)). 48 | 49 | Should be scalable already (WARNING: not proven yet, large-scale testing and mathematical proof is needed). 50 | 51 | ## Major open issues 52 | Major open issues in the order from more important to less important (the order is not strict): 53 | * Nodes selection for anonymous routing (will likely require DHT re-implementation) 54 | * Make AEZ implementation secure (timings attacks in particular) 55 | * Compute performance characteristics of ES-DHT 56 | * Conduct security audit for Ronion 57 | * Conduct security audit of a project as the whole 58 | 59 | ## How to install 60 | ``` 61 | npm install @detox/core 62 | ``` 63 | 64 | ## How to use 65 | Node.js: 66 | ```javascript 67 | var detox_core = require('@detox/core') 68 | 69 | detox_core.ready(function () { 70 | // Do stuff 71 | }); 72 | ``` 73 | Browser: 74 | ```javascript 75 | requirejs(['@detox/core'], function (detox_core) { 76 | detox_core.ready(function () { 77 | // Do stuff 78 | }); 79 | }) 80 | ``` 81 | 82 | ## API 83 | ### detox_core.ready(callback) 84 | * `callback` - Callback function that is called when library is ready for use 85 | 86 | ### detox_core.generate_seed() : Uint8Array 87 | Generates random seed that can be later used in `detox_core.Core` constructor. 88 | 89 | ### detox_core.Core(bootstrap_nodes : string[], ice_servers : Object[], packets_per_second = 1 : number, bucket_size = 2 : number, options = {} : Object) : detox_core.Core 90 | Constructor for Core object, offers methods for connecting to and interacting with Detox network. 91 | 92 | * `bootstrap_nodes` - array of strings in format `node_id:address:port` 93 | * `ice_servers` - array of objects as in [RTCPeerConnection constructor](https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/RTCPeerConnection) 94 | * `packets_per_second` - packets are sent at constant rate (which together with fixed packet size of 512 bytes can be used to identify bandwidth requirements for specific connection), `1` is minimal supported rate, actual rate is negotiated between 2 sides on connection 95 | * `bucket_size` - size of the bucket used in DHT internals (directly affects number of active WebRTC connections) 96 | * `max_pending_segments` - row much segments can be in pending state per one address in router 97 | * `options` - more options that are less frequently used (see source code for defaults): 98 | * `dht_keypair_seed` - seed that corresponds to temporary user identity in DHT network 99 | * `state_history_size` - how many DHT versions of local history will be kept 100 | * `values_cache_size` - how many DHT values will be kept in cache 101 | * `fraction_of_nodes_from_same_peer` - max fraction of nodes originated from single peer allowed on lookup start 102 | * `lookup_number` - number of nodes to be returned if exact match was not found 103 | * `timeouts` - various timeouts and intervals used internally, refer to source code and corresponding components for details and default values: 104 | * `GET_PROOF_REQUEST_TIMEOUT` (used in DHT) 105 | * `GET_STATE_REQUEST_TIMEOUT` (used in DHT) 106 | * `GET_VALUE_TIMEOUT` (used in DHT) 107 | * `PUT_VALUE_TIMEOUT` (used in DHT) 108 | * `STATE_UPDATE_INTERVAL` (used in DHT) 109 | * `CONNECTION_TIMEOUT` (used by Core and Transport) 110 | * `LAST_USED_TIMEOUT` (used by Core) 111 | * `ANNOUNCE_INTERVAL` (used by Core) 112 | * `STALE_AWARE_OF_NODE_TIMEOUT` (used by Core) 113 | * `GET_MORE_AWARE_OF_NODES_INTERVAL` (used by Nodes manager) 114 | * `RANDOM_LOOKUPS_INTERVAL` (used by Core) 115 | * `ROUTING_PATH_SEGMENT_TIMEOUT` (used by Router) 116 | * `max_pending_segments` - how any routing segments can be in pending state per one address 117 | * `aware_of_nodes_limit` - how many aware of nodes should be kept in memory 118 | * `min_number_of_peers_for_ready` - how many peers should be connected in order to consider instance ready for use and fire `ready` event 119 | * `connected_nodes_limit` - keep total number of active connections under this number (soft limit) 120 | 121 | ### detox_core.Core.start_bootstrap_node(bootstrap_seed : Uint8Array, ip : string, port : number, public_address = ip : string, public_port = port : number) 122 | * `bootstrap_seed` - seed for generating bootstrap node's keypairs 123 | * `ip` - IP on which bootstrap server will be started 124 | * `port` - port on which bootstrap server will be started 125 | * `public_address` - publicly available address that will be returned to other node, typically domain name (instead of using IP) 126 | * `public_port` - publicly available port on `public_address` 127 | Start bootstrap server (HTTP) listening on specified IP and port, optionally referred externally by specified address (like domain name) and port. 128 | 129 | ### detox_core.Core.get_bootstrap_nodes() : string[] 130 | Returns array of collected bootstrap nodes obtained during DHT operation in the same format as `bootstrap_nodes` argument in constructor. 131 | 132 | ### detox_core.Core.announce(real_key_seed : Uint8Array, number_of_introduction_nodes : number, number_of_intermediate_nodes : number) : Uint8Array|null 133 | Announce itself to the DHT network (without this it is still possible to interact with network and connect to friends, but friends will not be able to discover this node). 134 | 135 | Listen for events to identify when/if announcement succeeded. 136 | 137 | * `real_key_seed` - seed that corresponds to long-term user identity for connecting with friends (from which bootstrap node's `node_id` is derived) 138 | * `number_of_introduction_nodes` - non-zero number of nodes that will act as introduction nodes 139 | * `number_of_intermediate_nodes` - non-zero number of intermediate nodes between this node and introduction node (not including it) used during routing path construction for anonymity 140 | 141 | ### detox_core.Core.connect_to(real_key_seed : Uint8Array, target_id : Uint8Array, application : Uint8Array, secret : Uint8Array, number_of_intermediate_nodes : number) : Uint8Array|null 142 | Connecting to a friend with `target_id` and `secret`. 143 | 144 | Listen for events to identify when/if connection succeeded. NOTE: there is no way to know if a friend refused to answer or simply not available. 145 | 146 | * `real_key_seed` - seed that corresponds to long-term user identity for connecting with friends 147 | * `target_id` - long-term public key of a friend 148 | * `application` - Application-specific string up to 64 bytes that both friends should understand 149 | * `secret` - secret that will be sent to a friend, up to 32 bytes, typically used for friend requests and identification as kind of a password 150 | * `number_of_intermediate_nodes` - number of intermediate nodes between this node and rendezvous node (not including it) used during routing path construction for anonymity 151 | 152 | Returns real public key or `null` in case of failure. 153 | 154 | ### detox_core.Core.get_max_data_size() : number 155 | Returns how much data can be sent at once. 156 | 157 | NOTE: this is a maximum supported limit, because of network architecture sending large portions of data might take a lot of time. 158 | 159 | ### detox_core.Core.send_to(real_public_key : Uint8Array, target_id : Uint8Array, command : number, data : Uint8Array) 160 | Send data to previously connected friend. 161 | 162 | * `real_public_key` - own real long-term public key as returned by `announce()` and `connect_to()` methods 163 | * `target_id` - long-term public key of a friend 164 | * `command` - command for data, can be any number from the range `0..245` 165 | * `data` - data being sent, size limit can be obtained with `get_max_data_size()` method, roughly 65KiB 166 | 167 | ### detox_core.Core.destroy() 168 | Stops bootstrap server, destroys all connections. 169 | 170 | ### detox_core.Core.on(event: string, callback: Function) : detox_core.Core 171 | Register event handler. 172 | 173 | ### detox_core.Core.once(event: string, callback: Function) : detox_core.Core 174 | Register one-time event handler (just `on()` + `off()` under the hood). 175 | 176 | ### detox_core.Core.off(event: string[, callback: Function]) : detox_core.Core 177 | Unregister event handler. 178 | 179 | ### Event: ready 180 | No payload. 181 | Event is fired when Core instance is ready to be used. 182 | 183 | ### Event: introduction 184 | Payload is `data` object with properties `real_public_key`, `target_id`, `application`, `secret` and `number_of_intermediate_nodes`. 185 | Event is fired when a `target_id` friend is asking for introduction for `real_public_key` using application `application` (exactly 64 bytes as used in `connect_to` method, if supplied application was smaller that 64 bytes then zeroes are appended) with `secret` (exactly 32 bytes as used in `connect_to` method, if supplied secret was smaller that 32 bytes then zeroes are appended). 186 | If node decides to accept introduction and establish connection, it sets `number_of_intermediate_nodes` property to the number of intermediate nodes between this node and rendezvous node of a friend (not including it) used during routing path construction for anonymity. 187 | 188 | ### Event: data 189 | Payload consists of four arguments: `real_public_key` (`Uint8Array`), `target_id` (`Uint8Array`), `command` (`number`) and `data` (`Uint8Array`). 190 | Event is fired when a friend have sent data using `send_to()` method. 191 | 192 | ### Event: announcement_failed 193 | Payload consists of two arguments: `real_public_key` (`Uint8Array`) and `reason` (`number`), which is one of `detox_core.Core.ANNOUNCEMENT_ERROR_*` constants. 194 | Event is fired when announcement failed. 195 | 196 | ### Event: announced 197 | Payload is a single argument `real_public_key` (`Uint8Array`). 198 | Event is fired when announcement succeeded. 199 | 200 | ### Event: connection_failed 201 | Payload consists of three arguments: `real_public_key` (`Uint8Array`), `target_id` (`Uint8Array`) and `reason` (`number`), which is one of `detox_core.Core.CONNECTION_ERROR_*` constants. 202 | Event is fired when connection to `target_id` failed. 203 | 204 | ### Event: connection_progress 205 | Payload consists of three arguments: `real_public_key` (`Uint8Array`), `target_id` (`Uint8Array`) and `stage` (`number`), which is one of `detox_core.Core.CONNECTION_PROGRESS_*` constants. 206 | Event is fired when there is a progress in the process of connecting to `target_id`. 207 | 208 | ### Event: connected 209 | Payload consists of two `Uint8Array` arguments: `real_public_key` and `target_id`. 210 | Event is fired when connection to `target_id` succeeded. 211 | 212 | ### Event: disconnected 213 | Payload consists of two `Uint8Array` arguments: `real_public_key` and `target_id`. 214 | Event is fired when `target_id` disconnected for whatever reason. 215 | 216 | ### Event: connected_nodes_count 217 | Payload is a single argument `count` (`number`). 218 | Event is fired when new direct connection with node is established or destroyed. 219 | 220 | ### Event: aware_of_nodes_count 221 | Payload is a single argument `count` (`number`). 222 | Event is fired when number of nodes which current node is aware of changes. 223 | 224 | ### Event: routing_paths_count 225 | Payload is a single argument `count` (`number`). 226 | Event is fired when new routing path is established or destroyed. 227 | 228 | ### Event: application_connections_count 229 | Payload is a single argument `count` (`number`). 230 | Event is fired when new application connection established or destroyed. 231 | 232 | ## Contribution 233 | Feel free to create issues and send pull requests (for big changes create an issue first and link it from the PR), they are highly appreciated! 234 | 235 | When reading LiveScript code make sure to configure 1 tab to be 4 spaces (GitHub uses 8 by default), otherwise code might be hard to read. 236 | 237 | ## License 238 | Implementation: Free Public License 1.0.0 / Zero Clause BSD License 239 | 240 | https://opensource.org/licenses/FPL-1.0.0 241 | 242 | https://tldrlegal.com/license/bsd-0-clause-license 243 | 244 | Specification and design: public domain 245 | -------------------------------------------------------------------------------- /spec.md: -------------------------------------------------------------------------------- 1 | # Detox specification 2 | 3 | Specification version: 0.5.1 4 | 5 | Author: Nazar Mokrynskyi 6 | 7 | License: Detox specification (this document) is hereby placed in the public domain 8 | 9 | ### Introduction 10 | This document is a textual specification of Detox. The goal of this document is to give enough guidance to permit a complete and correct implementation. 11 | 12 | Refer to the design document if you need a high level overview. 13 | 14 | The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in IETF [RFC 2119](http://www.ietf.org/rfc/rfc2119.txt). 15 | 16 | #### Glossary 17 | * Initiator: the node that initiates communication 18 | * Responder: the node with which initiator wants to communicate 19 | * Routing path: a sequence of nodes that form a path through which initiator and responder are connected and can anonymously send encrypted data 20 | * Friend: a node that have established friendship with another node, so that these 2 nodes are friends with each other 21 | * Peer: the node that has direct connection with local node 22 | 23 | #### Multiplexing/demultiplexing 24 | Multiplexing is happening by appending the original data with data length header (depending on maximum supported data length can be a number in either 1 or more bytes in big endian format) and then splitting the result into chunks of specified size. 25 | If there is not enough data to fill the chunk, we append zero length data until the size of the result is bigger or equal to the size of the chunk (basically, appending data length header containing only zeroes). 26 | 27 | In order to demultiplex chunks back into original data we read data size from buffer of received chunks and check if we've received enough data already. 28 | If yes - remove data length header and actual data from the buffer, return the data and read next data length header. If not - wait till next chunk. 29 | 30 | ### Data channel 31 | Data channel in Detox network is WebRTC's RTCDataChannel, where data are sent in packets of 512 bytes at a time after at a fixed rate configured during startup. 32 | 33 | Each piece of data being sent is prepended with a command and then multiplexed into data channel and demultiplexed back on receiving side. 34 | 35 | Command is 1 byte unsigned integer from range `0..255`. This range is split into few sub-ranges for different purposes. 36 | 37 | One way to split commands is by compression: 38 | * compressed commands are from range `0..19` 39 | * uncompressed commands are from range `20..255` 40 | 41 | Maximum data length for uncompressed commands is 65534 bytes (one byte is occupied by command), hence data length header for multiplexing/demultiplexing will be 2 bytes. 42 | Data for compressed commands are prepended by one byte containing `0` if data were not compressed and `1` if data were compressed. This is because sometimes compressed data can be longer than uncompressed. Because of this one byte, maximum data length for uncompressed commands is 65533 bytes. 43 | Compression process is described in section "Data channel compression" below. 44 | 45 | So each piece of data sent over data channel have at least 3 bytes overhead (4 bytes in case of compressed commands), if there is no useful data - empty data blocks (essentially just data length headers) are sent. 46 | 47 | Another way to split commands is by their purpose: 48 | * commands from range `0..9` are DHT commands `DHT_COMMAND_*` 49 | * commands from range `10..19` are translated into compressed core commands `COMPRESSED_CORE_COMMAND_*` from range `0..9` 50 | * command `20` is routing command and is consumed by Router directly 51 | * commands from range `21..255` are translated into uncompressed core commands `UNCOMPRESSED_CORE_COMMAND_*` from range `0..234` 52 | 53 | These types of commands will be described in detail in corresponding sections below. 54 | 55 | Data MUST be sent at fixed rate specified by user (in packets per second, which is internally translated into delay in milliseconds) in alternating direction: first packet forward, first packet back, second packet forward, second packet back and so on. 56 | 57 | Packet is sent in response after delay that corresponds to packets rate, but not earlier than next packet is received in response. 58 | This way it is possible to have different packets rates on 2 sides and don't exceed packets rate in general without any need for additional packets rate negotiation. 59 | If strict alternating order of packets is not honored, it is considered to be protocol violation and connection MUST be dropped immediately. 60 | 61 | #### Data channel compression 62 | While most of data sent through data channel are encrypted and look random, DHT commands and compressed core commands will contain data that can be greatly compressed. 63 | They are also quite lengthy, which means that they often don't fit into single data channel packet or even two of them. 64 | 65 | In order to improve data transfer efficiency, data for these command are compressed with zlib using dictionary. 66 | Dictionary is composed from last 5 pieces of compressed data sent concatenated together with most recent at the end. Compression in each direction is independent. 67 | 68 | This way common identifiers and commands are compressed very efficiently with low CPU load and much more often commands fit into single data channel packet, which means much lower latency. 69 | 70 | In case when compression results in data length that is larger than max allowed data size for compressed data, uncompressed data is sent instead. 71 | 72 | ### Core commands 73 | There are some commands that do not belong to DHT or routing tasks directly, they are called core commands here and can be compressed and uncompressed. 74 | 75 | Following compressed commands are available: 76 | 77 | | Command name | Numeric value | 78 | |---------------------------------|---------------| 79 | | COMPRESSED_CORE_COMMAND_SIGNAL | 0 | 80 | 81 | `COMPRESSED_CORE_COMMAND_SIGNAL` is used for transport layer when there is a need to connect to peer's peer and contains following data: 82 | * 32 bytes - source ID, own DHT public key 83 | * 32 bytes - target ID, DHT public key of peer's peer 84 | * 1 byte - 0 or SDP answer or 1 for SDP offer 85 | * X bytes - SDP itself from WebRTC (no Trickle ICE) 86 | * 64 bytes - ed25519 signature of SDP that corresponds to own DHT public key 87 | 88 | When peer receives `COMPRESSED_CORE_COMMAND_SIGNAL` it will first check command data for correctness. 89 | If correct, target ID is compared to own DHT public key and if matches, sends `COMPRESSED_CORE_COMMAND_SIGNAL` command back with own SDP answer details. 90 | If target ID doesn't match to own DHT public key, but matches one of peer's DHT public key, command is forwarded there. 91 | 92 | This way peer can facilitate connection between 2 of its peers, it is actively used during lookup process in DHT. 93 | 94 | Following uncompressed commands are available: 95 | 96 | | Command name | Numeric value | 97 | |------------------------------------------------|---------------| 98 | | UNCOMPRESSED_CORE_COMMAND_FORWARD_INTRODUCTION | 0 | 99 | | UNCOMPRESSED_CORE_COMMAND_GET_NODES_REQUEST | 1 | 100 | | UNCOMPRESSED_CORE_COMMAND_GET_NODES_RESPONSE | 2 | 101 | | UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE | 3 | 102 | 103 | Uncompressed core commands will be described in other sections below, since they are a part of more complex procedures. 104 | 105 | ### DHT 106 | DHT is based on [ES-DHT](https://github.com/nazar-pc/es-dht) framework, make yourself familiar with ES-DHT first as this document will not cover it. 107 | 108 | DHT implementation in Detox makes following choices on top of ES-DHT framework: 109 | * ID space is 256 bits 110 | * uses ed25519 public key as node ID in DHT 111 | * uses ed25519 signatures for mutable values stored in DHT and Blake2b truncated to 256 bits for immutable values 112 | * bootstrap nodes additionally run HTTP server alongside regular DHT operations, so that other nodes can connect to them directly on startup 113 | 114 | Here is the list of DHT commands: 115 | 116 | | Command name | Numeric value | 117 | |-----------------------|---------------| 118 | | DHT_COMMAND_RESPONSE | 0 | 119 | | DHT_COMMAND_GET_STATE | 1 | 120 | | DHT_COMMAND_GET_PROOF | 2 | 121 | | DHT_COMMAND_GET_VALUE | 3 | 122 | | DHT_COMMAND_PUT_VALUE | 4 | 123 | 124 | * `DHT_COMMAND_RESPONSE` - generic command sent in response to `DHT_COMMAND_GET_STATE`, `DHT_COMMAND_GET_PROOF` and `DHT_COMMAND_GET_VALUE` commands 125 | * `DHT_COMMAND_GET_STATE` - is used to get latest state of a peer 126 | * `DHT_COMMAND_GET_PROOF` - is used to get a proof that certain peer's peer is in specific state version 127 | * `DHT_COMMAND_GET_VALUE` - is used to get mutable or immutable value 128 | * `DHT_COMMAND_PUT_VALUE` - is used to put mutable or immutable value 129 | 130 | Each command's payload has following structure: 131 | * 2 bytes transaction ID 132 | * the rest is command data 133 | 134 | Response to commands that expect such a response will need to include the same transaction ID as during request. 135 | 136 | #### Get state 137 | In order to get state of a peer (see ES-DHT for details on what state is and why is it needed), `DHT_COMMAND_GET_STATE` command is sent to a peer with following data: 138 | * 0 bytes or 32 bytes - either no data (to get latest state) or state version to get specific state 139 | 140 | Response to `DHT_COMMAND_GET_STATE` command contains following data: 141 | * 32 bytes - state version 142 | * 1 byte - unsigned integer, proof height (number of proof blocks) 143 | * 33 bytes * proof_height - proof itself, proofs that peer's ID is in this state version 144 | * 32 bytes * number of peers - IDs of peer's peers that correspond to state version 145 | 146 | Upon receiving response, node MUST check response for correctness. 147 | 148 | #### Get proof 149 | During lookup process it might be necessary to get a proof that peer's peer corresponds to state version (proof will also contain peer's peer state version). 150 | Proof can be requested using `DHT_COMMAND_GET_PROOF` command with following data: 151 | * 32 bytes - state version for which to get proof 152 | * 32 bytes - peer's peer ID for which to get proof 153 | 154 | Response to `DHT_COMMAND_GET_PROOF` command contains following data: 155 | * 33 bytes * X - proof of height X 156 | 157 | Upon receiving response, node MUST check response for correctness. 158 | 159 | #### Get value 160 | Whenever node wants to get value from DHT, `DHT_COMMAND_GET_VALUE` command is sent with data: 161 | * 32 bytes - key of the value 162 | 163 | Response to `DHT_COMMAND_GET_VALUE` command contains following data: 164 | * 0 bytes or X bytes - no data in case value with specified key is unknown or value data structure (see "Put value" section below) 165 | 166 | Upon receiving response, node MUST check response for correctness. 167 | 168 | #### Put value 169 | There 2 types of values in DHT: mutable and immutable. 170 | 171 | Immutable values are easy: key of the value is Blake2b hash of the value truncated to 256 bits. 172 | 173 | Mutable values are more complex: key for mutable value is ed25519 public key and value itself is represented by following data structure: 174 | * 4 bytes - unsigned integer in big endian format, value version (allows value updating over time, higher version number overrides lower) 175 | * X bytes - value itself 176 | * 64 bytes - ed25519 signature for version concatenated with value that corresponds to value's key 177 | 178 | In order to put value into DHT node through lookup searches for nodes closest to value's key and sends `DHT_COMMAND_PUT_VALUE` to each of nodes with following data: 179 | * 32 bytes - value's key 180 | * X bytes - value's data structure in case of mutable value or simply value itself in case of immutable value. 181 | 182 | Upon receiving `DHT_COMMAND_PUT_VALUE` request, node MUST check it for correctness. 183 | 184 | Max value length is 1024 bytes. 185 | 186 | ### Bootstrap node 187 | Bootstrap node is a node that besides regular routing tasks also performs bootstraping for other nodes. 188 | 189 | Bootstrap node runs a simple HTTP server that accepts POST request with the same payload as `COMPRESSED_CORE_COMMAND_SIGNAL` command data (see "Core commands" section above), but with target ID containing all zeroes. 190 | Upon receiving such request, bootstrap node has 2 options: 191 | * either consume these signaling data and generate response payload the same as `COMPRESSED_CORE_COMMAND_SIGNAL` command data 192 | * or forward request as `COMPRESSED_CORE_COMMAND_SIGNAL` to one of its peer while replacing target ID with actual ID of a peer 193 | 194 | In either case, response to POST HTTP request will contain: 195 | * X bytes - payload according to `COMPRESSED_CORE_COMMAND_SIGNAL` format 196 | * 64 bytes - ed25519 signature that corresponds to bootstrap node's key pair (which might be different from DHT key pair, so that load balancing can be applied) 197 | 198 | This way node that joins the network can connect to 1 new node in DHT at a time using one bootstrap node. 199 | 200 | When DHT node connects to new peer, it will send `UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE` command in response with following data: 201 | * X bytes - string in format `node_id:address:port` where `node_id` is bootstrap node's ed25519 public key, `address` and `port` can be used for HTTP connections to bootstrap node 202 | 203 | `UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE` is a way for bootstrap node to advertise to other DHT nodes that it supports bootstrapping and will not accept routing commands. 204 | 205 | Bootstrap node doesn't provide any routing tasks, it only supports DHT commands, `UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE` and `COMPRESSED_CORE_COMMAND_SIGNAL`. 206 | Essentially, all routing features MUST be disabled and node only operates as DHT bootstrap node. 207 | 208 | #### Router 209 | Anonymous router is based on [Ronion](https://github.com/nazar-pc/ronion) framework, make yourself familiar with Ronion first as this document will not cover it. 210 | 211 | Following choices were made for this particular implementation of Ronion: 212 | * packet size is 509 bytes (512 of data channel packet - 3 for data channel packet header) 213 | * address in 32 bytes DHT public key (see keypairs section below) 214 | * `Noise_NK_25519_ChaChaPoly_BLAKE2b` from [Noise Protocol Framework](https://noiseprotocol.org/) is used for encryption/decryption (payload on `CREATE_REQUEST`, `CREATE_RESPONSE` and `EXTEND_REQUEST` is Noise's handshake message) 215 | * [AEZ block cipher](http://web.cs.ucdavis.edu/%7Erogaway/aez/) is used for re-wrapping (keys for wrapping/unwrapping with AEZ are received by encrypting 32 zero bytes with empty additional data using send/receive Noise CipherState used for encryption/decryption, together with 16 bytes MAC it will give identical 48 bytes keys for wrapping and unwrapping on both sides; nonce is 12 zero bytes and before each wrapping/unwrapping it is incremented starting from the last byte and moving to the first one) 216 | * data MUST only be sent between initiator and responder, all other data sent by other nodes on routing path MUST be ignored 217 | 218 | Here is the list of commands supported on Router level: 219 | 220 | | Command name | Numeric value | 221 | |--------------------------------------------------|---------------| 222 | | ROUTING_COMMAND_ANNOUNCE | 0 | 223 | | ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST | 1 | 224 | | ROUTING_COMMAND_FIND_INTRODUCTION_NODES_RESPONSE | 2 | 225 | | ROUTING_COMMAND_INITIALIZE_CONNECTION | 3 | 226 | | ROUTING_COMMAND_INTRODUCTION | 4 | 227 | | ROUTING_COMMAND_CONFIRM_CONNECTION | 5 | 228 | | ROUTING_COMMAND_CONNECTED | 6 | 229 | | ROUTING_COMMAND_DATA | 7 | 230 | | ROUTING_COMMAND_PING | 8 | 231 | 232 | * `ROUTING_COMMAND_ANNOUNCE` - is used for announcement node to the network (see "Announcement to the network" section below) 233 | * `ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST` - is used for requesting introduction nodes from rendezvous node (see "Discovery and connection to a friend" section below) 234 | * `ROUTING_COMMAND_FIND_INTRODUCTION_NODES_RESPONSE` - response for `ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST` (see "Discovery and connection to a friend" section below) 235 | * `ROUTING_COMMAND_INITIALIZE_CONNECTION` - is used for instructing rendezvous node to initialize connection to target node through introduction node (see "Discovery and connection to a friend" section below) 236 | * `ROUTING_COMMAND_INTRODUCTION` - is used by introduction node to send introduction to target node (see "Discovery and connection to a friend" section below) 237 | * `ROUTING_COMMAND_CONFIRM_CONNECTION` - is used by target node to respond to introduction and establish connection through rendezvous node (see "Discovery and connection to a friend" section below) 238 | * `ROUTING_COMMAND_CONNECTED` - is used by rendezvous node to confirm that connection to target node is established (see "Discovery and connection to a friend" section below) 239 | * `ROUTING_COMMAND_DATA` - is used for data sending and forwarding (see "Sending data to a friend" section below) 240 | * `ROUTING_COMMAND_PING` - is used for ensuring connection is still working (see "Announcement to the network" section below) 241 | 242 | #### One-way encryption 243 | In some cases one-way encryption is used when there is a need to send encrypted piece of data, but there is no two-way communication yet (like during connection to a friend). 244 | 245 | In this case Noise's `Noise_N_25519_ChaChaPoly_BLAKE2b` is used and the output is as follows: 246 | * 48 bytes Noise handshake message 247 | * ciphertext, same length as plaintext 248 | * 16 bytes MAC 249 | 250 | ### Selection of nodes for routing path creation 251 | When routing path is created (see "Routing path creation" section below), we need a set of nodes through which to create this routing path. 252 | 253 | The first node in routing path MUST be always the random node to which direct connection is already established, at the same time each new routing path MUST start from unique node (e.g. there MUST NOT be 2 routing paths started from the same node). The rest of the nodes MUST be those to which direct connections are not yet established. 254 | 255 | Node can send `UNCOMPRESSED_CORE_COMMAND_GET_NODES_REQUEST` transport command with empty contents to the other nodes and in response it will receive `UNCOMPRESSED_CORE_COMMAND_GET_NODES_RESPONSE` transport command that contains concatenated list of up to 10 unique random IDs of nodes queried node is aware of. 256 | This list SHOULD NOT include nodes that are directly connected to queried node, instead it consists of 3 to 5 random peers of queried node's peers and up to 7 nodes that were collected from `UNCOMPRESSED_CORE_COMMAND_GET_NODES_RESPONSE` by queried node. 257 | 258 | When routing path is created, necessary number of nodes is selected from these known nodes. 259 | 260 | TODO: This is a very naive approach and must be improved in future iterations of the spec! 261 | 262 | ### Routing path creation 263 | Routing path creation is regular Router routing path with pair of multiplexers/demultiplexers on both sides. 264 | 265 | Nodes for routing path are selected as described in "Selection of nodes for routing path creation" section above. 266 | 267 | Multiplexers/demultiplexers are only used for sending and receiving data, routing commands defined in Ronion specification MUST fit into single packet with max size of 509 bytes, so that they take at most 1 data channel packet (actual size of payload that fits into single packet is 488 bytes as Routing is encrypted and has its own overhead). 268 | In case command payload is 0 bytes, it should be replaced with zero byte payload, otherwise demultiplexer will treat it as useless padding and will discard such command. 269 | 270 | ### Announcement to the network 271 | Announcement to the network is done anonymously through introduction nodes. 272 | 273 | Node that wants to introduce itself to the network first creates a few routing paths to introduction nodes. 274 | 275 | Once connections are established, node generates announcement message, which is a mutable DHT value so that: 276 | * key is a long-term Ed25519 public key that node wants to announce itself under 277 | * version number is Unix timestamp in seconds 278 | * value is a concatenated IDs of all of the introduction nodes to which routing paths were created 279 | 280 | Announcement message contains following data: 281 | * 32 bytes - long-term Ed25519 public key 282 | * X bytes - mutable value data structure (see "Put value" section above) 283 | 284 | Announcement message is not published to DHT directly, instead node sends `ROUTING_COMMAND_ANNOUNCE` routing command to introduction nodes through previously created routing paths with announcement message as payload. 285 | When node receives `ROUTING_COMMAND_ANNOUNCE` (and validates its correctness) it becomes aware that it is now acting as introduction node for someone and MUST publish to announcement message to DHT directly, also each 30 minutes introduction node MUST re-send announcement message to DHT. 286 | 287 | Node that announced itself SHOULD send `ROUTING_COMMAND_PING` routing command with empty contents to introduction node at least once per 60 seconds to make sure connection is kept alive, otherwise routing path can be destroyed by introduction node and it will stop forwarding introductions (see "Discovery and connection to a friend" section below). 288 | 289 | When `ROUTING_COMMAND_PING` is received, node MUST send the same `ROUTING_COMMAND_PING` routing command with empty contents back. 290 | 291 | ### Discovery and connection to a friend 292 | Discovery and connection to a friend also happens anonymously using rendezvous node selected by the node that wants to connect and introduction node selected by a friend during announcement to the network. 293 | 294 | First of all, node creates routing path to rendezvous node. 295 | 296 | Once connection is established, `ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST` routing command is sent to rendezvous node with data that contains long-term public key of a friend. 297 | Rendezvous node uses DHT to find an item using long-term public key as DHT key. 298 | 299 | Once search is done rendezvous node responds with `ROUTING_COMMAND_FIND_INTRODUCTION_NODES_RESPONSE` routing command which contains data as follows: 300 | * 1 byte status code (see below) 301 | * 0 or more Ed25519 public keys of introduction nodes for requested long-term public key 302 | 303 | Status codes: 304 | 305 | | Code name | Numeric value | 306 | |-----------------------------|---------------| 307 | | OK | 0 | 308 | | ERROR_NO_INTRODUCTION_NODES | 1 | 309 | 310 | * `OK` - introduction nodes were found successfully 311 | * `ERROR_NO_INTRODUCTION_NODES` - introduction nodes were not found 312 | 313 | Once introduction nodes are found, random introduction node is selected and introduction message is created as follows: 314 | * 64 bytes - Ed25519 signature of Ed25519 public key of introduction node (not part of introduction payload) concatenated with introduction payload using node's long-term keypair 315 | * 240 bytes - introduction payload 316 | 317 | Introduction payload is created as follows: 318 | * 32 bytes - Own long-term public key 319 | * 32 bytes - Ed25519 public key of rendezvous node 320 | * 32 bytes - rendezvous token (one-time randomly generated string) 321 | * 48 bytes - Noise handshake message for end-to-end encryption with a friend (the same `Noise_NK_25519_ChaChaPoly_BLAKE2b` is used as in routing, long-term public key is used as remote public static key) 322 | * 64 bytes - application (to be interpreted by applications on both sides of conversation, if shorter than 64 bytes MUST be padded with zeroes) 323 | * 32 bytes - secret (to be interpreted by remote node, SHOULD be negotiated beforehand) 324 | 325 | Once introduction message is created, it is one-way encrypted (see "One-way encryption" section above) with long-term public key of a friend. 326 | 327 | After this `ROUTING_COMMAND_INITIALIZE_CONNECTION` routing command is sent to rendezvous node with contents as follows: 328 | * 32 bytes - rendezvous token, the same as in introduction payload 329 | * 32 bytes - introduction node, the same as in introduction payload 330 | * 32 bytes - target node to which to connect, the same as was requested in `ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST` 331 | * 368 bytes - encrypted introduction message 332 | 333 | When node receives `ROUTING_COMMAND_INITIALIZE_CONNECTION` routing command it becomes aware that it is now acting as rendezvous node for someone. 334 | Now rendezvous node MUST connect to specified introduction node and send `UNCOMPRESSED_CORE_COMMAND_FORWARD_INTRODUCTION` transport command with contents as follows: 335 | * 32 bytes - target node to which to connect 336 | * 368 bytes - encrypted introduction message 337 | 338 | When introduction node receives `UNCOMPRESSED_CORE_COMMAND_FORWARD_INTRODUCTION` transport command it will send `ROUTING_COMMAND_INTRODUCTION` routing command to target node through routing path established before `ROUTING_COMMAND_ANNOUNCE` with encrypted introduction message as contents. 339 | 340 | When target node receives `ROUTING_COMMAND_INTRODUCTION` from one of introduction nodes it will: 341 | * decrypt introduction message 342 | * verify signature with introduction node ID command came from 343 | * check if application is known and supported 344 | * check if secret is valid taking into account ID of the node that wants to communicate 345 | 346 | If secret and application are fine and node wants to establish communication, it will creates new routing path to rendezvous node and will sent `ROUTING_COMMAND_CONFIRM_CONNECTION` routing command with contents as follows: 347 | * 64 bytes - Ed25519 signature of rendezvous token using node's long-term keypair 348 | * 32 bytes - rendezvous token 349 | * 48 bytes - Noise handshake message for end-to-end encryption with a friend (acts as responder, long-term public key is used as local private static key) 350 | 351 | Once rendezvous node receives `ROUTING_COMMAND_CONFIRM_CONNECTION` command, it will: 352 | * check if rendezvous token is known 353 | * verify that signature is valid using target node ID from `ROUTING_COMMAND_INITIALIZE_CONNECTION` 354 | 355 | If rendezvous token is known and signature is valid, rendezvous node will sent `ROUTING_COMMAND_CONFIRM_CONNECTION` routing command back to the node that sent `ROUTING_COMMAND_INITIALIZE_CONNECTION` with contents as follows: 356 | * 64 bytes - Ed25519 signature of rendezvous token using node's long-term keypair 357 | * 32 bytes - rendezvous token 358 | * 48 bytes - Noise handshake message for end-to-end encryption with a friend (acts as responder, long-term public key is used as local private static key) 359 | 360 | Once `ROUTING_COMMAND_CONFIRM_CONNECTION` is received, connection to a friend is considered established and any `ROUTING_COMMAND_DATA` routing commands received on routing paths MUST be blindly forwarded by rendezvous node to target node and back. 361 | 362 | NOTE: When two nodes initiated connection to each other approximately at the same time, race condition is resolved by discarding connection initiated by the node whose public key is smaller (comparing bytes starting from the first one). 363 | 364 | ### Friendship requests 365 | Friendship is not specified as special entity or something, but there is a way to make it work. 366 | 367 | Node can create special `secret` (see "Discovery and connection to a friend" section above) that will be commonly used for friendship requests. 368 | This way when `ROUTING_COMMAND_INTRODUCTION` routing command is received with this `secret`, application might interpret it as friendship request instead of rejecting immediately. 369 | 370 | In case this `secret` happens to appear in hands of spammers, it can be changed and all requests with old one will be ignored. Moreover, application can have multiple such `secret`s for different purposes, so that it doesn't have to revoke all of them at once. 371 | 372 | ### Sending data to a friend 373 | In order to make sure data packets always fit into single data channel packet multiplexing/demultiplexing is used with max data length of 65535 bytes and packet size of 472 bytes: 374 | * 512 of data channel packet 375 | * \- 3 for data channel packet header 376 | * \- 16 for block-level MAC (we encrypt each block with one-way encryption, see "One-way encryption" section above, as it will be sent through rendezvous node, which MUST NOT be able to read contents) 377 | * \- 2 for Ronion's segment ID 378 | * \- 1 for Ronion's command 379 | * \- 2 for Ronion's command data length 380 | * \- 16 for Ronion's MAC 381 | 382 | This way each encrypted block of data will be encrypted and will occupy exactly 1 data channel packet, so that even rendezvous node will not know what data of which size it forwards. 383 | 384 | Data are send to rendezvous node using `ROUTING_COMMAND_DATA` routing command by 2 sides of the conversation and rendezvous node transparently forwards data to the other side just like if 2 friends have direct routing path between them. 385 | 386 | Each piece of data has command and payload. Command is 1 byte unsigned integer and payload is what needs to be sent. Command interpretation and payload format depends on application. 387 | 388 | In order to send data to a friend, node: 389 | * concatenates command byte and payload 390 | * multiplexes the result into chunks of 472 bytes each 391 | * encrypts each chunk with Noise's ChiperState established using Noise handshake messages exchanged during connection process (see "Discovery and connection to a friend" section above) 392 | * sends each encrypted chunk using `ROUTING_COMMAND_DATA` routing command to rendezvous node 393 | 394 | When `ROUTING_COMMAND_DATA` routing command is received, node: 395 | * decrypts contents with Noise's CipherState 396 | * feeds the result into demultiplexer 397 | * when demultiplexer returns data, then first byte is command and the rest is payload 398 | 399 | ### Protocol violations and errors handling 400 | Whenever peer explicitly violates protocol (sends incorrect proof, incorrect mutable value or similar), node MUST disconnect from such peer immediately and blacklist it. 401 | 402 | If peer violates protocol implicitly (by not responding in time, failing to forward signaling data or similar), warning with timestamp SHOULD be remembered. 403 | 404 | High number of warnings over short period of time will also result in disconnection and blacklisting of such peer. 405 | 406 | TODO: specification doesn't yet specify exact blacklisting timeout or detailed warnings heuristics, this will need to be defined in future versions of the specification. 407 | 408 | ### Acknowledgements 409 | Detox is heavily inspired by [Tor](https://www.torproject.org/) and [Tox](https://tox.chat/). 410 | -------------------------------------------------------------------------------- /src/externs.js: -------------------------------------------------------------------------------- 1 | /** 2 | * @package Detox core 3 | * @author Nazar Mokrynskyi 4 | * @license 0BSD 5 | */ 6 | /** 7 | * @param {Array} dependencies 8 | * @param {Function} wrapper 9 | */ 10 | var define = function (dependencies, wrapper) {}; 11 | /** 12 | * @param {string} module 13 | */ 14 | var require = function (module) {}; 15 | var exports = {}; 16 | var module = {}; 17 | module.exports = {}; 18 | /** 19 | * @param {number} size 20 | * 21 | * @return {!Uint8Array} 22 | */ 23 | crypto.randomBytes = function(size) {}; 24 | var Buffer = {}; 25 | /** 26 | * @param {!Uint8Array} source 27 | */ 28 | Buffer.from = function (source) {}; 29 | -------------------------------------------------------------------------------- /src/index.ls: -------------------------------------------------------------------------------- 1 | /** 2 | * @package Detox core 3 | * @author Nazar Mokrynskyi 4 | * @license 0BSD 5 | */ 6 | /* 7 | * Implements version 0.5.0 of the specification 8 | */ 9 | const DHT_COMMANDS_OFFSET = 10 # 0..9 are reserved as Core commands 10 | const ROUTING_COMMANDS = 20 # 10..19 are reserved as DHT commands 11 | const UNCOMPRESSED_COMMANDS_OFFSET = ROUTING_COMMANDS # Core and DHT commands are compressed 12 | const UNCOMPRESSED_CORE_COMMANDS_OFFSET = 21 13 | 14 | const COMPRESSED_CORE_COMMAND_SIGNAL = 0 15 | 16 | const UNCOMPRESSED_CORE_COMMAND_FORWARD_INTRODUCTION = 0 17 | const UNCOMPRESSED_CORE_COMMAND_GET_NODES_REQUEST = 1 18 | const UNCOMPRESSED_CORE_COMMAND_GET_NODES_RESPONSE = 2 19 | const UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE = 3 20 | 21 | const ROUTING_COMMAND_ANNOUNCE = 0 22 | const ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST = 1 23 | const ROUTING_COMMAND_FIND_INTRODUCTION_NODES_RESPONSE = 2 24 | const ROUTING_COMMAND_INITIALIZE_CONNECTION = 3 25 | const ROUTING_COMMAND_INTRODUCTION = 4 26 | const ROUTING_COMMAND_CONFIRM_CONNECTION = 5 27 | const ROUTING_COMMAND_CONNECTED = 6 28 | const ROUTING_COMMAND_DATA = 7 29 | const ROUTING_COMMAND_PING = 8 30 | 31 | const PUBLIC_KEY_LENGTH = 32 32 | const SIGNATURE_LENGTH = 64 33 | # Handshake message length for Noise_NK_25519_ChaChaPoly_BLAKE2b 34 | const HANDSHAKE_MESSAGE_LENGTH = 48 35 | # ChaChaPoly+BLAKE2b 36 | const MAC_LENGTH = 16 37 | # Length of the application name used during introduction 38 | const APPLICATION_LENGTH = 64 39 | const DEFAULT_TIMEOUTS = 40 | # How long node should wait for rendezvous node to receive incoming connection from intended responder 41 | 'CONNECTION_TIMEOUT' : 10 42 | # After specified number of seconds since last data sending or receiving connection or route is considered unused and can be closed 43 | 'LAST_USED_TIMEOUT' : 60 44 | # Re-announce each 5 minutes 45 | 'ANNOUNCE_INTERVAL' : 5 * 60 46 | # After 5 minutes aware of node is considered stale and needs refreshing or replacing with a new one 47 | 'STALE_AWARE_OF_NODE_TIMEOUT' : 5 * 60 48 | # New aware of nodes will be fetched and old refreshed each 30 seconds 49 | 'GET_MORE_AWARE_OF_NODES_INTERVAL' : 30 50 | # Do random lookups about each 60 seconds 51 | 'RANDOM_LOOKUPS_INTERVAL' : 60 52 | # Max time in seconds allowed for routing path segment creation after which creation is considered failed 53 | 'ROUTING_PATH_SEGMENT_TIMEOUT' : 10 54 | 55 | const CONNECTION_OK = 0 56 | const CONNECTION_ERROR_NO_INTRODUCTION_NODES = 1 57 | const CONNECTION_ERROR_CANT_FIND_INTRODUCTION_NODES = 2 58 | const CONNECTION_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES = 3 59 | const CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE = 4 60 | const CONNECTION_ERROR_OUT_OF_INTRODUCTION_NODES = 5 61 | 62 | const CONNECTION_PROGRESS_CONNECTED_TO_RENDEZVOUS_NODE = 0 63 | const CONNECTION_PROGRESS_FOUND_INTRODUCTION_NODES = 1 64 | const CONNECTION_PROGRESS_INTRODUCTION_SENT = 2 65 | 66 | const ANNOUNCEMENT_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES = 0 67 | const ANNOUNCEMENT_ERROR_NO_INTRODUCTION_NODES_CONFIRMED = 1 68 | 69 | /** 70 | * @param {!Uint8Array} signal 71 | * @param {!Uint8Array} signature 72 | * 73 | * @return {!Uint8Array} 74 | */ 75 | function compose_bootstrap_response (signal, signature) 76 | new Uint8Array(signal.length + SIGNATURE_LENGTH) 77 | ..set(signal) 78 | ..set(signature, signal.length) 79 | /** 80 | * @param {!Uint8Array} data 81 | * 82 | * @return {!Array} [signal, signature] 83 | */ 84 | function parse_bootstrap_response (data) 85 | signal = data.subarray(0, data.length - SIGNATURE_LENGTH) 86 | signature = data.subarray(data.length - SIGNATURE_LENGTH) 87 | [signal, signature] 88 | /** 89 | * @param {!Uint8Array} source_id 90 | * @param {!Uint8Array} target_id 91 | * @param {!Uint8Array} sdp 92 | * @param {!Uint8Array} signature 93 | * 94 | * @return {!Uint8Array} 95 | */ 96 | function compose_signal (source_id, target_id, sdp, signature) 97 | new Uint8Array(PUBLIC_KEY_LENGTH * 2 + sdp.length + SIGNATURE_LENGTH) 98 | ..set(source_id) 99 | ..set(target_id, PUBLIC_KEY_LENGTH) 100 | ..set(sdp, PUBLIC_KEY_LENGTH * 2) 101 | ..set(signature, PUBLIC_KEY_LENGTH * 2 + sdp.length) 102 | /** 103 | * @param {!Uint8Array} data 104 | * 105 | * @return {!Array} [source_id, target_id, sdp, signature] 106 | */ 107 | function parse_signal (data) 108 | source_id = data.subarray(0, PUBLIC_KEY_LENGTH) 109 | target_id = data.subarray(PUBLIC_KEY_LENGTH, PUBLIC_KEY_LENGTH * 2) 110 | sdp = data.subarray(PUBLIC_KEY_LENGTH * 2, data.length - SIGNATURE_LENGTH) 111 | signature = data.subarray(data.length - SIGNATURE_LENGTH) 112 | [source_id, target_id, sdp, signature] 113 | /** 114 | * @param {number} code 115 | * @param {!Uint8Array} target_id 116 | * @param {!Array} nodes 117 | * 118 | * @return {!Uint8Array} 119 | */ 120 | function compose_find_introduction_nodes_response (code, target_id, nodes) 121 | result = new Uint8Array(1 + PUBLIC_KEY_LENGTH + nodes.length * PUBLIC_KEY_LENGTH) 122 | ..set([code]) 123 | ..set(target_id, 1) 124 | for node, i in nodes 125 | result.set(node, 1 + PUBLIC_KEY_LENGTH + i * PUBLIC_KEY_LENGTH) 126 | result 127 | /** 128 | * @param {!Uint8Array} data 129 | * 130 | * @return {!Array} [code, target_id, nodes] 131 | */ 132 | function parse_find_introduction_nodes_response (data) 133 | code = data[0] 134 | target_id = data.subarray(1, 1 + PUBLIC_KEY_LENGTH) 135 | nodes = [] 136 | data = data.subarray(1 + PUBLIC_KEY_LENGTH) 137 | for i from 0 til data.length / PUBLIC_KEY_LENGTH 138 | nodes.push(data.subarray(i * PUBLIC_KEY_LENGTH, (i + 1) * PUBLIC_KEY_LENGTH)) 139 | [code, target_id, nodes] 140 | /** 141 | * @param {!Uint8Array} target_id 142 | * @param {!Uint8Array} rendezvous_node 143 | * @param {!Uint8Array} rendezvous_token 144 | * @param {!Uint8Array} handshake_message 145 | * @param {!Uint8Array} application 146 | * @param {!Uint8Array} secret 147 | * 148 | * @return {!Uint8Array} 149 | */ 150 | function compose_introduction_payload (target_id, rendezvous_node, rendezvous_token, handshake_message, application, secret) 151 | new Uint8Array(PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH + APPLICATION_LENGTH + PUBLIC_KEY_LENGTH) 152 | ..set(target_id) 153 | ..set(rendezvous_node, PUBLIC_KEY_LENGTH) 154 | ..set(rendezvous_token, PUBLIC_KEY_LENGTH * 2) 155 | ..set(handshake_message, PUBLIC_KEY_LENGTH * 3) 156 | ..set(application, PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH) 157 | ..set(secret, PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH + APPLICATION_LENGTH) 158 | /** 159 | * @param {!Uint8Array} introduction_payload 160 | * 161 | * @return {!Array} [target_id, rendezvous_node, rendezvous_token, handshake_message, application, secret] 162 | */ 163 | function parse_introduction_payload (introduction_payload) 164 | target_id = introduction_payload.subarray(0, PUBLIC_KEY_LENGTH) 165 | rendezvous_node = introduction_payload.subarray(PUBLIC_KEY_LENGTH, PUBLIC_KEY_LENGTH * 2) 166 | rendezvous_token = introduction_payload.subarray(PUBLIC_KEY_LENGTH * 2, PUBLIC_KEY_LENGTH * 3) 167 | handshake_message = introduction_payload.subarray(PUBLIC_KEY_LENGTH * 3, PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH) 168 | application = introduction_payload.subarray(PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH, PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH + APPLICATION_LENGTH) 169 | secret = introduction_payload.subarray(PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH + APPLICATION_LENGTH, PUBLIC_KEY_LENGTH * 3 + HANDSHAKE_MESSAGE_LENGTH + APPLICATION_LENGTH + PUBLIC_KEY_LENGTH) 170 | [target_id, rendezvous_node, rendezvous_token, handshake_message, application, secret] 171 | /** 172 | * @param {!Uint8Array} rendezvous_token 173 | * @param {!Uint8Array} introduction_node 174 | * @param {!Uint8Array} target_id 175 | * @param {!Uint8Array} introduction_message 176 | * 177 | * @return {!Uint8Array} 178 | */ 179 | function compose_initialize_connection_data (rendezvous_token, introduction_node, target_id, introduction_message) 180 | new Uint8Array(PUBLIC_KEY_LENGTH * 3 + introduction_message.length) 181 | ..set(rendezvous_token) 182 | ..set(introduction_node, PUBLIC_KEY_LENGTH) 183 | ..set(target_id, PUBLIC_KEY_LENGTH * 2) 184 | ..set(introduction_message, PUBLIC_KEY_LENGTH * 3) 185 | /** 186 | * @param {!Uint8Array} message 187 | * 188 | * @return {!Array} [rendezvous_token, introduction_node, target_id, introduction_message] 189 | */ 190 | function parse_initialize_connection_data (message) 191 | rendezvous_token = message.subarray(0, PUBLIC_KEY_LENGTH) 192 | introduction_node = message.subarray(PUBLIC_KEY_LENGTH, PUBLIC_KEY_LENGTH * 2) 193 | target_id = message.subarray(PUBLIC_KEY_LENGTH * 2, PUBLIC_KEY_LENGTH * 3) 194 | introduction_message = message.subarray(PUBLIC_KEY_LENGTH * 3) 195 | [rendezvous_token, introduction_node, target_id, introduction_message] 196 | /** 197 | * @param {!Uint8Array} signature 198 | * @param {!Uint8Array} rendezvous_token 199 | * @param {!Uint8Array} handshake_message 200 | * 201 | * @return {!Uint8Array} 202 | */ 203 | function compose_confirm_connection_data (signature, rendezvous_token, handshake_message) 204 | new Uint8Array(SIGNATURE_LENGTH + PUBLIC_KEY_LENGTH + HANDSHAKE_MESSAGE_LENGTH) 205 | ..set(signature) 206 | ..set(rendezvous_token, SIGNATURE_LENGTH) 207 | ..set(handshake_message, SIGNATURE_LENGTH + PUBLIC_KEY_LENGTH) 208 | /** 209 | * @param {!Uint8Array} message 210 | * 211 | * @return {!Array} [signature, rendezvous_token, handshake_message] 212 | */ 213 | function parse_confirm_connection_data (message) 214 | signature = message.subarray(0, SIGNATURE_LENGTH) 215 | rendezvous_token = message.subarray(SIGNATURE_LENGTH, SIGNATURE_LENGTH + PUBLIC_KEY_LENGTH) 216 | handshake_message = message.subarray(SIGNATURE_LENGTH + PUBLIC_KEY_LENGTH) 217 | [signature, rendezvous_token, handshake_message] 218 | /** 219 | * @param {!Uint8Array} target_id 220 | * @param {!Uint8Array} introduction_message 221 | * 222 | * @return {!Uint8Array} 223 | */ 224 | function compose_introduce_to_data (target_id, introduction_message) 225 | new Uint8Array(PUBLIC_KEY_LENGTH + introduction_message.length) 226 | ..set(target_id) 227 | ..set(introduction_message, PUBLIC_KEY_LENGTH) 228 | /** 229 | * @param {!Uint8Array} message 230 | * 231 | * @return {!Array} [target_id, introduction_message] 232 | */ 233 | function parse_introduce_to_data (message) 234 | target_id = message.subarray(0, PUBLIC_KEY_LENGTH) 235 | introduction_message = message.subarray(PUBLIC_KEY_LENGTH) 236 | [target_id, introduction_message] 237 | 238 | /** 239 | * @param {!Function=} fetch 240 | */ 241 | function Wrapper (detox-crypto, detox-dht, detox-nodes-manager, detox-routing, detox-transport, detox-utils, fixed-size-multiplexer, async-eventer, fetch = window['fetch']) 242 | hex2array = detox-utils['hex2array'] 243 | array2hex = detox-utils['array2hex'] 244 | string2array = detox-utils['string2array'] 245 | array2string = detox-utils['array2string'] 246 | random_bytes = detox-utils['random_bytes'] 247 | random_int = detox-utils['random_int'] 248 | pull_random_item_from_array = detox-utils['pull_random_item_from_array'] 249 | are_arrays_equal = detox-utils['are_arrays_equal'] 250 | concat_arrays = detox-utils['concat_arrays'] 251 | timeoutSet = detox-utils['timeoutSet'] 252 | intervalSet = detox-utils['intervalSet'] 253 | error_handler = detox-utils['error_handler'] 254 | ArrayMap = detox-utils['ArrayMap'] 255 | ArraySet = detox-utils['ArraySet'] 256 | sample = detox-utils['sample'] 257 | empty_array = new Uint8Array(0) 258 | null_id = new Uint8Array(PUBLIC_KEY_LENGTH) 259 | /** 260 | * @param {Uint8Array} seed 261 | * 262 | * @return {!Object} 263 | */ 264 | function create_keypair (seed) 265 | detox-crypto['create_keypair'](seed) 266 | /** 267 | * @return {!Uint8Array} 268 | */ 269 | function fake_node_id 270 | create_keypair(null)['ed25519']['public'] 271 | /** 272 | * @constructor 273 | * 274 | * @param {!Array} bootstrap_nodes Array of strings in format `node_id:address:port` 275 | * @param {!Array} ice_servers 276 | * @param {number} packets_per_second Each packet send in each direction has exactly the same size and packets are sent at fixed rate (>= 1) 277 | * @param {number} bucket_size 278 | * @param {Object=} options More options that are less frequently used 279 | * 280 | * @return {!Core} 281 | * 282 | * @throws {Error} 283 | */ 284 | !function Core (bootstrap_nodes, ice_servers, packets_per_second = 1, bucket_size = 2, options = {}) 285 | if !(@ instanceof Core) 286 | return new Core(bootstrap_nodes, ice_servers, packets_per_second, bucket_size, options) 287 | async-eventer.call(@) 288 | 289 | @_options = Object.assign( 290 | { 291 | 'dht_keypair_seed' : null 292 | 'state_history_size' : 1000 293 | 'values_cache_size' : 1000 294 | 'fraction_of_nodes_from_same_peer' : 0.2 295 | 'lookup_number' : Math.max(bucket_size, 5) 296 | 'max_pending_segments' : 10 297 | 'aware_of_nodes_limit' : 1000 298 | 'min_number_of_peers_for_ready' : bucket_size # TODO: Use this option 299 | 'connected_nodes_limit' : 50 300 | } 301 | options 302 | { 303 | 'timeouts' : Object.assign({}, DEFAULT_TIMEOUTS, options['timeouts'] || {}) 304 | } 305 | ) 306 | 307 | @_announced_keypairs = ArrayMap() 308 | @_dht_keypair = create_keypair(@_options['dht_key_seed']) 309 | # Convenient shortcuts 310 | @_dht_public_key = @_dht_keypair['ed25519']['public'] 311 | @_dht_private_key = @_dht_keypair['ed25519']['private'] 312 | @_max_data_size = detox-transport['MAX_DATA_SIZE'] 313 | @_max_compressed_data_size = detox-transport['MAX_COMPRESSED_DATA_SIZE'] 314 | 315 | @_connected_nodes_count = 0 316 | @_connections_in_progress = ArrayMap() 317 | @_waiting_for_signal = ArrayMap() 318 | @_get_nodes_requested = ArraySet() 319 | @_routing_paths = ArrayMap() 320 | # Mapping from responder ID to routing path and from routing path to responder ID, so that we can use responder ID for external API 321 | @_id_to_routing_path = ArrayMap() 322 | @_routing_path_to_id = ArrayMap() 323 | @_connections_timeouts = ArrayMap() 324 | @_routes_timeouts = ArrayMap() 325 | @_pending_connections = ArrayMap() 326 | @_announcements_from = ArrayMap() 327 | @_forwarding_mapping = ArrayMap() 328 | @_pending_pings = ArraySet() 329 | @_encryptor_instances = ArrayMap() 330 | @_multiplexers = ArrayMap() 331 | @_demultiplexers = ArrayMap() 332 | @_pending_sending = ArrayMap() 333 | @_application_connections = ArraySet() 334 | 335 | @_cleanup_interval = intervalSet(@_options['timeouts']['LAST_USED_TIMEOUT'], !~> 336 | # Unregister unused routing paths 337 | unused_older_than = +(new Date) - @_options['timeouts']['LAST_USED_TIMEOUT'] * 1000 338 | @_routes_timeouts.forEach (last_updated, source_id) !~> 339 | if last_updated < unused_older_than 340 | if @_routing_paths.has(source_id) 341 | [node_id, route_id] = @_routing_paths.get(source_id) 342 | @_unregister_routing_path(node_id, route_id) 343 | @_routes_timeouts.delete(source_id) 344 | # Destroy connections that are no longer used 345 | @_connections_timeouts.forEach (last_updated, node_id) !~> 346 | if last_updated < unused_older_than 347 | @_connections_timeouts.delete(node_id) 348 | @_transport['destroy_connection'](node_id) 349 | ) 350 | # On 4/5 of the way to dropping connection 351 | @_keep_announce_routes_interval = intervalSet(@_options['timeouts']['LAST_USED_TIMEOUT'] / 5 * 4, !~> 352 | @_announced_keypairs.forEach ([real_keypair, number_of_introduction_nodes, number_of_intermediate_nodes, announced_to, last_announcement], real_public_key) !~> 353 | if announced_to.size < number_of_introduction_nodes && last_announcement 354 | # Give at least 3x time for announcement process to complete and to announce to some node 355 | reannounce_if_older_than = +(new Date) - @_options['timeouts']['CONNECTION_TIMEOUT'] * 3 356 | if last_announcement < reannounce_if_older_than 357 | @_announce(real_public_key) 358 | announced_to.forEach (introduction_node) !~> 359 | full_introduction_node_id = concat_arrays(real_public_key, introduction_node) 360 | [node_id, route_id] = @_id_to_routing_path.get(full_introduction_node_id) 361 | if @_send_ping(node_id, route_id) 362 | source_id = concat_arrays(node_id, route_id) 363 | @_pending_pings.add(source_id) 364 | ) 365 | @_get_more_nodes_interval = intervalSet(@_options['timeouts']['GET_MORE_AWARE_OF_NODES_INTERVAL'], !~> 366 | @_get_more_aware_of_nodes() 367 | ) 368 | 369 | @_transport = detox-transport['Transport']( 370 | @_dht_public_key 371 | ice_servers 372 | packets_per_second 373 | UNCOMPRESSED_COMMANDS_OFFSET 374 | @_options['timeouts']['CONNECTION_TIMEOUT'] 375 | ) 376 | .'on'('connected', (peer_id) !~> 377 | @_dht['add_peer'](peer_id) 378 | @_nodes_manager['add_connected_node'](peer_id) 379 | if @_bootstrap_node 380 | @_send_uncompressed_core_command(peer_id, UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE, string2array(@_bootstrap_node)) 381 | ) 382 | .'on'('disconnected', (peer_id) !~> 383 | @_dht['del_peer'](peer_id) 384 | @_nodes_manager['del_connected_node'](peer_id) 385 | @_get_nodes_requested.delete(peer_id) 386 | ) 387 | .'on'('data', (peer_id, command, command_data) !~> 388 | @_update_connection_timeout(peer_id, false) 389 | if command >= UNCOMPRESSED_CORE_COMMANDS_OFFSET 390 | if @_bootstrap_node && command != UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE 391 | return 392 | @_handle_uncompressed_core_command(peer_id, command - UNCOMPRESSED_CORE_COMMANDS_OFFSET, command_data) 393 | else if command == ROUTING_COMMANDS 394 | if @_bootstrap_node 395 | return 396 | @_router['process_packet'](peer_id, command_data) 397 | else if command >= DHT_COMMANDS_OFFSET 398 | @_dht['receive'](peer_id, command - DHT_COMMANDS_OFFSET, command_data) 399 | else 400 | if @_bootstrap_node && command != COMPRESSED_CORE_COMMAND_SIGNAL 401 | return 402 | @_handle_compressed_core_command(peer_id, command, command_data) 403 | ) 404 | @_dht = detox-dht['DHT']( 405 | @_dht_public_key 406 | bucket_size 407 | @_options['state_history_size'] 408 | @_options['values_cache_size'] 409 | @_options['fraction_of_nodes_from_same_peer'] 410 | @_options['timeouts'] 411 | ) 412 | .'on'('peer_error', (peer_id) !~> 413 | @_peer_error(peer_id) 414 | ) 415 | .'on'('peer_warning', (peer_id) !~> 416 | @_peer_warning(peer_id) 417 | ) 418 | .'on'('connect_to', (peer_peer_id, peer_id) ~> 419 | new Promise (resolve, reject) !~> 420 | if @_nodes_manager['has_connected_node'](peer_peer_id) 421 | resolve() 422 | return 423 | connection = @_transport['get_connection'](peer_peer_id) 424 | if !connection 425 | connection = @_transport['create_connection'](true, peer_peer_id) 426 | if !connection 427 | reject() 428 | return 429 | connection['on']('signal', (sdp) !~> 430 | signature = detox-crypto['sign'](sdp, @_dht_public_key, @_dht_private_key) 431 | command_data = compose_signal(@_dht_public_key, peer_peer_id, sdp, signature) 432 | @_send_compressed_core_command(peer_id, COMPRESSED_CORE_COMMAND_SIGNAL, command_data) 433 | ) 434 | connection 435 | .'once'('connected', !-> 436 | connection['off']('disconnected', disconnected) 437 | resolve() 438 | ) 439 | .'once'('disconnected', disconnected) 440 | !~function disconnected 441 | reject() 442 | ) 443 | .'on'('send', (peer_id, command, command_data) !~> 444 | @_send_dht_command(peer_id, command, command_data) 445 | ) 446 | .'on'('peer_updated', (peer_id, peer_peers) !~> 447 | @_nodes_manager['set_peer'](peer_id, peer_peers) 448 | ) 449 | @_router = detox-routing['Router'](@_dht_keypair['x25519']['private'], @_options['max_pending_segments'], @_options['timeouts']['ROUTING_PATH_SEGMENT_TIMEOUT']) 450 | .'on'('activity', (node_id, route_id) !~> 451 | source_id = concat_arrays(node_id, route_id) 452 | if !@_routing_paths.has(source_id) 453 | @_routing_paths.set(source_id, [node_id, route_id]) 454 | @_routes_timeouts.set(source_id, +(new Date)) 455 | ) 456 | .'on'('send', (node_id, data) !~> 457 | @_send_routing_command(node_id, data) 458 | ) 459 | .'on'('data', (node_id, route_id, command, data) !~> 460 | source_id = concat_arrays(node_id, route_id) 461 | switch command 462 | case ROUTING_COMMAND_ANNOUNCE 463 | public_key = @_verify_announcement_message(data) 464 | if !public_key 465 | return 466 | # If re-announcement, make sure to stop old interval 467 | if @_announcements_from.has(public_key) 468 | clearInterval(@_announcements_from.get(public_key)[2]) 469 | announce_interval = intervalSet(@_options['timeouts']['ANNOUNCE_INTERVAL'], !~> 470 | if !@_routing_paths.has(source_id) 471 | return 472 | @_publish_announcement_message(data) 473 | ) 474 | @_announcements_from.set(public_key, [node_id, route_id, announce_interval]) 475 | @_publish_announcement_message(data) 476 | case ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST 477 | target_id = data 478 | if target_id.length != PUBLIC_KEY_LENGTH 479 | return 480 | /** 481 | * @param {number} code 482 | * @param {!Array} nodes 483 | */ 484 | send_response = (code, nodes) !~> 485 | data = compose_find_introduction_nodes_response(code, target_id, nodes) 486 | @_send_to_routing_path(node_id, route_id, ROUTING_COMMAND_FIND_INTRODUCTION_NODES_RESPONSE, data) 487 | @_find_introduction_nodes(target_id) 488 | .then (introduction_nodes) !-> 489 | if !introduction_nodes.length 490 | send_response(CONNECTION_ERROR_NO_INTRODUCTION_NODES, []) 491 | else 492 | send_response(CONNECTION_OK, introduction_nodes) 493 | .catch (error) !-> 494 | error_handler(error) 495 | send_response(CONNECTION_ERROR_NO_INTRODUCTION_NODES, []) 496 | case ROUTING_COMMAND_INITIALIZE_CONNECTION 497 | [rendezvous_token, introduction_node, target_id, introduction_message] = parse_initialize_connection_data(data) 498 | if @_pending_connections.has(rendezvous_token) 499 | # Ignore subsequent usages of the same rendezvous token 500 | return 501 | connection_timeout = timeoutSet(@_options['timeouts']['CONNECTION_TIMEOUT'], !~> 502 | @_pending_connections.delete(rendezvous_token) 503 | ) 504 | @_pending_connections.set(rendezvous_token, [node_id, route_id, target_id, connection_timeout]) 505 | @_send_uncompressed_core_command( 506 | introduction_node 507 | UNCOMPRESSED_CORE_COMMAND_FORWARD_INTRODUCTION 508 | compose_introduce_to_data(target_id, introduction_message) 509 | ) 510 | case ROUTING_COMMAND_CONFIRM_CONNECTION 511 | [signature, rendezvous_token, handshake_message] = parse_confirm_connection_data(data) 512 | pending_connection = @_pending_connections.get(rendezvous_token) 513 | if !pending_connection 514 | return 515 | [target_node_id, target_route_id, target_id, connection_timeout] = pending_connection 516 | if !detox-crypto['verify'](signature, rendezvous_token, target_id) 517 | return 518 | @_pending_connections.delete(rendezvous_token) 519 | clearTimeout(connection_timeout) 520 | @_send_to_routing_path(target_node_id, target_route_id, ROUTING_COMMAND_CONNECTED, data) 521 | target_source_id = concat_arrays(target_node_id, target_route_id) 522 | @_forwarding_mapping.set(source_id, [target_node_id, target_route_id]) 523 | @_forwarding_mapping.set(target_source_id, [node_id, route_id]) 524 | case ROUTING_COMMAND_INTRODUCTION 525 | routing_path_details = @_routing_path_to_id.get(source_id) 526 | if !routing_path_details 527 | # If routing path unknown - ignore 528 | return 529 | [real_public_key, introduction_node] = routing_path_details 530 | if !@_announced_keypairs.has(real_public_key) 531 | return 532 | [real_keypair, , , announced_to] = @_announced_keypairs.get(real_public_key) 533 | if !announced_to.has(introduction_node) 534 | return 535 | try 536 | introduction_message_decrypted = detox-crypto['one_way_decrypt'](real_keypair['x25519']['private'], data) 537 | signature = introduction_message_decrypted.subarray(0, SIGNATURE_LENGTH) 538 | introduction_payload = introduction_message_decrypted.subarray(SIGNATURE_LENGTH) 539 | [ 540 | target_id 541 | rendezvous_node 542 | rendezvous_token 543 | handshake_message 544 | application 545 | secret 546 | ] = parse_introduction_payload(introduction_payload) 547 | for_signature = concat_arrays(introduction_node, introduction_payload) 548 | if !detox-crypto['verify'](signature, for_signature, target_id) 549 | return 550 | full_target_id = concat_arrays(real_public_key, target_id) 551 | if @_id_to_routing_path.has(full_target_id) 552 | # If already have connection to this node - silently ignore: 553 | # might be a tricky attack when DHT public key is the same as real public key 554 | return 555 | if @_connections_in_progress.has(full_target_id) 556 | connection_in_progress = @_connections_in_progress.get(full_target_id) 557 | if connection_in_progress.initiator && !connection_in_progress.discarded 558 | for item, key in real_public_key 559 | if item == target_id[key] 560 | continue 561 | if item > target_id[key] 562 | # If this node's public_key if bigger, then connection initiated by this node will win and the other side will 563 | # discard its initiated connection 564 | return 565 | else 566 | # Otherwise our connection is discarded and we proceed with connection initiated by the other side 567 | connection_in_progress.discarded = true 568 | break 569 | else 570 | connection_in_progress = 571 | initiator : false 572 | @_connections_in_progress.set(full_target_id, connection_in_progress) 573 | data = 574 | 'real_public_key' : real_public_key 575 | 'target_id' : target_id 576 | 'secret' : secret 577 | 'application' : application 578 | 'number_of_intermediate_nodes' : null 579 | @'fire'('introduction', data) 580 | .then !~> 581 | number_of_intermediate_nodes = data['number_of_intermediate_nodes'] 582 | if number_of_intermediate_nodes == null 583 | throw 'No event handler for introduction' 584 | # Here we allow user to connect to rendezvous node directly if he wants too 585 | if number_of_intermediate_nodes 586 | nodes = @_nodes_manager['get_nodes_for_routing_path'](number_of_intermediate_nodes, [rendezvous_node]) 587 | nodes.push(rendezvous_node) 588 | if !nodes 589 | return 590 | else 591 | nodes = [rendezvous_node] 592 | first_node = nodes[0] 593 | @_construct_routing_path(nodes) 594 | .then (route_id) !~> 595 | encryptor_instance = detox-crypto['Encryptor'](false, real_keypair['x25519']['private']) 596 | encryptor_instance['put_handshake_message'](handshake_message) 597 | response_handshake_message = encryptor_instance['get_handshake_message']() 598 | @_encryptor_instances.set(full_target_id, encryptor_instance) 599 | @_register_routing_path(real_public_key, target_id, first_node, route_id) 600 | @_connections_in_progress.delete(full_target_id) 601 | @_register_application_connection(real_public_key, target_id) 602 | signature = detox-crypto['sign'](rendezvous_token, real_public_key, real_keypair['ed25519']['private']) 603 | @_send_to_routing_node( 604 | real_public_key 605 | target_id 606 | ROUTING_COMMAND_CONFIRM_CONNECTION 607 | compose_confirm_connection_data(signature, rendezvous_token, response_handshake_message) 608 | ) 609 | # Error handler is already present in `_construct_routing_path()` method 610 | .catch !~> 611 | @_connections_in_progress.delete(full_target_id) 612 | if connection_in_progress.initiator && connection_in_progress.discarded 613 | @'fire'('connection_failed', real_public_key, target_id, CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE) 614 | .catch (error) !~> 615 | error_handler(error) 616 | @_connections_in_progress.delete(full_target_id) 617 | if connection_in_progress.initiator && connection_in_progress.discarded 618 | @'fire'('connection_failed', real_public_key, target_id, CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE) 619 | catch error 620 | error_handler(error) 621 | case ROUTING_COMMAND_DATA 622 | if @_forwarding_mapping.has(source_id) 623 | [target_node_id, target_route_id] = @_forwarding_mapping.get(source_id) 624 | @_send_to_routing_path(target_node_id, target_route_id, ROUTING_COMMAND_DATA, data) 625 | else if @_routing_path_to_id.has(source_id) 626 | [real_public_key, target_id] = @_routing_path_to_id.get(source_id) 627 | full_target_id = concat_arrays(real_public_key, target_id) 628 | encryptor_instance = @_encryptor_instances.get(full_target_id) 629 | if !encryptor_instance 630 | return 631 | demultiplexer = @_demultiplexers.get(full_target_id) 632 | if !demultiplexer 633 | return 634 | data_decrypted = encryptor_instance['decrypt'](data) 635 | demultiplexer['feed'](data_decrypted) 636 | while demultiplexer['have_more_data']() 637 | data_with_header = demultiplexer['get_data']() 638 | command = data_with_header[0] 639 | @'fire'('data', real_public_key, target_id, command, data_with_header.subarray(1)) 640 | case ROUTING_COMMAND_PING 641 | if @_routing_path_to_id.has(source_id) 642 | if @_pending_pings.has(source_id) 643 | # Don't ping back if we have sent ping ourselves 644 | @_pending_pings.delete(source_id) 645 | return 646 | # Send ping back 647 | @_send_ping(node_id, route_id) 648 | ) 649 | @_nodes_manager = detox-nodes-manager(bootstrap_nodes, @_options['aware_of_nodes_limit'], @_options['timeouts']['STALE_AWARE_OF_NODE_TIMEOUT']) 650 | .'on'('connected_nodes_count', (connected_nodes_count) !~> 651 | @_connected_nodes_count = connected_nodes_count 652 | @'fire'('connected_nodes_count', @_connected_nodes_count) 653 | if connected_nodes_count > @_options['connected_nodes_limit'] 654 | # TODO: This should be greatly improved, should also take into account peer warnings 655 | nodes_used_in_forwarding = ArraySet() 656 | @_forwarding_mapping.forEach ([node_id]) !-> 657 | nodes_used_in_forwarding.add(node_id) 658 | candidates_for_removal = @_nodes_manager['get_candidates_for_disconnection'](nodes_used_in_forwarding) 659 | if candidates_for_removal.length 660 | random_connected_node = pull_random_item_from_array(candidates_for_removal) 661 | @_transport['destroy_connection'](random_connected_node) 662 | ) 663 | .'on'('aware_of_nodes_count', (aware_of_nodes_count) !~> 664 | @'fire'('aware_of_nodes_count', aware_of_nodes_count) 665 | ) 666 | .'on'('peer_error', (peer_id) !~> 667 | @_peer_error(peer_id) 668 | ) 669 | .'on'('peer_warning', (peer_id) !~> 670 | @_peer_warning(peer_id) 671 | ) 672 | # As we wrap encrypted data into encrypted routing path, we'll have more overhead: MAC on top of encrypted block of multiplexed data 673 | @_max_packet_data_size = @_router['get_max_packet_data_size']() - MAC_LENGTH # 472 bytes 674 | if !bootstrap_nodes.length 675 | setTimeout !~> 676 | @'fire'('ready') 677 | else 678 | @_dht['once']('peer_updated', !~> 679 | @'fire'('ready') 680 | @_get_more_aware_of_nodes() 681 | ) 682 | @_do_random_lookup() 683 | Core.'CONNECTION_ERROR_CANT_FIND_INTRODUCTION_NODES' = CONNECTION_ERROR_CANT_FIND_INTRODUCTION_NODES 684 | Core.'CONNECTION_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES' = CONNECTION_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES 685 | Core.'CONNECTION_ERROR_NO_INTRODUCTION_NODES' = CONNECTION_ERROR_NO_INTRODUCTION_NODES 686 | Core.'CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE' = CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE 687 | Core.'CONNECTION_ERROR_OUT_OF_INTRODUCTION_NODES' = CONNECTION_ERROR_OUT_OF_INTRODUCTION_NODES 688 | 689 | Core.'CONNECTION_PROGRESS_CONNECTED_TO_RENDEZVOUS_NODE' = CONNECTION_PROGRESS_CONNECTED_TO_RENDEZVOUS_NODE 690 | Core.'CONNECTION_PROGRESS_FOUND_INTRODUCTION_NODES' = CONNECTION_PROGRESS_FOUND_INTRODUCTION_NODES 691 | Core.'CONNECTION_PROGRESS_INTRODUCTION_SENT' = CONNECTION_PROGRESS_INTRODUCTION_SENT 692 | 693 | Core.'ANNOUNCEMENT_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES' = ANNOUNCEMENT_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES 694 | Core.'ANNOUNCEMENT_ERROR_NO_INTRODUCTION_NODES_CONFIRMED' = ANNOUNCEMENT_ERROR_NO_INTRODUCTION_NODES_CONFIRMED 695 | Core:: = 696 | /** 697 | * Start HTTP server listening on specified ip:port, so that current node will be capable of acting as bootstrap node for other users 698 | * 699 | * @param {!Uint8Array} bootstrap_seed Seed for generating bootstrap node's keypairs 700 | * @param {string} ip 701 | * @param {number} port 702 | * @param {string} public_address Publicly available address that will be returned to other node, typically domain name (instead of using IP) 703 | * @param {number} public_port Publicly available port on `public_address` 704 | */ 705 | 'start_bootstrap_node' : (bootstrap_seed, ip, port, public_address = ip, public_port = port) !-> 706 | keypair = detox-crypto['create_keypair'](bootstrap_seed)['ed25519'] 707 | @_http_server = require('http')['createServer'] (request, response) !~> 708 | response['setHeader']('Access-Control-Allow-Origin', '*') 709 | response['setHeader']('Connection', 'close') 710 | /** 711 | * @param {number} status 712 | * @param {!Uint8Array=} data 713 | */ 714 | !function exit (status, data) 715 | if closed 716 | return 717 | response['writeHead'](status) 718 | if data 719 | response['write'](Buffer.from(data)) 720 | response['end']() 721 | content_length = request.headers['content-length'] 722 | if !( 723 | request.method == 'POST' && 724 | content_length && 725 | content_length <= @_max_compressed_data_size 726 | ) 727 | exit(400) 728 | return 729 | body = [] 730 | closed = false 731 | var timeout 732 | request 733 | .'on'('data', (chunk) !-> 734 | body.push(chunk) 735 | ) 736 | .'on'('end', !~> 737 | body := concat_arrays(body) 738 | [source_id, target_id, sdp, signature] = parse_signal(body) 739 | if !( 740 | detox-crypto['verify'](signature, sdp, source_id) && 741 | are_arrays_equal(target_id, null_id) 742 | ) 743 | exit(400) 744 | return 745 | if !@_connected_nodes_count || !random_int(0, @_connected_nodes_count) 746 | random_connected_node = null 747 | else 748 | random_connected_node = @_nodes_manager['get_random_connected_nodes'](1)?[0] 749 | if random_connected_node 750 | waiting_for_signal_key = concat_arrays(source_id, random_connected_node) 751 | if @_waiting_for_signal.has(waiting_for_signal_key) 752 | exit(503) 753 | return 754 | command_data = compose_signal(source_id, random_connected_node, sdp, signature) 755 | @_send_compressed_core_command(random_connected_node, COMPRESSED_CORE_COMMAND_SIGNAL, command_data) 756 | @_waiting_for_signal.set(waiting_for_signal_key, (sdp, signature, command_data) !~> 757 | clearTimeout(timeout) 758 | if detox-crypto['verify'](signature, sdp, random_connected_node) 759 | data = compose_bootstrap_response( 760 | command_data 761 | detox-crypto['sign'](command_data, keypair['public'], keypair['private']) 762 | ) 763 | exit(200, Buffer.from(data)) 764 | else 765 | exit(502) 766 | ) 767 | timeout := timeoutSet(@_options['timeouts']['CONNECTION_TIMEOUT'], !~> 768 | @_waiting_for_signal.delete(waiting_for_signal_key) 769 | exit(504) 770 | ) 771 | else 772 | connection = @_transport['create_connection'](false, source_id) 773 | if !connection 774 | exit(503) 775 | return 776 | connection 777 | .'once'('signal', (sdp) ~> 778 | signature = detox-crypto['sign'](sdp, @_dht_public_key, @_dht_private_key) 779 | command_data = compose_signal(@_dht_public_key, source_id, sdp, signature) 780 | data = compose_bootstrap_response( 781 | command_data 782 | detox-crypto['sign'](command_data, keypair['public'], keypair['private']) 783 | ) 784 | exit(200, Buffer.from(data)) 785 | false 786 | ) 787 | .'signal'(sdp) 788 | ) 789 | .'on'('close', !-> 790 | clearTimeout(timeout) 791 | closed := true 792 | ) 793 | @_http_server 794 | .'on'('error', error_handler) 795 | .'listen'(port, ip, !~> 796 | node_id = array2hex(keypair['public']) 797 | @_bootstrap_node = "#node_id:#public_address:#public_port" 798 | ) 799 | # Stop doing any routing tasks immediately 800 | @_destroy_router() 801 | /** 802 | * Get an array of bootstrap nodes obtained during DHT operation in the same format as `bootstrap_nodes` argument in constructor 803 | * 804 | * @return {!Array} 805 | */ 806 | 'get_bootstrap_nodes' : -> 807 | @_nodes_manager['get_bootstrap_nodes']() 808 | /** 809 | * @param {Function=} callback 810 | */ 811 | _bootstrap : (callback) !-> 812 | bootstrap_nodes = @'get_bootstrap_nodes'() 813 | waiting_for = bootstrap_nodes.length 814 | if !waiting_for 815 | callback?() 816 | return 817 | !~function done 818 | --waiting_for 819 | if waiting_for 820 | return 821 | callback?() 822 | bootstrap_nodes.forEach (bootstrap_node) !~> 823 | bootstrap_node = bootstrap_node.split(':') 824 | bootstrap_node_id = hex2array(bootstrap_node[0]) 825 | bootstrap_node_address = bootstrap_node[1] + ':' + bootstrap_node[2] 826 | random_id = random_bytes(PUBLIC_KEY_LENGTH) 827 | connection = @_transport['create_connection'](true, random_id) 828 | if !connection 829 | return 830 | connection 831 | .'on'('signal', (sdp) !~> 832 | signature = detox-crypto['sign'](sdp, @_dht_public_key, @_dht_private_key) 833 | init = 834 | 'method' : 'POST' 835 | 'headers' : 836 | 'Connection' : 'close' 837 | # TODO: When https://github.com/bitinn/node-fetch/pull/457 is merged and released, remove `.buffer` as unnecessary 838 | 'body' : compose_signal(@_dht_public_key, null_id, sdp, signature).buffer 839 | # TODO: Abort fetch on destroying once https://github.com/bitinn/node-fetch/pull/437 is released 840 | # Prefer HTTPS connection if possible, otherwise fallback to insecure (primarily for development purposes) 841 | fetch("https://#bootstrap_node_address", init) 842 | .catch (error) -> 843 | if typeof location == 'undefined' || location['protocol'] == 'http:' 844 | fetch("http://#bootstrap_node_address", init) 845 | else 846 | throw error 847 | .then (response) -> 848 | if !response['ok'] 849 | throw 'Request failed, status code ' + response['status'] 850 | response['arrayBuffer']() 851 | .then (buffer) -> 852 | new Uint8Array(buffer) 853 | .then (data) !~> 854 | [signal, signature] = parse_bootstrap_response(data) 855 | if !detox-crypto['verify'](signature, signal, bootstrap_node_id) 856 | throw 'Bad bootstrap node response' 857 | [source_id, target_id, sdp, signature] = parse_signal(signal) 858 | if !( 859 | detox-crypto['verify'](signature, sdp, source_id) && 860 | are_arrays_equal(target_id, @_dht_public_key) 861 | ) 862 | throw 'Bad response' 863 | if @_transport['get_connection'](source_id) 864 | throw 'Already connected' 865 | @_transport['update_peer_id'](random_id, source_id) 866 | connection['signal'](sdp) 867 | .catch (error) !-> 868 | # No error handing here, there might be too many network-related errors here that we can do nothing about 869 | connection['destroy']() 870 | ) 871 | .'once'('connected', !~> 872 | connection['off']('disconnected', disconnected) 873 | done() 874 | ) 875 | .'once'('disconnected', disconnected) 876 | !function disconnected 877 | done() 878 | /** 879 | * @param {!Uint8Array} real_key_seed Seed used to generate real long-term keypair 880 | * @param {number} number_of_introduction_nodes 881 | * @param {number} number_of_intermediate_nodes How many hops should be made until introduction node (not including it) 882 | * 883 | * @return {Uint8Array} Real public key or `null` in case of failure 884 | */ 885 | 'announce' : (real_key_seed, number_of_introduction_nodes, number_of_intermediate_nodes) -> 886 | if @_bootstrap_node 887 | return null 888 | real_keypair = create_keypair(real_key_seed) 889 | real_public_key = real_keypair['ed25519']['public'] 890 | # Ignore repeated announcement 891 | if @_announced_keypairs.has(real_public_key) 892 | return null 893 | @_announced_keypairs.set( 894 | real_public_key 895 | [real_keypair, number_of_introduction_nodes, number_of_intermediate_nodes, ArraySet()] 896 | ) 897 | @_announce(real_public_key) 898 | real_public_key 899 | /** 900 | * @param {!Uint8Array} real_public_key 901 | */ 902 | _announce : (real_public_key) !-> 903 | [ 904 | real_keypair 905 | number_of_introduction_nodes 906 | number_of_intermediate_nodes 907 | announced_to 908 | ] = @_announced_keypairs.get(real_public_key) 909 | old_introduction_nodes = [] 910 | announced_to.forEach (introduction_node) !-> 911 | old_introduction_nodes.push(introduction_node) 912 | number_of_introduction_nodes = number_of_introduction_nodes - old_introduction_nodes.length 913 | if !number_of_introduction_nodes 914 | return 915 | @_update_last_announcement(real_public_key, +(new Date)) 916 | # Last node is introduction node, hence +1 917 | nodes_for_routes_to_introduction_nodes = [] 918 | for let _ from 0 til number_of_intermediate_nodes 919 | nodes = @_nodes_manager['get_nodes_for_routing_path'](number_of_intermediate_nodes + 1, old_introduction_nodes) 920 | if nodes 921 | nodes_for_routes_to_introduction_nodes.push(nodes) 922 | if !nodes_for_routes_to_introduction_nodes.length 923 | @_update_last_announcement(real_public_key, 1) 924 | @'fire'('announcement_failed', real_public_key, ANNOUNCEMENT_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES) 925 | return 926 | introductions_pending = nodes_for_routes_to_introduction_nodes.length 927 | introduction_nodes_confirmed = [] 928 | /** 929 | * @param {!Uint8Array=} introduction_node 930 | */ 931 | !~function announced (introduction_node) 932 | if introduction_node 933 | introduction_nodes_confirmed.push(introduction_node) 934 | --introductions_pending 935 | if introductions_pending 936 | return 937 | if !introduction_nodes_confirmed.length 938 | @_update_last_announcement(real_public_key, 1) 939 | @'fire'('announcement_failed', real_public_key, ANNOUNCEMENT_ERROR_NO_INTRODUCTION_NODES_CONFIRMED) 940 | return 941 | # Add old introduction nodes to the list 942 | introduction_nodes_confirmed := introduction_nodes_confirmed.concat(old_introduction_nodes) 943 | announcement_message = @_generate_announcement_message( 944 | real_public_key 945 | real_keypair['ed25519']['private'] 946 | introduction_nodes_confirmed 947 | ) 948 | for introduction_node in introduction_nodes_confirmed 949 | @_send_to_routing_node(real_public_key, introduction_node, ROUTING_COMMAND_ANNOUNCE, announcement_message) 950 | # TODO: Check using independent routing path that announcement indeed happened 951 | @'fire'('announced', real_public_key) 952 | combined_introduction_nodes = old_introduction_nodes.concat( 953 | for nodes in nodes_for_routes_to_introduction_nodes 954 | nodes[* - 1] 955 | ) 956 | for let nodes in nodes_for_routes_to_introduction_nodes 957 | first_node = nodes[0] 958 | introduction_node = nodes[* - 1] 959 | @_construct_routing_path(nodes) 960 | .then (route_id) !~> 961 | @_register_routing_path(real_public_key, introduction_node, first_node, route_id) 962 | announced_to.add(introduction_node) 963 | announced(introduction_node) 964 | # Error handler is already present in `_construct_routing_path()` method 965 | .catch !~> 966 | announced() 967 | /** 968 | * @param {!Uint8Array} real_public_key 969 | * @param {number} value 970 | */ 971 | _update_last_announcement : (real_public_key, value) !-> 972 | @_announced_keypairs.get(real_public_key)[4] = value 973 | /** 974 | * @param {!Uint8Array} real_key_seed Seed used to generate real long-term keypair 975 | * @param {!Uint8Array} target_id Real Ed25519 pubic key of interested node 976 | * @param {!Uint8Array} application Up to 64 bytes 977 | * @param {!Uint8Array} secret Up to 32 bytes 978 | * @param {number} number_of_intermediate_nodes How many hops should be made until rendezvous node (not including it) 979 | * 980 | * @return {Uint8Array} Real public key or `null` in case of failure 981 | */ 982 | 'connect_to' : (real_key_seed, target_id, application, secret, number_of_intermediate_nodes) -> 983 | if @_bootstrap_node 984 | return null 985 | real_keypair = create_keypair(real_key_seed) 986 | real_public_key = real_keypair['ed25519']['public'] 987 | # Don't connect to itself 988 | if are_arrays_equal(real_public_key, target_id) 989 | return null 990 | full_target_id = concat_arrays(real_public_key, target_id) 991 | # Don't initiate 2 concurrent connections to the same node, it will not end up well 992 | if @_connections_in_progress.has(full_target_id) 993 | return real_public_key 994 | # `discarded` is used when alternative connection from a friend is happening at the same time and this connection establishing is discarded 995 | connection_in_progress = 996 | initiator : true 997 | discarded : false 998 | @_connections_in_progress.set(full_target_id, connection_in_progress) 999 | if @_id_to_routing_path.has(full_target_id) 1000 | # Already connected, do nothing 1001 | return null 1002 | !~function connection_failed (code) 1003 | if connection_in_progress.discarded 1004 | return 1005 | @_connections_in_progress.delete(full_target_id) 1006 | @'fire'('connection_failed', real_public_key, target_id, code) 1007 | nodes = @_nodes_manager['get_nodes_for_routing_path'](number_of_intermediate_nodes + 1) 1008 | if !nodes 1009 | connection_failed(CONNECTION_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES) 1010 | return null 1011 | first_node = nodes[0] 1012 | rendezvous_node = nodes[* - 1] 1013 | @_construct_routing_path(nodes) 1014 | .then (route_id) !~> 1015 | @'fire'('connection_progress', real_public_key, target_id, CONNECTION_PROGRESS_CONNECTED_TO_RENDEZVOUS_NODE) 1016 | !~function found_introduction_nodes (new_node_id, new_route_id, command, data) 1017 | if !( 1018 | are_arrays_equal(first_node, new_node_id) && 1019 | are_arrays_equal(route_id, new_route_id) && 1020 | command == ROUTING_COMMAND_FIND_INTRODUCTION_NODES_RESPONSE 1021 | ) 1022 | return 1023 | [code, introduction_target_id, introduction_nodes] = parse_find_introduction_nodes_response(data) 1024 | if !are_arrays_equal(target_id, introduction_target_id) 1025 | return 1026 | clearTimeout(find_introduction_nodes_timeout) 1027 | if code != CONNECTION_OK 1028 | # Routing path is already established, but we don't need it anymore 1029 | @_router['destroy_routing_path'](first_node, route_id) 1030 | @_nodes_manager['del_first_node_in_routing_path'](first_node) 1031 | connection_failed(code) 1032 | return 1033 | @'fire'('connection_progress', real_public_key, target_id, CONNECTION_PROGRESS_FOUND_INTRODUCTION_NODES) 1034 | !~function try_to_introduce 1035 | if connection_in_progress.discarded 1036 | return 1037 | if !introduction_nodes.length 1038 | # Routing path is already established, but we don't need it anymore 1039 | @_router['destroy_routing_path'](first_node, route_id) 1040 | @_nodes_manager['del_first_node_in_routing_path'](first_node) 1041 | connection_failed(CONNECTION_ERROR_OUT_OF_INTRODUCTION_NODES) 1042 | return 1043 | introduction_node = pull_random_item_from_array(introduction_nodes) 1044 | rendezvous_token = random_bytes(PUBLIC_KEY_LENGTH) 1045 | x25519_public_key = detox-crypto['convert_public_key'](target_id) 1046 | encryptor_instance = detox-crypto['Encryptor'](true, x25519_public_key) 1047 | handshake_message = encryptor_instance['get_handshake_message']() 1048 | introduction_payload = compose_introduction_payload( 1049 | real_public_key 1050 | rendezvous_node 1051 | rendezvous_token 1052 | handshake_message 1053 | application 1054 | secret 1055 | ) 1056 | for_signature = concat_arrays(introduction_node, introduction_payload) 1057 | signature = detox-crypto['sign'](for_signature, real_public_key, real_keypair['ed25519']['private']) 1058 | introduction_message = concat_arrays(signature, introduction_payload) 1059 | introduction_message_encrypted = detox-crypto['one_way_encrypt'](x25519_public_key, introduction_message) 1060 | !~function path_confirmation (new_node_id, new_route_id, command, data) 1061 | if !( 1062 | are_arrays_equal(first_node, new_node_id) && 1063 | are_arrays_equal(route_id, new_route_id) && 1064 | command == ROUTING_COMMAND_CONNECTED 1065 | ) 1066 | return 1067 | [signature, rendezvous_token_received, handshake_message_received] = parse_confirm_connection_data(data) 1068 | if !( 1069 | are_arrays_equal(rendezvous_token_received, rendezvous_token) && 1070 | detox-crypto['verify'](signature, rendezvous_token, target_id) 1071 | ) 1072 | return 1073 | encryptor_instance['put_handshake_message'](handshake_message_received) 1074 | @_encryptor_instances.set(full_target_id, encryptor_instance) 1075 | clearTimeout(path_confirmation_timeout) 1076 | @_router['off']('data', path_confirmation) 1077 | @_register_routing_path(real_public_key, target_id, first_node, route_id) 1078 | @_connections_in_progress.delete(full_target_id) 1079 | @_register_application_connection(real_public_key, target_id) 1080 | @_router['on']('data', path_confirmation) 1081 | @_send_to_routing_path( 1082 | first_node 1083 | route_id 1084 | ROUTING_COMMAND_INITIALIZE_CONNECTION 1085 | compose_initialize_connection_data(rendezvous_token, introduction_node, target_id, introduction_message_encrypted) 1086 | ) 1087 | @'fire'('connection_progress', real_public_key, target_id, CONNECTION_PROGRESS_INTRODUCTION_SENT) 1088 | path_confirmation_timeout = timeoutSet(@_options['timeouts']['CONNECTION_TIMEOUT'], !~> 1089 | @_router['off']('data', path_confirmation) 1090 | encryptor_instance['destroy']() 1091 | try_to_introduce() 1092 | ) 1093 | try_to_introduce() 1094 | @_router['on']('data', found_introduction_nodes) 1095 | @_send_to_routing_path(first_node, route_id, ROUTING_COMMAND_FIND_INTRODUCTION_NODES_REQUEST, target_id) 1096 | find_introduction_nodes_timeout = timeoutSet(@_options['timeouts']['CONNECTION_TIMEOUT'], !~> 1097 | @_router['off']('data', found_introduction_nodes) 1098 | # Routing path is already established, but we don't need it anymore 1099 | @_router['destroy_routing_path'](first_node, route_id) 1100 | @_nodes_manager['del_first_node_in_routing_path'](first_node) 1101 | connection_failed(CONNECTION_ERROR_CANT_FIND_INTRODUCTION_NODES) 1102 | ) 1103 | # Error handler is already present in `_construct_routing_path()` method 1104 | .catch !~> 1105 | connection_failed(CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE) 1106 | real_public_key 1107 | 'get_max_data_size' : -> 1108 | @_max_data_size 1109 | /** 1110 | * @param {!Uint8Array} real_public_key Own real long-term public key as returned by `announce()` and `connect_to()` methods 1111 | * @param {!Uint8Array} target_id Should be connected already 1112 | * @param {number} command Command from range `0..255` 1113 | * @param {!Uint8Array} data Size limit can be obtained with `get_max_data_size()` method, roughly 65KiB 1114 | */ 1115 | 'send_to' : (real_public_key, target_id, command, data) !-> 1116 | if @_bootstrap_node 1117 | return 1118 | full_target_id = concat_arrays(real_public_key, target_id) 1119 | encryptor_instance = @_encryptor_instances.get(full_target_id) 1120 | if !encryptor_instance || data.length > @_max_data_size 1121 | return 1122 | multiplexer = @_multiplexers.get(full_target_id) 1123 | if !multiplexer 1124 | return 1125 | data_with_header = concat_arrays([command], data) 1126 | multiplexer['feed'](data_with_header) 1127 | if @_pending_sending.has(full_target_id) 1128 | # Timer is already in progress 1129 | return 1130 | # It might sometimes happen that we send command with small piece of data and the rest of the block is wasted. Sending data after 0 timeout 1131 | # allows for a few synchronous `send_to` calls to share the same block if possible in order to use space more efficiently 1132 | @_pending_sending.set( 1133 | full_target_id 1134 | setTimeout !~> 1135 | @_pending_sending.delete(full_target_id) 1136 | while multiplexer['have_more_blocks']() 1137 | data_block = multiplexer['get_block']() 1138 | data_block_encrypted = encryptor_instance['encrypt'](data_block) 1139 | @_send_to_routing_node(real_public_key, target_id, ROUTING_COMMAND_DATA, data_block_encrypted) 1140 | ) 1141 | 'destroy' : !-> 1142 | if @_destroyed 1143 | return 1144 | @_destroyed = true 1145 | # Bootstrap node immediately destroys router, no need to do it again 1146 | if !@_bootstrap_node 1147 | @_destroy_router() 1148 | else if @_http_server 1149 | @_http_server.close() 1150 | clearTimeout(@_random_lookup_timeout) 1151 | @_transport['destroy']() 1152 | @_dht['destroy']() 1153 | @_nodes_manager['destroy']() 1154 | _destroy_router : !-> 1155 | clearInterval(@_cleanup_interval) 1156 | clearInterval(@_keep_announce_routes_interval) 1157 | clearInterval(@_get_more_nodes_interval) 1158 | @_routing_paths.forEach ([node_id, route_id]) !~> 1159 | @_unregister_routing_path(node_id, route_id) 1160 | @_pending_connections.forEach ([, , , connection_timeout]) !~> 1161 | clearTimeout(connection_timeout) 1162 | @_router['destroy']() 1163 | /** 1164 | * Request more nodes to be aware of from some of the nodes already connected to 1165 | */ 1166 | _get_more_aware_of_nodes : !-> 1167 | if @_bootstrap_node || !@_nodes_manager['more_aware_of_nodes_needed']() 1168 | return 1169 | nodes = @_nodes_manager['get_random_connected_nodes'](5) 1170 | if !nodes 1171 | return 1172 | for node_id in nodes 1173 | @_get_more_nodes_from(node_id) 1174 | /** 1175 | * @param {!Uint8Array} peer_id 1176 | */ 1177 | _get_more_nodes_from : (peer_id) !-> 1178 | @_get_nodes_requested.add(peer_id) 1179 | @_send_uncompressed_core_command(peer_id, UNCOMPRESSED_CORE_COMMAND_GET_NODES_REQUEST, empty_array) 1180 | _do_random_lookup : !-> 1181 | if @_destroyed 1182 | return 1183 | # TODO: Selective disconnect from the network by an active attacker is dangerous 1184 | # Bootstrap if disconnected from the network 1185 | if @_dht['get_peers']().length < @'get_bootstrap_nodes'().length 1186 | @_bootstrap !~> 1187 | if @_dht['get_peers']().length 1188 | @_do_random_lookup() 1189 | else 1190 | timeoutSet(1, !~> 1191 | @_do_random_lookup() 1192 | ) 1193 | return 1194 | # Instead of fixed interval we use exponential distribution sampling 1195 | timeout = sample(@_options['timeouts']['RANDOM_LOOKUPS_INTERVAL']) 1196 | # Start first lookup immediately 1197 | @_random_lookup_timeout = timeoutSet(if @_random_lookup_timeout then timeout else 0, !~> 1198 | @_dht['lookup'](fake_node_id(), @_options['lookup_number']) 1199 | .then !~> 1200 | @_do_random_lookup() 1201 | .catch !~> 1202 | @_do_random_lookup() 1203 | ) 1204 | /** 1205 | * @param {!Array} nodes 1206 | * 1207 | * @return {!Promise} 1208 | */ 1209 | _construct_routing_path : (nodes) -> 1210 | first_node = nodes[0] 1211 | @_router['construct_routing_path'](nodes) 1212 | ..catch (error) !~> 1213 | error_handler(error) 1214 | @_nodes_manager['del_first_node_in_routing_path'](first_node) 1215 | /** 1216 | * @param {!Uint8Array} real_public_key 1217 | * @param {!Uint8Array} target_id Last node in routing path, responder 1218 | * @param {!Uint8Array} node_id First node in routing path, used for routing path identification 1219 | * @param {!Uint8Array} route_id ID of the route on `node_id` 1220 | */ 1221 | _register_routing_path : (real_public_key, target_id, node_id, route_id) !-> 1222 | source_id = concat_arrays(node_id, route_id) 1223 | if @_routing_path_to_id.has(source_id) 1224 | # Something went wrong, ignore 1225 | return 1226 | full_target_id = concat_arrays(real_public_key, target_id) 1227 | @_id_to_routing_path.set(full_target_id, [node_id, route_id]) 1228 | @_routing_path_to_id.set(source_id, [real_public_key, target_id]) 1229 | # Make sure each chunk after encryption will fit perfectly into DHT packet 1230 | # Multiplexer/demultiplexer pair is not needed for introduction node, but for simplicity we'll create it anyway 1231 | @_multiplexers.set(full_target_id, fixed-size-multiplexer['Multiplexer'](@_max_data_size, @_max_packet_data_size)) 1232 | @_demultiplexers.set(full_target_id, fixed-size-multiplexer['Demultiplexer'](@_max_data_size, @_max_packet_data_size)) 1233 | @'fire'('routing_paths_count', @_id_to_routing_path.size) 1234 | /** 1235 | * @param {!Uint8Array} node_id First node in routing path, used for routing path identification 1236 | * @param {!Uint8Array} route_id ID of the route on `node_id` 1237 | */ 1238 | _unregister_routing_path : (node_id, route_id) !-> 1239 | source_id = concat_arrays(node_id, route_id) 1240 | if !@_routing_paths.has(source_id) 1241 | return 1242 | @_nodes_manager['del_first_node_in_routing_path'](node_id) 1243 | @_routing_paths.delete(source_id) 1244 | @_router['destroy_routing_path'](node_id, route_id) 1245 | @_forwarding_mapping.delete(source_id) 1246 | @_pending_pings.delete(source_id) 1247 | @_announcements_from.forEach ([node_id, route_id, announce_interval], target_id) !~> 1248 | source_id_local = concat_arrays(node_id, route_id) 1249 | if !are_arrays_equal(source_id, source_id_local) 1250 | return 1251 | clearInterval(announce_interval) 1252 | @_announcements_from.delete(target_id) 1253 | if !@_routing_path_to_id.has(source_id) 1254 | return 1255 | [real_public_key, target_id] = @_routing_path_to_id.get(source_id) 1256 | full_target_id = concat_arrays(real_public_key, target_id) 1257 | @_routing_path_to_id.delete(source_id) 1258 | @_id_to_routing_path.delete(full_target_id) 1259 | if @_pending_sending.has(full_target_id) 1260 | clearTimeout(@_pending_sending.get(full_target_id)) 1261 | @_pending_sending.delete(full_target_id) 1262 | if @_announced_keypairs.has(real_public_key) 1263 | announced_to = @_announced_keypairs.get(real_public_key)[3] 1264 | announced_to.delete(target_id) 1265 | encryptor_instance = @_encryptor_instances.get(full_target_id) 1266 | if encryptor_instance 1267 | encryptor_instance['destroy']() 1268 | @_encryptor_instances.delete(full_target_id) 1269 | @_multiplexers.delete(full_target_id) 1270 | @_demultiplexers.delete(full_target_id) 1271 | @_unregister_application_connection(real_public_key, target_id) 1272 | @'fire'('routing_paths_count', @_id_to_routing_path.size) 1273 | /** 1274 | * @param {!Uint8Array} real_public_key 1275 | * @param {!Uint8Array} target_id Last node in routing path, responder 1276 | */ 1277 | _register_application_connection : (real_public_key, target_id) !-> 1278 | full_target_id = concat_arrays(real_public_key, target_id) 1279 | @_application_connections.add(full_target_id) 1280 | @'fire'('connected', real_public_key, target_id) 1281 | @'fire'('application_connections_count', @_application_connections.size) 1282 | /** 1283 | * @param {!Uint8Array} real_public_key 1284 | * @param {!Uint8Array} target_id Last node in routing path, responder 1285 | */ 1286 | _unregister_application_connection : (real_public_key, target_id) !-> 1287 | full_target_id = concat_arrays(real_public_key, target_id) 1288 | if @_application_connections.has(full_target_id) 1289 | @_application_connections.delete(full_target_id) 1290 | @'fire'('disconnected', real_public_key, target_id) 1291 | @'fire'('application_connections_count', @_application_connections.size) 1292 | /** 1293 | * @param {!Uint8Array} peer_id 1294 | * @param {number} command 0..9 1295 | * @param {!Uint8Array} command_data 1296 | */ 1297 | _handle_compressed_core_command : (peer_id, command, command_data) !-> 1298 | switch command 1299 | case COMPRESSED_CORE_COMMAND_SIGNAL 1300 | [source_id, target_id, sdp, signature] = parse_signal(command_data) 1301 | if !detox-crypto['verify'](signature, sdp, source_id) 1302 | @_peer_error(peer_id) 1303 | return 1304 | # If we are waiting for command - consume with callback 1305 | waiting_for_signal_key = concat_arrays(target_id, source_id) 1306 | waiting_for_signal_callback = @_waiting_for_signal.get(waiting_for_signal_key) 1307 | if waiting_for_signal_callback 1308 | @_waiting_for_signal.delete(waiting_for_signal_key) 1309 | waiting_for_signal_callback(sdp, signature, command_data) 1310 | return 1311 | # If command targets our peer, forward it 1312 | if @_nodes_manager['has_connected_node'](target_id) && are_arrays_equal(peer_id, source_id) 1313 | @_send_compressed_core_command(target_id, COMPRESSED_CORE_COMMAND_SIGNAL, command_data) 1314 | return 1315 | # If command doesn't target ourselves - exit 1316 | if !are_arrays_equal(target_id, @_dht_public_key) 1317 | return 1318 | # Otherwise consume signal 1319 | connection = @_transport['get_connection'](source_id) 1320 | if !connection 1321 | connection = @_transport['create_connection'](false, source_id) 1322 | if !connection 1323 | return 1324 | connection['on']('signal', (sdp) !~> 1325 | signature = detox-crypto['sign'](sdp, @_dht_public_key, @_dht_private_key) 1326 | command_data = compose_signal(@_dht_public_key, source_id, sdp, signature) 1327 | @_send_compressed_core_command(peer_id, COMPRESSED_CORE_COMMAND_SIGNAL, command_data) 1328 | ) 1329 | connection['signal'](sdp) 1330 | /** 1331 | * @param {!Uint8Array} peer_id 1332 | * @param {number} command 0..9 1333 | * @param {!Uint8Array} command_data 1334 | */ 1335 | _handle_uncompressed_core_command : (peer_id, command, command_data) !-> 1336 | switch command 1337 | case UNCOMPRESSED_CORE_COMMAND_FORWARD_INTRODUCTION 1338 | [target_id, introduction_message] = parse_introduce_to_data(command_data) 1339 | if !@_announcements_from.has(target_id) 1340 | return 1341 | [target_node_id, target_route_id] = @_announcements_from.get(target_id) 1342 | @_send_to_routing_path(target_node_id, target_route_id, ROUTING_COMMAND_INTRODUCTION, introduction_message) 1343 | case UNCOMPRESSED_CORE_COMMAND_GET_NODES_REQUEST 1344 | command_data = concat_arrays(@_nodes_manager['get_aware_of_nodes'](peer_id)) 1345 | @_send_uncompressed_core_command(peer_id, UNCOMPRESSED_CORE_COMMAND_GET_NODES_RESPONSE, command_data) 1346 | case UNCOMPRESSED_CORE_COMMAND_GET_NODES_RESPONSE 1347 | if !@_get_nodes_requested.has(peer_id) 1348 | return 1349 | @_get_nodes_requested.delete(peer_id) 1350 | if !command_data.length 1351 | return 1352 | if command_data.length % PUBLIC_KEY_LENGTH != 0 1353 | @_peer_error(peer_id) 1354 | return 1355 | number_of_nodes = command_data.length / PUBLIC_KEY_LENGTH 1356 | nodes = [] 1357 | for i from 0 til number_of_nodes 1358 | new_node_id = command_data.subarray(i * PUBLIC_KEY_LENGTH, (i + 1) * PUBLIC_KEY_LENGTH) 1359 | if are_arrays_equal(@_dht_public_key, new_node_id) 1360 | @_peer_error(peer_id) 1361 | nodes.push(new_node_id) 1362 | @_nodes_manager['set_aware_of_nodes'](peer_id, nodes) 1363 | case UNCOMPRESSED_CORE_COMMAND_BOOTSTRAP_NODE 1364 | @_nodes_manager['add_bootstrap_node'](peer_id, array2string(command_data)) 1365 | /** 1366 | * @param {!Uint8Array} peer_id 1367 | * @param {number} command 0..9 1368 | * @param {!Uint8Array} command_data 1369 | */ 1370 | _send_compressed_core_command : (peer_id, command, command_data) !-> 1371 | @_send(peer_id, command, command_data) 1372 | /** 1373 | * @param {!Uint8Array} peer_id 1374 | * @param {number} command 0..9 1375 | * @param {!Uint8Array} command_data 1376 | */ 1377 | _send_dht_command : (peer_id, command, command_data) !-> 1378 | @_send(peer_id, command + DHT_COMMANDS_OFFSET, command_data) 1379 | /** 1380 | * @param {!Uint8Array} peer_id 1381 | * @param {!Uint8Array} data 1382 | */ 1383 | _send_routing_command : (peer_id, data) !-> 1384 | @_send(peer_id, ROUTING_COMMANDS, data) 1385 | /** 1386 | * @param {!Uint8Array} peer_id 1387 | * @param {number} command 0..234 1388 | * @param {!Uint8Array} command_data 1389 | */ 1390 | _send_uncompressed_core_command : (peer_id, command, command_data) !-> 1391 | @_send(peer_id, command + UNCOMPRESSED_CORE_COMMANDS_OFFSET, command_data) 1392 | /** 1393 | * @param {!Uint8Array} node_id 1394 | * @param {number} command 0..255 1395 | * @param {!Uint8Array} command_data 1396 | */ 1397 | _send : (node_id, command, command_data) !-> 1398 | if @_nodes_manager['has_connected_node'](node_id) 1399 | @_update_connection_timeout(node_id, true) 1400 | @_transport['send'](node_id, command, command_data) 1401 | return 1402 | !~function connected (new_node_id) 1403 | if !are_arrays_equal(node_id, new_node_id) 1404 | return 1405 | @_update_connection_timeout(node_id, true) 1406 | @_transport['send'](node_id, command, command_data) 1407 | @_transport['on']('connected', connected) 1408 | @_dht['lookup'](node_id, @_options['lookup_number']) 1409 | .then !~> 1410 | @_transport['off']('connected', connected) 1411 | .catch !~> 1412 | @_transport['off']('connected', connected) 1413 | /** 1414 | * @param {!Uint8Array} real_public_key 1415 | * @param {!Uint8Array} target_id 1416 | * @param {number} command 0..245 1417 | * @param {!Uint8Array} data 1418 | */ 1419 | _send_to_routing_node : (real_public_key, target_id, command, data) !-> 1420 | full_target_id = concat_arrays(real_public_key, target_id) 1421 | if !@_id_to_routing_path.has(full_target_id) 1422 | return 1423 | [node_id, route_id] = @_id_to_routing_path.get(full_target_id) 1424 | @_send_to_routing_path(node_id, route_id, command, data) 1425 | /** 1426 | * @param {!Uint8Array} node_id 1427 | * @param {!Uint8Array} route_id 1428 | * @param {number} command 1429 | * @param {!Uint8Array} data 1430 | */ 1431 | _send_to_routing_path : (node_id, route_id, command, data) -> 1432 | if data.length == 0 1433 | # Just to make sure demultiplexer will not discard this command 1434 | data = new Uint8Array(1) 1435 | @_router['send_data'](node_id, route_id, command, data) 1436 | /** 1437 | * @param {!Uint8Array} node_id 1438 | * @param {!Uint8Array} route_id 1439 | * 1440 | * @return {boolean} `true` if ping was sent (not necessary delivered) 1441 | */ 1442 | _send_ping : (node_id, route_id) -> 1443 | source_id = concat_arrays(node_id, route_id) 1444 | if @_pending_pings.has(source_id) || !@_routing_paths.has(source_id) 1445 | return false 1446 | @_send_to_routing_path(node_id, route_id, ROUTING_COMMAND_PING, empty_array) 1447 | true 1448 | /** 1449 | * @param {!Uint8Array} node_id 1450 | * @param {boolean} send 1451 | */ 1452 | _update_connection_timeout : (node_id, send) !-> 1453 | # TODO: Check `send == true` separately from `send == false` 1454 | @_connections_timeouts.set(node_id, +(new Date)) 1455 | /** 1456 | * Generate message with introduction nodes that can later be published by any node connected to DHT (typically other node than this for anonymity) 1457 | * 1458 | * @param {!Uint8Array} real_public_key Ed25519 public key (real one, different from supplied in DHT constructor) 1459 | * @param {!Uint8Array} real_private_key Corresponding Ed25519 private key 1460 | * @param {!Array} introduction_nodes Array of public keys of introduction points 1461 | * 1462 | * @return {!Uint8Array} 1463 | */ 1464 | _generate_announcement_message : (real_public_key, real_private_key, introduction_nodes) -> 1465 | time = parseInt(+(new Date) / 1000, 10) # In seconds, should be enough if kept as unsigned 32-bit integer which we actually do 1466 | concat_arrays(@_dht['make_mutable_value'](real_public_key, real_private_key, time, concat_arrays(introduction_nodes))) 1467 | /** 1468 | * @param {!Uint8Array} message 1469 | * 1470 | * @return {Uint8Array} Public key if signature is correct, `null` otherwise 1471 | */ 1472 | _verify_announcement_message : (message) -> 1473 | real_public_key = message.subarray(0, PUBLIC_KEY_LENGTH) 1474 | data = message.subarray(PUBLIC_KEY_LENGTH) 1475 | payload = @_dht['verify_value'](real_public_key, data) 1476 | # If value is not valid or length doesn't fit certain number of introduction nodes exactly 1477 | if !payload || (payload[1].length % PUBLIC_KEY_LENGTH) 1478 | null 1479 | else 1480 | real_public_key 1481 | /** 1482 | * Publish message with introduction nodes (typically happens on different node than `_generate_announcement_message()`) 1483 | * 1484 | * @param {!Uint8Array} message 1485 | */ 1486 | _publish_announcement_message : (message) !-> 1487 | real_public_key = message.subarray(0, PUBLIC_KEY_LENGTH) 1488 | data = message.subarray(PUBLIC_KEY_LENGTH) 1489 | @_dht['put_value'](real_public_key, data, @_options['lookup_number']) 1490 | /** 1491 | * Find nodes in DHT that are acting as introduction points for specified public key 1492 | * 1493 | * @param {!Uint8Array} target_public_key 1494 | * 1495 | * @return {!Promise} Resolves with `!Array` 1496 | */ 1497 | _find_introduction_nodes : (target_public_key) -> 1498 | @_dht['get_value'](target_public_key, @_options['lookup_number']).then (introduction_nodes_bulk) -> 1499 | if introduction_nodes_bulk.length % PUBLIC_KEY_LENGTH != 0 1500 | throw '' 1501 | introduction_nodes = [] 1502 | for i from 0 til introduction_nodes_bulk.length / PUBLIC_KEY_LENGTH 1503 | introduction_nodes.push(introduction_nodes_bulk.subarray(i * PUBLIC_KEY_LENGTH, (i + 1) * PUBLIC_KEY_LENGTH)) 1504 | introduction_nodes 1505 | _peer_error : (peer_id) !-> 1506 | @_dht['del_peer'](peer_id) 1507 | @_transport['destroy_connection'](peer_id) 1508 | # TODO 1509 | _peer_warning : (peer_id) !-> 1510 | # TODO 1511 | 1512 | Core:: = Object.assign(Object.create(async-eventer::), Core::) 1513 | Object.defineProperty(Core::, 'constructor', {value: Core}) 1514 | { 1515 | 'ready' : (callback) !-> 1516 | <-! detox-crypto['ready'] 1517 | <-! detox-dht['ready'] 1518 | <-! detox-routing['ready'] 1519 | callback() 1520 | /** 1521 | * Generate random seed that can be used as keypair seed 1522 | * 1523 | * @return {!Uint8Array} 32 bytes 1524 | */ 1525 | 'generate_seed' : -> 1526 | random_bytes(PUBLIC_KEY_LENGTH) 1527 | 'Core' : Core 1528 | } 1529 | 1530 | # NOTE: `node-fetch` dependency is the last one and only specified for CommonJS, make sure to insert new dependencies before it 1531 | if typeof define == 'function' && define['amd'] 1532 | # AMD 1533 | define(['@detox/crypto', '@detox/dht', '@detox/nodes-manager', '@detox/routing', '@detox/transport', '@detox/utils', 'fixed-size-multiplexer', 'async-eventer'], Wrapper) 1534 | else if typeof exports == 'object' 1535 | # CommonJS 1536 | module.exports = Wrapper(require('@detox/crypto'), require('@detox/dht'), require('@detox/nodes-manager'), require('@detox/routing'), require('@detox/transport'), require('@detox/utils'), require('fixed-size-multiplexer'), require('async-eventer'), require('node-fetch')) 1537 | else 1538 | # Browser globals 1539 | @'detox_core' = Wrapper(@'detox_crypto', @'detox_dht', @'detox_nodes_manager', @'detox_routing', @'detox_transport', @'detox_utils', @'fixed_size_multiplexer', @'async_eventer') 1540 | -------------------------------------------------------------------------------- /src/index.min.js: -------------------------------------------------------------------------------- 1 | /* 2 | 0BSD 3 | */ 4 | (function(){function aa(e,l){var h=new Uint8Array(e.length+t);h.set(e);h.set(l,e.length);return h}function J(h,l,y,v){var n=new Uint8Array(2*e+y.length+t);n.set(h);n.set(l,e);n.set(y,2*e);n.set(v,2*e+y.length);return n}function R(h){return[h.subarray(0,e),h.subarray(e,2*e),h.subarray(2*e,h.length-t),h.subarray(h.length-t)]}function Ca(h){var l;var y=h[0];var v=h.subarray(1,1+e);var n=[];h=h.subarray(1+e);var p=0;for(l=h.length/e;p=T?b.i&&c!==S||b.ua(a,c-T,d):c===U?b.i||b.g.process_packet(a,d):c>=V?b.f.receive(a,c-V,d):b.i&&c!==E||b.ta(a,c,d)});this.f=l.DHT(this.h,k,this.a.state_history_size,this.a.values_cache_size,this.a.fraction_of_nodes_from_same_peer, 10 | this.a.timeouts).on("peer_error",function(a){b.H(a)}).on("peer_warning",function(){}).on("connect_to",function(a,c){return new Promise(function(f,d){function e(){d()}if(b.b.has_connected_node(a))f();else{var m=b.c.get_connection(a);if(!m){m=b.c.create_connection(!0,a);if(!m){d();return}m.on("signal",function(f){var d=h.sign(f,b.h,b.L);f=J(b.h,a,f,d);b.P(c,E,f)})}m.once("connected",function(){m.off("disconnected",e);f()}).once("disconnected",e)}})}).on("send",function(a,c,d){b.xa(a,c,d)}).on("peer_updated", 11 | function(a,c){b.b.set_peer(a,c)});this.g=v.Router(this.V.x25519["private"],this.a.max_pending_segments,this.a.timeouts.ROUTING_PATH_SEGMENT_TIMEOUT).on("activity",function(a,c){var f=u(a,c);b.s.has(f)||b.s.set(f,[a,c]);b.Y.set(f,+new Date)}).on("send",function(a,c){b.ya(a,c)}).on("data",function(a,c,d,k){var f;var g=u(a,c);switch(d){case ea:var m=b.Aa(k);if(!m)break;b.v.has(m)&&clearInterval(b.v.get(m)[2]);d=P(b.a.timeouts.ANNOUNCE_INTERVAL,function(){b.s.has(g)&&b.ia(k)});b.v.set(m,[a,c,d]);b.ia(k); 12 | break;case fa:var q=k;if(q.length!==e)break;var W=function(d,f){var k=q,m;var r=m=new Uint8Array(1+e+f.length*e);r.set([d]);r.set(k,1);d=0;for(k=f.length;dq[E])return;H.B=!0;break}}}}else H={S:!1},b.j.set(B,H);k={real_public_key:A,target_id:q,secret:z,application:v,number_of_intermediate_nodes:null};b.fire("introduction", 15 | k).then(function(){var a=k.number_of_intermediate_nodes;if(null===a)throw"No event handler for introduction";if(a){if(a=b.b.get_nodes_for_routing_path(a,[w]),a.push(w),!a)return}else a=[w];var c=a[0];b.ba(a).then(function(a){var d=h.Encryptor(!1,p.x25519["private"]);d.put_handshake_message(da);var f=d.get_handshake_message();b.C.set(B,d);b.ca(A,q,c,a);b.j["delete"](B);b.ka(A,q);a=h.sign(x,A,p.ed25519["private"]);b.Z(A,q,la,Fa(a,x,f))})["catch"](function(){b.j["delete"](B);H.S&&H.B&&b.fire("connection_failed", 16 | A,q,Q)})})["catch"](function(a){M(a);b.j["delete"](B);H.S&&H.B&&b.fire("connection_failed",A,q,Q)})}}}catch(Ka){m=Ka,M(m)}break;case Y:if(b.D.has(g))r=b.D.get(g),d=r[0],l=r[1],b.u(d,l,Y,k);else if(b.o.has(g)&&(r=b.o.get(g),A=r[0],q=r[1],B=u(A,q),d=b.C.get(B))&&(m=b.U.get(B)))for(d=d.decrypt(k),m.feed(d);m.have_more_data();)C=m.get_data(),d=C[0],b.fire("data",A,q,d,C.subarray(1));break;case oa:if(b.o.has(g)&&b.I.has(g))b.I["delete"](g);else b.la(a,c)}});this.b=y(a,this.a.aware_of_nodes_limit,this.a.timeouts.STALE_AWARE_OF_NODE_TIMEOUT).on("connected_nodes_count", 17 | function(a){b.K=a;b.fire("connected_nodes_count",b.K);if(a>b.a.connected_nodes_limit){var c=L();b.D.forEach(function(a){c.add(a[0])});a=b.b.get_candidates_for_disconnection(c);a.length&&(a=pa(a),b.c.destroy_connection(a))}}).on("aware_of_nodes_count",function(a){b.fire("aware_of_nodes_count",a)}).on("peer_error",function(a){b.H(a)}).on("peer_warning",function(){});this.ha=this.g.get_max_packet_data_size()-La;if(a.length)this.f.once("peer_updated",function(){b.fire("ready");b.ga()});else setTimeout(function(){b.fire("ready")}); 18 | this.G()}null==N&&(N=window.fetch);var Ma=p.hex2array;var Na=p.array2hex;var Ja=p.string2array;var Oa=p.array2string;var Z=p.random_bytes;var Pa=p.random_int;var pa=p.pull_random_item_from_array;var D=p.are_arrays_equal;var u=p.concat_arrays;var I=p.timeoutSet;var P=p.intervalSet;var M=p.error_handler;var z=p.ArrayMap;var L=p.ArraySet;var Qa=p.sample;var qa=new Uint8Array(0);var ra=new Uint8Array(e);w.CONNECTION_ERROR_CANT_FIND_INTRODUCTION_NODES=sa;w.CONNECTION_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES= 19 | ta;w.CONNECTION_ERROR_NO_INTRODUCTION_NODES=X;w.CONNECTION_ERROR_CANT_CONNECT_TO_RENDEZVOUS_NODE=Q;w.CONNECTION_ERROR_OUT_OF_INTRODUCTION_NODES=ua;w.CONNECTION_PROGRESS_CONNECTED_TO_RENDEZVOUS_NODE=va;w.CONNECTION_PROGRESS_FOUND_INTRODUCTION_NODES=wa;w.CONNECTION_PROGRESS_INTRODUCTION_SENT=xa;w.ANNOUNCEMENT_ERROR_NOT_ENOUGH_INTERMEDIATE_NODES=ya;w.ANNOUNCEMENT_ERROR_NO_INTRODUCTION_NODES_CONFIRMED=za;w.prototype={start_bootstrap_node:function(a,c,d,k,e){var b=this;null==k&&(k=c);null==e&&(e=d);var f= 20 | h.create_keypair(a).ed25519;this.W=require("http").createServer(function(a,c){function d(a,b){m||(c.writeHead(a),b&&c.write(Buffer.from(b)),c.end())}var k;c.setHeader("Access-Control-Allow-Origin","*");c.setHeader("Connection","close");var e=a.headers["content-length"];if("POST"===a.method&&e&&e<=b.wa){var g=[];var m=!1;a.on("data",function(a){g.push(a)}).on("end",function(){var a;g=u(g);var c=R(g);var e=c[0];var m=c[1];var q=c[2];var l=c[3];if(h.verify(l,q,e)&&D(m,ra))if(b.K&&Pa(0,b.K)?a=null!=(c= 21 | b.b.get_random_connected_nodes(1))?c[0]:void 0:a=null,a){var x=u(e,a);b.J.has(x)?d(503):(q=J(e,a,q,l),b.P(a,E,q),b.J.set(x,function(b,c,e){clearTimeout(k);h.verify(c,b,a)?(b=aa(e,h.sign(e,f["public"],f["private"])),d(200,Buffer.from(b))):d(502)}),k=I(b.a.timeouts.CONNECTION_TIMEOUT,function(){b.J["delete"](x);d(504)}))}else(c=b.c.create_connection(!1,e))?c.once("signal",function(a){var c=h.sign(a,b.h,b.L);a=J(b.h,e,a,c);a=aa(a,h.sign(a,f["public"],f["private"]));d(200,Buffer.from(a));return!1}).signal(q): 22 | d(503);else d(400)}).on("close",function(){clearTimeout(k);m=!0})}else d(400)});this.W.on("error",M).listen(d,c,function(){var a=Na(f["public"]);b.i=a+":"+k+":"+e});this.da()},get_bootstrap_nodes:function(){return this.b.get_bootstrap_nodes()},na:function(a){function c(){--d;d||"function"==typeof a&&a()}var d,k=this;var g=this.get_bootstrap_nodes();(d=g.length)?g.forEach(function(a){function b(){c()}var d;a=a.split(":");var g=Ma(a[0]);var l=a[1]+":"+a[2];var u=Z(e);if(d=k.c.create_connection(!0,u))d.on("signal", 23 | function(a){var b=h.sign(a,k.h,k.L);var c={method:"POST",headers:{Connection:"close"},body:J(k.h,ra,a,b).buffer};N("https://"+l,c)["catch"](function(a){if("undefined"===typeof location||"http:"===location.protocol)return N("http://"+l,c);throw a;}).then(function(a){if(!a.ok)throw"Request failed, status code "+a.status;return a.arrayBuffer()}).then(function(a){return new Uint8Array(a)}).then(function(a){a=[a.subarray(0,a.length-t),a.subarray(a.length-t)];var b=a[0];a=a[1];if(!h.verify(a,b,g))throw"Bad bootstrap node response"; 24 | a=R(b);b=a[0];var c=a[1];var f=a[2];a=a[3];if(!h.verify(a,f,b)||!D(c,k.h))throw"Bad response";if(k.c.get_connection(b))throw"Already connected";k.c.update_peer_id(u,b);d.signal(f)})["catch"](function(){d.destroy()})}).once("connected",function(){d.off("disconnected",b);c()}).once("disconnected",b)}):"function"==typeof a&&a()},announce:function(a,c,d){if(this.i)return null;a=O(a);var e=a.ed25519["public"];if(this.m.has(e))return null;this.m.set(e,[a,c,d,L()]);this.ea(e);return e},ea:function(a){function c(c){var d; 25 | c&&n.push(c);--q;if(!q)if(n.length){n=n.concat(t);var e=b.qa(a,h.ed25519["private"],n);var f=0;for(d=n.length;fthis.N||!(b=this.X.get(h))||(d=u([d],e),b.feed(d),this.A.has(h)||this.A.set(h,setTimeout(function(){for(f.A["delete"](h);b.have_more_blocks();){var d=b.get_block();d=k.encrypt(d);f.Z(a,c,Y,d)}})))}},destroy:function(){this.fa||(this.fa=!0,this.i?this.W&&this.W.close():this.da(),clearTimeout(this.ja),this.c.destroy(),this.f.destroy(),this.b.destroy())},da:function(){var a=this;clearInterval(this.oa);clearInterval(this.va); 31 | clearInterval(this.sa);this.s.forEach(function(c){a.ma(c[0],c[1])});this.w.forEach(function(a){clearTimeout(a[3])});this.g.destroy()},ga:function(){var a,c;if(!this.i&&this.b.more_aware_of_nodes_needed()&&(a=this.b.get_random_connected_nodes(5))){var d=0;for(c=a.length;d 5 | * @license 0BSD 6 | */ 7 | (function(){ 8 | var detoxCrypto, lib, test, NUMBER_OF_NODES, bootstrap_ip, bootstrap_address, bootstrap_port, command, data, application; 9 | require('@detox/simple-peer-mock').register(); 10 | detoxCrypto = require('@detox/crypto'); 11 | lib = require('..'); 12 | test = require('tape'); 13 | NUMBER_OF_NODES = 50; 14 | bootstrap_ip = '127.0.0.1'; 15 | bootstrap_address = 'localhost'; 16 | bootstrap_port = 16882; 17 | command = 38; 18 | data = Buffer.from('9551397153de34cc344549f724c3c448f0a61a96e05b700213f1d05cb6f81785d8b28523a19cfb0c65df3d484127c3486772ebcc1f6df452c76e51ec156fd488240bffc743170d943d764622e2ccf79518dbd54d322bdb88b398fc17545fb975eb8f4fa284ecdab825a3cb8245bae7dd5e8cb53a675f3aaa5cbced903145f4a5830a272d41474e42218fac332319d9bb792b3594bb2f0112824823da341a1eeb170b1871bb0971a4034c0038e0db2f3aaf0a53a1ae9b252ed01cfb17a667bd446b98f801d633beab6c215b6a7c82bcc04b0515f8b47e50d0a86325895c0877ab0ec1bf5071130f31fad5c1384bda7f57e829e849db57a08bc320ee0e59a5d202c07430ed8d1a7c0c1ce58f9263ed909f8135fede90154cece08ee79b76f05ae1309ced4cf120a555ba94a96aa23d61b8ee6896f01eef655291269686c96cc9a13248a13a3870a10c6eb6f836a1f7b93eb4355f9828546dd7487c6e69248d349cc7123320b7afa3b480dfd2fe9e27cbfedbbb192802d2a654a60fbfed7c87e7e6a586e0b8ec30df4bc622e797220626268fc11700c4b62406000718e7a11d38b319a3d5728e8c41ad0f2bb6085129e066ab3d92df365326106a28ea546183f891eab0d6d24838b99a982417a4e91b3f27a92dee29e06b5baad2fcb7b1cd56ba0a546b44e07f281a1420d0f4e7861b0a7ae56d2b71989086420409ad2dc24523b02ff61cbc8f189a28c3e0b82fd4063b9bf6c7f6b5dbe8d47cf96ba6c4d3f3b9debd0705b5a7bcccdd46c8f7842f83e118d7dac74f2f722426391669eba4fbadf0897c881ebe518fcf1b88e7805056c1b231a4bb307a22c1435ac297e2993348de689b10ce1bb73b26b554091af8a48b968aabff4a772065e9b14182fe06fb1ca8a653a367cc31439f7210e998795f32fb4eed2a01f435ce6d385b07d7140bd43a9ddb6e92f6c9460f0b26c10329e7f781de5fef6d131d54ed708a3847', 'hex'); 19 | application = Buffer.from('Detox test'); 20 | lib.ready(function(){ 21 | test('Core', function(t){ 22 | var generated_seed, bootstrap_node_seed, bootstrap_node_id, bootstrap_node_info, x$, node_1_real_seed, node_1_real_public_key, node_1_secret, y$, node_3_real_seed, node_3_real_public_key, nodes, wait_for, options, promise, i$, to$; 23 | generated_seed = lib.generate_seed(); 24 | t.ok(generated_seed instanceof Uint8Array, 'Seed is Uint8Array'); 25 | t.equal(generated_seed.length, 32, 'Seed length is 32 bytes'); 26 | bootstrap_node_seed = new Uint8Array(32); 27 | bootstrap_node_id = Buffer.from(detoxCrypto.create_keypair(bootstrap_node_seed).ed25519['public']).toString('hex'); 28 | bootstrap_node_info = bootstrap_node_id + ":" + bootstrap_address + ":" + bootstrap_port; 29 | x$ = node_1_real_seed = new Uint8Array(32); 30 | x$.set([1, 1]); 31 | node_1_real_public_key = detoxCrypto.create_keypair(node_1_real_seed).ed25519['public']; 32 | node_1_secret = Buffer.from('c2fd7c6349f0bb25ed28', 'hex'); 33 | y$ = node_3_real_seed = new Uint8Array(32); 34 | y$.set([3, 1]); 35 | node_3_real_public_key = detoxCrypto.create_keypair(node_3_real_seed).ed25519['public']; 36 | nodes = []; 37 | wait_for = NUMBER_OF_NODES; 38 | options = { 39 | timeouts: { 40 | STATE_UPDATE_INTERVAL: 2, 41 | GET_MORE_AWARE_OF_NODES_INTERVAL: 2, 42 | ROUTING_PATH_SEGMENT_TIMEOUT: 30, 43 | LAST_USED_TIMEOUT: 15, 44 | ANNOUNCE_INTERVAL: 5, 45 | RANDOM_LOOKUPS_INTERVAL: 100 46 | }, 47 | connected_nodes_limit: 30, 48 | lookup_number: 3 49 | }; 50 | promise = Promise.resolve(); 51 | for (i$ = 0, to$ = NUMBER_OF_NODES; i$ < to$; ++i$) { 52 | (fn$.call(this, i$)); 53 | } 54 | function destroy_nodes(){ 55 | var i$, ref$, len$, node; 56 | console.log('Destroying nodes...'); 57 | for (i$ = 0, len$ = (ref$ = nodes).length; i$ < len$; ++i$) { 58 | node = ref$[i$]; 59 | node.destroy(); 60 | } 61 | console.log('Destroyed'); 62 | t.end(); 63 | } 64 | function ready_callback(){ 65 | var announcement_retry, connection_retry, node_1, node_3; 66 | announcement_retry = 3; 67 | connection_retry = 5; 68 | node_1 = nodes[1]; 69 | node_3 = nodes[3]; 70 | node_1.once('announced', function(){ 71 | node_1.off('announcement_failed'); 72 | t.pass('Announced successfully'); 73 | node_1.on('introduction', function(data){ 74 | t.equal(data.application.subarray(0, application.length).join(','), application.join(','), 'Correct application on introduction'); 75 | t.equal(data.secret.subarray(0, node_1_secret.length).join(','), node_1_secret.join(','), 'Correct secret on introduction'); 76 | data.number_of_intermediate_nodes = 1; 77 | }); 78 | node_3.once('connected', function(arg$, target_id){ 79 | node_1.off('introduction'); 80 | node_3.off('connection_failed'); 81 | t.equal(target_id.join(','), node_1_real_public_key.join(','), 'Connected to intended node successfully'); 82 | node_1.once('data', function(arg$, arg1$, received_command, received_data){ 83 | t.equal(received_command, command, 'Received command correctly'); 84 | t.equal(received_data.join(','), data.join(','), 'Received data correctly'); 85 | destroy_nodes(); 86 | }); 87 | console.log('Sending data...'); 88 | node_3.send_to(node_3_real_public_key, node_1_real_public_key, command, data); 89 | }); 90 | node_3.on('connection_failed', function(arg$, arg1$, reason){ 91 | if (connection_retry) { 92 | --connection_retry; 93 | console.log('Connection failed with code ' + reason + ', retry in 5s...'); 94 | setTimeout(function(){ 95 | console.log('Connecting...'); 96 | node_3.connect_to(node_3_real_seed, node_1_real_public_key, application, node_1_secret, 1); 97 | }, 5000); 98 | return; 99 | } 100 | t.fail('Connection failed with code ' + reason); 101 | destroy_nodes(); 102 | }); 103 | console.log('Preparing for connection (5s)...'); 104 | setTimeout(function(){ 105 | console.log('Connecting...'); 106 | node_3.connect_to(node_3_real_seed, node_1_real_public_key, application, node_1_secret, 1); 107 | }, 5000); 108 | }).on('announcement_failed', function(arg$, reason){ 109 | if (announcement_retry) { 110 | --announcement_retry; 111 | console.log('Announcement failed with code ' + reason + ', retry...'); 112 | return; 113 | } 114 | t.fail('Announcement failed with code ' + reason); 115 | destroy_nodes(); 116 | }); 117 | console.log('Preparing for announcement (2s)...'); 118 | setTimeout(function(){ 119 | console.log('Announcing...'); 120 | node_1.announce(node_1_real_seed, 3, 1); 121 | }, 2000); 122 | } 123 | function fn$(i){ 124 | promise = promise.then(function(){ 125 | return new Promise(function(resolve){ 126 | var x$, dht_seed, instance; 127 | x$ = dht_seed = new Uint8Array(32); 128 | x$.set([i % 255, (i - i % 255) / 255]); 129 | if (i === 0) { 130 | instance = lib.Core([], [], 10, 20, Object.assign({}, options, { 131 | dht_keypair_seed: dht_seed, 132 | connected_nodes_limit: NUMBER_OF_NODES 133 | })); 134 | instance.start_bootstrap_node(bootstrap_node_seed, bootstrap_ip, bootstrap_port, bootstrap_address); 135 | } else { 136 | instance = lib.Core([bootstrap_node_info], [], 10, 2, Object.assign({ 137 | dht_keypair_seed: dht_seed 138 | }, options)); 139 | } 140 | instance.once('ready', function(){ 141 | t.pass('Node ' + i + ' is ready, #' + (NUMBER_OF_NODES - wait_for + 1) + '/' + NUMBER_OF_NODES); 142 | if (i === 2) { 143 | t.same(instance.get_bootstrap_nodes(), [bootstrap_node_info], 'Bootstrap nodes are returned correctly'); 144 | t.equal(instance.get_max_data_size(), Math.pow(2, 16) - 2, 'Max data size returned correctly'); 145 | } 146 | --wait_for; 147 | if (!wait_for) { 148 | ready_callback(); 149 | } 150 | }); 151 | nodes.push(instance); 152 | setTimeout(resolve, 100); 153 | }); 154 | }); 155 | } 156 | }); 157 | }); 158 | }).call(this); 159 | -------------------------------------------------------------------------------- /tests/index.ls: -------------------------------------------------------------------------------- 1 | /** 2 | * @package Detox core 3 | * @author Nazar Mokrynskyi 4 | * @license 0BSD 5 | */ 6 | require('@detox/simple-peer-mock').register() 7 | 8 | detox-crypto = require('@detox/crypto') 9 | lib = require('..') 10 | test = require('tape') 11 | 12 | const NUMBER_OF_NODES = 50 13 | 14 | bootstrap_ip = '127.0.0.1' 15 | bootstrap_address = 'localhost' 16 | bootstrap_port = 16882 17 | 18 | command = 38 19 | data = Buffer.from( 20 | '9551397153de34cc344549f724c3c448f0a61a96e05b700213f1d05cb6f81785d8b28523a19cfb0c65df3d484127c3486772ebcc1f6df452c76e51ec156fd488240bffc743170d943d764622e2ccf79518dbd54d322bdb88b398fc17545fb975eb8f4fa284ecdab825a3cb8245bae7dd5e8cb53a675f3aaa5cbced903145f4a5830a272d41474e42218fac332319d9bb792b3594bb2f0112824823da341a1eeb170b1871bb0971a4034c0038e0db2f3aaf0a53a1ae9b252ed01cfb17a667bd446b98f801d633beab6c215b6a7c82bcc04b0515f8b47e50d0a86325895c0877ab0ec1bf5071130f31fad5c1384bda7f57e829e849db57a08bc320ee0e59a5d202c07430ed8d1a7c0c1ce58f9263ed909f8135fede90154cece08ee79b76f05ae1309ced4cf120a555ba94a96aa23d61b8ee6896f01eef655291269686c96cc9a13248a13a3870a10c6eb6f836a1f7b93eb4355f9828546dd7487c6e69248d349cc7123320b7afa3b480dfd2fe9e27cbfedbbb192802d2a654a60fbfed7c87e7e6a586e0b8ec30df4bc622e797220626268fc11700c4b62406000718e7a11d38b319a3d5728e8c41ad0f2bb6085129e066ab3d92df365326106a28ea546183f891eab0d6d24838b99a982417a4e91b3f27a92dee29e06b5baad2fcb7b1cd56ba0a546b44e07f281a1420d0f4e7861b0a7ae56d2b71989086420409ad2dc24523b02ff61cbc8f189a28c3e0b82fd4063b9bf6c7f6b5dbe8d47cf96ba6c4d3f3b9debd0705b5a7bcccdd46c8f7842f83e118d7dac74f2f722426391669eba4fbadf0897c881ebe518fcf1b88e7805056c1b231a4bb307a22c1435ac297e2993348de689b10ce1bb73b26b554091af8a48b968aabff4a772065e9b14182fe06fb1ca8a653a367cc31439f7210e998795f32fb4eed2a01f435ce6d385b07d7140bd43a9ddb6e92f6c9460f0b26c10329e7f781de5fef6d131d54ed708a3847' 21 | 'hex' 22 | ) 23 | application = Buffer.from('Detox test') 24 | 25 | <-! lib.ready 26 | test('Core', (t) !-> 27 | generated_seed = lib.generate_seed() 28 | t.ok(generated_seed instanceof Uint8Array, 'Seed is Uint8Array') 29 | t.equal(generated_seed.length, 32, 'Seed length is 32 bytes') 30 | 31 | bootstrap_node_seed = new Uint8Array(32) 32 | bootstrap_node_id = Buffer.from(detox-crypto.create_keypair(bootstrap_node_seed).ed25519.public).toString('hex') 33 | bootstrap_node_info = "#bootstrap_node_id:#bootstrap_address:#bootstrap_port" 34 | 35 | node_1_real_seed = new Uint8Array(32) 36 | ..set([1, 1]) 37 | node_1_real_public_key = detox-crypto.create_keypair(node_1_real_seed).ed25519.public 38 | node_1_secret = Buffer.from('c2fd7c6349f0bb25ed28', 'hex') 39 | node_3_real_seed = new Uint8Array(32) 40 | ..set([3, 1]) 41 | node_3_real_public_key = detox-crypto.create_keypair(node_3_real_seed).ed25519.public 42 | 43 | nodes = [] 44 | 45 | wait_for = NUMBER_OF_NODES 46 | options = 47 | timeouts : 48 | STATE_UPDATE_INTERVAL : 2 49 | GET_MORE_AWARE_OF_NODES_INTERVAL : 2 50 | ROUTING_PATH_SEGMENT_TIMEOUT : 30 51 | LAST_USED_TIMEOUT : 15 52 | ANNOUNCE_INTERVAL : 5 53 | RANDOM_LOOKUPS_INTERVAL : 100 54 | connected_nodes_limit : 30 55 | lookup_number : 3 56 | promise = Promise.resolve() 57 | for let i from 0 til NUMBER_OF_NODES 58 | promise := promise.then -> 59 | new Promise (resolve) !-> 60 | dht_seed = new Uint8Array(32) 61 | ..set([i % 255, (i - i % 255) / 255]) 62 | if i == 0 63 | instance = lib.Core([], [], 10, 20, Object.assign({}, options, {dht_keypair_seed : dht_seed, connected_nodes_limit : NUMBER_OF_NODES})) 64 | instance.start_bootstrap_node(bootstrap_node_seed, bootstrap_ip, bootstrap_port, bootstrap_address) 65 | else 66 | instance = lib.Core([bootstrap_node_info], [], 10, 2, Object.assign({dht_keypair_seed : dht_seed}, options)) 67 | instance.once('ready', !-> 68 | t.pass('Node ' + i + ' is ready, #' + (NUMBER_OF_NODES - wait_for + 1) + '/' + NUMBER_OF_NODES) 69 | 70 | if i == 2 71 | # Only check the first node after bootstrap 72 | t.same(instance.get_bootstrap_nodes(), [bootstrap_node_info], 'Bootstrap nodes are returned correctly') 73 | 74 | t.equal(instance.get_max_data_size(), 2 ** 16 - 2, 'Max data size returned correctly') 75 | 76 | --wait_for 77 | if !wait_for 78 | ready_callback() 79 | ) 80 | nodes.push(instance) 81 | setTimeout(resolve, 100) 82 | 83 | !function destroy_nodes 84 | console.log 'Destroying nodes...' 85 | for node in nodes 86 | node.destroy() 87 | console.log 'Destroyed' 88 | t.end() 89 | 90 | !function ready_callback 91 | announcement_retry = 3 92 | connection_retry = 5 93 | node_1 = nodes[1] 94 | node_3 = nodes[3] 95 | node_1 96 | .once('announced', !-> 97 | node_1.off('announcement_failed') 98 | t.pass('Announced successfully') 99 | 100 | node_1.on('introduction', (data) !-> 101 | t.equal(data.application.subarray(0, application.length).join(','), application.join(','), 'Correct application on introduction') 102 | t.equal(data.secret.subarray(0, node_1_secret.length).join(','), node_1_secret.join(','), 'Correct secret on introduction') 103 | data.number_of_intermediate_nodes = 1 104 | ) 105 | node_3.once('connected', (, target_id) !-> 106 | node_1.off('introduction') 107 | node_3.off('connection_failed') 108 | t.equal(target_id.join(','), node_1_real_public_key.join(','), 'Connected to intended node successfully') 109 | 110 | node_1.once('data', (, , received_command, received_data) !-> 111 | t.equal(received_command, command, 'Received command correctly') 112 | t.equal(received_data.join(','), data.join(','), 'Received data correctly') 113 | 114 | destroy_nodes() 115 | ) 116 | 117 | console.log 'Sending data...' 118 | node_3.send_to(node_3_real_public_key, node_1_real_public_key, command, data) 119 | ) 120 | node_3.on('connection_failed', (, , reason) !-> 121 | if connection_retry 122 | --connection_retry 123 | console.log 'Connection failed with code ' + reason + ', retry in 5s...' 124 | setTimeout (!-> 125 | console.log 'Connecting...' 126 | node_3.connect_to(node_3_real_seed, node_1_real_public_key, application, node_1_secret, 1) 127 | ), 5000 128 | return 129 | t.fail('Connection failed with code ' + reason) 130 | 131 | destroy_nodes() 132 | ) 133 | 134 | console.log 'Preparing for connection (5s)...' 135 | # Hack to make sure at least one announcement reaches corresponding DHT node at this point 136 | setTimeout (!-> 137 | console.log 'Connecting...' 138 | node_3.connect_to(node_3_real_seed, node_1_real_public_key, application, node_1_secret, 1) 139 | ), 5000 140 | ) 141 | .on('announcement_failed', (, reason) !-> 142 | if announcement_retry 143 | --announcement_retry 144 | console.log 'Announcement failed with code ' + reason + ', retry...' 145 | return 146 | t.fail('Announcement failed with code ' + reason) 147 | 148 | destroy_nodes() 149 | ) 150 | 151 | console.log 'Preparing for announcement (2s)...' 152 | setTimeout (!-> 153 | console.log 'Announcing...' 154 | node_1.announce(node_1_real_seed, 3, 1) 155 | ), 2000 156 | ) 157 | -------------------------------------------------------------------------------- /why.md: -------------------------------------------------------------------------------- 1 | # Why? 2 | This document aims to describe primary ideas that drive Detox development as well as major differences and design choices comparing to networks like Tor and Loopix. 3 | 4 | ### Reachability (browser environment) 5 | Detox project aims to be reachable by as many people as possible. Anonymity heavily depends on anonymity set and as more people join the network, it is easier to blend with the crowd. 6 | In order to achieve this, decision was made to design Detox in a way that will allow it to work in modern web browser without any additional software or configuration required. 7 | 8 | This seems to be the first implementation of this kind, in contrast to Tor or Loopix, connection to the network is one click away from the user. 9 | 10 | Besides obvious benefits, this also results in some drawbacks. For instance, only protocols available from user-space JavaScript in browsers like HTTP, WebSocket or WebRTC are available. 11 | Detox uses HTTP for connection to bootstrap nodes and WebRTC for all of its communications. 12 | 13 | ### Resistance to traffic analysis 14 | Detox network forms a single-layer mesh of nodes, where some nodes are connected to some other nodes. 15 | Each such connection is encrypted and data are transmitted at constant rate. 16 | Basically, Detox takes brute force approach for protection against traffic analysis. 17 | 18 | This is an obvious benefit, since passive observer can't differentiate whether node is just connected to the network or actually talks to anyone. 19 | The drawback here is very limited bandwidth and potentially most of bandwidth being wasted. 20 | 21 | Tor in comparison is vulnerable to traffic analysis, but is incomparably more efficient and can provide incomparably higher bandwidth. 22 | Loopix with its cover traffic is also resistant to GPA. 23 | 24 | ### Initiator and responder anonymity and unobservability 25 | Each node in Detox network has one temporary keypair used for interacting with other nodes on DHT level and the long-term keypair (potentially multiple or zero in case of bootstrap node) for end-to-end communications. 26 | 27 | Temporary keypair is independent from long-term keypair and is never used directly to announce long-term keypair, so that these keypairs can't be easily linked together and on next connection to the network temporary keypair will be different. 28 | 29 | Announcement to the network consists of building routing paths (onion route) to introduction nodes. 30 | 31 | In order to connect to responder several things are required: responder's public key and secret must be known to initiator and responder should announce itself to the network. How public key and secret are shared is out of scope of Detox network design and defined by applications. 32 | 33 | If public key and secret are shared in an anonymous way, responder will be anonymous not only to the network in general, but even to initiator. 34 | Initiator can generate a fresh long-term keypair for one conversation without announcing itself to the network, in which case they will also be anonymous not only to the network in general, but also to responder. 35 | 36 | It is inspired by Tor's hidden services, but there is nothing besides hidden services in Detox. 37 | Comparing to Loopix there are no providers, all nodes are equally participating in routing traffic for other nodes and any particular node can't know where on routing path it is located, who is initiator and who is responder. 38 | 39 | In contrast to both Tor and Loopix packets in Detox are fixed size and do not depend on the length of routing path, encryption/decryption on each segment is done using AEZ wide block cipher with no ciphertext expansion. 40 | Routing paths are stateful so packet doesn't contain information about next node in routing path and routing path of length 1 and 10 will look identically. 41 | 42 | ### Online state disclosure 43 | In order for responder to be able to receive incoming connections, it needs to announce itself to the network. 44 | This means that adversary can query DHT for particular public key and see that the node with particular long-term public key is currently online or was online relatively recently. 45 | Also adversary that controls introduction node will know when node with particular node ID went online or offline. 46 | 47 | Node can choose not to announce itself to the network. In this case node will only be able to act as initiator, incoming connections will not be possible. 48 | 49 | This is similar to the way Tor hidden services work. 50 | Loopix in contrast has concept of providers, so providers will know when user is online or offline, but not the other nodes in the network. 51 | 52 | ### Resistance to Eclipse attack 53 | There is some minor protection against this by using ES-DHT that allows making lookups over snapshot of DHT and adversary can't generate fake nodes on the fly as lookup progresses. 54 | Also IDs are ed25519 public keys, so it is not possible to deliberately select specific node ID. 55 | More work is needed in this direction, possibilities include the use of crypto-puzzles for temporary DHT keypairs, digital signatures and Loopix-like self-checks. 56 | 57 | ### Resistance to Sybil attacks 58 | Not really implemented at this point. Current measures include identifying nodes that do not follow protocol and disconnecting from them. 59 | More work is needed in this direction. There are 3 main limited resources: CPU, RAM and network, that we need to balance carefully so that it is not too expensive for regular user, but would become unfeasible for an attacker. 60 | Possibilities include the use of crypto-puzzles for temporary DHT keypairs, so that maintaining node that participates in DHT would require adaptive amount of computational resources. 61 | 62 | ### Latency 63 | Connection from initiator to responder should take under one minute of time. Sending message with size that fits one packet should take around 1 second for each segment in routing path, but may depend on network congestion. 64 | 65 | Without final implementation and testing of production-like deployment it is hard to tell how exactly latency compares to Tor or Loopix (which can additionally be flexible), but generally it is expected to be on the lower side. 66 | --------------------------------------------------------------------------------