├── package.json ├── LICENSE ├── CYPHERLINK.md └── README.md /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "cyphernet", 3 | "description": "", 4 | "version": "0.0.0", 5 | "homepage": "https://github.com/dominictarr/cyphernet", 6 | "repository": { 7 | "type": "git", 8 | "url": "git://github.com/dominictarr/cyphernet.git" 9 | }, 10 | "dependencies": { 11 | }, 12 | "devDependencies": { 13 | }, 14 | "scripts": { 15 | "test": "set -e; for t in test/*.js; do node $t; done" 16 | }, 17 | "author": "Dominic Tarr (http://dominictarr.com)", 18 | "license": "MIT" 19 | } 20 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2013 Dominic Tarr 2 | 3 | Permission is hereby granted, free of charge, 4 | to any person obtaining a copy of this software and 5 | associated documentation files (the "Software"), to 6 | deal in the Software without restriction, including 7 | without limitation the rights to use, copy, modify, 8 | merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom 10 | the Software is furnished to do so, 11 | subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice 14 | shall be included in all copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES 18 | OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR 20 | ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 21 | TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 22 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 23 | -------------------------------------------------------------------------------- /CYPHERLINK.md: -------------------------------------------------------------------------------- 1 | # Weird Idea 2 | 3 | so… I've had this weird idea recently... 4 | 5 | In git (and in other secure + distributed systems) you have 6 | a tree of hashes where each object is identified by it's hash 7 | and objects contain pointers to other objects. 8 | They just have the hash of other objects stored inside them. 9 | 10 | (this is known as a Content Addressable Store, 11 | but there does not seem to be any terminology 12 | for the design pattern of hash pointers) 13 | 14 | Have been calling these hash pointers 15 | CYPHERLINKS (because it sounds much cooler) 16 | 17 | git is based on cypherlinks, 18 | and so is bittorrent 19 | and bitcoin! - couchdb uses cypherlinks to version documents. 20 | 21 | The way that couchdb uses cypher links is much simpler than git. 22 | 23 | Git has 3 types of objects - commits, trees, and blobs. 24 | blobs are just snapshots of files, 25 | trees are snapshots of a directory and link to blobs and subtrees, 26 | commits link to a tree and a previous commit (unless it's the first version) 27 | 28 | In couchdb - each document links to the previous version of that document. 29 | 30 | [camlistore](http://camlistore.org/) implements much of this idea! 31 | 32 | the cypherlink has some interesting properties: 33 | 34 | * cypherlinks are immutable - if you "update" a document, it's now a different document. 35 | instead, associate it with the previous version - cypherlink to the previous version. 36 | 37 | * cypherlinks always point back in time. You have to know the value of an object before you can link it. 38 | 39 | * Thus, it's impossible to create cycles - because this is impossible: `A === Hash(A + B)` for any non empty `A, B` 40 | 41 | Previously, people (including myself) have been working on "git for data", 42 | replicating the architecture of git (commits, merges, etc) 43 | 44 | But now, I'm thinking that the best idea is just to build a system for working with cypherlinks, 45 | and then using that to build git, if you want, or some arbitrary system. 46 | 47 | The other thing that you get with cypherlinks, is easy replication. 48 | You just have to figure out what you have that I don't have, and then send me that. 49 | You can use a [merkle tree](http://en.wikipedia.org/wiki/Merkle_tree) for that, 50 | I wrote a [module for that](https://github.com/dominictarr/level-merkle) last weekend. 51 | 52 | ## the new idea 53 | 54 | All data is stored in the same "bucket", immutable, and addressed by it's hash. 55 | 56 | ### Query by Searching 57 | 58 | Store objects in JSON, by their hashes, indexing everything. 59 | Sure it might take 3 times the space to store your data, 60 | but you don't have to write queries. 61 | 62 | ### Traverse the cypherlinks 63 | 64 | Build well defined sets of objects by traversing cypherlinks. 65 | If `X` cypherlinks to `Y`, `Y` is a part of `X`. 66 | You can index and traverse backlinks too, 67 | but you can't validate those documents, 68 | or verify that you have all of them. 69 | 70 | If you want to certify a collection of backlinked objects, 71 | create a new document that cypherlinks to them. 72 | 73 | ### replicate with merkle trees 74 | 75 | With a merkle tree, it's easy to replicate any (sub) set of objects. 76 | Hashing is pretty fast, 77 | (0.4 seconds to hash 30mb on my laptop - that could be 1.5 million records!) 78 | so it's would be pretty easy to build a hash tree just per replication. 79 | 80 | You could select a subset by search, or by traversing a cyphertree, 81 | then you could replicate that like git. 82 | 83 | Except it would be pretty straightforward to build say, a blog, 84 | where there is a markdown object, some JSON metadata that pointed to it, 85 | then you sign it with crypto, and later people check this out, and add comments 86 | that link back to the article. 87 | 88 | They could send these comments back to your article, and by adding a new tree, 89 | you'd collect everything into that. 90 | 91 | ## getting really crazy... 92 | 93 | But it doesn't need to stop there. 94 | hashed objects do not collide, so you could just put all the objects in the same namespace! 95 | I mean, all the objects on the internet! 96 | 97 | There needs be no bounded database, 98 | just trees of cryptographically signed data floating in the ether... 99 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # cyphernet 2 | 3 | secure replicatable tree database. 4 | 5 | ## a README Project 6 | 7 | Currently, this is a readme project. 8 | Really, cyphernet will be many projects that fit together. 9 | Please check the issues to take part in discussions on aspects of this project. 10 | 11 | ## Synopsis 12 | 13 | cyphernet is a abstraction of ideas present within various distributed systems 14 | such as git, bittorrent, and bitcoin. There are three key aspects: 15 | 16 | * objects are just binary blobs identified by their hash. This is known as a content addressable database. 17 | 18 | * hashes inside an (e.g. JSON) object are pointers or "links" to other objects. 19 | thus, the database is a tree of hash pointers, called "cypherlinks". 20 | 21 | * arbitary subsets of any two databases may be exchanged via merkle trees. 22 | (merkle trees allow two remote nodes to rapidly compare two sets of objects, 23 | and may then replicate by merging their sets) 24 | 25 | ## interesting properties 26 | 27 | ### Database is a tree, not a graph. 28 | 29 | It is not possible for a cycle to form within a graph, unless there is a hash collision 30 | (highly unlikely). So the graph must be a tree, and it is not necessary to check for cycles. 31 | 32 | ### Hashes always point backwards in time 33 | 34 | You cannot create a hash, and then the object that produces the hash. 35 | Instead, you must have the object, and then hash it. 36 | Therefore cypherlinks always point backwards in time to things that 37 | already existed before the document that they are contained within 38 | was created. 39 | 40 | ### Immutability, Security, Distributability. 41 | 42 | Since every object is referenced by it's hash, it's impossible to change (immutable). 43 | If any one changed a piece of data, then it would change the hash. 44 | If you have the hash of a document, then you can always verify it. 45 | So, if you ask for a particular document ("follow a cypherlink"), 46 | then it doesn't matter who you receive that document from! 47 | 48 | This is the reverse of the security model of the www. 49 | In most network applications, security is implemented by authorizing the connection. 50 | But in a cyphernet, you use cryptographic hashes to authorise the _documents_, 51 | but it is not necessary to authorize the connection. 52 | 53 | This is rather like "authenticating" your friends by the sound of their voice, 54 | not just because the call has come from the correct phone number. You can still verify 55 | your friends identity if they call from a different phone, or even if you see them in person. 56 | 57 | ### Replication Partners 58 | 59 | The cyphernet is still potentially secure when completely distributed. 60 | There is no need to replicate via a central, or known server. 61 | However, how to track peers, and initiate connections is out of scope of this project. 62 | cyphernet is focusing on datastructures, security, and replication. 63 | Network topology is another problem. There are other projects that 64 | deal with creating connections between remote peers. 65 | [peerjs](http://peerjs.com), [cjdns](https://github.com/cjdelisle/cjdns) and [ZeroTierOne](https://www.zerotier.com/) 66 | 67 | 68 | ### Defining a subset to replicate 69 | 70 | The set of documents that define an "application" or "service" could be defined 71 | in any number of ways. In git, a repository contains the data for just one 72 | project, but you can replicate (checkout) just the branches you require. 73 | With a blog, you'd want to replicate the text and images on each post, plus 74 | the comments. On a social network, you'd want to replicate your mutual friends. 75 | 76 | When two nodes connect, they will exchange a handshake that describes the set 77 | of objects they wish to exchange. Then each pair traverses their database for 78 | objects in that set. Then the difference between the two sets is found via 79 | merkle tree. Finially, only the objects each node is missing is sent over 80 | the wire! 81 | 82 | _How_ a set is defined is up to the application, but most will involve 83 | traversing the tree, or querying the indexed properties. 84 | 85 | ### Users & Authors 86 | 87 | Using asymmetric key cryptography it is possible to verify the authorship of 88 | documents. A user uploads an object, and then creates another signature 89 | object, which links to the first object, the user's public key, and a signature 90 | (of the object, with the key) 91 | 92 | Signatures, and Keys, be replicated and stored within the database like any 93 | other object. 94 | 95 | ## License 96 | 97 | MIT 98 | --------------------------------------------------------------------------------