├── .github └── FUNDING.yml ├── .gitignore ├── .travis.yml ├── .travis ├── deploy.sh └── id_rsa.enc ├── LICENSE ├── README.md ├── book.toml └── src ├── CNAME ├── SUMMARY.md ├── feed ├── datamodel.md ├── datatypes.md ├── index.md └── messages.md ├── hub.md ├── identity.md ├── images ├── current.jpg ├── first_pass.jpg ├── future_stack.jpg ├── low_coord_high_duplication.jpg └── scuttle-shell.jpg ├── introduction.md ├── peer.md ├── same-as.md └── threads.md /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | open_collective: sunrise-choir 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /book 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: rust 2 | sudo: false 3 | cache: 4 | - cargo 5 | rust: 6 | - stable 7 | before_script: 8 | - "(test -x $HOME/.cargo/bin/cargo-install-update || cargo install cargo-update)" 9 | - (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.1" mdbook) 10 | - cargo install-update -a 11 | script: 12 | - cd . && mdbook build && mdbook test 13 | deploy: 14 | - provider: script 15 | skip_cleanup: true # do not delete the built assets 16 | script: ./.travis/deploy.sh 17 | on: 18 | branch: master 19 | notifications: 20 | email: false 21 | -------------------------------------------------------------------------------- /.travis/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # setup ssh-agent and provide the GitHub deploy key 5 | eval "$(ssh-agent -s)" 6 | openssl aes-256-cbc -K $encrypted_be30558f84bc_key -iv $encrypted_be30558f84bc_iv -in .travis/id_rsa.enc -out id_rsa -d 7 | chmod 600 id_rsa 8 | ssh-add id_rsa 9 | 10 | wget https://github.com/X1011/git-directory-deploy/raw/e37ac94cda4bfc5773c0f01d89d8c875a21ab4f9/deploy.sh 11 | chmod +x deploy.sh 12 | GIT_DEPLOY_DIR=book GIT_DEPLOY_BRANCH=gh-pages GIT_DEPLOY_REPO=git@github.com:sunrise-choir/spec ./deploy.sh 13 | -------------------------------------------------------------------------------- /.travis/id_rsa.enc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunrise-choir/spec/07c89afac14d01dccb2d2a0779baf370327421d3/.travis/id_rsa.enc -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Attribution 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution 4.0 International Public License 58 | 59 | By exercising the Licensed Rights (defined below), You accept and agree 60 | to be bound by the terms and conditions of this Creative Commons 61 | Attribution 4.0 International Public License ("Public License"). To the 62 | extent this Public License may be interpreted as a contract, You are 63 | granted the Licensed Rights in consideration of Your acceptance of 64 | these terms and conditions, and the Licensor grants You such rights in 65 | consideration of benefits the Licensor receives from making the 66 | Licensed Material available under these terms and conditions. 67 | 68 | 69 | Section 1 -- Definitions. 70 | 71 | a. Adapted Material means material subject to Copyright and Similar 72 | Rights that is derived from or based upon the Licensed Material 73 | and in which the Licensed Material is translated, altered, 74 | arranged, transformed, or otherwise modified in a manner requiring 75 | permission under the Copyright and Similar Rights held by the 76 | Licensor. For purposes of this Public License, where the Licensed 77 | Material is a musical work, performance, or sound recording, 78 | Adapted Material is always produced where the Licensed Material is 79 | synched in timed relation with a moving image. 80 | 81 | b. Adapter's License means the license You apply to Your Copyright 82 | and Similar Rights in Your contributions to Adapted Material in 83 | accordance with the terms and conditions of this Public License. 84 | 85 | c. Copyright and Similar Rights means copyright and/or similar rights 86 | closely related to copyright including, without limitation, 87 | performance, broadcast, sound recording, and Sui Generis Database 88 | Rights, without regard to how the rights are labeled or 89 | categorized. For purposes of this Public License, the rights 90 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 91 | Rights. 92 | 93 | d. Effective Technological Measures means those measures that, in the 94 | absence of proper authority, may not be circumvented under laws 95 | fulfilling obligations under Article 11 of the WIPO Copyright 96 | Treaty adopted on December 20, 1996, and/or similar international 97 | agreements. 98 | 99 | e. Exceptions and Limitations means fair use, fair dealing, and/or 100 | any other exception or limitation to Copyright and Similar Rights 101 | that applies to Your use of the Licensed Material. 102 | 103 | f. Licensed Material means the artistic or literary work, database, 104 | or other material to which the Licensor applied this Public 105 | License. 106 | 107 | g. Licensed Rights means the rights granted to You subject to the 108 | terms and conditions of this Public License, which are limited to 109 | all Copyright and Similar Rights that apply to Your use of the 110 | Licensed Material and that the Licensor has authority to license. 111 | 112 | h. Licensor means the individual(s) or entity(ies) granting rights 113 | under this Public License. 114 | 115 | i. Share means to provide material to the public by any means or 116 | process that requires permission under the Licensed Rights, such 117 | as reproduction, public display, public performance, distribution, 118 | dissemination, communication, or importation, and to make material 119 | available to the public including in ways that members of the 120 | public may access the material from a place and at a time 121 | individually chosen by them. 122 | 123 | j. Sui Generis Database Rights means rights other than copyright 124 | resulting from Directive 96/9/EC of the European Parliament and of 125 | the Council of 11 March 1996 on the legal protection of databases, 126 | as amended and/or succeeded, as well as other essentially 127 | equivalent rights anywhere in the world. 128 | 129 | k. You means the individual or entity exercising the Licensed Rights 130 | under this Public License. Your has a corresponding meaning. 131 | 132 | 133 | Section 2 -- Scope. 134 | 135 | a. License grant. 136 | 137 | 1. Subject to the terms and conditions of this Public License, 138 | the Licensor hereby grants You a worldwide, royalty-free, 139 | non-sublicensable, non-exclusive, irrevocable license to 140 | exercise the Licensed Rights in the Licensed Material to: 141 | 142 | a. reproduce and Share the Licensed Material, in whole or 143 | in part; and 144 | 145 | b. produce, reproduce, and Share Adapted Material. 146 | 147 | 2. Exceptions and Limitations. For the avoidance of doubt, where 148 | Exceptions and Limitations apply to Your use, this Public 149 | License does not apply, and You do not need to comply with 150 | its terms and conditions. 151 | 152 | 3. Term. The term of this Public License is specified in Section 153 | 6(a). 154 | 155 | 4. Media and formats; technical modifications allowed. The 156 | Licensor authorizes You to exercise the Licensed Rights in 157 | all media and formats whether now known or hereafter created, 158 | and to make technical modifications necessary to do so. The 159 | Licensor waives and/or agrees not to assert any right or 160 | authority to forbid You from making technical modifications 161 | necessary to exercise the Licensed Rights, including 162 | technical modifications necessary to circumvent Effective 163 | Technological Measures. For purposes of this Public License, 164 | simply making modifications authorized by this Section 2(a) 165 | (4) never produces Adapted Material. 166 | 167 | 5. Downstream recipients. 168 | 169 | a. Offer from the Licensor -- Licensed Material. Every 170 | recipient of the Licensed Material automatically 171 | receives an offer from the Licensor to exercise the 172 | Licensed Rights under the terms and conditions of this 173 | Public License. 174 | 175 | b. No downstream restrictions. You may not offer or impose 176 | any additional or different terms or conditions on, or 177 | apply any Effective Technological Measures to, the 178 | Licensed Material if doing so restricts exercise of the 179 | Licensed Rights by any recipient of the Licensed 180 | Material. 181 | 182 | 6. No endorsement. Nothing in this Public License constitutes or 183 | may be construed as permission to assert or imply that You 184 | are, or that Your use of the Licensed Material is, connected 185 | with, or sponsored, endorsed, or granted official status by, 186 | the Licensor or others designated to receive attribution as 187 | provided in Section 3(a)(1)(A)(i). 188 | 189 | b. Other rights. 190 | 191 | 1. Moral rights, such as the right of integrity, are not 192 | licensed under this Public License, nor are publicity, 193 | privacy, and/or other similar personality rights; however, to 194 | the extent possible, the Licensor waives and/or agrees not to 195 | assert any such rights held by the Licensor to the limited 196 | extent necessary to allow You to exercise the Licensed 197 | Rights, but not otherwise. 198 | 199 | 2. Patent and trademark rights are not licensed under this 200 | Public License. 201 | 202 | 3. To the extent possible, the Licensor waives any right to 203 | collect royalties from You for the exercise of the Licensed 204 | Rights, whether directly or through a collecting society 205 | under any voluntary or waivable statutory or compulsory 206 | licensing scheme. In all other cases the Licensor expressly 207 | reserves any right to collect such royalties. 208 | 209 | 210 | Section 3 -- License Conditions. 211 | 212 | Your exercise of the Licensed Rights is expressly made subject to the 213 | following conditions. 214 | 215 | a. Attribution. 216 | 217 | 1. If You Share the Licensed Material (including in modified 218 | form), You must: 219 | 220 | a. retain the following if it is supplied by the Licensor 221 | with the Licensed Material: 222 | 223 | i. identification of the creator(s) of the Licensed 224 | Material and any others designated to receive 225 | attribution, in any reasonable manner requested by 226 | the Licensor (including by pseudonym if 227 | designated); 228 | 229 | ii. a copyright notice; 230 | 231 | iii. a notice that refers to this Public License; 232 | 233 | iv. a notice that refers to the disclaimer of 234 | warranties; 235 | 236 | v. a URI or hyperlink to the Licensed Material to the 237 | extent reasonably practicable; 238 | 239 | b. indicate if You modified the Licensed Material and 240 | retain an indication of any previous modifications; and 241 | 242 | c. indicate the Licensed Material is licensed under this 243 | Public License, and include the text of, or the URI or 244 | hyperlink to, this Public License. 245 | 246 | 2. You may satisfy the conditions in Section 3(a)(1) in any 247 | reasonable manner based on the medium, means, and context in 248 | which You Share the Licensed Material. For example, it may be 249 | reasonable to satisfy the conditions by providing a URI or 250 | hyperlink to a resource that includes the required 251 | information. 252 | 253 | 3. If requested by the Licensor, You must remove any of the 254 | information required by Section 3(a)(1)(A) to the extent 255 | reasonably practicable. 256 | 257 | 4. If You Share Adapted Material You produce, the Adapter's 258 | License You apply must not prevent recipients of the Adapted 259 | Material from complying with this Public License. 260 | 261 | 262 | Section 4 -- Sui Generis Database Rights. 263 | 264 | Where the Licensed Rights include Sui Generis Database Rights that 265 | apply to Your use of the Licensed Material: 266 | 267 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 268 | to extract, reuse, reproduce, and Share all or a substantial 269 | portion of the contents of the database; 270 | 271 | b. if You include all or a substantial portion of the database 272 | contents in a database in which You have Sui Generis Database 273 | Rights, then the database in which You have Sui Generis Database 274 | Rights (but not its individual contents) is Adapted Material; and 275 | 276 | c. You must comply with the conditions in Section 3(a) if You Share 277 | all or a substantial portion of the contents of the database. 278 | 279 | For the avoidance of doubt, this Section 4 supplements and does not 280 | replace Your obligations under this Public License where the Licensed 281 | Rights include other Copyright and Similar Rights. 282 | 283 | 284 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 285 | 286 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 287 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 288 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 289 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 290 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 291 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 292 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 293 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 294 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 295 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 296 | 297 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 298 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 299 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 300 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 301 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 302 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 303 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 304 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 305 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 306 | 307 | c. The disclaimer of warranties and limitation of liability provided 308 | above shall be interpreted in a manner that, to the extent 309 | possible, most closely approximates an absolute disclaimer and 310 | waiver of all liability. 311 | 312 | 313 | Section 6 -- Term and Termination. 314 | 315 | a. This Public License applies for the term of the Copyright and 316 | Similar Rights licensed here. However, if You fail to comply with 317 | this Public License, then Your rights under this Public License 318 | terminate automatically. 319 | 320 | b. Where Your right to use the Licensed Material has terminated under 321 | Section 6(a), it reinstates: 322 | 323 | 1. automatically as of the date the violation is cured, provided 324 | it is cured within 30 days of Your discovery of the 325 | violation; or 326 | 327 | 2. upon express reinstatement by the Licensor. 328 | 329 | For the avoidance of doubt, this Section 6(b) does not affect any 330 | right the Licensor may have to seek remedies for Your violations 331 | of this Public License. 332 | 333 | c. For the avoidance of doubt, the Licensor may also offer the 334 | Licensed Material under separate terms or conditions or stop 335 | distributing the Licensed Material at any time; however, doing so 336 | will not terminate this Public License. 337 | 338 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 339 | License. 340 | 341 | 342 | Section 7 -- Other Terms and Conditions. 343 | 344 | a. The Licensor shall not be bound by any additional or different 345 | terms or conditions communicated by You unless expressly agreed. 346 | 347 | b. Any arrangements, understandings, or agreements regarding the 348 | Licensed Material not stated herein are separate from and 349 | independent of the terms and conditions of this Public License. 350 | 351 | 352 | Section 8 -- Interpretation. 353 | 354 | a. For the avoidance of doubt, this Public License does not, and 355 | shall not be interpreted to, reduce, limit, restrict, or impose 356 | conditions on any use of the Licensed Material that could lawfully 357 | be made without permission under this Public License. 358 | 359 | b. To the extent possible, if any provision of this Public License is 360 | deemed unenforceable, it shall be automatically reformed to the 361 | minimum extent necessary to make it enforceable. If the provision 362 | cannot be reformed, it shall be severed from this Public License 363 | without affecting the enforceability of the remaining terms and 364 | conditions. 365 | 366 | c. No term or condition of this Public License will be waived and no 367 | failure to comply consented to unless expressly agreed to by the 368 | Licensor. 369 | 370 | d. Nothing in this Public License constitutes or may be interpreted 371 | as a limitation upon, or waiver of, any privileges and immunities 372 | that apply to the Licensor or You, including from the legal 373 | processes of any jurisdiction or authority. 374 | 375 | 376 | ======================================================================= 377 | 378 | Creative Commons is not a party to its public 379 | licenses. Notwithstanding, Creative Commons may elect to apply one of 380 | its public licenses to material it publishes and in those instances 381 | will be considered the “Licensor.” The text of the Creative Commons 382 | public licenses is dedicated to the public domain under the CC0 Public 383 | Domain Dedication. Except for the limited purpose of indicating that 384 | material is shared under a Creative Commons public license or as 385 | otherwise permitted by the Creative Commons policies published at 386 | creativecommons.org/policies, Creative Commons does not authorize the 387 | use of the trademark "Creative Commons" or any other trademark or logo 388 | of Creative Commons without its prior written consent including, 389 | without limitation, in connection with any unauthorized modifications 390 | to any of its public licenses or any other arrangements, 391 | understandings, or agreements concerning use of licensed material. For 392 | the avoidance of doubt, this paragraph does not form part of the 393 | public licenses. 394 | 395 | Creative Commons may be contacted at creativecommons.org. 396 | 397 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SSB Spec Repository 2 | 3 | This repo contains a spec for secure scuttlebutt, written as an [mdBook](https://rust-lang-nursery.github.io/mdBook/cli/serve.html). Unlike the [protocol guide](https://github.com/ssbc/scuttlebutt-protocol-guide), this is not to be a linear read to quickly get something working. Rather it aims to be a comprehensive reference of how scuttlebutt works. 4 | -------------------------------------------------------------------------------- /book.toml: -------------------------------------------------------------------------------- 1 | [book] 2 | authors = ["AljoschaMeyer"] 3 | multilingual = false 4 | src = "src" 5 | title = "SSB Specification" 6 | description = "Specification of the ssb protocol." 7 | -------------------------------------------------------------------------------- /src/CNAME: -------------------------------------------------------------------------------- 1 | spec.scuttlebutt.nz 2 | -------------------------------------------------------------------------------- /src/SUMMARY.md: -------------------------------------------------------------------------------- 1 | # Summary 2 | 3 | [Introduction](./introduction.md) 4 | 5 | - [Feed](./feed/index.md) 6 | - [Common Datatypes](./feed/datatypes.md) 7 | - [Datamodel](./feed/datamodel.md) 8 | - [Feeds and Messages](./feed/messages.md) 9 | -------------------------------------------------------------------------------- /src/feed/datamodel.md: -------------------------------------------------------------------------------- 1 | # Datamodel 2 | 3 | Ssb accepts schemaless data, adhering to a certain format specified in this section. This is the data that application developers care about. Internally, ssb then adds some more metadata to maintain the append-only logs, discussed in the [next chapter](./messages). This chapter only describes the data model of the schemaless content data. 4 | 5 | It is important to clearly distinguish between the abstract data model of a format, and various encodings. For example, the strings `1`, `1.0` and `0.1e1` are all different [json](http://json.org/) encodings of the same abstract value. And there can even be different encoding formats for the same abstract value, e.g. the bytes `0xf93c00` also encode the floating point number 1.0, but in [cbor](https://tools.ietf.org/html/rfc7049) rather than json. 6 | 7 | Ssb messages all use the same abstract data model, but there are different situations where they use different encodings. An encoding used for signing the data has different requirements than an encoding used for transmitting the data, which again has different requirements than an encoding used for persisten storage. 8 | 9 | Encodings for persistent storage are not specified in this document, but is crucial for all ssb implementations to compute the exact same signatures, and to send data in a format that can be understood by other implementations. We call these encodings *signing encoding* and *transport encoding* respectively. 10 | 11 | The ssb protocol was initially implemented in javascript, and it relied heavily on implicit behavior of the [node js](https://nodejs.org/en/) runtime. People (including the authors of this specification) have called the message format "bizarre" and worse, and that is entirely justified. But when during reading you inevitably thing "How could anybody end up with this, I could have done a much better job.", then remember: Maybe you could have, but you didn't. 12 | 13 | ### Abstract Data Model 14 | 15 | The data model describes all of the free-form data that can be carried in a message. It is close to the [json](http://json.org/) data model, but with a few differences. The definition came about as the set of javascript values created by `JSON.parse(json)` for which [`json === JSON.stringify(JSON.parse(json))`](%EyGGCcjAbaShKFCMxXKYiZjQe17SR298D0SLTuKmZpo=.sha256) (javascript code). 16 | 17 | Defined in a language-agnostic way: 18 | 19 | ###### Null 20 | `null` is a value that carries [no information](https://en.wikipedia.org/wiki/Unit_type). 21 | 22 | ###### Booleans 23 | `true` and `false` are values, called *booleans*. 24 | 25 | ###### Strings 26 | An ordered sequence of bytes which form valid [utf8](https://en.wikipedia.org/wiki/UTF-8) of length between `0` and `2^53 - 1` (inclusive) is a value, called a *string*. Such a string may include null bytes. 27 | 28 | ###### Floats 29 | An [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754) double precision (64 bit) floating point number that is none of `Infinity`, `-Infinity`, `-0` or `NaN` is a value, called a *float*. 30 | 31 | ###### Arrays 32 | Let `n` be a natural number less than `2^32`, and let `v_0, ..., v_n` be [values](#abstract-data-model). 33 | 34 | The ordered sequence `[v_0, ..., v_n]` is a value, called an *array*. 35 | 36 | ###### Objects 37 | Let `n` be a natural number less than `2^32`, let `s_0, ..., s_n` be pairwise distinct [strings](#strings), and let `v_0, ..., v_n` be [values](#abstract-data-model). 38 | 39 | The (unordered) set of pairs `(s_i, v_i)` for all `0 <= i <= n` is a value, called a *object*. The pairs are called *entries*, the strings are called *keys*, and the values are called *values*. 40 | 41 | ### Signing Encoding 42 | 43 | The encoding to turn values into a signeable array of bytes is based on json (the set of valid encodings is a subset of [ECMA-404 json](https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf)). There are multiple valid encodings of a single value, because some of the entries of an objects can be encoded in an arbitary order. But up to object entry order, the encoding is unique. When receiving a message over the network, the order of the freely-orderable object entries in the transport encoding is the order that must be used for verifying the signature. Thus the network encoding induces a unique signing encoding to use for signature checking. 44 | 45 | The signing encoding is defined as follows: 46 | 47 | ###### Signing Encoding Null 48 | `null` is encoded as the utf-8 string `null` (`[0x6E, 0x75, 0x6C, 0x6C]`). 49 | 50 | ###### Signing Encoding Booleans 51 | `true` is encoded as the utf-8 string `true` (`[0x74, 0x72, 0x75, 0x65]`). 52 | `false` is encoded as the utf-8 string `false` (`[0x66, 0x61, 0x6c, 0x73, 0x65]`). 53 | 54 | ###### Signing Encoding Strings 55 | A string containing the unicode code points `c_0, ..., c_n` is is encoded as follows: 56 | 57 | - begin with the utf-8 string `"` (`0x22`) 58 | - for each code point `c_i` in `c_0, ..., c_n`: 59 | - if `c_i` is unicode code point `0x000022` (quotation mark `"`), append the utf-8 string `\"` (`[0x5C, 0x22]`) 60 | - else if `c_i` is unicode code point `0x00005C` (reverse solidus `\`), append the utf-8 string `\\` (`[0x5C, 0x5C]`) 61 | - else if `c_i` is unicode code point `0x000008` (backspace), append the utf-8 string `\b` (`[0x5C, 0x62]`) 62 | - else if `c_i` is unicode code point `0x00000C` (form feed), append the utf-8 string `\f` (`[0x5C, 0x66]`) 63 | - else if `c_i` is unicode code point `0x00000A` (line feed), append the utf-8 string `\n` (`[0x5C, 0x6E]`) 64 | - else if `c_i` is unicode code point `0x00000D` (carriage return), append the utf-8 string `\r` (`[0x5C, 0x72]`) 65 | - else if `c_i` is unicode code point `0x000009` (line tabulation), append the utf-8 string `\t` (`[0x5C, 0x74]`) 66 | - else if `c_i` is a unicode code point below `0x000020` (space), append the utf-8 string `\u` (`[0x5C, 0x75, ]`), where `` are the two utf-8 bytes of the hexadecimal encoding of the code point, using lower-case letters `a` - `f` (`0x61` - `0x66`) for alphabetic hex digits 67 | - else append the utf-8 representation of `c_i` without any modifications 68 | - append the utf-8 string `"` (`0x22`) 69 | 70 | ###### Signing Encoding Floats 71 | 72 | A float `m` is encoded as follows: 73 | 74 | - if `m == 0`, the encoding is the utf-8 string `0` (`0x30`) 75 | - else if `m` is negative, the encoding is the utf-8 string `-`(`[0x2d, ]`), where `` is the encoding of the same float with the sign bit flipped 76 | - else (largely quoting from the [ECMAScript specification, applying NOTE 2](https://www.ecma-international.org/ecma-262/6.0/#sec-tostring-applied-to-the-number-type) from here on): 77 | - let `n`, `k` and `s` be integers such that: 78 | - `k >= 1` 79 | - `10 ^ (k - 1) <= s <= 10 ^ k` 80 | - `s * (10 ^ (n - k))` is `m` (or [round-to-even-s](https://en.wikipedia.org/wiki/Rounding#Round_half_to_even) to `m` if it is not precisely representable in a 64 bit float) 81 | - `k` is as small as possible 82 | - if there are multiple values for `s`, choose the one for which `s * (10 ^ (n - k))` is closest in value to `m` 83 | - if there are two such possible values of `s`, choose the one that is even 84 | - Intuitively, `s` is the integer you get by removing the point and all trailing zeros from the decimal representation of `m`, `k` is the number of digits in the decimal representation of `s`, and `n` specifies how to print the number: If `n` is greater than `0`, there are `n` digits left of the point, else there are `abs(n)` many zeros right of the point. The choice of `s` uniquely determines `k` and `n`, the tricky part is finding the unique `s` that rounds correctly and for which `k` is minimal. 85 | - if `k <= n <= 21`, the encoding is the utf-8 string ``, where `` is the utf-8 encoding of the digits of the decimal representation of `s`, and `.` (`[, 0x2E, ]`), where `` is the utf-8 encoding of the most significant `n` digits of the decimal representation of `s`, and `` is the utf-8 encoding of the remaining `k - n` digits of the decimal representation of `s` 87 | - else if `-6 < n <= 0`, the encoding is the utf-8 string `0.` (`[0x30, 0x2E, , ]`), where `` are `-n` many zero digits (`0x30`), and `` is the utf-8 encoding of the digits of the decimal representation of `s` 88 | - else if `k == 1`, the encoding is `e` (`[, 0x65, , ]`), where `` is the utf-8 encoding of the single digit of `s`, `` is the utf-8 string `+` (`0x2B`) if `n - 1` is positive or the utf-8 string `-` (`0x2D`) if `n - 1` is negative, and `` is the utf-8 encoding of the decimal representation of the absolute value of `n - 1` 89 | - else, the encoding is the utf-8 string `.e` (`[, 0x2E, , 0x65, , ]`), where `` is the utf-8 encoding of the most significant digit of the decimal representation of `s`, `` is the utf-8 encoding of the remaining `k - 1` digits of the decimal representation of `s`, `` is the utf-8 string `+` (`0x2B`) if `n - 1` is positive or the utf-8 string `-` (`0x2D`) if `n - 1` is negative, and `` is the utf-8 encoding of the decimal representation of the absolute value of `n - 1` 90 | - good to know: The maximum length for such an encoding is 25 bytes 91 | 92 | ###### Signing Encoding Arrays 93 | 94 | Let `n` be a natural number less than `2^32`, let `v_0, ..., v_n` be [values](#abstract-data-model), and let `e_0, ..., e_n` be functions that take a natural number as an argument and return the [encodings](#signing-encoding) of `v_0, ..., v_n` respectively using the supplied number as the indentation level. 95 | 96 | At indentation level `indent`, the array `[v_0, ..., v_n]` is encoded as follows: 97 | 98 | - if the array is empty (`n == 0`), the encoding is the utf-8 string `[]` (`[0x5B, 0x5D]`) 99 | - else, do the following: 100 | - begin with the utf-8 string `[` (`[0x5B, 0x0A]`) 101 | - for each `v_i` in `v_0, ..., v_(n - 1)` (so skip this if `n == 1`): 102 | - append `indent + 2` many space characters (`0x20`) 103 | - append `e_i(indent + 2)` 104 | - append the utf-8 string `,` (`[0x2C, 0x0A]`) 105 | - append `indent + 2` many space characters (`0x20`) 106 | - append `e_n(indent + 2)` 107 | - append the utf-8 string `` (`0x0A`) 108 | - append `indent` many space characters (`0x20`) 109 | - append the utf-8 string `]` (`0x5D`) 110 | 111 | ###### Signing Encoding Objects 112 | 113 | Let `n` be a natural number less than `2^32`, let `s_0, ..., s_n` be pairwise distinct [strings](#strings), let `v_0, ..., v_n` be [values](#abstract-data-model), and let `e_0, ..., e_n` be functions that take a natural number as an argument and return the [encodings](#signing-encoding) of `v_0, ..., v_n` respectively using the supplied number as the indentation level. 114 | 115 | At indentation level `indent`, the object `{ s_0: v_0, ..., s_n: v_n}` is encoded as follows: 116 | 117 | - if the object is empty (`n == 0`), the encoding is the utf-8 string `{}` (`[0x7B, 0x7D]`) 118 | - else, do the following: 119 | - begin with the utf-8 string `{` (`[0x7B, 0x0A]`) 120 | - for each pair `(s_i, v_i)` for `i` in `0, ..., n - 1` (so skip this if `n == 1`): 121 | - append `indent + 2` many space characters (`0x20`) 122 | - append the encoding of the string `s_i` 123 | - append the utf-8 string `:` (`[0x3A, 0x20]`) 124 | - append `e_i(indent + 2)` 125 | - append the utf-8 string `,` (`[0x2C, 0x0A]`) 126 | - append `indent + 2` many space characters (`0x20`) 127 | - append the encoding of the string `s_i` 128 | - append the utf-8 string `:` (`[0x3A, 0x20]`) 129 | - append `e_i(indent + 2)` 130 | - append the utf-8 string `` (`0x0A`) 131 | - append `indent` many space characters (`0x20`) 132 | - append the utf-8 string `}` (`0x7D`) 133 | - The order in which to serialize the entries `s_i: v_i` is not fully specified, but there are some constraints: 134 | - intuitively: Natural numbers less than 2^32 - 1 are sorted ascendingly 135 | - formally: 136 | - a key is called a `more-or-less integer key` if it is either the string `"0"`, or a nonzero decimal digit (1 - 9 (`0x31` - `0x39`)) followed by zero or more arbitrary decimal digits (0 - 9 (`0x30` - `0x39`)) and consists solely of such digits 137 | - a key is called an `int key` if it: 138 | - is a `more-or-less integer key` 139 | - its numeric value is strictly less than `2^32 - 1` 140 | - no, there's no off-by one error here: `0xfffffffe` is an `int key`, whereas `0xffffffff` is not 141 | - Why? We have no idea either 142 | - all entries with `int key`s must be serialized before all other entries 143 | - among themselves, the `int key`s are sorted: 144 | - by length first (ascending), using 145 | - numeric value as a tie breaker (the key whose raw bytes interpreted as a natural number are larger is serialized later) 146 | - note that this coincides with sorting the decimally encoded numbers by numeric value 147 | - all other entries may be serialized in an arbitrary order 148 | 149 | The string handling is equivalent to [ECMAScript 2015 QuoteJSONString](https://www.ecma-international.org/ecma-262/6.0/#sec-quotejsonstring), but defined over utf-8 strings instead of utf-16 ones. 150 | 151 | The float handling is equivalent to (and quotes from) [ECMAScript 2015 ToString Applied to the Number Type](https://www.ecma-international.org/ecma-262/6.0/#sec-tostring-applied-to-the-number-type), with step 5 replaced as specified in NOTE 2 to result in unique encodings. This spec provides a declarative description of the encoding process, for an algorithmic perspective, there are some papers on the subject such as: 152 | 153 | - [Steele Jr, Guy L., and Jon L. White. "How to print floating-point numbers accurately." ACM SIGPLAN Notices. Vol. 25. No. 6. ACM, 1990.](https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf) 154 | - [Loitsch, Florian. "Printing floating-point numbers quickly and accurately with integers." ACM Sigplan Notices. Vol. 45. No. 6. ACM, 2010.](https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf) 155 | - [Andrysco, Marc, Ranjit Jhala, and Sorin Lerner. "Printing floating-point numbers: a faster, always correct method." ACM SIGPLAN Notices. Vol. 51. No. 1. ACM, 2016.](https://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf) 156 | - [Adams, Ulf. "Ryū: fast float-to-string conversion." Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2018.](https://dl.acm.org/citation.cfm?id=3192369) 157 | 158 | The array and object handling is equivalent to `JSON.stringify(value, null, 2)`, specified in [ECMAScript 2015](https://www.ecma-international.org/ecma-262/6.0/#sec-json.stringify) (except for the object entry ordering, which is not specified in ECMAScript, but implemented this way in v8 and spidermonkey). 159 | 160 | ### Hash Computation 161 | 162 | To compute the hash of a value, you can not use the signing encoding directly, but the hash computation is based on it. The signing encoding always results in valid unicode. Represent this unicode in [utf-16](https://en.wikipedia.org/wiki/UTF-16). This encoding is a sequence of code units, each consisting of two bytes. The data to hash is obtained from these code units by only keeping the less significant byte. 163 | 164 | Example: Suppose you want to compute the hash for `"ß"`, the corresponding utf8 is `[0x22, 0xC3, 0x9F, 0x22]`. In big-endian utf16, this is `[(0x22, 0x00), (0xDF, 0x00), (0x22, 0x00)]`, in little-endian utf16, this is `[(0x00, 0x22), (0x00, 0xDF), (0x00, 0x22)]`. In both cases, the sequence of less signifiant bytes per code unit is `[0x22, 0xDF, 0x22]`. That is the byte array over which to compute the hash. 165 | 166 | Note that this means that two strings with different utf-8 encodings can result in the same hash, due to the information in the more significant byte of the utf-16 encoding being dropped. 167 | 168 | ### Length Computation 169 | 170 | Ssb places a limit on the size of messages. To compute the length of a value, compute the signing encoding (which is always valid unicode), reencode that unicode as utf16, then count the number of code units. 171 | 172 | ### JSON Transport Encoding 173 | 174 | In addition to the signing format, messages can be encoded as [ECMA-404 json](https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf), with the following differences: 175 | 176 | - numbers may not be negative zero 177 | - numbers may not round to positive infinity, negative infinity or negative zero IEEE 754 64 bit floating point numbers 178 | - strings may not be longer than `2^53 - 1` bytes 179 | - arrays and object may not contain more than `2^32 - 1` entries 180 | - objects may not contain multiple entries with the same key 181 | - in strings, unicode escape sequences of code points greater than `U+FFFF` must be interpreted as a single code point, not as an explicit surrogate pair 182 | - escape sequences of surrogate code points must be matched: Each escape sequence for a high surrogate must be followed by an escape sequence for a low surrogate, and any escape sequence for a low surrogate must be preceded by an escape sequence for a high surrogate 183 | 184 | The signing format itself is a subset of this, but this format can be more compact (by omitting all whitespace). This compact form is used by ssb server implementations for message exchange with other servers. 185 | -------------------------------------------------------------------------------- /src/feed/datatypes.md: -------------------------------------------------------------------------------- 1 | # Common Datatypes 2 | 3 | A few datatypes appear throughout this spec, it makes sense to introduce them in one single place. 4 | 5 | An overarching theme of various parts of the ssb protocol(s) is that of future-proofness. The protocol will need to adapt itself to new circumstances, such as new transport channels or broken cryptographic primitives. A simple way to keep flexibility are self-describing multiformats, where the data is prefixed by an identifier (and in some cases its expected length). New data types can be introduced by simply assigning a new identifier. Older software can detect data types it doesn't understand yet, and react accordingly. 6 | 7 | Each format consists of some logical data type, and then one or more encodings. These encodings can serve different purposes, for example they might be optimized for machine-readability, human-readability, uniqueness, backwards-compatibility, etc. 8 | 9 | ## Multikey 10 | 11 | A multikey is the public key of some [digital signature](https://en.wikipedia.org/wiki/Digital_signature) scheme, annotated with an identifier for the scheme itself. The only currently supported cryptographic primitive is [ed25519](http://ed25519.cr.yp.to/) (which has a public key length of 32 bytes). 12 | 13 | ### Multikey Encoding 14 | 15 | The encoding of a multikey is defined as the concatenation of: 16 | 17 | - the character `@` (`0x40`) 18 | - the [canonic](https://tools.ietf.org/html/rfc4648#section-3.5) base64 encoding of the key itself 19 | - [ietf rfc 4648, section 4](https://tools.ietf.org/html/rfc4648#section-4), disallowing superflous `=` characters inside the data or after the necessary padding `=`s 20 | - it may not omit `=` characters either - the amount of encoding bytes must always be a multiple of four 21 | - the character `.` (`0x2E`) 22 | - the primitive-specific suffix 23 | - for ed25519, this is `ed25519` (`[0x65, 0x64, 0x32, 0x35, 0x35, 0x31, 0x39]`) 24 | 25 | Typically, this encoding is stored in a json string. 26 | 27 | ## Multifeed 28 | 29 | A multifeed represents a scuttlebutt feed. It consists of a feed kind indicator and some additional data depending on the feed kind. The only currently supported feed kind is `multikey`, whose additional data surprisingly enough is a multikey. 30 | 31 | ### Multifeed Encoding 32 | 33 | The encoding of a multikey multifeed is the same as the encoding of the multikey. The encoding of possible future multifeed kinds will use a different first char than `@`. 34 | 35 | ## Multihash 36 | 37 | A multihash is a pair of: 38 | 39 | - the hash target 40 | - the hash digest of some [cryptographically secure hash function](https://en.wikipedia.org/wiki/Cryptographic_hash_function), annotated with an identifier for the hash function itself. 41 | 42 | The only currently supported hash targets are `Message` and `Blob` 43 | 44 | The only currently supported cryptographic primitive is [sha256](https://en.wikipedia.org/wiki/SHA-2) (which has a digest length of 32 bytes). 45 | 46 | ### Multihash Encoding 47 | 48 | The encoding of a multihash is defined as the concatenation of: 49 | 50 | - depending on the hash target: 51 | - `Message`: the character `%` (`0x25`) 52 | - `Blob`: the character `&` (`0x26`) 53 | - the [canonic](https://tools.ietf.org/html/rfc4648#section-3.5) base64 encoding of the digest itself 54 | - [ietf rfc 4648, section 4](https://tools.ietf.org/html/rfc4648#section-4), disallowing superflous `=` characters inside the data or after the necessary padding `=`s 55 | - it may not omit `=` characters either - the amount of encoding bytes must always be a multiple of four 56 | - the character `.` (`0x2E`) 57 | - the primitive-specific suffix 58 | - for sha256, this is `sha256` (`[0x73, 0x68, 0x61, 0x32, 0x35, 0x36]`) 59 | 60 | Typically, this encoding is stored in a json string. 61 | 62 | ## Multibox 63 | 64 | A multibox is a cyphertext, annotated with an identifier for the algorithm that produced it. The algorithm identifiers are natural numbers between 0 and 2^64 - 1 (inclusive). Even identifiers are reserved for assignment by the ssb protocol devs, odd identifiers are open for experimentation and/or custom usage. 65 | 66 | The only currently specified algorithm is [private-box](https://ssbc.github.io/scuttlebutt-protocol-guide/#private-messages), using the identifier 0. 67 | 68 | ### Multibox Encoding 69 | 70 | The encoding of a multibox is defined as the concatenation of: 71 | 72 | - the [canonic](https://tools.ietf.org/html/rfc4648#section-3.5) base64 encoding of the cyphertext 73 | - [ietf rfc 4648, section 4](https://tools.ietf.org/html/rfc4648#section-4), disallowing superflous `=` 74 | - it may not omit `=` characters either - the amount of encoding bytes must always be a multiple of four 75 | - the characters `.box` (`[0x2E, 0x62, 0x6F, 0x78]`) 76 | - the uppercase [base32](https://tools.ietf.org/html/rfc4648#section-6) encoding without padding of the identifier, without leading zeros, using the following table (the canonic subset of [Crockford's base32](http://www.crockford.com/wrmg/base32.html)): 77 | 78 | | Value | Symbol | 79 | |-------|--------| 80 | | 0 | 0 | 81 | | 1 | 1 | 82 | | 2 | 2 | 83 | | 3 | 3 | 84 | | 4 | 4 | 85 | | 5 | 5 | 86 | | 6 | 6 | 87 | | 7 | 7 | 88 | | 8 | 8 | 89 | | 9 | 9 | 90 | | 10 | A | 91 | | 11 | B | 92 | | 12 | C | 93 | | 13 | D | 94 | | 14 | E | 95 | | 15 | F | 96 | | 16 | G | 97 | | 17 | H | 98 | | 18 | J | 99 | | 19 | K | 100 | | 20 | M | 101 | | 21 | N | 102 | | 22 | P | 103 | | 23 | Q | 104 | | 24 | R | 105 | | 25 | S | 106 | | 26 | T | 107 | | 27 | V | 108 | | 28 | W | 109 | | 29 | X | 110 | | 30 | Y | 111 | | 31 | Z | 112 | 113 | Due to omitting leading zeros, the suffix for private box (identifier 0) is simply `"box"`. 114 | 115 | For large identifiers (between 2^60 and 2^64 - 1 inclusive), 13 characters are needed, and the most-significant bit of the leftmost character does not contribute to the decoded value. Identifiers must be encoded such that this ignored bit is set to zero. Put another way: If the identifier encoding takes up 13 characters, it must begin with one of `(1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F)`. 116 | -------------------------------------------------------------------------------- /src/feed/index.md: -------------------------------------------------------------------------------- 1 | # Feed spec 2 | 3 | - [Common Datatypes](./datatypes.md) 4 | - [Datamodel](./datamodel.md) 5 | - [Feeds and Messages](./messages.md) 6 | -------------------------------------------------------------------------------- /src/feed/messages.md: -------------------------------------------------------------------------------- 1 | # Feeds and Messages 2 | 3 | Ssb is designed around a data model optimized for simple data replication and strong cryptographic integrity guarantees in a social setting. The central entities in this model are *feeds* and *messages*. Messages are the pieces of data inserted into the system, feeds describe *who* authored a piece of data. 4 | 5 | Messages are carry schemaless, free-form [pieces of data](./datamodel), together with some metadata necessary for message replication and verification. Each message belongs to exactly one feed, and each message contains a backlink to the previous message posted to that feed. Feeds thus form linked lists. Links are implemented via [cryptographically secure hashes](https://en.wikipedia.org/wiki/Cryptographic_hash_function), in that sense feeds behave mostly like blockchains. But whereas blockchains are traditionally used to create a global, single source of truth, ssb chooses a different approach. 6 | 7 | Instead of one single, global linked list for all data, each ssb user has their own linked list. To ensure that no malicious actor can append data to other user's list, all messages are [signed](https://en.wikipedia.org/wiki/Digital_signature). We call this data structure - a signed hash-based linked list - a *sigchain*. 8 | 9 | The sigchain-per-user design has some very desirable properties: 10 | 11 | - data can be moved across untrusted machines, you still know that some piece of data has indeed be posted by a certain identity 12 | - immutability of data, nothing gets lost 13 | - the order between messages can not be confused 14 | - replicating data becomes simple: just exchange the number of messages you know about for a certain feed, and your peer can send you everything that is newer 15 | 16 | Of course, this design also has some drawbacks: 17 | 18 | - the simple replication scheme does not allow subscription to only parts of the data - it's all or nothing 19 | - immutability and non-repudiation are not appropriate for all use-cases 20 | - a single identity can not append data from multiple machines in parallel, as that would result in a tree rather than a linked list 21 | 22 | In some sense, ssb can be seen as an experiment whether the advantages of a sigchain-based distributed database outweigh the disadvantages. So far, it appears to be working sufficiently well. 23 | 24 | The remainder of this chapter describes the exact format of the metadata ssb maintains to build the sigchain. Conceptually, to form the sigchain, the metadata of each message must include: 25 | 26 | - the feed the message belongs to 27 | - the hash of the previous message from the same feed, or `null` 28 | - the free-form content of the message 29 | - a signature to prove that the author knew the private key of the feed 30 | 31 | The actual metadata formats also need to include some extra information. 32 | 33 | ## Metadata 34 | 35 | The abstract model for metadata is a tuple containing the following entries: 36 | 37 | - `previous`: either a [multihash](./datatypes.md#multihash), or the distinct value `null` 38 | - `author`: a [multikey](./datatypes.md#multikey) 39 | - `sequence`: an unsigned 53 bit integer 40 | - `timestamp`: an IEEE 754 64 bit float except the infinities, negative zero, and `NaN`s 41 | - `content`: a [data value](./datamodel.md#abstract-data-model) that is either: 42 | - an object containing an entry `"type"`, whose value is a string that takes between 3 and 53 (inclusive) code units when encoded as utf16 43 | - a [multibox](./datatypes.md#multibox) 44 | - `swapped`: a boolean indicating how to encode the metadata 45 | - `signature`: A signature of the message, generated by the cryptographic primitive of the `author` 46 | - this can only be computed once all other metadata is known 47 | 48 | ### Json Encoding 49 | 50 | Metadata can be encoded as json, just like regular data. The json encoding is a json object containing the entries listed above, with the following additional regulations: 51 | 52 | - the `previous` multihash must use the [message hash encoding](./datatypes.md#multihash-encoding) as a string 53 | - the `author` multikey must use the [multikey encoding](./datatypes.md#multikey-encoding) as a string 54 | - the `sequence` is serialized as a float, since that's the only number type available 55 | - it must not be negative 56 | - it must not contain a decimal point 57 | - the `signature` value is a string whose content is the concatenation of: 58 | - the [canonic](https://tools.ietf.org/html/rfc4648#section-3.5) base64 encoding of the message's signature itself (see next section) 59 | - [ietf rfc 4648, section 4](https://tools.ietf.org/html/rfc4648#section-4), disallowing superflous `=` characters inside the data or after the necessary padding `=`s 60 | - the characters `.sig.` (`[0x2E, 0x73, 0x69, 0x67, 0x2E]`) 61 | - a primitive-specific suffix, depending of the primitive of the `author` 62 | - for ed25519, this is `ed25519` (`[0x65, 0x64, 0x32, 0x35, 0x35, 0x31, 0x39]`) 63 | - the `swapped` boolean is omitted 64 | - an entry `"hash": "sha256"` is added 65 | - if `swapped`, the order of entries must be `previous, sequence, author, timestamp, hash, content, signature`, else it must be `previous, author, sequence, timestamp, hash, content, signature` 66 | - if `content` is a multibox, it must use the [multibox encoding](./datatypes.md#multibox-encoding) as a string 67 | 68 | When creating new messages, the (purely logical) value of `swapped` should be `false`. Or you might set it to `true`, or generate it randomly - nobody can stop you, and everything will still work. 69 | 70 | ### Messages 71 | 72 | To this json encoding corresponds exactly one data value with a fixed order of object entries. This value is referred to as a *message*. 73 | 74 | ### Computing a Message's Signature 75 | 76 | To obtain the signature of a message, first compute the json encoding specified above, but without the `"signature"` entry. Then compute the signing encoding of the corresponding data value, using the entry order of the json as a tie-breaker for the entry-order of the signing encoding. Finally use the signing primitive of the message's `author` on the signing encoding, to obtain the signature. 77 | 78 | ### Computing the Hash of a Message 79 | 80 | To compute the hash of a message, use the [value hash computation](./datamode.md/#hash-computation), with the signing encoding where the object entry order that would produce the correct signature. 81 | 82 | ### Computing the Size of a Message 83 | 84 | To compute the size of a message, use the [value length computation](./datamode.md/#length-computation), with the signing encoding where the object entry order that would produce the correct signature. 85 | 86 | ### Message Validation 87 | 88 | A message is considered valid if and only if all of the following conditions are met: 89 | 90 | - its json encoding is a possible output of the [message creation algorithm](#json-encoding) 91 | - in particular, the claimed signature must match the data and public key 92 | - its length is smaller than `16385` 93 | - if `previous` is `null`, the sequence number of the message must be the float `1`, else it must be one larger than that of the message whose hash is the value of `previous` 94 | -------------------------------------------------------------------------------- /src/hub.md: -------------------------------------------------------------------------------- 1 | # App Hub spec 2 | 3 | TODO 4 | -------------------------------------------------------------------------------- /src/identity.md: -------------------------------------------------------------------------------- 1 | # Shared Identity spec 2 | 3 | TODO 4 | -------------------------------------------------------------------------------- /src/images/current.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunrise-choir/spec/07c89afac14d01dccb2d2a0779baf370327421d3/src/images/current.jpg -------------------------------------------------------------------------------- /src/images/first_pass.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunrise-choir/spec/07c89afac14d01dccb2d2a0779baf370327421d3/src/images/first_pass.jpg -------------------------------------------------------------------------------- /src/images/future_stack.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunrise-choir/spec/07c89afac14d01dccb2d2a0779baf370327421d3/src/images/future_stack.jpg -------------------------------------------------------------------------------- /src/images/low_coord_high_duplication.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunrise-choir/spec/07c89afac14d01dccb2d2a0779baf370327421d3/src/images/low_coord_high_duplication.jpg -------------------------------------------------------------------------------- /src/images/scuttle-shell.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunrise-choir/spec/07c89afac14d01dccb2d2a0779baf370327421d3/src/images/scuttle-shell.jpg -------------------------------------------------------------------------------- /src/introduction.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | TODO (mw) re-write this. 4 | 5 | This document is the specification of the [secure-scuttlebutt]((https://www.scuttlebutt.nz/)) protocol (ssb). The primary audience are developers who need a thorough understanding of protocol internals, for example because they want to implement it themselves. The specificaiton is not intended for developers who want to build applications on top of ssb. Ssb provides nice abstractions to those developer, hiding the nitty-gritty details. This spec however is all about those details. 6 | 7 | It is possible to read through the spec from end to end, but it's not an easy read. It is structured into different sections, each dealing with a different aspect of the protocol. The sections begin with a high-level overview of the problem space, and then delve into the details. In the description of those details, the highest importance is given to unambiguity. If you can not tell *exactly* how something works, it is a bug in the spec. 8 | 9 | 10 | -------------------------------------------------------------------------------- /src/peer.md: -------------------------------------------------------------------------------- 1 | # Network Peer spec 2 | 3 | TODO 4 | -------------------------------------------------------------------------------- /src/same-as.md: -------------------------------------------------------------------------------- 1 | # Same-as spec 2 | 3 | TODO 4 | -------------------------------------------------------------------------------- /src/threads.md: -------------------------------------------------------------------------------- 1 | # Threads spec 2 | 3 | TODO 4 | --------------------------------------------------------------------------------