├── .github └── workflows │ ├── stale.yml │ └── generated-pr.yml ├── LICENSE └── README.md /.github/workflows/stale.yml: -------------------------------------------------------------------------------- 1 | name: Close Stale Issues 2 | 3 | on: 4 | schedule: 5 | - cron: '0 0 * * *' 6 | workflow_dispatch: 7 | 8 | permissions: 9 | issues: write 10 | pull-requests: write 11 | 12 | jobs: 13 | stale: 14 | uses: ipdxco/unified-github-workflows/.github/workflows/reusable-stale-issue.yml@v1 15 | -------------------------------------------------------------------------------- /.github/workflows/generated-pr.yml: -------------------------------------------------------------------------------- 1 | name: Close Generated PRs 2 | 3 | on: 4 | schedule: 5 | - cron: '0 0 * * *' 6 | workflow_dispatch: 7 | 8 | permissions: 9 | issues: write 10 | pull-requests: write 11 | 12 | jobs: 13 | stale: 14 | uses: ipdxco/unified-github-workflows/.github/workflows/reusable-generated-pr.yml@v1 15 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Protocol Labs Inc. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # unsigned-varint 2 | 3 | [![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](https://protocol.ai) 4 | [![](https://img.shields.io/badge/project-multiformats-blue.svg?style=flat-square)](https://github.com/multiformats/multiformats) 5 | [![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](https://webchat.freenode.net/?channels=%23ipfs) 6 | [![](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) 7 | 8 | > unsigned varint in use in multiformat specs 9 | 10 | This unsigned varint (VARiable INTeger) format is for the use in all the multiformats. 11 | 12 | ## Format 13 | 14 | Our unsigned varint is a variant of unsigned [LEB128](https://en.wikipedia.org/wiki/LEB128) with some additional restrictions. 15 | 16 | Specifically, unlike LEB128: 17 | 18 | - Signed numbers are not supported. 19 | - Leading zeros must be trimmed when encoding and must be rejected when decoding. The only number that can end in a `0x00` is `0`. 20 | 21 | ### Spec 22 | 23 | The encoding is: 24 | - unsigned integers are serialized 7 bits at a time, starting with the least significant bits 25 | - the most significant bit (msb) in each output byte indicates if there is a continuation byte (msb = 1) 26 | - there are no signed integers 27 | - integers are minimally encoded 28 | 29 | Examples: 30 | 31 | ``` 32 | 1 (0x01) => 00000001 (0x01) 33 | 127 (0x7f) => 01111111 (0x7f) 34 | 128 (0x80) => 10000000 00000001 (0x8001) 35 | 255 (0xff) => 11111111 00000001 (0xff01) 36 | 300 (0x012c) => 10101100 00000010 (0xac02) 37 | 16384 (0x4000) => 10000000 10000000 00000001 (0x808001) 38 | ``` 39 | 40 | ``` 41 | byte # |             0 |           1 |     2 | 42 | bit # |c 6 5 4 3 2 1 0 |c 5 4 3 2 1 0 7 |c 4 3 2 1 0 7 6 | 43 | |----------------|----------------|----------------| 44 | 16384 => |1 0 0 0 0 0 0 0 |1 0 0 0 0 0 0 0 |0 0 0 0 0 0 0 1 | 45 | ``` 46 | 47 | Code that generates this. 48 | 49 | ```go 50 | package main 51 | 52 | // test program. we can use the go one. 53 | import ( 54 | "encoding/binary" // varint is here 55 | "fmt" 56 | ) 57 | 58 | func main() { 59 | ints := []uint64{1, 127, 128, 255, 300, 16384} 60 | for _, i := range ints { 61 | buf := make([]byte, 10) 62 | n := binary.PutUvarint(buf, uint64(i)) 63 | 64 | hexStr := fmt.Sprintf("%x", i) 65 | if len(hexStr)%2 == 1 { 66 | hexStr = "0" + hexStr 67 | } 68 | fmt.Print(i, " (0x"+hexStr+")\t=> ") 69 | for c := 0; c < n; c++ { 70 | fmt.Printf("%08b ", int(buf[c])) 71 | } 72 | fmt.Printf("(0x%x)\n", buf[:n]) 73 | } 74 | } 75 | ``` 76 | 77 | 78 | 79 | ### Practical maximum of 9 bytes (for security) 80 | 81 | For security, to avoid memory attacks, we use a "practical max" of 9 bytes. Though there is no theoretical limit, and future specs can grow this number if it is truly necessary to have code or length values equal to or larger than `2^63`. 82 | 83 | For the forseeable future: 84 | 85 | - Implementations MUST restrict the size of the varint to a max of 9 bytes (63 bits). 86 | - A multiformat spec MAY explicitly declare a smaller maximum when using varints. 87 | - A multiformat spec MAY NOT explicitly declare a larger maximum when using varints without first changing this spec. 88 | 89 | ### Incompatibilities with Go varints 90 | 91 | This MSB-based unsigned varint is based on the [varint of the Go standard library](https://golang.org/src/encoding/binary/varint.go), which itself was based on the protocol buffers one. 92 | 93 | However, we have two modifications: 94 | 95 | - Multiformats varint only supports unsigned integers, the Go varint supports signed (using zig-zag encoding). 96 | - Multiformats varints must be minimally encoded. That is, numbers must be encoded in the least number of bytes possible. 97 | 98 | > What do we mean by minimally encoded? 99 | 100 | Multiformat varints must be encoded in as few bytes as possible. To illustrate 101 | the issue, take `{0x81 0x00}`. This is a valid golang varint encoding of 0x1. 102 | However, the _minimal_ encoding of 0x1 is `{0x1}`. 103 | 104 | ## Implementations 105 | 106 | * [Rust](https://github.com/paritytech/unsigned-varint) 107 | * [JS](https://github.com/chrisdickinson/varint) 108 | * [PHP](https://github.com/team-acaisia/php-varint) 109 | * [Go](https://github.com/multiformats/go-varint) 110 | 111 | ## Contribute 112 | 113 | Contributions welcome. Please check out [the issues](https://github.com/multiformats/unsigned-varint/issues). 114 | 115 | Check out our [contributing document](https://github.com/multiformats/multiformats/blob/master/contributing.md) for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). 116 | 117 | Small note: If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification. 118 | 119 | ## License 120 | 121 | This repository is only for documents. All of these are licensed under the [CC-BY-SA 3.0](https://ipfs.io/ipfs/QmVreNvKsQmQZ83T86cWSjPu2vR3yZHGPm5jnxFuunEB9u) license © 2016 Protocol Labs Inc. Any code is under a [MIT](LICENSE) © 2016 Protocol Labs Inc. 122 | --------------------------------------------------------------------------------