├── .github └── workflows │ └── publish.yml ├── .gitmodules ├── CODE_OF_CONDUCT.md ├── LICENSE-CC ├── LICENSE-GPL ├── README.md ├── algorithms ├── aggregateKey.tex ├── attemptToFinalizeAtRound.tex ├── bestFinalCandidate.tex ├── bestPrevoteCandidate.tex ├── blockProductionLottery.tex ├── buildBlock.tex ├── decodeMortality.tex ├── derivePrimary.tex ├── encodeMortality.tex ├── epochRandomness.tex ├── finalizable.tex ├── grandpaGhost.tex ├── importAndValidateBlock.tex ├── initiateGrandpa.tex ├── interactWithRuntime.tex ├── invokeBlockAuthoring.tex ├── maintainTransactionPool.tex ├── medianAlgorithm.tex ├── payoutStakersRuntimeFunctionBenchmark.tex ├── playGrandpaRound.tex ├── processCatchupRequest.tex ├── processCatchupResponse.tex ├── queryingStateLightClients.tex ├── requestJudgementRuntimeFunctionBenchmark.tex ├── slotTime.tex ├── transferRuntimeFunctionBenchmark.tex ├── validateTransactionsAndStore.tex ├── verifyAuthorshipRight.tex ├── verifySlotWinner.tex ├── warpSyncLightClients.tex └── withdrawUnbondedRuntimeFunctionBenchmark.tex ├── bibliography.bib ├── docs ├── chap-host-api.md ├── chap-networking.md ├── chap-overview.md ├── chap-runtime-api.md ├── chap-state.md ├── chap-sync.md ├── chapter-anv.md ├── id-consensus.md ├── id-cryptography-encoding.md ├── id-extrinsics.md ├── id-glossary.md ├── id-polkadot-protocol.md ├── id-weights.md ├── part-polkadot-host.md ├── part-polkadot-runtime.md ├── sect-block-production.md ├── sect-finality.md ├── sect-lightclient.md └── sect-metadata.md └── kaitai_structures ├── block.ksy ├── block_body.ksy ├── block_header.ksy ├── digest.ksy ├── metadata.ksy ├── metadata_extrinsic.ksy ├── metadata_pallet.ksy ├── metadata_type.ksy ├── metadata_type_definition.ksy ├── metadata_type_fields.ksy ├── metadata_type_variants.ksy ├── pallet_constant.ksy ├── pallet_storage.ksy ├── scale.ksy └── storage_definition.ksy /.github/workflows/publish.yml: -------------------------------------------------------------------------------- 1 | name: Publish website 2 | 3 | on: 4 | workflow_dispatch: 5 | push: 6 | branches: 7 | - main 8 | 9 | jobs: 10 | build: 11 | if: github.repository_owner == 'w3f' 12 | name: build and deploy 13 | runs-on: ubuntu-latest 14 | steps: 15 | - uses: actions/checkout@master 16 | - uses: browser-actions/setup-chrome@v1 17 | - uses: actions/setup-node@v2 18 | with: 19 | node-version: "18.x" 20 | - name: Initialize submodules 21 | run: | 22 | git submodule update --init --recursive 23 | git submodule update --remote 24 | - name: Build PDF 25 | run: | 26 | cd website && npm i && npm run build_pdf && cd .. 27 | - name: Publish 28 | run: | 29 | git config --global user.email "grants-deployer@users.noreply.github.com" 30 | git config --global user.name "grants-deployer" 31 | echo "machine github.com login grants-deployer password ${{ secrets.ACCESS_KEY }}" > ~/.netrc 32 | cd website && npm run build && GIT_USER=grants-deployer PUBLISHING=true npm run deploy 33 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "website"] 2 | path = website 3 | url = https://github.com/w3f/polkadot-spec-website 4 | branch = main 5 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | We as members, contributors, and leaders pledge to make participation in our 6 | community a harassment-free experience for everyone, regardless of age, body 7 | size, visible or invisible disability, ethnicity, sex characteristics, gender 8 | identity and expression, level of experience, education, socio-economic status, 9 | nationality, personal appearance, race, religion, or sexual identity 10 | and orientation. 11 | 12 | We pledge to act and interact in ways that contribute to an open, welcoming, 13 | diverse, inclusive, and healthy community. 14 | 15 | ## Our Standards 16 | 17 | Examples of behavior that contributes to a positive environment for our 18 | community include: 19 | 20 | * Demonstrating empathy and kindness toward other people 21 | * Being respectful of differing opinions, viewpoints, and experiences 22 | * Giving and gracefully accepting constructive feedback 23 | * Accepting responsibility and apologizing to those affected by our mistakes, 24 | and learning from the experience 25 | * Focusing on what is best not just for us as individuals, but for the 26 | overall community 27 | 28 | Examples of unacceptable behavior include: 29 | 30 | * The use of sexualized language or imagery, and sexual attention or 31 | advances of any kind 32 | * Trolling, insulting or derogatory comments, and personal or political attacks 33 | * Public or private harassment 34 | * Publishing others' private information, such as a physical or email 35 | address, without their explicit permission 36 | * Other conduct which could reasonably be considered inappropriate in a 37 | professional setting 38 | 39 | ## Enforcement Responsibilities 40 | 41 | Community leaders are responsible for clarifying and enforcing our standards of 42 | acceptable behavior and will take appropriate and fair corrective action in 43 | response to any behavior that they deem inappropriate, threatening, offensive, 44 | or harmful. 45 | 46 | Community leaders have the right and responsibility to remove, edit, or reject 47 | comments, commits, code, wiki edits, issues, and other contributions that are 48 | not aligned to this Code of Conduct, and will communicate reasons for moderation 49 | decisions when appropriate. 50 | 51 | ## Scope 52 | 53 | This Code of Conduct applies within all community spaces, and also applies when 54 | an individual is officially representing the community in public spaces. 55 | Examples of representing our community include using an official e-mail address, 56 | posting via an official social media account, or acting as an appointed 57 | representative at an online or offline event. 58 | 59 | ## Enforcement 60 | 61 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 62 | reported to the community leaders responsible for enforcement at 63 | spec@web3.foundation. 64 | All complaints will be reviewed and investigated promptly and fairly. 65 | 66 | All community leaders are obligated to respect the privacy and security of the 67 | reporter of any incident. 68 | 69 | ## Enforcement Guidelines 70 | 71 | Community leaders will follow these Community Impact Guidelines in determining 72 | the consequences for any action they deem in violation of this Code of Conduct: 73 | 74 | ### 1. Correction 75 | 76 | **Community Impact**: Use of inappropriate language or other behavior deemed 77 | unprofessional or unwelcome in the community. 78 | 79 | **Consequence**: A private, written warning from community leaders, providing 80 | clarity around the nature of the violation and an explanation of why the 81 | behavior was inappropriate. A public apology may be requested. 82 | 83 | ### 2. Warning 84 | 85 | **Community Impact**: A violation through a single incident or series 86 | of actions. 87 | 88 | **Consequence**: A warning with consequences for continued behavior. No 89 | interaction with the people involved, including unsolicited interaction with 90 | those enforcing the Code of Conduct, for a specified period of time. This 91 | includes avoiding interactions in community spaces as well as external channels 92 | like social media. Violating these terms may lead to a temporary or 93 | permanent ban. 94 | 95 | ### 3. Temporary Ban 96 | 97 | **Community Impact**: A serious violation of community standards, including 98 | sustained inappropriate behavior. 99 | 100 | **Consequence**: A temporary ban from any sort of interaction or public 101 | communication with the community for a specified period of time. No public or 102 | private interaction with the people involved, including unsolicited interaction 103 | with those enforcing the Code of Conduct, is allowed during this period. 104 | Violating these terms may lead to a permanent ban. 105 | 106 | ### 4. Permanent Ban 107 | 108 | **Community Impact**: Demonstrating a pattern of violation of community 109 | standards, including sustained inappropriate behavior, harassment of an 110 | individual, or aggression toward or disparagement of classes of individuals. 111 | 112 | **Consequence**: A permanent ban from any sort of public interaction within 113 | the community. 114 | 115 | ## Attribution 116 | 117 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 118 | version 2.0, available at 119 | https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. 120 | 121 | Community Impact Guidelines were inspired by [Mozilla's code of conduct 122 | enforcement ladder](https://github.com/mozilla/diversity). 123 | 124 | [homepage]: https://www.contributor-covenant.org 125 | 126 | For answers to common questions about this code of conduct, see the FAQ at 127 | https://www.contributor-covenant.org/faq. Translations are available at 128 | https://www.contributor-covenant.org/translations. 129 | -------------------------------------------------------------------------------- /LICENSE-CC: -------------------------------------------------------------------------------- 1 | Attribution-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-ShareAlike 4.0 International Public 58 | License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-ShareAlike 4.0 International Public License ("Public 63 | License"). To the extent this Public License may be interpreted as a 64 | contract, You are granted the Licensed Rights in consideration of Your 65 | acceptance of these terms and conditions, and the Licensor grants You 66 | such rights in consideration of benefits the Licensor receives from 67 | making the Licensed Material available under these terms and 68 | conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. Share means to provide material to the public by any means or 126 | process that requires permission under the Licensed Rights, such 127 | as reproduction, public display, public performance, distribution, 128 | dissemination, communication, or importation, and to make material 129 | available to the public including in ways that members of the 130 | public may access the material from a place and at a time 131 | individually chosen by them. 132 | 133 | l. Sui Generis Database Rights means rights other than copyright 134 | resulting from Directive 96/9/EC of the European Parliament and of 135 | the Council of 11 March 1996 on the legal protection of databases, 136 | as amended and/or succeeded, as well as other essentially 137 | equivalent rights anywhere in the world. 138 | 139 | m. You means the individual or entity exercising the Licensed Rights 140 | under this Public License. Your has a corresponding meaning. 141 | 142 | 143 | Section 2 -- Scope. 144 | 145 | a. License grant. 146 | 147 | 1. Subject to the terms and conditions of this Public License, 148 | the Licensor hereby grants You a worldwide, royalty-free, 149 | non-sublicensable, non-exclusive, irrevocable license to 150 | exercise the Licensed Rights in the Licensed Material to: 151 | 152 | a. reproduce and Share the Licensed Material, in whole or 153 | in part; and 154 | 155 | b. produce, reproduce, and Share Adapted Material. 156 | 157 | 2. Exceptions and Limitations. For the avoidance of doubt, where 158 | Exceptions and Limitations apply to Your use, this Public 159 | License does not apply, and You do not need to comply with 160 | its terms and conditions. 161 | 162 | 3. Term. The term of this Public License is specified in Section 163 | 6(a). 164 | 165 | 4. Media and formats; technical modifications allowed. The 166 | Licensor authorizes You to exercise the Licensed Rights in 167 | all media and formats whether now known or hereafter created, 168 | and to make technical modifications necessary to do so. The 169 | Licensor waives and/or agrees not to assert any right or 170 | authority to forbid You from making technical modifications 171 | necessary to exercise the Licensed Rights, including 172 | technical modifications necessary to circumvent Effective 173 | Technological Measures. For purposes of this Public License, 174 | simply making modifications authorized by this Section 2(a) 175 | (4) never produces Adapted Material. 176 | 177 | 5. Downstream recipients. 178 | 179 | a. Offer from the Licensor -- Licensed Material. Every 180 | recipient of the Licensed Material automatically 181 | receives an offer from the Licensor to exercise the 182 | Licensed Rights under the terms and conditions of this 183 | Public License. 184 | 185 | b. Additional offer from the Licensor -- Adapted Material. 186 | Every recipient of Adapted Material from You 187 | automatically receives an offer from the Licensor to 188 | exercise the Licensed Rights in the Adapted Material 189 | under the conditions of the Adapter's License You apply. 190 | 191 | c. No downstream restrictions. You may not offer or impose 192 | any additional or different terms or conditions on, or 193 | apply any Effective Technological Measures to, the 194 | Licensed Material if doing so restricts exercise of the 195 | Licensed Rights by any recipient of the Licensed 196 | Material. 197 | 198 | 6. No endorsement. Nothing in this Public License constitutes or 199 | may be construed as permission to assert or imply that You 200 | are, or that Your use of the Licensed Material is, connected 201 | with, or sponsored, endorsed, or granted official status by, 202 | the Licensor or others designated to receive attribution as 203 | provided in Section 3(a)(1)(A)(i). 204 | 205 | b. Other rights. 206 | 207 | 1. Moral rights, such as the right of integrity, are not 208 | licensed under this Public License, nor are publicity, 209 | privacy, and/or other similar personality rights; however, to 210 | the extent possible, the Licensor waives and/or agrees not to 211 | assert any such rights held by the Licensor to the limited 212 | extent necessary to allow You to exercise the Licensed 213 | Rights, but not otherwise. 214 | 215 | 2. Patent and trademark rights are not licensed under this 216 | Public License. 217 | 218 | 3. To the extent possible, the Licensor waives any right to 219 | collect royalties from You for the exercise of the Licensed 220 | Rights, whether directly or through a collecting society 221 | under any voluntary or waivable statutory or compulsory 222 | licensing scheme. In all other cases the Licensor expressly 223 | reserves any right to collect such royalties. 224 | 225 | 226 | Section 3 -- License Conditions. 227 | 228 | Your exercise of the Licensed Rights is expressly made subject to the 229 | following conditions. 230 | 231 | a. Attribution. 232 | 233 | 1. If You Share the Licensed Material (including in modified 234 | form), You must: 235 | 236 | a. retain the following if it is supplied by the Licensor 237 | with the Licensed Material: 238 | 239 | i. identification of the creator(s) of the Licensed 240 | Material and any others designated to receive 241 | attribution, in any reasonable manner requested by 242 | the Licensor (including by pseudonym if 243 | designated); 244 | 245 | ii. a copyright notice; 246 | 247 | iii. a notice that refers to this Public License; 248 | 249 | iv. a notice that refers to the disclaimer of 250 | warranties; 251 | 252 | v. a URI or hyperlink to the Licensed Material to the 253 | extent reasonably practicable; 254 | 255 | b. indicate if You modified the Licensed Material and 256 | retain an indication of any previous modifications; and 257 | 258 | c. indicate the Licensed Material is licensed under this 259 | Public License, and include the text of, or the URI or 260 | hyperlink to, this Public License. 261 | 262 | 2. You may satisfy the conditions in Section 3(a)(1) in any 263 | reasonable manner based on the medium, means, and context in 264 | which You Share the Licensed Material. For example, it may be 265 | reasonable to satisfy the conditions by providing a URI or 266 | hyperlink to a resource that includes the required 267 | information. 268 | 269 | 3. If requested by the Licensor, You must remove any of the 270 | information required by Section 3(a)(1)(A) to the extent 271 | reasonably practicable. 272 | 273 | b. ShareAlike. 274 | 275 | In addition to the conditions in Section 3(a), if You Share 276 | Adapted Material You produce, the following conditions also apply. 277 | 278 | 1. The Adapter's License You apply must be a Creative Commons 279 | license with the same License Elements, this version or 280 | later, or a BY-SA Compatible License. 281 | 282 | 2. You must include the text of, or the URI or hyperlink to, the 283 | Adapter's License You apply. You may satisfy this condition 284 | in any reasonable manner based on the medium, means, and 285 | context in which You Share Adapted Material. 286 | 287 | 3. You may not offer or impose any additional or different terms 288 | or conditions on, or apply any Effective Technological 289 | Measures to, Adapted Material that restrict exercise of the 290 | rights granted under the Adapter's License You apply. 291 | 292 | 293 | Section 4 -- Sui Generis Database Rights. 294 | 295 | Where the Licensed Rights include Sui Generis Database Rights that 296 | apply to Your use of the Licensed Material: 297 | 298 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 299 | to extract, reuse, reproduce, and Share all or a substantial 300 | portion of the contents of the database; 301 | 302 | b. if You include all or a substantial portion of the database 303 | contents in a database in which You have Sui Generis Database 304 | Rights, then the database in which You have Sui Generis Database 305 | Rights (but not its individual contents) is Adapted Material, 306 | including for purposes of Section 3(b); and 307 | 308 | c. You must comply with the conditions in Section 3(a) if You Share 309 | all or a substantial portion of the contents of the database. 310 | 311 | For the avoidance of doubt, this Section 4 supplements and does not 312 | replace Your obligations under this Public License where the Licensed 313 | Rights include other Copyright and Similar Rights. 314 | 315 | 316 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 317 | 318 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 319 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 320 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 321 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 322 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 323 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 324 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 325 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 326 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 327 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 328 | 329 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 330 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 331 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 332 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 333 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 334 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 335 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 336 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 337 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 338 | 339 | c. The disclaimer of warranties and limitation of liability provided 340 | above shall be interpreted in a manner that, to the extent 341 | possible, most closely approximates an absolute disclaimer and 342 | waiver of all liability. 343 | 344 | 345 | Section 6 -- Term and Termination. 346 | 347 | a. This Public License applies for the term of the Copyright and 348 | Similar Rights licensed here. However, if You fail to comply with 349 | this Public License, then Your rights under this Public License 350 | terminate automatically. 351 | 352 | b. Where Your right to use the Licensed Material has terminated under 353 | Section 6(a), it reinstates: 354 | 355 | 1. automatically as of the date the violation is cured, provided 356 | it is cured within 30 days of Your discovery of the 357 | violation; or 358 | 359 | 2. upon express reinstatement by the Licensor. 360 | 361 | For the avoidance of doubt, this Section 6(b) does not affect any 362 | right the Licensor may have to seek remedies for Your violations 363 | of this Public License. 364 | 365 | c. For the avoidance of doubt, the Licensor may also offer the 366 | Licensed Material under separate terms or conditions or stop 367 | distributing the Licensed Material at any time; however, doing so 368 | will not terminate this Public License. 369 | 370 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 371 | License. 372 | 373 | 374 | Section 7 -- Other Terms and Conditions. 375 | 376 | a. The Licensor shall not be bound by any additional or different 377 | terms or conditions communicated by You unless expressly agreed. 378 | 379 | b. Any arrangements, understandings, or agreements regarding the 380 | Licensed Material not stated herein are separate from and 381 | independent of the terms and conditions of this Public License. 382 | 383 | 384 | Section 8 -- Interpretation. 385 | 386 | a. For the avoidance of doubt, this Public License does not, and 387 | shall not be interpreted to, reduce, limit, restrict, or impose 388 | conditions on any use of the Licensed Material that could lawfully 389 | be made without permission under this Public License. 390 | 391 | b. To the extent possible, if any provision of this Public License is 392 | deemed unenforceable, it shall be automatically reformed to the 393 | minimum extent necessary to make it enforceable. If the provision 394 | cannot be reformed, it shall be severed from this Public License 395 | without affecting the enforceability of the remaining terms and 396 | conditions. 397 | 398 | c. No term or condition of this Public License will be waived and no 399 | failure to comply consented to unless expressly agreed to by the 400 | Licensor. 401 | 402 | d. Nothing in this Public License constitutes or may be interpreted 403 | as a limitation upon, or waiver of, any privileges and immunities 404 | that apply to the Licensor or You, including from the legal 405 | processes of any jurisdiction or authority. 406 | 407 | 408 | ======================================================================= 409 | 410 | Creative Commons is not a party to its public 411 | licenses. Notwithstanding, Creative Commons may elect to apply one of 412 | its public licenses to material it publishes and in those instances 413 | will be considered the “Licensor.” The text of the Creative Commons 414 | public licenses is dedicated to the public domain under the CC0 Public 415 | Domain Dedication. Except for the limited purpose of indicating that 416 | material is shared under a Creative Commons public license or as 417 | otherwise permitted by the Creative Commons policies published at 418 | creativecommons.org/policies, Creative Commons does not authorize the 419 | use of the trademark "Creative Commons" or any other trademark or logo 420 | of Creative Commons without its prior written consent including, 421 | without limitation, in connection with any unauthorized modifications 422 | to any of its public licenses or any other arrangements, 423 | understandings, or agreements concerning use of licensed material. For 424 | the avoidance of doubt, this paragraph does not form part of the 425 | public licenses. 426 | 427 | Creative Commons may be contacted at creativecommons.org. -------------------------------------------------------------------------------- /LICENSE-GPL: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | > [!NOTE] 2 | > The Polkadot Protocol Specifications are not actively maintained by the Web3 Foundation as of 02 October 2024. Please follow the [RFC process](https://github.com/polkadot-fellows/RFCs) for information on latest protocol updates. 3 | 4 | # Polkadot Protocol Specification 5 | 6 | - [Introduction](#introduction) 7 | - [Process](#process) 8 | - [Local Development](#local-development) 9 | - [Formatting](#formatting) 10 | - [LaTeX](#latex) 11 | - [Numeration System](#numeration-system) 12 | - [Chapters](#chapters) 13 | - [Sections](#sections) 14 | - [Definitions](#definitions) 15 | - [Algorithms](#algorithms) 16 | - [Tables and Images](#tables-and-images) 17 | - [References](#references) 18 | - [Bibliography](#bibliography) 19 | - [Broken Links](#broken-links) 20 | - [License](#license) 21 | 22 | ## Introduction 23 | 24 | This repository contains the official specification for the [Polkadot Protocol](https://polkadot.network/). The latest releases of the *Polkadot Protocol Specification* can be found on [spec.polkadot.network](https://spec.polkadot.network) or our [GitHub Releases page](https://github.com/w3f/polkadot-spec/releases). 25 | 26 | ## Process 27 | 28 | Everyone, but specifically implementers and members of the [Technical Fellowship](https://wiki.polkadot.network/docs/learn-polkadot-opengov#the-technical-fellowship), are welcome to open PRs to this repo and contribute to the specification. A PR is merged as soon as two members of the Spec Committee accept the PR. 29 | 30 | ### Spec Committee 31 | 32 | The Spec Committee consists of Researchers, Researcher Engineers, and Implementers of the Polkadot Protocol that actively contribute (= opening/contributing to PRs or issues regarding the spec) to the specification. For now and compared to the [Technical Fellowship](https://github.com/polkadot-fellows/manifesto), the Spec Committee's responsibility is clearly defined as maintaining the specification of the protocol. In case of inactivity for three months, members will be removed again. 33 | 34 | Web3 Foundation 35 | - [Bhargav Bhatt](https://github.com/bhargavbh) 36 | - [David Hawig](https://github.com/Noc2) 37 | - [Seyed Lavasani](https://github.com/drskalman) 38 | 39 | Quadrivium 40 | - [Kamil Salakhiev](https://github.com/kamilsa) 41 | 42 | Smoldot 43 | - [Pierre Krieger](https://github.com/tomaka) 44 | 45 | Parity 46 | - [Fateme Shirazi](https://github.com/FatemeShirazi) 47 | 48 | ## Local Development 49 | 50 | To clone, build and serve the website locally, run the following commands: 51 | 52 | ```bash 53 | git clone --recurse-submodules https://github.com/w3f/polkadot-spec 54 | cd polkadot-spec 55 | git submodule update --remote 56 | cd website 57 | npm i 58 | npm run build # or build_with_kaitai to also rebuild kaitai SVG files 59 | npm run serve 60 | ``` 61 | 62 | If you already have the repo cloned, remember to update it and the submodule before making any changes: 63 | 64 | ```bash 65 | git pull 66 | git submodule update --remote 67 | ``` 68 | 69 | Because of the "complex" build, you can't see the changes in real time if you directly edit the `docs` folder. There are two workarounds: 70 | - use a Markdown plugin or editor (e.g., the extension [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) for VSCode) to see the changes in real time. You won't be able to see the placeholders rendered and other elements, but you'll be able to render the markdown and latex; 71 | 72 | - build the website the first time, then run `npm run start` instead of `npm run serve,` and edit the files inside the `website/docs` folder. This will allow you to see the changes in real time, but you have to remember to bring the changes inside the `docs` folder before committing. Also, in this way, you can't put the placeholders and other elements inside the markdown files of `website/docs` as they wouldn't be rendered. 73 | 74 | You can find the markdown files inside the [`docs`](docs) folder. 75 | 76 | When building, the scripts inside [`preBuild`](https://github.com/w3f/polkadot-spec-website/tree/main/preBuild) will generate a `docs` folder, from which Docusaurus will render the website. Then, the rendered content will be modified by the [`plugins`](https://github.com/w3f/polkadot-spec-website/tree/main/plugins) in the browser. 77 | 78 | ## Formatting 79 | 80 | ### LaTeX 81 | 82 | You can use LaTeX inside the markdown files using the following syntax: 83 | ```md 84 | $ LaTeX here $ 85 | ``` 86 | or 87 | ```md 88 | $$ 89 | LaTeX here 90 | $$ 91 | ``` 92 | 93 | ### Numeration System 94 | 95 | Inside [`preBuild`](https://github.com/w3f/polkadot-spec-website/tree/main/preBuild), you can find the script [`numerationSystem`](https://github.com/w3f/polkadot-spec-website/tree/main/preBuild/numerationSystem/index.ts). This will assign to several entities a number and substitute the placeholders inside the markdown files. This is done to avoid having to manually update the numbers when adding new entities. 96 | 97 | This is the structure of the spec: 98 | ```md 99 | - Macro Chapter X 100 | 1. Chapter A 101 | - Section 1.1 102 | ... subsections 103 | - Section 1.2 104 | 2. Chapter B 105 | - Section 2.1 106 | ... subsections 107 | - Section 2.2 108 | - Macro Chapter Y 109 | etc. 110 | ``` 111 | Example: 112 | ```md 113 | - Polkadot Host 114 | 1. Overview 115 | 1.1 Light Client 116 | ... 117 | 2. State and Transitions 118 | ... 119 | ``` 120 | 121 | The entities involved are: 122 | - Chapters 123 | - Sections 124 | - Definitions 125 | - Algorithms 126 | - Tables 127 | - Images 128 | 129 | Those defined as "Macro Chapters" will not be numbered. 130 | 131 | #### Chapters 132 | To write a new chapter, use the following syntax: 133 | ```md 134 | --- 135 | title: -chap-num- Chapter Title 136 | --- 137 | 138 | ``` 139 | The placeholder `-chap-num-` will be replaced by the number assigned by [`numerationSystem`](https://github.com/w3f/polkadot-spec-website/tree/main/preBuild/numerationSystem/index.ts). 140 | 141 | If you add a chapter (or "Macro Chapter"), you must also add it to the [`sidebars.js`](https://github.com/w3f/polkadot-spec-website/tree/main/sidebars.js) file and adjust the numbers of the other chapters. 142 | 143 | #### Sections 144 | To write a new section, use the following syntax: 145 | ```md 146 | ## -sec-num- Section name {#id-section-name} 147 | ``` 148 | - Use a markdown header from H2 to H5 included, so the maximum depth is `a.b.c.d.e` (H2 is `a.b`). 149 | - Put the placeholder `-sec-num-` in the header, which will be replaced; 150 | - Add an id to the header, which will be used to reference the section. 151 | 152 | #### Definitions 153 | 154 | To write a definition: 155 | ```md 156 | ###### Definition -def-num- Runtime Pointer {#defn-runtime-pointer} 157 | ``` 158 | - Use a markdown H6 header (######); 159 | - Put the placeholder `-def-num-` in the header; 160 | - Add an id to the header. 161 | 162 | Then, you should include the definition content inside the custom admonition `:::definition` (you can find all the custom admonitions inside [`src/theme/Admonition/Types.js`](https://github.com/w3f/polkadot-spec-website/tree/main/src/theme/Admonition/Types.js)). 163 | 164 | So the final result will be the following: 165 | ```md 166 | ###### Definition -def-num- Runtime Pointer {#defn-runtime-pointer} 167 | :::definition 168 | 169 | Definition content here 170 | 171 | ::: 172 | ``` 173 | 174 | #### Algorithms 175 | 176 | To define an algorithm, use the same syntax as for definitions but with the placeholder `-algo-num-`: 177 | ```md 178 | ###### Algorithm -algo-num- Aggregate-Key {#algo-aggregate-key} 179 | ``` 180 | At the top of the page, you must include the [`Pseudocode`](https://github.com/w3f/polkadot-spec-website/tree/main/src/components/Pseudocode.jsx) component and the LaTeX algorithm you want to render: 181 | ```md 182 | --- 183 | title: -chap-num- States and Transitions 184 | --- 185 | import Pseudocode from '@site/src/components/Pseudocode'; 186 | import aggregateKey from '!!raw-loader!@site/src/algorithms/aggregateKey.tex'; 187 | ``` 188 | After this, you can build the algorithm using the admonition `:::algorithm`, and using the [`Pseudocode`](https://github.com/w3f/polkadot-spec-website/tree/main/src/components/Pseudocode.jsx) component (refer to the file to know more about its `props`). This will be the final result: 189 | ```md 190 | --- 191 | title: -chap-num- States and Transitions 192 | --- 193 | import Pseudocode from '@site/src/components/Pseudocode'; 194 | import aggregateKey from '!!raw-loader!@site/src/algorithms/aggregateKey.tex'; 195 | 196 | 197 | 198 | ###### Algorithm -algo-num- Aggregate-Key {#algo-aggregate-key} 199 | :::algorithm 200 | 205 | 206 | 207 | ::: 208 | ``` 209 | 210 | #### Tables and Images 211 | 212 | To define a table or an image, use the same syntax as for definitions and algorithms (always using an H6 header) but with the placeholder `-tab-num-` or `-img-num-`: 213 | ```md 214 | ###### Table -tab-num- Name {#tab-name} 215 | ``` 216 | or 217 | ```md 218 | ###### Image -img-num- Name {#img-name} 219 | ``` 220 | For these two entities, you won't need to use any component or admonition. 221 | 222 | #### References 223 | 224 | To reference any of the entities from anywhere in the website, you have to use the following syntax: 225 | ```md 226 | [Entity -xxx-num-ref-](entity-page#entity-id) 227 | ``` 228 | - Use a markdown link; 229 | - Put the placeholder `-xxx-num-ref-` in the text link, which will be replaced (`xxx` depends on the entity, for example `-def-num-ref-` for definitions); 230 | - The link should point to the header id of the definition, with the page name as a prefix (if entity-page.md is a page, ".md" must be omitted). 231 | 232 | ### Bibliography 233 | 234 | The cited works are defined inside [`bibliography.bib`](bibliography.bib). To cite a work, use the following syntax: 235 | ```md 236 | [@work-id] 237 | ``` 238 | Automatically, the bibliography will be generated at the end of the page. 239 | 240 | ### Broken Links 241 | 242 | During the `preBuild,` the external links in the markdown files will be checked.
243 | After the `build,` the internal links will be checked.
244 | If any link is broken, the console will show a warning.
245 | 246 | Refer to [`checkBrokenInternalLinks/index.ts`](https://github.com/w3f/polkadot-spec-website/tree/main/plugins/checkBrokenInternalLinks/index.ts) and [`checkBrokenExternalLinks/index.ts`](https://github.com/w3f/polkadot-spec-website/tree/main/preBuild/checkBrokenExternalLinks/index.ts). 247 | 248 | ## Resources 249 | 250 | Docusaurus docs: https://docusaurus.io/docs/category/guides 251 | 252 | Kaitai docs: https://doc.kaitai.io/user_guide.html 253 | 254 | ## License 255 | 256 | Any code in this repository is licensed under the [GPL 3.0](https://www.gnu.org/licenses/gpl-3.0.en.html), and any documentation or specification is licensed under the [CC BY-SA 2.0](https://creativecommons.org/licenses/by-sa/2.0/). 257 | -------------------------------------------------------------------------------- /algorithms/aggregateKey.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Aggregate-Key} 3 | \begin{algorithmic} 4 | \require{$P_N \coloneqq ($\textsc{TrieRoot}$ = N_1, \dots, N_j = N)$} 5 | \state $pk^{Agr}_N \leftarrow \phi$ 6 | \state $i \leftarrow 1$ 7 | \forall{$N_i \in P_N$} 8 | \state $pk^{Agr}_N \leftarrow pk^{Agr}_N || pk_{N_i} || \textrm{Index}_{N_i}(N_{i + 1})$ 9 | \endfor 10 | \state $pk^{Agr}_N \leftarrow pk^{Agr}_N || pk_{N}$ 11 | \return $pk^{Agr}_N$ 12 | \end{algorithmic} 13 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/attemptToFinalizeAtRound.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Attempt-To-Finalize-At-Round} 3 | \begin{algorithmic} 4 | \REQUIRE($r$) 5 | \STATE $L \leftarrow$ \textsc{Last-Finalized-Block} 6 | \STATE $E \leftarrow$ \call{Best-Final-Candidate}{$r$} 7 | \IF{$E \geqslant L$ \and ${V^{r, \textrm{pc}}_{\textrm{obs}(v)}}(E) > 2 / 3 |\mathbb{V}|$} 8 | \STATE{\textsc{Last-Finalized-Block}$\leftarrow E$} 9 | \IF{$M_v^{r, \textrm{Fin}} (E) \notin $\textsc{Received-Messages}} 10 | \STATE \call{Broadcast}{$M_v^{r, \textrm{Fin}}(E)$} 11 | \RETURN 12 | \ENDIF 13 | \ENDIF 14 | \end{algorithmic} 15 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/bestFinalCandidate.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Best-Final-Candidate} 3 | \begin{algorithmic} 4 | \input $r$ 5 | \state $B_v^{r, pv} \leftarrow$ \call{GRANDPA-GHOST}{$r$} 6 | \if{$r = 0$} 7 | \return $B_v^{r, pv}$ 8 | \else 9 | \state $\mathcal{C} \leftarrow \{ B' | B' \leqslant B_v^{r,pv} | \#V^{r, pc}_{\operatorname{obv}(v), pot}(B') > \frac{2}{3} |\mathbb{V}| \}$ 10 | \if{$\mathcal{C} = \phi$} 11 | \return $B_v^{r, pv}$ 12 | \else 13 | \return $E \in \mathcal{C} : H_n (E) = \operatorname{max}\left(H_n (B') | B' \in \mathcal{C}\right)$ 14 | \endif 15 | \endif 16 | \end{algorithmic} 17 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/bestPrevoteCandidate.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Best-PreVote-Candidate} 3 | \begin{algorithmic} 4 | \input $r$ 5 | \state $B^{r-1, pv}_v \leftarrow$ \call{GRANDPA-GHOST}{$r-1$} 6 | \if{\call{Received}{$M_{v_{primary}}^{r, prim}(B))$ \and $B^{r-1, pv}_v \geqslant B > L$}} 7 | \state $N \leftarrow B$ 8 | \else 9 | \state $N \leftarrow B^{r-1, pv}_v$ 10 | \endif 11 | \return \call{Best-Chain-Safe-Head}{N} 12 | \end{algorithmic} 13 | \end{algorithm} 14 | -------------------------------------------------------------------------------- /algorithms/blockProductionLottery.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Block-Production-Lottery} 3 | \begin{algorithmic} 4 | \require sk 5 | \state $r \leftarrow$ \call{Epoch-Randomness}{$n$} 6 | \for{$i := 1 ~\textbf{to}~ sc_n$} 7 | \state $(\pi, d) \leftarrow$ \call{VRF}{$r, i, sk$} 8 | \state $A[i] \leftarrow (d, \pi)$ 9 | \endfor 10 | \return{A} 11 | \end{algorithmic} 12 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/buildBlock.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Build-Block} 3 | \begin{algorithmic} 4 | \state $P_B \leftarrow $\call{Head}{$C_{Best}$} 5 | \state $\text{Head}(B) \leftarrow \left(H_p \leftarrow H_h(P_B), H_i \leftarrow H_i(P_B) + 1, H_r \leftarrow \phi, H_e \leftarrow \phi, H_d \leftarrow \phi \right)$ 6 | \state \call{Call-Runtime-Entry}{$\texttt{Core\_initialize\_block}, \text{Head}(B)$} 7 | \state \textsc{I-D}$ \leftarrow $\call{Call-Runtime-Entry}{$\texttt{BlockBuilder\_inherent\_extrinsics}, $\textsc{Inherent-Data}} 8 | \for{$E~\textbf{in} $\textsc{I-D}} 9 | \state \call{Call-Runtime-Entry}{$\texttt{BlockBuilder\_apply\_extrinsics}, E$} 10 | \endfor 11 | \while{\not \call{End-Of-Slot}{$s$}} 12 | \state $E \leftarrow$ \call{Next-Ready-Extrinsic}{} 13 | \state $R \leftarrow$ \call{Call-Runtime-Entry}{$\texttt{BlockBuilder\_apply\_extrinsics}, E$} 14 | \if{\call{Block-Is-Full}{$R$}} 15 | \break 16 | \endif 17 | \if{\call{Should-Drop}{$R$}} 18 | \state \call{Drop}{$E$} 19 | \endif 20 | \state $\text{Head}(B) \leftarrow$ \call{Call-Runtime-Entry}{$\texttt{BlockBuilder\_finalize\_block}, B$} 21 | \state $B \leftarrow$ \call{Add-Seal}{$B$} 22 | \endwhile 23 | \end{algorithmic} 24 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/decodeMortality.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Decode Mortality} 3 | \begin{algorithmic} 4 | \Require{$T_{mor}$} 5 | \Return $\textit{Immortal} \enspace \textbf{if} \enspace T^{b0}_{mor} = 0$ 6 | \State \textbf{init} $enc = T^{b0}_{mor} + (T^{b1}_{mor} \ll 8)$ 7 | \State \textbf{init} $M_{per} = 2 \ll (enc\ mod\ (1 \ll 4))$ 8 | \State \textbf{init} $factor =$ \call{Limit}{$M_{per} \gg 12, 1, \phi$} 9 | \State \textbf{init} $M_{pha} = (enc \gg 4) * factor$ 10 | \Return $(M_{per}, M_{pha})$ 11 | \end{algorithmic} 12 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/derivePrimary.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Derive-Primary} 3 | \begin{algorithmic} 4 | \input $r$ 5 | \return $r \bmod |\mathbb{V}|$ 6 | \end{algorithmic} 7 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/encodeMortality.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Encode Mortality} 3 | \begin{algorithmic} 4 | \Require{$M_{per}, M_{pha}$} 5 | \Return $0 \enspace \textbf{if} \enspace \textit{extrinsic is immortal}$ 6 | \State \textbf{init} $factor = $\call{Limit}{$M_{per} \gg 12, 1, \phi$} 7 | \State \textbf{init} $left = $\call{Limit}{\call{TZ}{$M_{per}$}$ - 1, 1, 15$} 8 | \State \textbf{init} $right = \frac{M_{pha}}{factor} \ll 4$ 9 | \Return $left|right$ 10 | \end{algorithmic} 11 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/epochRandomness.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Epoch-Randomness} 3 | \begin{algorithmic} 4 | \Require $n > 2$ 5 | \State \textbf{init} $\rho \leftarrow \phi$ 6 | \For{$B$ in \call{SubChain}{$\mathcal{E}_{n-2}$}} 7 | \State $\rho \leftarrow \rho || d_B$ 8 | \EndFor 9 | \Return \call{Blake2b}{\call{Epoch-Randomness}{$n-1$}$||n||\rho$} 10 | \end{algorithmic} 11 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/finalizable.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Finalizable} 3 | \begin{algorithmic} 4 | \REQUIRE($r$) 5 | \IF{$r$ \textbf{is not} Completable} 6 | \RETURN \textbf{False} 7 | \ENDIF 8 | \STATE $G \leftarrow$ \call{GRANDPA-GHOST}{$J^{r, pv}(B)$} 9 | \IF{$G = \phi$} 10 | \RETURN \textbf{False} 11 | \ENDIF 12 | \STATE $E_r \leftarrow$ \call{Best-Final-Candidate}{$r$} 13 | \IF{$E_r \neq \phi$ \and \call{Best-Final-Candidate}{$r - 1$} $\leqslant E_r \leqslant G$} 14 | \RETURN \textbf{True} 15 | \ELSE 16 | \RETURN \textbf{False} 17 | \ENDIF 18 | \end{algorithmic} 19 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/grandpaGhost.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{GRANDPA-GHOST} 3 | \begin{algorithmic} 4 | \input $r$ 5 | \if{$r = 0$} 6 | \state $G \leftarrow B_{last}$ 7 | \else 8 | \state $L \leftarrow$ \call{Best-Final-Candidate}{$r - 1$} 9 | \state $\mathcal{G} = \{ \forall B > L | \#V_{\operatorname{obs}(v)}^{r, pv}(B) \geqslant \frac{2}{3} |\mathbb{V}| \}$ 10 | \if{$\mathcal{G} = \phi$} 11 | \state $G \leftarrow L$ 12 | \else 13 | \state $G \in \mathcal{G} | H_n(G) = \operatorname{max}\left( H_n (B) | \forall B \in \mathcal{G} \right)$ 14 | \endif 15 | \endif 16 | \return $G$ 17 | \end{algorithmic} 18 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/importAndValidateBlock.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Import-and-Validate-Block} 3 | \begin{algorithmic} 4 | \require $B, \text{Just}(B)$ 5 | \state \call{Set-Storage-State-At}{$P(B)$} 6 | \if{$\text{Just}(B) \neq \emptyset$} 7 | \state \call{Verify-Block-Justification}{$B, \text{Just}(B)$} 8 | \if{$B~\textbf{is}~\text{Finalized}~\textbf{and}~P(B)~\textbf{is not}~\text{Finalized}$} 9 | \state \call{Mark-as-Final}{$P(B)$} 10 | \endif 11 | \endif 12 | \if{$H_p(B) \notin PBT$} 13 | \return 14 | \endif 15 | \state \call{Verify-Authorship-Right}{$\text{Head}(B)$} 16 | \state $B \leftarrow$ \call{Remove-Seal}{$B$} 17 | \state $R \leftarrow$ \call{Call-Runtime-Entry}{$\texttt{Core\_execute\_block}, B$} 18 | \state $B \leftarrow$ \call{Add-Seal}{$B$} 19 | \if{$R =$ \textsc{True}} 20 | \state \call{Persist-State}{} 21 | \endif 22 | \end{algorithmic} 23 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/initiateGrandpa.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Initiate-Grandpa} 3 | \begin{algorithmic} 4 | \input $r_{last}, B_{last}$ 5 | \state \textsc{Last-Finalized-Block} $\leftarrow B_{last}$ 6 | \state \textsc{Best-Final-Candidate}$(0) \leftarrow B_{last}$ 7 | \state \textsc{GRANDPA-GHOST}$(0) \leftarrow B_{last}$ 8 | \state \textsc{Last-Completed-Round}$ \leftarrow 0$ 9 | \state $r_n \leftarrow 1$ 10 | \state \call{Play-Grandpa-round}{$r_n$} 11 | \end{algorithmic} 12 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/interactWithRuntime.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Interact-With-Runtime} 3 | \begin{algorithmic} 4 | \require $F, H_b(B),(A_1,\ldots,A_n)$ 5 | \state $\mathcal{S}_B \leftarrow$ \call{Set-State-At}{$H_b(B)$} 6 | \state $A \leftarrow Enc_{SC}((A_1, \ldots, A_n))$ 7 | \state \call{Call-Runtime-Entrypoint}{$R_B, \mathcal{RE}_B, F, A, A_{len}$} 8 | \end{algorithmic} 9 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/invokeBlockAuthoring.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Invoke-Block-Authoring} 3 | \begin{algorithmic} 4 | \require $sk, pk, n, BT$ 5 | \state $A \leftarrow$ \call{Block-production-lottery}{$sk, n$} 6 | \for{$s \leftarrow 1 ~\textbf{to}~ sc_n$} 7 | \state \call{Wait-Until}{\call{Slot-Time}{$s$}} 8 | \state $(d, \pi) \leftarrow A[s]$ 9 | \if{$\tau > d$} 10 | \state $C_{Best} \leftarrow$ \call{Longest-Chain}{$BT$} 11 | \state $B_s \leftarrow$ \call{Build-Block}{$C_{Best}$} 12 | \state \call{Add-Digest-Item}{$B_s,\text{Pre-Runtime}, E_{id}(\text{BABE}), H_\text{BABE}(B_s)$} 13 | \state \call{Add-Digest-Item}{$B_s, \text{Seal}, S_B$} 14 | \state \call{Broadcast-Block}{$B_s$} 15 | \endif 16 | \endfor 17 | \end{algorithmic} 18 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/maintainTransactionPool.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Maintain-Transaction-Pool} 3 | \begin{algorithmic} 4 | \state Scan the pool for ready transactions 5 | \state Move them to the transaction queue 6 | \state Drop invalid transactions 7 | \end{algorithmic} 8 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/medianAlgorithm.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Median-Algorithm} 3 | \begin{algorithmic} 4 | \require $\mathfrak{E}, s_{sync}$ 5 | \state $T_s \leftarrow \{ \}$ 6 | \for{$B ~\textbf{in}~ \mathfrak{E}_j$} 7 | \state $t_{est}^{B} \leftarrow T_B + $\call{Slot-Offset}{$s_B, s_{sync}$}$ \times \mathcal{T}$ 8 | \state $T_s \leftarrow T_s \cup t_{est}^{B}$ 9 | \endfor 10 | \return \call{Median}{$T_s$} 11 | \end{algorithmic} 12 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/payoutStakersRuntimeFunctionBenchmark.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{"payout\_stakers"` Runtime function benchmark} 3 | \begin{algorithmic} 4 | \Ensure $\mathcal{W}$ 5 | \State \textbf{init} $collection = \{\}$ 6 | \For{$amount \leftarrow 1,MaxNominatorRewardedPerValidator$} 7 | \For{$era\_depth \leftarrow 1,HistoryDepth$} 8 | \State $validator \leftarrow$ \call{Generate-Validator}{} 9 | \State \call{Validate}{$validator$} 10 | \State $nominators \leftarrow$ \call{Generate-Nominators}{$amount$} 11 | \For{$nominator \in nominators$} 12 | \State \call{Nominate}{$validator, nominator$} 13 | \EndFor 14 | \State $era\_index \leftarrow$ \call{Create-Rewards}{$validator, nominators, era\_depth$} 15 | \State $time \leftarrow$ \call{Timer}{\call{Payout-Stakers}{$validator$}$, era\_index$} 16 | \State \call{Add-To}{$collection, time$} 17 | \EndFor 18 | \EndFor 19 | \State $\mathcal{W} \leftarrow$ \call{Compute-Weight}{$collection$} 20 | \Return $\mathcal{W}$ 21 | \end{algorithmic} 22 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/playGrandpaRound.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Play-Grandpa-Round} 3 | \begin{algorithmic} 4 | \REQUIRE($r$) 5 | \STATE $t_{r, v} \leftarrow$ Current local time 6 | \STATE $\textrm{primary} \leftarrow$ \call{Derive-Primary}{$r$} 7 | \IF{$v = \textrm{primary}$} 8 | \STATE \call{Broadcast}{$M_{v}^{r - 1, \textrm{Fin}}($\call{Best-Final-Candidate}{$r - 1$}$)$} 9 | \IF{\call{Best-Final-Candidate}{$r - 1$} $\geqslant$ \textsc{Last-Finalized-Block}} 10 | \STATE \call{Broadcast}{$M_{v}^{r - 1, \textrm{Prim}}($\call{Best-Final-Candidate}{$r - 1$}$)$} 11 | \ENDIF 12 | \ENDIF 13 | \STATE \call{Receive-Messages}{\textbf{until} Time $\geqslant t_{r_,v} + 2 \times T$ \or $r$ \textbf{is} completable} 14 | \STATE $L \leftarrow$ \call{Best-Final-Candidate}{$r - 1$} 15 | \STATE $N \leftarrow$ \call{Best-PreVote-Candidate}{$r$} 16 | \STATE \call{Broadcast}{$M_v^{r, \textrm{pv}} (N)$} 17 | \STATE \call{Receive-Messages}{\textbf{until} $B^{r,\textrm{pv}}_v \geqslant L$ \and $($ Time $\geqslant t_{r_,v} + 4 \times T$ \or $r$ \textbf{is} completable $)$} 18 | \STATE \call{Broadcast}{$M_v^{r, \textrm{pc}}(B_v^{r, \textrm{pv}})$} 19 | 20 | \REPEAT 21 | \STATE \call{Receive-Messages}{} 22 | \STATE \call{Attempt-To-Finalize-At-Round}{$r$} 23 | \UNTIL{$r$ \textbf{is} completable \and \call{Finalizable}{$r$} \and \textsc{Last-Finalized-Block} $\geqslant$ \call{Best-Final-Candidate}{$r - 1$}} 24 | 25 | \STATE \call{Play-Grandpa-round}{$r + 1$} 26 | 27 | \REPEAT 28 | \STATE \call{Receive-Messages}{} 29 | \STATE \call{Attempt-To-Finalize-At-Round}{$r$} 30 | \UNTIL{\textsc{Last-Finalized-Block} $\geqslant$ \call{Best-Final-Candidate}{$r$}} 31 | 32 | \IF{$r > $ \textsc{Last-Completed-Round}} 33 | \STATE \textsc{Last-Completed-Round} $\leftarrow r$ 34 | \ENDIF 35 | \end{algorithmic} 36 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/processCatchupRequest.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Process-Catchup-Request} 3 | \begin{algorithmic} 4 | \input $M_{i, v}^\text{Cat-q}(\text{id}_\mathbb{V}, r)$ 5 | \if{$M_{i, v}^\text{Cat-q}(\text{id}_\mathbb{V}, r).\text{id}_\mathbb{V} \neq \text{id}_\mathbb{V}$} 6 | \state \textbf{error} ``Catching up on different set'' 7 | \endif 8 | \if{$i \notin \mathbb{P}$} 9 | \state \textbf{error} ``Requesting catching up from a non-peer'' 10 | \endif 11 | \if{$r >$ \textsc{Last-Completed-Round}} 12 | \state \textbf{error} ``Catching up on a round in the future'' 13 | \endif 14 | \state \call{Send}{$i, M_{v, i}^\text{Cat-s}(\text{id}_\mathbb{V}, r)$} 15 | \end{algorithmic} 16 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/processCatchupResponse.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Process-Catchup-Response} 3 | \begin{algorithmic} 4 | \input $M_{v,i}^\text{Cat-s}(\text{id}_{\mathbb{V}}, r)$ 5 | \state $M_{v,i}^\text{Cat-s}(\text{id}_{\mathbb{V}}, r).\text{id}_{\mathbb{V}}, r, J^{r, pv}(B), J^{r, pc}(B), H_h(B'), H_i(B') \leftarrow \text{Dec}_{SC}(M_{v, i}^{Cat-s}(\text{id}_{\mathbb{V}}, r)$ 6 | \if{$M_{v, i}^\text{Cat-s}(\text{id}_{\mathbb{V}}, r).\text{id}_{\mathbb{V}} \neq \text{id}_{\mathbb{V}}$} 7 | \state \textbf{error} ``Catching up on different set'' 8 | \endif 9 | \if{$r \leqslant$ \textsc{Leading-Round}} 10 | \state \textbf{error} ``Catching up in to the past'' 11 | \endif 12 | \if{$J^{r, pv}(B)$ \textbf{is not} valid} 13 | \state \textbf{error} ``Invalid pre-vote justification'' 14 | \endif 15 | \if{$J^{r, pc}(B)$ \textbf{is not} valid} 16 | \state \textbf{error} ``Invalid pre-commit justification'' 17 | \endif 18 | \state $G \leftarrow$ \call{GRANDPA-GHOST}{$J^{r, pv}(B)$} 19 | \if{$G = \phi$} 20 | \state \textbf{error} ``GHOST-less Catch-up'' 21 | \endif 22 | \if{$r$ \textbf{is not} completable} 23 | \state \textbf{error} ``Catch-up round is not completable'' 24 | \endif 25 | \if{$J^{r, pc}(B)$ justifies $B'$ finalization} 26 | \state \textbf{error} ``Unjustified Catch-up target finalization'' 27 | \endif 28 | \state \textsc{Last-Completed-Round} $\leftarrow r$ 29 | \if{$i \in \mathbb{V}$} 30 | \state \call{Play-Grandpa-round}{$r + 1$} 31 | \endif 32 | \end{algorithmic} 33 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/queryingStateLightClients.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Querying-State-Light-Clients} 3 | \begin{algorithmic} 4 | \INPUT Query q, BlockHeight h, CommitmentRootHash $root$ 5 | \OUTPUT Maybe Result $res$ 6 | \STATE ($res$, $\pi$) $\leftarrow QueryFullNode (q, h)$ 7 | \IF{$validityCheck_{root}(res, \pi)$} 8 | \STATE $return$ $SOME$ $res$ 9 | \ENDIF 10 | \STATE {$throw$ $ERROR$} 11 | \end{algorithmic} 12 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/requestJudgementRuntimeFunctionBenchmark.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{"request\_judgement"` Runtime function benchmark} 3 | \begin{algorithmic} 4 | \Ensure $\mathcal{W}$ 5 | \State \textbf{init} $collection = \{\}$ 6 | \For{$amount \leftarrow 1,MaxRegistrars$} 7 | \State \call{Generate-Registrars}{$amount$} 8 | \State $caller \leftarrow$ \call{Create-Account}{$caller, 1$} 9 | \State \call{Set-Balance}{$caller, 100$} 10 | \State $time \leftarrow$ \call{Timer}{\call{Request-Judgement}{\call{Random}{$amount$}$, 100$}} 11 | \State \call{Add-To}{$collection, time$} 12 | \EndFor 13 | \State $\mathcal{W} \leftarrow$ \call{Compute-Weight}{$collection$} 14 | \Return $\mathcal{W}$ 15 | \end{algorithmic} 16 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/slotTime.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Slot-Time} 3 | \begin{algorithmic} 4 | \require $s$ 5 | \return{$t_\text{sync} + $\call{Slot-Offset}{$s_{sync}, s$}$ \times \mathcal{T}$} 6 | \end{algorithmic} 7 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/transferRuntimeFunctionBenchmark.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{"transfer" Runtime function benchmark} 3 | \begin{algorithmic} 4 | \Ensure{$collection$: a collection of time measurements of all benchmark iterations} 5 | \Function{Main}{} 6 | \State \textbf{init} $collection = \{ \}$ 7 | \State \textbf{init} $balance = 1'000$ 8 | \For{$index \gets 1,1'000$} 9 | \State $time \leftarrow$ \call{Run-Benchmark}{$index, balance$} 10 | \State \call{Add-To}{$collection, time$} 11 | \EndFor 12 | \State \textbf{init} $index = 1'000$ 13 | \For{$balance \gets 2,1'000$} 14 | \State $time \leftarrow$ \call{Run-Benchmark}{$index, balance$} 15 | \State \call{Add-To}{$collection, time$} 16 | \EndFor 17 | \State $\mathcal{W} \leftarrow$ \call{Compute-Weight}{$collection$} 18 | \Return $\mathcal{W}$ 19 | \EndFunction 20 | \Function{Run-Benchmark}{$index$, $balance$} 21 | \State $sender \leftarrow$ \call{Create-Account}{$caller, index$} 22 | \State $recipient \leftarrow$ \call{Create-Accouny}{$recipient, index$} 23 | \State \call{Set-Balance}{$sender, balance$} 24 | \State $time \leftarrow$ \call{Timer}{\call{Transfer}{$sender, recipient, balance$}} 25 | \Return $time$ 26 | \EndFunction 27 | \end{algorithmic} 28 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/validateTransactionsAndStore.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Validate-Transactions-and-Store} 3 | \begin{algorithmic} 4 | \state $L \leftarrow Dec_{SC}(M_T)$ 5 | \forall{$\{T \in L \mid T \notin TQ \mid T \notin TP\}$} 6 | \state $B_d \leftarrow$ \call{Head}{\call{Longest-Chain}{$BT$}} 7 | \state $N \leftarrow H_n(B_d)$ 8 | \state $R \leftarrow$ \call{Call-Runtime-Entry}{$\texttt{TaggedTransactionQueue\_validate\_transaction}, N, T$} 9 | \if{\call{Valid}{$R$}} 10 | \if{\call{Requires}{$R$}$ \subset \bigcup_{\forall T \in (TQ~\cup~B_i \mid \exists i_{\mid d > i})}$ \call{Provided-Tags}{$T$}} 11 | \state \call{Insert-At}{$TQ, T, $\call{Requires}{$R$}$, $\call{Priority}{$R$}} 12 | \else 13 | \state \call{Add-To}{$TP,T$} 14 | \endif 15 | \state \call{Maintain-Transaction-Pool}{} 16 | \if{\call{ShouldPropagate}{$R$}} 17 | \state \call{Propagate}{$T$} 18 | \endif 19 | \endif 20 | \endfor 21 | \end{algorithmic} 22 | \end{algorithm} 23 | -------------------------------------------------------------------------------- /algorithms/verifyAuthorshipRight.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Verify-Authorship-Right} 3 | \begin{algorithmic} 4 | \require $\text{Head}_{s(B)}$ 5 | \state $s \leftarrow$ \call{Slot-Number-At-Given-Time}{$T_B$} 6 | \state $\mathcal{E}_c \leftarrow$ \call{Current-Epoch}{} 7 | \state $(D_1, \ldots, D_{|H_d(B)|}) \leftarrow H_d(B)$ 8 | \state $D_s \leftarrow D_{|H_d(B)|}$ 9 | \state $H_d(B) \leftarrow \left(D_1, \ldots, D_{|H_d(B)| - 1}\right)$ 10 | \comment{remove the seal from the digest} 11 | \state $(id, \text{Sig}_B)\leftarrow \text{Dec}_{SC}(D_s)$ 12 | \if{$id \neq$ \textsc{Seal-Id}} 13 | \state \textbf{error} ``Seal missing'' 14 | \endif 15 | \state $\text{AuthorID} \leftarrow \text{AuthorityDirectory}^{\mathcal{E}_c}[H_{BABE}(B).\text{SingerIndex}]$ 16 | \state \call{Verify-Signature}{$\text{AuthorID}, H_h(B),\text{Sig}_B$} 17 | \if{$\exists B' \in BT : H_h(B) \neq H_h (B)$ \and $s_B = s_B'$ \and $\text{SignerIndex}_B = \text{SignerIndex}_{B'}$} 18 | \state \textbf{error} ``Block producer is equivocating'' 19 | \endif 20 | \state \call{Verify-Slot-Winner}{$(d_B, \pi_B), s_B, \text{AuthorID}$} 21 | \end{algorithmic} 22 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/verifySlotWinner.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Verify-Slot-Winner} 3 | \begin{algorithmic} 4 | \require $B$ 5 | \state $\mathcal{E}_c \leftarrow$ \textsc{Current-Epoch} 6 | \state $\rho \leftarrow$ \call{Epoch-Randomness}{$c$} 7 | \state \call{Verify-VRF}{$\rho, H_{BABE}(B).(d_B, \pi_B), H_{BABE}(B).s, c$} 8 | \if{$d_B \geqslant \tau$} 9 | \state \textbf{error} ``Block producer is not a winner of the slot'' 10 | \endif 11 | \end{algorithmic} 12 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/warpSyncLightClients.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{Warp-Sync-Light-Clients} 3 | \begin{algorithmic} 4 | \INPUT BlockHeader startblock, the initial block to start the sync. May not be the Genesis Block. 5 | \OUTPUT CommitmentRootHash $root$, State Tries Root hash of the latest finalized Block. 6 | \STATE \textsc{fullnode} $\leftarrow$ SelectFullNode 7 | \STATE \textsc{latestBlockHeader, grandpaJustifications} $\leftarrow$ SyncWithNode(\textsc{fullnode}) 8 | \STATE \textsc{isVerified} $\leftarrow$ verifyAuthoritySetChange(\textsc{grandpaJustifications}) $\land$ verifyFinality(\textsc{latestBlockHeader}) 9 | \IF{\textsc{isVerified}} 10 | \STATE $return$ $SOME$ getCommitmentRootHash(\textsc{latestBlockHeader}) 11 | \ENDIF 12 | \STATE {$throw$ $ERROR$} 13 | \end{algorithmic} 14 | \end{algorithm} -------------------------------------------------------------------------------- /algorithms/withdrawUnbondedRuntimeFunctionBenchmark.tex: -------------------------------------------------------------------------------- 1 | \begin{algorithm} 2 | \caption{"withdraw\_unbonded" Runtime function benchmark} 3 | \begin{algorithmic} 4 | \Ensure $\mathcal{W}$ 5 | \Function{Main}{} 6 | \State \textbf{init} $collection = \{\}$ 7 | \For{$balance \gets 1,100$} 8 | \State $stash \leftarrow$ \call{Create-Account}{$stash, 1$} 9 | \State $controller \leftarrow$ \call{Create-Account}{$controller, 1$} 10 | \State \call{Set-Balance}{$stash, 100$} 11 | \State \call{Set-Balance}{$controller, 1$} 12 | \State \call{Bond}{$stash, controller, balance$} 13 | \State \call{Pass-Era}{} 14 | \State \call{UnBond}{$controller, balance$} 15 | \State \call{Pass-Era}{} 16 | \State $time \leftarrow$ \call{Timer}{\call{Withdraw-Unbonded}{$controller$}} 17 | \State \call{Add-To}{$collection, time$} 18 | \EndFor 19 | \State $\mathcal{W} \leftarrow$ \call{Compute-Weight}{$collection$} 20 | \Return $\mathcal{W}$ 21 | \EndFunction 22 | \end{algorithmic} 23 | \end{algorithm} -------------------------------------------------------------------------------- /bibliography.bib: -------------------------------------------------------------------------------- 1 | @misc{stewart_grandpa_2019, 2 | title = {{GRANDPA}: {A} {Byzantine} {Finality} {Gadget}}, 3 | author = {Stewart, Alistair}, 4 | year = {2019}, 5 | } 6 | 7 | @techreport{w3f_blind_2019, 8 | address = {http://research.web3.foundation/en/latest/polkadot/BABE/Babe/}, 9 | type = {Technical {Specification}}, 10 | title = {Blind {Assignment} for {Blockchain} {Extension}}, 11 | url = {http://research.web3.foundation/en/latest/polkadot/BABE/Babe/}, 12 | institution = {Web 3.0 Foundation}, 13 | author = {W3F Research Group}, 14 | year = {2019} 15 | } 16 | 17 | @inproceedings{david_ouroboros_2018, 18 | title = {Ouroboros praos: {An} adaptively-secure, semi-synchronous proof-of-stake blockchain}, 19 | booktitle = {Annual {International} {Conference} on the {Theory} and {Applications} of {Cryptographic} {Techniques}}, 20 | publisher = {Springer}, 21 | author = {David, Bernardo and Gaži, Peter and Kiayias, Aggelos and Russell, Alexander}, 22 | year = {2018}, 23 | pages = {66--98} 24 | } 25 | 26 | @techreport{collet_extremely_2019, 27 | address = {http://cyan4973.github.io/xxHash/}, 28 | title = {Extremely fast non-cryptographic hash algorithm}, 29 | url = {http://cyan4973.github.io/xxHash/}, 30 | author = {Collet, Yann}, 31 | year = {2019} 32 | } 33 | 34 | @techreport{saarinen_blake2_2015, 35 | address = {https://tools.ietf.org/html/rfc7693}, 36 | type = {{RFC}}, 37 | title = {The {BLAKE}2 cryptographic hash and message authentication code ({MAC})}, 38 | url = {https://tools.ietf.org/html/rfc7693}, 39 | number = {7693}, 40 | institution = {-}, 41 | author = {Saarinen, Markku Juhani and Aumasson, Jean-Philippe}, 42 | year = {2015} 43 | } 44 | 45 | @article{wood_polkadot_2016, 46 | title = {Polkadot: {Vision} for a heterogeneous multi-chain framework}, 47 | journal = {White Paper}, 48 | author = {Wood, Gavin}, 49 | year = {2016}, 50 | } 51 | 52 | @techreport{parity_substrate_2019, 53 | address = {https://substrate.dev/rustdocs/}, 54 | type = {Rust {Doc}}, 55 | title = {Substrate {Reference} {Documentation}}, 56 | url = {https://substrate.dev/rustdocs/}, 57 | institution = {Parity Technologies}, 58 | author = {Parity Technologies}, 59 | year = {2019} 60 | } 61 | 62 | @techreport{protocol_labs_libp2p_2019, 63 | address = {https://github.com/libp2p/specs}, 64 | title = {libp2p {Specification}}, 65 | url = {https://github.com/libp2p/specs}, 66 | institution = {Protocol labs}, 67 | author = {Protocol labs}, 68 | year = {2019} 69 | } 70 | 71 | @techreport{perrin_noise_2018, 72 | address = {https://noiseprotocol.org/noise.html}, 73 | title = {The {Noise} {Protocol} {Framework}}, 74 | url = {https://noiseprotocol.org/noise.html}, 75 | author = {Perrin, Trevor}, 76 | year = {2018} 77 | } 78 | 79 | @article{liusvaara_edwards-curve_2017, 80 | title = {Edwards-{Curve} {Digital} {Signature} {Algorithm} ({EdDSA})}, 81 | author = {Liusvaara, Ilari and Josefsson, Simon}, 82 | year = {2017} 83 | } 84 | 85 | @inproceedings{josefsson_edwards-curve_2017, 86 | title = {Edwards-curve digital signature algorithm ({EdDSA})}, 87 | volume = {8032}, 88 | booktitle = {Internet {Research} {Task} {Force}, {Crypto} {Forum} {Research} {Group}, {RFC}}, 89 | author = {Josefsson, Simon and Liusvaara, Ilari}, 90 | year = {2017} 91 | } 92 | 93 | @techreport{burdges_schnorr_2019, 94 | title = {Schnorr {VRFs} and signatures on the {Ristretto} group}, 95 | url = {https://github.com/w3f/schnorrkel}, 96 | author = {Burdges, Jeff}, 97 | year = {2019} 98 | } 99 | 100 | @techreport{paritytech_genesis_state, 101 | address = {https://github.com/paritytech/polkadot/tree/master/node/service/res/}, 102 | title = {Polkadot {Genisis} {State}}, 103 | url = {https://github.com/paritytech/polkadot/tree/master/node/service/res/}, 104 | author = {Parity Tech}, 105 | year = {2020} 106 | } 107 | 108 | @misc{papadopoulos17, 109 | author = {Dimitrios Papadopoulos and Duane Wessels and Shumon Huque and Moni Naor and Jan Včelák and Leonid Reyzin and Sharon Goldberg}, 110 | title = {Making NSEC5 Practical for DNSSEC}, 111 | howpublished = {Cryptology ePrint Archive, Paper 2017/099}, 112 | year = {2017}, 113 | note = {\url{https://eprint.iacr.org/2017/099}}, 114 | url = {https://eprint.iacr.org/2017/099} 115 | } 116 | 117 | @electronic{goldberg17, 118 | author = {Goldberg, Sharon and Papadopoulos, Dimitrios and Vcelak, Jan}, 119 | edition = {draft-goldbe-vrf-01}, 120 | title = {Internet Draft - Verifiable Random Functions (VRFs)}, 121 | url = {https://tools.ietf.org/id/draft-goldbe-vrf-01.html}, 122 | year = {2017}, 123 | } 124 | 125 | 126 | @electronic{devalence_ed25519zebra_2020, 127 | author = {Henry de Valence}, 128 | title = {Explicitly Defining and Modifying Ed25519 Validation Rules}, 129 | url = {https://github.com/zcash/zips/blob/master/zip-0215.rst}, 130 | year = {2020} 131 | } -------------------------------------------------------------------------------- /docs/chap-overview.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Overview 3 | --- 4 | 5 | The Polkadot Protocol differentiates between different classes of Polkadot Hosts. Each class differs in its trust roots and how active or passively they interact with the network. 6 | 7 | ## -sec-num- Light Client {#sect-client-light} 8 | 9 | The light client is a mostly passive participant in the protocol. Light clients are designed to work in resource-constrained environments like browsers, mobile devices, or even on-chain. Its main objective is to follow the chain, make queries to the full node on specific information on the recent state of the blockchain, and add extrinsics (transactions). It does not maintain the full state but rather queries the full node on the latest finalized state and verifies the authenticity of the responses trustlessly. Details of specifications focused on Light Clients can be found in [Chapter -chap-num-ref-](sect-lightclient). 10 | 11 | ## -sec-num- Full Node {#sect-node-full} 12 | 13 | While the full node is still a mostly passive participant of the protocol, they follow the chain by receiving and verifying every block in the chain. It maintains a full state of the blockchain by executing the extrinsics in blocks. Their role in the consensus mechanism is limited to following the chain and not producing the blocks. 14 | 15 | - **Functional Requirements:** 16 | 17 | 1. The node must populate the state storage with the official genesis state, elaborated further in [Section -sec-num-ref-](id-cryptography-encoding#section-genesis). 18 | 19 | 2. The node should maintain a set of around 50 active peers at any time. New peers can be found using the discovery protocols ([Section -sec-num-ref-](chap-networking#sect-network-bootstrap)) 20 | 21 | 3. The node should open and maintain the various required streams ([Section -sec-num-ref-](chap-networking#sect-protocols-substreams)) with each of its active peers. 22 | 23 | 4. Furthermore, the node should send block requests ([Section -sec-num-ref-](chap-networking#sect-msg-block-request)) to these peers to receive all blocks in the chain and execute each of them. 24 | 25 | 5. The node should exchange neighbor packets ([Section -sec-num-ref-](chap-networking#sect-grandpa-neighbor-msg)). 26 | 27 | ## -sec-num- Authoring Node {#sect-node-authoring} 28 | 29 | The authoring node covers all the features of the full node, but instead of just passively following the protocol, it is an active participant, producing blocks and voting in Grandpa. 30 | 31 | - **Functional Requirements:** 32 | 33 | 1. Verify that the Host’s session key is included in the current Epoch’s authority set ([Section -sec-num-ref-](chap-sync#sect-authority-set)). 34 | 35 | 2. Run the BABE lottery ([Chapter -chap-num-ref-](sect-block-production)) and wait for the next assigned slot in order to produce a block. 36 | 37 | 3. Gossip any produced blocks to all connected peers ([Section -sec-num-ref-](chap-networking#sect-msg-block-announce)). 38 | 39 | 4. Run the catch-up protocol ([Section -sec-num-ref-](sect-finality#sect-grandpa-catchup)) to make sure that the node is participating in the current round and not a past round. 40 | 41 | 5. Run the GRANDPA rounds protocol ([Chapter -chap-num-ref-](sect-finality)). 42 | 43 | ## -sec-num- Relaying Node {#sect-node-relaying} 44 | 45 | The relaying node covers all the features of the authoring node but also participants in the availability and validity process to process new parachain blocks as described in [Chapter -chap-num-ref-](chapter-anv). 46 | -------------------------------------------------------------------------------- /docs/chap-sync.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Synchronization 3 | --- 4 | 5 | import Pseudocode from '@site/src/components/Pseudocode'; 6 | import importAndValidateBlock from '!!raw-loader!@site/src/algorithms/importAndValidateBlock.tex'; 7 | 8 | Many applications that interact with the Polkadot network, to some extent, must be able to retrieve certain information about the network. Depending on the utility, this includes validators that interact with Polkadot’s consensus and need access to the full state, either from the past or just the most up-to-date state, or light clients that are only interested in the minimum information required in order to verify some claims about the state of the network, such as the balance of a specific account. To allow implementations to quickly retrieve the required information, different types of synchronization protocols are available, respectively Full, Fast, and Warp sync suited for different needs. 9 | 10 | The associated network messages are specified in [Section -sec-num-ref-](chap-networking#sect-network-messages). 11 | 12 | ## -sec-num- Warp Sync {#sect-sync-warp} 13 | 14 | Warp sync ([Section -sec-num-ref-](chap-networking#sect-msg-warp-sync)) only downloads the block headers where authority set changes occurred, so-called fragments ([Definition -def-num-ref-](chap-networking#defn-warp-sync-proof)), and by verifying the GRANDPA justifications ([Definition -def-num-ref-](chap-networking#defn-grandpa-justifications-compact)). This protocol allows nodes to arrive at the desired state much faster than fast sync. 15 | 16 | ## -sec-num- Fast Sync {#sect-sync-fast} 17 | 18 | Fast sync works by downloading the block header history and validating the authority set changes ([Section -sec-num-ref-](chap-sync#sect-authority-set)) in order to arrive at a specific (usually the most recent) header. After the desired header has been reached and verified, the state can be downloaded and imported ([Section -sec-num-ref-](chap-networking#sect-msg-state-request)). Once this process has been completed, the node will automatically switch to a full sync. 19 | 20 | ## -sec-num- Full Sync {#id-full-sync} 21 | 22 | The full sync protocol is the "default" protocol that’s suited for many types of implementations, such as archive nodes (nodes that store everything), validators that participate in Polkadots consensus and light clients that only verify claims about the state of the network. Full sync works by listening to announced blocks ([Section -sec-num-ref-](chap-networking#sect-msg-block-announce)) and requesting the blocks from the announcing peers or just the block headers in case of light clients. 23 | 24 | The full sync protocol usually downloads the entire chain, but no such requirements must be met. If an implementation only wants the latest, finalized state, it can combine it with protocols such as fast sync ([Section -sec-num-ref-](chap-sync#sect-sync-fast)) and/or warp sync ([Section -sec-num-ref-](chap-sync#sect-sync-warp)) to make synchronization as fast as possible. 25 | 26 | ### -sec-num- Consensus Authority Set {#sect-authority-set} 27 | 28 | Because Polkadot is a proof-of-stake protocol, each of its consensus engines has its own set of nodes represented by known public keys, which have the authority to influence the protocol in pre-defined ways explained in this Section. To verify the validity of each block, the Polkadot node must track the current list of authorities ([Definition -def-num-ref-](chap-sync#defn-authority-list)) for that block. 29 | 30 | ###### Definition -def-num- Authority List {#defn-authority-list} 31 | :::definition 32 | 33 | The **authority list** of block ${B}$ for consensus engine ${C}$ noted as $\text{Auth}_{{C}}{\left({B}\right)}$ is an array that contains the following pair of types for each of its authorities ${A}\in\text{Auth}_{{C}}{\left({B}\right)}$: 34 | 35 | $$ 36 | {\left({p}{k}_{{A}},{w}_{{A}}\right)} 37 | $$ 38 | 39 | ${p}{k}_{{A}}$ is the session public key ([Definition -def-num-ref-](id-cryptography-encoding#defn-session-key)) of authority ${A}$. And ${w}_{{A}}$ is an unsigned 64-bit integer indicating the authority weight. The value of $\text{Auth}_{{C}}{\left({B}\right)}$ is part of the Polkadot state. The value for $\text{Auth}_{{C}}{\left({B}_{{0}}\right)}$ is set in the genesis state ([Section -sec-num-ref-](id-cryptography-encoding#section-genesis)) and can be retrieved using a runtime entrypoint corresponding to consensus engine ${C}$. 40 | 41 | The authorities and their corresponding weights can be retrieved from the Runtime ([Section -sec-num-ref-](chap-runtime-api#sect-rte-grandpa-auth)). 42 | 43 | ::: 44 | 45 | :::info 46 | In Polkadot, the authorities are unweighted, i.e., the weights for all authorities are set to 1. The proportionality in terms of stakes is managed by the [NPOS](https://wiki.polkadot.network/docs/learn-phragmen) (Nominated Proof-of-Stake) algorithm in Polkadot. Once validators are elected for an era using the NPOS algorithm, they are considered equal in the BABE and GRANDPA consensus algorithms. 47 | 48 | ::: 49 | 50 | ### -sec-num- Runtime-to-Consensus Engine Message {#sect-consensus-message-digest} 51 | 52 | The authority list ([Definition -def-num-ref-](chap-sync#defn-authority-list)) is part of the Polkadot state, and the Runtime has the authority to update this list in the course of any state transitions. The Runtime informs the corresponding consensus engine about the changes in the authority set by adding the appropriate consensus message in the form of a digest item ([Definition -def-num-ref-](chap-state#defn-digest)) to the block header of block ${B}$ which caused the transition in the authority set. 53 | 54 | The Polkadot Host must inspect the digest header of each block and delegate consensus messages to their consensus engines. The BABE and GRANDPA consensus engine must react based on the type of consensus messages it receives. The active GRANDPA authorities can only vote for blocks that occurred after the finalized block in which they were selected. Any votes for blocks before they came into effect would get rejected. 55 | 56 | ## -sec-num- Importing and Validating Block {#sect-block-validation} 57 | 58 | Block validation is the process by which a node asserts that a block is fit to be added to the blockchain. This means that the block is consistent with the current state of the system and transitions to a new valid state. 59 | 60 | New blocks can be received by the Polkadot Host via other peers ([Section -sec-num-ref-](chap-networking#sect-msg-block-request)) or from the Host’s own consensus engine ([Chapter -chap-num-ref-](sect-block-production)). Both the Runtime and the Polkadot Host then need to work together to assure block validity. A block is deemed valid if the block author had authorship rights for the slot in which the block was produced as well as if the transactions in the block constitute a valid transition of states. The former criterion is validated by the Polkadot Host according to the block production consensus protocol. The latter can be verified by the Polkadot Host invoking entry into the Runtime as ([Section -sec-num-ref-](chap-runtime-api#sect-rte-core-execute-block)) as a part of the validation process. Any state changes created by this function on successful execution are persisted. 61 | 62 | The Polkadot Host implements [Import-and-Validate-Block](chap-sync#algo-import-and-validate-block) to assure the validity of the block. 63 | 64 | ###### Algorithm -algo-num- Import-and-Validate-Block {#algo-import-and-validate-block} 65 | :::algorithm 66 | 71 | 72 | **where** 73 | - $\text{Remove-Seal}$ removes the Seal digest from the block ([Definition -def-num-ref-](chap-state#defn-digest)) before submitting it to the Runtime. 74 | 75 | - $\text{Add-Seal}$ adds the Seal digest back to the block ([Definition -def-num-ref-](chap-state#defn-digest)) for later propagation. 76 | 77 | - $\text{Persist-State}$ implies the persistence of any state changes created by ${\mathtt{\text{Core\_execute\_block}}}$ ([Section -sec-num-ref-](chap-runtime-api#sect-rte-core-execute-block)) on successful execution. 78 | 79 | - $\text{PBT}$ is the pruned block tree ([Definition -def-num-ref-](chap-state#defn-block-tree)). 80 | 81 | - $\text{Verify-Authorship-Right}$ is part of the block production consensus protocol and is described in [Verify-Authorship-Right](sect-block-production#algo-verify-authorship-right). 82 | 83 | - *Finalized block* and *finality* are defined in [Chapter -chap-num-ref-](sect-finality). 84 | ::: 85 | -------------------------------------------------------------------------------- /docs/id-consensus.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Consensus 3 | --- 4 | 5 | import Pseudocode from '@site/src/components/Pseudocode'; 6 | import epochRandomness from '!!raw-loader!@site/src/algorithms/epochRandomness.tex'; 7 | 8 | ## -sec-num- BABE digest messages {#id-babe-digest-messages} 9 | 10 | The Runtime is required to provide the BABE authority list and randomness to the host via a consensus message in the header of the first block of each epoch. 11 | 12 | The digest published in Epoch ${\mathcal{{{E}}}}_{{n}}$ is enacted in ${\mathcal{{{E}}}}_{{{n}+{1}}}$. The randomness in this digest is computed based on all the VRF outputs up to including Epoch ${\mathcal{{{E}}}}_{{{n}-{2}}}$ while the authority set is based on all transaction included up to Epoch ${\mathcal{{{E}}}}_{{{n}-{1}}}$. 13 | 14 | The computation of the randomness seed is described in [Epoch-Randomness](id-consensus#algo-epoch-randomness), which uses the concept of epoch subchain as described in host specification and the value ${d}_{{B}}$, which is the VRF output computed for slot ${s}_{{B}}$. 15 | 16 | ###### Algorithm -algo-num- Epoch Randomness {#algo-epoch-randomness} 17 | :::algorithm 18 | 23 | 24 | where ${n}$ is the epoch index. 25 | ::: 26 | -------------------------------------------------------------------------------- /docs/id-extrinsics.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Extrinsics 3 | --- 4 | 5 | import Pseudocode from '@site/src/components/Pseudocode'; 6 | import encodeMortality from '!!raw-loader!@site/src/algorithms/encodeMortality.tex'; 7 | import decodeMortality from '!!raw-loader!@site/src/algorithms/decodeMortality.tex'; 8 | 9 | ## -sec-num- Introduction {#id-introduction-5} 10 | 11 | An extrinsic is a SCALE encoded array consisting of a version number, signature, and varying data types indicating the resulting Runtime function to be called, including the parameters required for that function to be executed. 12 | 13 | ## -sec-num- Preliminaries {#id-preliminaries-3} 14 | 15 | ###### Definition -def-num- Extrinsic {#defn-extrinsic} 16 | :::definition 17 | 18 | An extrinsic , ${t}{x}$, is a tuple consisting of the extrinsic version, ${T}_{{v}}$ ([Definition -def-num-ref-](id-extrinsics#defn-extrinsic-version)), and the body of the extrinsic, ${T}_{{b}}$. 19 | 20 | $$ 21 | {t}{x}\:={\left({T}_{{v}},{T}_{{b}}\right)} 22 | $$ 23 | 24 | The value of ${T}_{{b}}$ varies for each version. The current version 4 is described in [Section -sec-num-ref-](id-extrinsics#sect-version-four). 25 | 26 | ::: 27 | ###### Definition -def-num- Extrinsic Version {#defn-extrinsic-version} 28 | :::definition 29 | 30 | ${T}_{{v}}$ is a 8-bit bitfield and defines the extrinsic version. The required format of an extrinsic body, ${T}_{{b}}$, is dictated by the Runtime. Older or unsupported versions are rejected. 31 | 32 | The most significant bit of ${T}_{{v}}$ indicates whether the transaction is **signed** (${1}$) or **unsigned** (${0}$). The remaining 7-bits represent the version number. As an example, for extrinsic format version 4, a signed extrinsic represents ${T}_{{v}}$ as `132` while an unsigned extrinsic represents it as `4`. 33 | 34 | ::: 35 | ## -sec-num- Extrinsics Body {#id-extrinsics-body} 36 | 37 | ### -sec-num- Version 4 {#sect-version-four} 38 | 39 | Version 4 of the Polkadot extrinsic format is defined as follows: 40 | 41 | $$ 42 | {T}_{{b}}\:={\left({A}_{{i}},{Sig},{E},{M}_{{i}},{F}_{{i}}{\left({m}\right)}\right)} 43 | $$ 44 | 45 | **where** 46 | - ${A}_{{i}}$ is the multi-address of the sender defined in [Definition -def-num-ref-](id-extrinsics#defn-extrinsic-address). 47 | 48 | - ${Sig}$: the signature of the sender ([Definition -def-num-ref-](id-extrinsics#defn-extrinsic-signature)). 49 | 50 | - ${E}$: the extra data for the extrinsic ([Definition -def-num-ref-](id-extrinsics#defn-extra-data)). 51 | 52 | - ${M}_{{i}}$: the indicator of the Polkadot module ([Definition -def-num-ref-](id-extrinsics#defn-module-indicator)). 53 | 54 | - ${F}_{{i}}{\left({m}\right)}$: the indicator of the function of the Polkadot module ([Definition -def-num-ref-](id-extrinsics#defn-function-indicator)). 55 | 56 | ###### Definition -def-num- Extrinsic Address {#defn-extrinsic-address} 57 | :::definition 58 | 59 | Account Id, ${A}_{{i}}$, is the 32-byte address of the sender of the extrinsic as described in the [external SS58 address format](https://github.com/paritytech/substrate/wiki/External-Address-Format-(SS58)). 60 | 61 | ::: 62 | ###### Definition -def-num- Extrinsic Signature {#defn-extrinsic-signature} 63 | :::definition 64 | 65 | The signature, ${Sig}$, is a varying data type indicating the used signature type, followed by the signature created by the extrinsic author. The following types are supported: 66 | 67 | $$ 68 | Sig := 69 | \begin{cases} 70 | 0, & \text{Ed25519, followed by: } (b_0, \ldots,b_{63}) \\ 71 | 1, & \text{Sr25519, followed by: } (b_0, \ldots,b_{63}) \\ 72 | 2, & \text{Ecdsa, followed by: } (b_0, \ldots,b_{64}) 73 | \end{cases} 74 | $$ 75 | 76 | 77 | Signature types vary in size, but each individual type is always fixed-size and therefore does not contain a length prefix. `Ed25519` and `Sr25519` signatures are 512-bit while `Ecdsa` is 520-bit, where the last 8 bits are the recovery ID. 78 | 79 | The signature is created by signing payload ${P}$. 80 | 81 | $$ 82 | \begin{aligned} 83 | P &:= \begin{cases} 84 | Raw, & \text{if } \|Raw\| \leq 256 \\ 85 | \text{Blake2}(Raw), & \text{if } \|Raw\| > 256 \\ 86 | \end{cases} \\ 87 | Raw &:= (M_i, F_i(m), E, R_v, F_v, H_h(G), H_h(B)) 88 | \end{aligned} 89 | $$ 90 | 91 | **where** 92 | - ${M}_{{i}}$: the module indicator ([Definition -def-num-ref-](id-extrinsics#defn-module-indicator)). 93 | 94 | - ${F}_{{i}}{\left({m}\right)}$: the function indicator of the module ([Definition -def-num-ref-](id-extrinsics#defn-function-indicator)). 95 | 96 | - ${E}$: the extra data ([Definition -def-num-ref-](id-extrinsics#defn-extra-data)). 97 | 98 | - ${R}_{{v}}$: a UINT32 containing the specification version (`spec_version`) of the Runtime ([Section -sec-num-ref-](chap-runtime-api#defn-rt-core-version)), which can be updated and is therefore subject to change. 99 | 100 | - ${F}_{{v}}$: a UINT32 containing the transaction version (`transaction_version`) of the Runtime ([Section -sec-num-ref-](chap-runtime-api#defn-rt-core-version)), which can be updated and is therefore subject to change. 101 | 102 | - ${H}_{{h}}{\left({G}\right)}$: a 32-byte array containing the genesis hash. 103 | 104 | - ${H}_{{h}}{\left({B}\right)}$: a 32-byte array containing the hash of the block which starts the mortality period, as described in [Definition -def-num-ref-](id-extrinsics#defn-extrinsic-mortality). 105 | 106 | ::: 107 | ###### Definition -def-num- Extra Data {#defn-extra-data} 108 | :::definition 109 | 110 | Extra data, ${E}$, is a tuple containing additional metadata about the extrinsic and the system it is meant to be executed in. 111 | 112 | $$ 113 | {E}\:={\left({T}_{mor},{N},{P}_{{t}}\right)} 114 | $$ 115 | 116 | **where** 117 | - ${T}_{mor}$: contains the SCALE encoded mortality of the extrinsic ([Definition -def-num-ref-](id-extrinsics#defn-extrinsic-mortality)). 118 | 119 | - ${N}$: a compact integer containing the nonce of the sender. The nonce must be incremented by one for each extrinsic created, otherwise, the Polkadot network will reject the extrinsic. 120 | 121 | - ${P}_{{t}}$: a compact integer containing the transactor pay including tip. 122 | 123 | ::: 124 | ###### Definition -def-num- Module Indicator {#defn-module-indicator} 125 | :::definition 126 | 127 | ${M}_{{i}}$ is an indicator for the Runtime to which Polkadot *module*, ${m}$, the extrinsic should be forwarded to. 128 | 129 | ${M}_{{i}}$ is a varying data type pointing to every module exposed to the network. 130 | 131 | $$ 132 | M_i := \begin{cases} 133 | 0, & \text{System} \\ 134 | 1, & \text{Utility} \\ 135 | \ldots & \\ 136 | 7, & \text{Balances} \\ 137 | \ldots & 138 | \end{cases} 139 | $$ 140 | 141 | ::: 142 | ###### Definition -def-num- Function Indicator {#defn-function-indicator} 143 | :::definition 144 | 145 | ${F}_{{i}}{\left({m}\right)}$ is a tuple which contains an indicator, ${m}_{{i}}$, for the Runtime to which *function* within the Polkadot *module*, ${m}$, the extrinsic should be forwarded to. This indicator is followed by the concatenated and SCALE encoded parameters of the corresponding function, ${p}{a}{r}{a}{m}{s}$. 146 | 147 | $$ 148 | {F}_{{i}}{\left({m}\right)}\:={\left({m}_{{i}},{p}{a}{r}{a}{m}{s}\right)} 149 | $$ 150 | 151 | The value of ${m}_{{i}}$ varies for each Polkadot module since every module offers different functions. As an example, the `Balances` module has the following functions: 152 | 153 | $$ 154 | Balances_i := \begin{cases} 155 | 0, & \text{transfer} \\ 156 | 1, & \text{set\_balance} \\ 157 | 2, & \text{force\_transfer} \\ 158 | 3, & \text{transfer\_keep\_alive} \\ 159 | \ldots & 160 | \end{cases} 161 | $$ 162 | 163 | ::: 164 | ### -sec-num- Mortality {#id-mortality} 165 | 166 | ###### Definition -def-num- Extrinsic Mortality {#defn-extrinsic-mortality} 167 | :::definition 168 | 169 | Extrinsic **mortality** is a mechanism which ensures that an extrinsic is only valid within a certain period of the ongoing Polkadot lifetime. Extrinsics can also be immortal, as clarified in [Section -sec-num-ref-](id-extrinsics#sect-mortality-encoding). 170 | 171 | The mortality mechanism works with two related values: 172 | 173 | - ${M}_{{{per}}}$: the period of validity in terms of block numbers from the block hash specified as ${H}_{{h}}{\left({B}\right)}$ in the payload ([Definition -def-num-ref-](id-extrinsics#defn-extrinsic-signature)). The requirement is ${M}_{{{per}}}\geq{4}$ and ${M}_{{{per}}}$ must be the power of two, such as `32`, `64`, `128`, etc. 174 | 175 | - ${M}_{{{pha}}}$: the phase in the period that this extrinsic’s lifetime begins. This value is calculated with a formula, and validators can use this value in order to determine which block hash is included in the payload. The requirement is ${M}_{{{pha}}}<{M}_{{{per}}}$. 176 | 177 | In order to tie a transaction’s lifetime to a certain block (${H}_{{i}}{\left({B}\right)}$) after it was issued, without wasting precious space for block hashes, block numbers are divided into regular periods and the lifetime is instead expressed as a "phase" (${M}_{{{pha}}}$) from these regular boundaries: 178 | 179 | $$ 180 | {M}_{pha}={H}_{{i}}{\left({B}\right)} \; mod \; {M}_{{{per}}} 181 | $$ 182 | 183 | ${M}_{{{per}}}$ and ${M}_{{{pha}}}$ are then included in the extrinsic, as clarified in [Definition -def-num-ref-](id-extrinsics#defn-extra-data), in the SCALE encoded form of ${T}_{mor}$ ([Section -sec-num-ref-](id-extrinsics#sect-mortality-encoding)). Polkadot validators can use ${M}_{{{pha}}}$ to figure out the block hash included in the payload, which will therefore result in a valid signature if the extrinsic is within the specified period or an invalid signature if the extrinsic "died". 184 | 185 | ::: 186 | #### -sec-num- Example {#id-example} 187 | 188 | The extrinsic author choses ${M}_{{{per}}}={256}$ at block `10'000`, resulting with ${M}_{{{pha}}}={16}$. The extrinsic is then valid for blocks ranging from `10'000` to `10'256`. 189 | 190 | #### -sec-num- Encoding {#sect-mortality-encoding} 191 | 192 | ${T}_{mor}$ refers to the SCALE encoded form of type ${M}_{{{per}}}$ and ${M}_{{{pha}}}$. ${T}_{mor}$ is the size of two bytes if the extrinsic is considered mortal, or simply one bytes with a value equal to zero if the extrinsic is considered immortal. 193 | 194 | $$ 195 | {T}_{mor}\:={E}{n}{c}_{{{S}{C}}}{\left({M}_{{{per}}},{M}_{{{pha}}}\right)} 196 | $$ 197 | 198 | The SCALE encoded representation of mortality ${T}_{mor}$ deviates from most other types, as it’s specialized to be the smallest possible value, as described in [Encode Mortality](id-extrinsics#algo-mortality-encode) and [Decode Mortality](id-extrinsics#algo-mortality-decode). 199 | 200 | If the extrinsic is immortal, specify a single byte with a value equal to zero. 201 | 202 | :::algorithm 203 | ###### Algorithm -algo-num- Encode Mortality {#algo-mortality-encode} 204 | 209 | 210 | ###### Algorithm -algo-num- Decode Mortality {#algo-mortality-decode} 211 | 216 | 217 | **where** 218 | - ${T}^{{{b}{0}}}_{\left\lbrace{mor}\right\rbrace}$: the first byte of ${T}_{mor}$. 219 | 220 | - ${T}^{{{b}{1}}}_{\left\lbrace{mor}\right\rbrace}$: the second byte of ${T}_{mor}$. 221 | 222 | - Limit(${num}$, ${min}$, ${max}$): Ensures that ${num}$ is between ${min}$ and ${max}$. If ${min}$ or ${max}$ is defined as $\phi$, then there is no requirement for the specified minimum/maximum. 223 | 224 | - TZ(${num}$): returns the number of trailing zeros in the binary representation of ${num}$. For example, the binary representation of `40` is `0010 1000`, which has three trailing zeros. 225 | 226 | - $\gg$: performs a binary right shift operation. 227 | 228 | - $\ll$: performs a binary left shift operation. 229 | 230 | - ${\mid}$ : performs a bitwise OR operation. 231 | ::: 232 | -------------------------------------------------------------------------------- /docs/id-glossary.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Glossary" 3 | --- 4 | 5 | ${P}_{{n}}$ 6 | - A path graph or a path of ${n}$ nodes. 7 | 8 | ${\left({b}_{{0}},{b}_{{1}},\ldots,{b}_{{{n}-{1}}}\right)}$ 9 | - A sequence of bytes or byte array of length ${n}$ 10 | 11 | $𝔹_{{n}}$ 12 | - A set of all byte arrays of length ${n}$ 13 | 14 | ${I}={\left({B}_{{n}}…{B}_{{0}}\right)}_{{{256}}}$ 15 | - A non-negative integer in base 256 16 | 17 | ${B}={\left({b}_{{0}},{b}_{{1}},…,{b}_{{n}}\right)}$ 18 | - The little-endian representation of a non-negative interger ${I}={\left({B}_{{n}}…{B}_{{0}}\right)}_{{{256}}}$ such that ${b}_{{i}}≔{B}_{{i}}$ 19 | 20 | $\textrm{Enc}_{LE}$ 21 | - The little-endian encoding function. 22 | 23 | ${C}$ 24 | - A blockchain is defined as a directed path graph. 25 | 26 | **Block**
27 | - A node of the directed path graph (blockchain) C 28 | 29 | **Genesis Block**
30 | - The unique sink of blockchain C 31 | 32 | **Head**
33 | - The source of blockchain C 34 | 35 | ${P}{\left({B}\right)}$ 36 | - The parent of block ${B}$ 37 | 38 | **UNIX time**
39 | - The number of milliseconds that have elapsed since the Unix epoch as a 64-bit integer 40 | 41 | ${BT}$ 42 | - The block tree of a blockchain 43 | 44 | ${G}$ 45 | - The genesis block, the root of the block tree BT 46 | 47 | $\textrm{CHAIN}({B})$ 48 | - The path graph from ${G}$ to ${B}$ in ${BT}$. 49 | 50 | ${Head}{\left({C}\right)}$ 51 | - The head of chain C. 52 | 53 | ${\left|{C}\right|}$ 54 | - The length of chain ${C}$ as a path graph 55 | 56 | $\textrm{SubChain}{\left({B}',{B}\right)}$ 57 | - The subgraph of ${Chain}{\left({B}\right)}$ path graph containing both ${B}$ and ${B}'$. 58 | 59 | $ℂ_{{B}}{\left({BT}\right)}$ 60 | - The set of all subchains of ${BT}$ rooted at block ${B}$. 61 | 62 | $ℂ,ℂ{\left({BT}\right)}$ 63 | - $ℂ_{{G}}{\left({BT}\right)}$ i.e. the set of all chains of ${BT}$ rooted at genesis block 64 | 65 | $\textrm{Longest-Chain}{\left({BT}\right)}$ 66 | - The longest sub path graph of ${BT}$ i.e. ${C}:{\left|{C}\right|}=\max_{{{C}_{{i}}∈ℂ}}{\left|{C}_{{i}}\right|}$ 67 | 68 | $\textrm{Longest-Path}{\left({BT}\right)}$ 69 | - The longest sub path graph of ${\left({P}\right)}{BT}$ with earliest block arrival time 70 | 71 | $\textrm{Deepest-Leaf}{\left({BT}\right)}$ 72 | - $\textrm{Head}{\textrm{Longest-Path}}{(BT)}$ i.e. the head of $\textrm{Longest-Path}{\left({BT}\right)}$ 73 | 74 | ${B}>{B}'$ 75 | - ${B}$ is a descendant of ${B}'$ in the block tree 76 | 77 | $\textrm{StoredValue}{\left({k}\right)}$ 78 | - The function to retrieve the value stored under a specific key in the state storage. 79 | 80 | **State trie, trie**
81 | - The Merkle radix-16 Tree, which stores hashes of storage entries. 82 | 83 | $\textrm{KeyEncode}{\left({k}\right)}$ 84 | - The function to encode keys for labeling branches of the trie. 85 | 86 | $𝒩$ 87 | - The set of all nodes in the Polkadot state trie. 88 | 89 | ${N}$ 90 | - An individual node in the trie. 91 | 92 | $𝒩_{{b}}$ 93 | - A branch node of the trie which has at least one and at most 16 children 94 | 95 | $𝒩_{{l}}$ 96 | - A childless leaf node of the trie 97 | 98 | ${pk}_{{N}}^{{{Agr}}}$ 99 | - The aggregated prefix key of node N 100 | 101 | ${pk}_{{N}}$ 102 | - The (suffix) partial key of node N 103 | 104 | $\textrm{Index}_{{N}}$ 105 | - A function returning an integer in range of {0, . . . ,15} representing the index of a child node of node ${N}$ among the children of ${N}$ 106 | 107 | ${v}_{{N}}$ 108 | - Node value containing the header of node ${N}$, its partial key and the digest of its childern values 109 | 110 | ${Head}_{{N}}$ 111 | - The node header of trie node ${N}$ storing information about the node’s type and kay 112 | 113 | ${H}{\left({N}\right)}$ 114 | - The Merkle value of node ${N}$. 115 | 116 | $\textrm{ChildrenBitmap}$ 117 | - The binary function indicates which child of a given node is present in the trie. 118 | 119 | ${sv}_{{N}}$ 120 | - The subvalue of a trie node ${N}$. 121 | 122 | **Child storage**
123 | - A sub storage of the state storage which has the same structure, although being stored separately 124 | 125 | **Child trie**
126 | - State trie of a child storage 127 | 128 | **Transaction Queue**
129 | - See [Definition -def-num-ref-](chap-state#defn-transaction-queue). 130 | 131 | ${H}_{{p}}$ 132 | - The 32-byte Blake2b hash of the header of the parent of the block. 133 | 134 | ${H}_{{i}},{H}_{{i}}{\left({B}\right)}$ 135 | - Block number, the incremental integer index of the current block in the chain. 136 | 137 | ${H}_{{r}}$ 138 | - The hash of the root of the Merkle trie of the state storage at a given block 139 | 140 | ${H}_{{e}}$ 141 | - An auxiliary field in the block header used by Runtime to validate the integrity of the extrinsics composing the block body. 142 | 143 | ${H}_{{d}}$, ${H}_{{d}}{\left({B}\right)}$ 144 | - A block header used to store any chain-specific auxiliary data. 145 | 146 | ${H}_{{h}}{\left({B}\right)}$ 147 | - The hash of the header of block ${B}$ 148 | 149 | ${Body}{\left({B}\right)}$ 150 | - The body of block ${B}$ consisting of a set of extrinsics 151 | 152 | ${M}^{{r},{stage}}_{v}$ 153 | - Vote message broadcasted by the voter v as part of the finality protocol 154 | 155 | ${M}_{{v}}^{{{r},{Fin}}}{\left({B}\right)}$ 156 | - The commit message broadcasted by voter ${v}$ indicating that they have finalized bock ${B}$ in round ${r}$ 157 | 158 | ${v}$ 159 | - GRANDPA voter node, which casts votes in the finality protocol 160 | 161 | ${{k}_{{v}}^{pr}}$ 162 | - The private key of voter ${v}$ 163 | 164 | ${v}_{{{i}{d}}}$ 165 | - The public key of voter ${v}$ 166 | 167 | $𝕍_{{B}},𝕍$ 168 | - The set of all GRANDPA voters for at block ${B}$ 169 | 170 | ${GS}$ 171 | - GRANDPA protocol state consisting of the set of voters, the number of times voters set has changed, and the current round number. 172 | 173 | ${r}$ 174 | - The voting round counter in the finality protocol 175 | 176 | ${V}_{{{B}}}$ 177 | - A GRANDPA vote casted in favor of block B 178 | 179 | ${{V}_{{v}}^{{{r},{pv}}}}$ 180 | - A GRANDPA vote casted by voter ${v}$ during the pre-vote stage of round ${r}$ 181 | 182 | ${{V}_{{v}}^{{{r},{pc}}}}$ 183 | - A GRANDPA vote casted by voter ${v}$ during the pre-commit stage of round ${r}$ 184 | 185 | ${J}^{{{r},{stage}}}{\left({B}\right)}$ 186 | - The justification for pre-committing or committing to block ${B}$ in round ${r}$ of finality protocol 187 | 188 | ${Sign}^{{{r},{stage}}}_{\left\lbrace{v}_{{i}}\right\rbrace}{\left({B}\right)}$ 189 | - The signature of voter ${v}$ on their vote to block B, broadcasted during the specified stage of finality round ${r}$ 190 | 191 | $ℰ^{{{r},{stage}}}$ 192 | - The set of all equivocator voters in sub-round ‘‘stage'' of round ${r}$ 193 | 194 | $ℰ^{{{r},{stage}}}_{\left\lbrace{o}{b}{s}{\left({v}\right)}\right\rbrace}$ 195 | - The set of all equivocator voters in sub-round ‘‘stage'' of round ${r}$ observed by voter ${v}$ 196 | 197 | ${V}{D}^{{{r},{stage}}}_{\left\lbrace{o}{b}{s}{\left({v}\right)}\right\rbrace}{\left({B}\right)}$ 198 | - The set of observed direct votes for block B in round ${r}$ 199 | 200 | ${V}^{{{r},{stage}}}_{\left\lbrace{o}{b}{s}{\left({v}\right)}\right\rbrace}$ 201 | - The set of total votes observed by voter v in sub-round ‘‘stage'' of round r 202 | 203 | ${V}^{{{r},{stage}}}_{\left\lbrace{o}{b}{s}{\left({v}\right)}\right\rbrace}{\left({B}\right)}$ 204 | - The set of all observed votes by ${v}$ in the sub-round “stage” of round ${r}$ (directly or indirectly) for block ${B}$ 205 | 206 | ${B}^{{{r},{pv}}}_{v}$ 207 | - The currently pre-voted block in round ${r}$. The GRANDPA GHOST of round ${r}$ 208 | 209 | **Account key**, ${\left({sk}^{{a}},{pk}^{{a}}\right)}$
210 | - A key pair of types accepted by the Polkadot protocol which can be used to sign transactions 211 | 212 | ${Enc}_{{{SC}}}{\left({A}\right)}$ 213 | - SCALE encoding of value ${A}$ 214 | 215 | ${T}≔{\left({A}_{{1}},\ldots,{A}_{{n}}\right)}$ 216 | - A tuple of values ${A}_{{i}}$'s each of different type 217 | 218 | **Varying Data Types** $𝒯={\left\lbrace{T}_{{1}},…,{T}_{{n}}\right\rbrace}$
219 | - A data type representing any of varying types ${T}_{{1}},…,{T}_{{n}}$. 220 | 221 | ${S}≔{A}_{{1}},…,{A}_{{n}}$ 222 | - Sequence of values ${A}_{{i}}$ of the same type 223 | 224 | ${Enc}^{{{Len}}}_{\left\lbrace{S}{C}\right\rbrace}{\left({n}\right)}$ 225 | - SCALE length encoding, aka. compact encoding of non-negative interger ${n}$ of arbitrary size. 226 | 227 | ${Enc}_{{{HE}}}{\left({P}{K}\right)}$ 228 | - Hex encoding 229 | -------------------------------------------------------------------------------- /docs/id-polkadot-protocol.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Polkadot Protocol 3 | --- 4 | :::note 5 | The specifications are not actively maintained by the Web3 Foundation as of 02/10/2024. Please follow the [RCF Process](https://github.com/polkadot-fellows/RFCs) for latest protocol updates. 6 | ::: 7 | 8 | Formally, Polkadot is a replicated sharded state machine designed to resolve the scalability and interoperability among blockchains. In Polkadot vocabulary, shards are called *parachains* and Polkadot *relay chain* is part of the protocol ensuring global consensus among all the parachains. The Polkadot relay chain protocol, henceforward called *Polkadot protocol*, can itself be considered as a replicated state machine on its own. As such, the protocol can be specified by identifying the state machine and the replication strategy. 9 | 10 | From a more technical point of view, the Polkadot protocol has been divided into two parts, the [Polkadot Runtime](part-polkadot-runtime) and the [Polkadot Host](part-polkadot-host). The Runtime comprises the state transition logic for the Polkadot protocol and is designed and be upgradable via the consensus engine without requiring hard forks of the blockchain. The Polkadot Host provides the necessary functionality for the Runtime to execute its state transition logic, such as an execution environment, I/O, consensus and network interoperability between parachains. The Polkadot Host is planned to be stable and mostly static for the lifetime duration of the Polkadot protocol, the goal being that most changes to the protocol are primarily conducted by applying Runtime updates and not having to coordinate with network participants on manual software updates. 11 | -------------------------------------------------------------------------------- /docs/part-polkadot-host.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Polkadot Host 3 | --- 4 | 5 | import DocCardList from '@theme/DocCardList'; 6 | 7 | With the current document, we aim to specify the Polkadot Host part of the Polkadot protocol as a replicated state machine. After defining the different types of hosts in [Chapter -chap-num-ref-](chap-overview), we proceed to specify the representation of a valid state of the Protocol in [Chapter -chap-num-ref-](chap-state). We also identify the protocol states by explaining the Polkadot state transition and discussing the detail based on which the Polkadot Host interacts with the state transition function, i.e., Runtime, in the same chapter. Following, we specify the input messages triggering the state transition and the system behavior. In [Chapter -chap-num-ref-](chap-networking), we specify the communication protocols and network messages required for the Polkadot Host to communicate with other nodes in the network, such as exchanging blocks and consensus messages. In [Chapter -chap-num-ref-](sect-block-production) and [Chapter -chap-num-ref-](sect-finality), we specify the consensus protocol, which is responsible for keeping all the replicas in the same state. Finally, the initial state of the machine is identified and discussed in [Section -sec-num-ref-](id-cryptography-encoding#section-genesis). A Polkadot Host implementation that conforms with this part of the specification should successfully be able to sync its states with the Polkadot network. 8 | 9 | 10 | -------------------------------------------------------------------------------- /docs/part-polkadot-runtime.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Polkadot Runtime 3 | --- 4 | 5 | import DocCardList from '@theme/DocCardList'; 6 | 7 | Description of various useful Runtime internals 8 | 9 | 10 | -------------------------------------------------------------------------------- /docs/sect-block-production.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Block Production 3 | --- 4 | 5 | import Pseudocode from '@site/src/components/Pseudocode'; 6 | import blockProductionLottery from '!!raw-loader!@site/src/algorithms/blockProductionLottery.tex'; 7 | import slotTime from '!!raw-loader!@site/src/algorithms/slotTime.tex'; 8 | import medianAlgorithm from '!!raw-loader!@site/src/algorithms/medianAlgorithm.tex'; 9 | import invokeBlockAuthoring from '!!raw-loader!@site/src/algorithms/invokeBlockAuthoring.tex'; 10 | import verifyAuthorshipRight from '!!raw-loader!@site/src/algorithms/verifyAuthorshipRight.tex'; 11 | import verifySlotWinner from '!!raw-loader!@site/src/algorithms/verifySlotWinner.tex'; 12 | import buildBlock from '!!raw-loader!@site/src/algorithms/buildBlock.tex'; 13 | 14 | ## -sec-num- Introduction {#id-introduction-3} 15 | 16 | The Polkadot Host uses BABE protocol for block production. It is designed based on [Ouroboros Praos](https://eprint.iacr.org/2017/573.pdf). BABE execution happens in sequential non-overlapping phases known as an ***epoch***. Each epoch is divided into a predefined number of slots. All slots in each epoch are sequentially indexed starting from 0. At the beginning of each epoch, the BABE node needs to run [Block-Production-Lottery](sect-block-production#algo-block-production-lottery) to find out in which slots it should produce a block and gossip to the other block producers. In turn, the block producer node should keep a copy of the block tree and grow it as it receives valid blocks from other block producers. A block producer prunes the tree in parallel by eliminating branches that do not include the most recently finalized blocks ([Definition -def-num-ref-](chap-state#defn-pruned-tree)). 17 | 18 | ### -sec-num- Block Producer {#id-block-producer} 19 | 20 | A **block producer**, noted by ${\mathcal{{P}}}_{{j}}$, is a node running the Polkadot Host, which is authorized to keep a transaction queue and which it gets a turn in producing blocks. 21 | 22 | ### -sec-num- Block Authoring Session Key Pair {#id-block-authoring-session-key-pair} 23 | 24 | **Block authoring session key pair** ${\left({s}{{k}_{{j}}^{{s}}},{p}{{k}_{{j}}^{{s}}}\right)}$ is an SR25519 key pair which the block producer ${\mathcal{{P}}}_{{j}}$ signs by their account key ([Definition -def-num-ref-](id-cryptography-encoding#defn-account-key)) and is used to sign the produced block as well as to compute its lottery values in [Block-Production-Lottery](sect-block-production#algo-block-production-lottery). 25 | 26 | ###### Definition -def-num- Epoch and Slot {#defn-epoch-slot} 27 | ::::definition 28 | 29 | A block production **epoch**, formally referred to as ${\mathcal{{E}}}$, is a period with a pre-known starting time and fixed-length during which the set of block producers stays constant. Epochs are indexed sequentially, and we refer to the ${n}^{{{t}{h}}}$ epoch since genesis by ${\mathcal{{E}}}_{{n}}$. Each epoch is divided into equal-length periods known as block production **slots**, sequentially indexed in each epoch. The index of each slot is called a **slot number**. The equal length duration of each slot is called the **slot duration** and indicated by ${\mathcal{{T}}}$. Each slot is awarded to a subset of block producers during which they are allowed to generate a block. 30 | 31 | :::info 32 | Substrate refers to an epoch as a "session" in some places. However, epoch should be the preferred and official name for these periods. 33 | ::: 34 | 35 | :::: 36 | ###### Definition -def-num- Epoch and Slot Duration {#defn-epoch-duration} 37 | :::definition 38 | 39 | We refer to the number of slots in epoch ${\mathcal{{E}}}_{{n}}$ by ${s}{c}_{{n}}$. ${s}{c}_{{n}}$ is set to the `duration` field in the returned data from the call of the Runtime entry `BabeApi_configuration` ([Section -sec-num-ref-](chap-runtime-api#sect-rte-babeapi-epoch)) at genesis. For a given block ${B}$, we use the notation **${s}_{{B}}$** to refer to the slot during which ${B}$ has been produced. Conversely, for slot ${s}$, ${\mathcal{{B}}}_{{c}}$ is the set of Blocks generated at slot ${s}$. 40 | 41 | [Definition -def-num-ref-](sect-block-production#defn-epoch-subchain) provides an iterator over the blocks produced during a specific epoch. 42 | 43 | ::: 44 | ###### Definition -def-num- Epoch Subchain {#defn-epoch-subchain} 45 | :::definition 46 | 47 | By ${\text{SubChain}{\left({\mathcal{{E}}}_{{n}}\right)}}$ for epoch ${\mathcal{{E}}}_{{n}}$, we refer to the path graph of ${B}{T}$ containing all the blocks generated during the slots of epoch ${\mathcal{{E}}}_{{n}}$. When there is more than one block generated at a slot, we choose the one which is also on $\text{Longest-Chain}{\left({B}{T}\right)}$. 48 | 49 | ::: 50 | ###### Definition -def-num- Equivocation {#defn-producer-equivocation} 51 | :::definition 52 | 53 | A block producer **equivocates** if they produce more than one block at the same slot. The proof of equivocation is the given distinct headers that were signed by the validator and which include the slot number. 54 | 55 | The Polkadot Host must detect equivocations committed by other validators and submit those to the Runtime as described in [Section -sec-num-ref-](chap-runtime-api#sect-babeapi_submit_report_equivocation_unsigned_extrinsic). 56 | 57 | ::: 58 | ###### Definition -def-num- BABE Consensus Message {#defn-consensus-message-babe} 59 | :::definition 60 | 61 | $\text{CM}_{{b}}$, the consensus message for BABE, is of the following format: 62 | 63 | $$ 64 | \text{CM}_{{b}}={\left\lbrace\begin{matrix}{1}&{\left(\text{Auth}_{{C}},{r}\right)}\\{2}&{A}_{{i}}\\{3}&{D}\end{matrix}\right.} 65 | $$ 66 | 67 | **where** 68 | 69 | | | | 70 | |---|-----| 71 | | **1** | implies *next epoch data*: The Runtime issues this message on every first block of an epoch. The supplied authority set [Definition -def-num-ref-](chap-sync#defn-authority-list), ${\text{Auth}_C}$, and randomness [Definition -def-num-ref-](sect-block-production#defn-epoch-randomness), ${r}$, are used in the next epoch $\mathcal E_n + 1$. In case the epochs $\mathcal E_n + 1$ to $\mathcal E_n + k$ are skipped (i.e., BABE does not produce blocks), then the epoch data ${\left(\text{Auth}_{{C}},{r}\right)}$ is used by the epoch $\mathcal E_n + k + 1$.| 72 | | **2** | implies *on disabled*: A 32-bit integer, ${A_i}$, indicating the individual authority in the current authority list that should be immediately disabled until the next authority set changes. This message's initial intention was to cause an immediate suspension of all authority functionality with the specified authority. | 73 | | **3** | implies *next epoch descriptor*: These messages are only issued on configuration change and in the first block of an epoch. The supplied configuration data are intended to be used from the next epoch onwards. | 74 | - $D$ is a varying datatype of the following format: 75 | $$ 76 | D = \{1, (c,2_{\text{nd}})\} 77 | $$ 78 | where ${c}$ is the probability that a slot will not be empty [Definition -def-num-ref-](sect-block-production#defn-babe-constant). It is encoded as a tuple of two unsigned 64-bit integers ${c_{nominator},c_{denominator}}$ which are used to compute the rational ${c = \frac{c_{nominator}}{c_{denominator}}}$. 79 | - ${2_{\text{nd}}}$ describes what secondary slot [Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots), if any, is to be used. It is encoded as one-byte varying datatype: 80 | $$ 81 | s_{\text{2nd}} = \begin{cases} 82 | 0 \rightarrow \text{no secondary slot} \\ 83 | 1 \rightarrow \text{plain secondary slot} \\ 84 | 2 \rightarrow \text{secondary slot with VRF output} 85 | \end{cases} 86 | $$ 87 | 88 | 89 | ::: 90 | ## -sec-num- Block Production Lottery {#sect-block-production-lottery} 91 | 92 | The BABE constant ([Definition -def-num-ref-](sect-block-production#defn-babe-constant)) is initialized at genesis to the value returned by calling `BabeApi_configuration` ([Section -sec-num-ref-](chap-runtime-api#sect-rte-babeapi-epoch)). For efficiency reasons, it is generally updated by the Runtime through the *next config data* consensus message in the digest ([Definition -def-num-ref-](chap-state#defn-digest)) of the first block of an epoch for the next epoch. 93 | 94 | A block producer aiming to produce a block during ${\mathcal{{E}}}_{{n}}$ should run ([Block-Production-Lottery](sect-block-production#algo-block-production-lottery)) to identify the slots it is awarded. These are the slots during which the block producer is allowed to build a block. The ${s}{k}$ is the block producer lottery secret key and ${n}$ is the index of the epoch for whose slots the block producer is running the lottery. 95 | 96 | In order to ensure consistent block production, BABE uses secondary slots in case no authority wins the (primary) block production lottery. Unlike the lottery, secondary slot assignees are known upfront publically ([Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots)). The Runtime provides information on how or if secondary slots are executed ([Section -sec-num-ref-](chap-runtime-api#sect-rte-babeapi-epoch)), explained further in [Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots). 97 | 98 | ###### Definition -def-num- BABE Constant {#defn-babe-constant} 99 | :::definition 100 | 101 | The **BABE constant** is the probability that a slot will not be empty and used in the winning threshold calculation ([Definition -def-num-ref-](sect-block-production#defn-winning-threshold)). It’s expressed as a rational, ${\left({x},{y}\right)}$, where ${x}$ is the numerator and ${y}$ is the denominator. 102 | 103 | ::: 104 | ###### Definition -def-num- Winning Threshold {#defn-winning-threshold} 105 | :::definition 106 | 107 | The **Winning threshold** denoted by ${T}_{{{\mathcal{{E}}}_{{n}}}}$ is the threshold that is used alongside the result of [Block-Production-Lottery](sect-block-production#algo-block-production-lottery) to decide if a block producer is the winner of a specific slot. ${T}_{{{\mathcal{{E}}}_{{n}}}}$ is calculated as follows: 108 | 109 | $$ 110 | {A}_{{w}}={\sum_{{{n}={1}}}^{{{\left|\text{Auth}_{{C}}{\left({B}\right)}\right|}}}}{\left({w}_{{A}}\in\text{Auth}_{{C}}{\left({B}\right)}_{{n}}\right)} 111 | $$ 112 | $$ 113 | {T}_{{{\mathcal{{E}}}_{{n}}}}\:={1}-{\left({1}-{c}\right)}^{{\frac{{w}_{{a}}}{{A}_{{w}}}}} 114 | $$ 115 | 116 | where ${A}_{{w}}$ is the total sum of all authority weights in the authority set ([Definition -def-num-ref-](chap-sync#defn-authority-list)) for epoch ${\mathcal{{E}}}_{{n}}$, ${w}_{{a}}$ is the weight of the block author and ${c}\in{\left({0},{1}\right)}$ is the BABE constant ([Definition -def-num-ref-](sect-block-production#defn-babe-constant)). 117 | 118 | The numbers should be treated as 64-bit rational numbers. 119 | 120 | ::: 121 | ### -sec-num- Primary Block Production Lottery {#id-primary-block-production-lottery} 122 | 123 | ###### Definition -def-num- BABE Slot VRF transcript, output, and proof {#defn-babe-vrf-transcript} 124 | :::definition 125 | 126 | The BABE block production lottery requires a specific transcript structure ([Definition -def-num-ref-](id-cryptography-encoding#defn-vrf-transcript)). That structure is used by both primary slots ([Block-Production-Lottery](sect-block-production#algo-block-production-lottery)) and secondary slots ([Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots)). 127 | 128 | $$ 129 | {t}_{{1}}\leftarrow\text{Transcript}{\left(\text{'BABE'}\right)} 130 | $$ 131 | $$ 132 | {t}_{{2}}\leftarrow\text{append}{\left({t}_{{1}},\text{'slot number'},{s}\right)} 133 | $$ 134 | $$ 135 | {t}_{{3}}\leftarrow\text{append}{\left({t}_{{2}},\text{'current epoch'},{e}_{{i}}\right)} 136 | $$ 137 | $$ 138 | {t}_{{4}}\leftarrow\text{append}{\left({t}_{{3}},\text{'chain randomness'},{r}\right)} 139 | $$ 140 | $$ 141 | {t}_{{5}}\leftarrow\text{append}{\left({t}_{{4}},\text{'vrf-nm-pk'},{p}_{{k}}\right)} 142 | $$ 143 | $$ 144 | {t}_{{6}}\leftarrow\text{meta-ad}{\left({t}_{{5}},\text{'VRFHash'},\text{False}\right)} 145 | $$ 146 | $$ 147 | {t}_{{7}}\leftarrow\text{meta-ad}{\left({t}_{{6}},{64}_{\text{le}},\text{True}\right)} 148 | $$ 149 | $$ 150 | {h}\leftarrow\text{prf}{\left({t}_{{7}},\text{False}\right)} 151 | $$ 152 | $$ 153 | {d}={s}_{{k}}\cdot{h} 154 | $$ 155 | $$ 156 | {\pi}\leftarrow\text{dleq\_prove}{\left({t}_{{7}},{h}\right)} 157 | $$ 158 | 159 | The operators are defined in [Definition -def-num-ref-](id-cryptography-encoding#defn-strobe-operations), $\text{dleq\_prove}$ in [Definition -def-num-ref-](id-cryptography-encoding#defn-vrf-dleq-prove). The computed outputs, ${d}$ and ${\pi}$, are included in the block Pre-Digest ([Definition -def-num-ref-](sect-block-production#defn-babe-header)). 160 | 161 | ::: 162 | 163 | A block producer aiming to produce a block during ${\mathcal{{E}}}_{{n}}$ should run the $\text{Block-Production-Lottery}$ algorithm to identify the slots it is awarded. These are the slots during which the block producer is allowed to build a block. The session secret key, ${s}{k}$, is the block producer lottery secret key, and ${n}$ is the index of the epoch for whose slots the block producer is running the lottery. 164 | 165 | ###### Algorithm -algo-num- Block Production Lottery {#algo-block-production-lottery} 166 | :::algorithm 167 | 172 | 173 | where $\text{Epoch-Randomness}$ is defined in ([Definition -def-num-ref-](sect-block-production#defn-epoch-randomness)), ${s}{c}_{{n}}$ is defined in [Definition -def-num-ref-](sect-block-production#defn-epoch-duration) , $\text{VRF}$ creates the BABE VRF transcript ([Definition -def-num-ref-](sect-block-production#defn-babe-vrf-transcript)) and ${e}_{{i}}$ is the epoch index, retrieved from the Runtime ([Section -sec-num-ref-](chap-runtime-api#sect-rte-babeapi-epoch)). ${s}_{{k}}$ and ${p}_{{k}}$ is the secret key, respectively, the public key of the authority. For any slot ${s}$ in epoch ${n}$ where ${o}<{T}_{{{\mathcal{{E}}}_{{n}}}}$ ([Definition -def-num-ref-](sect-block-production#defn-winning-threshold)), the block producer is required to produce a block. 174 | ::: 175 | 176 | :::info 177 | The secondary slots ([Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots)) are running alongside the primary block production lottery and mainly serve as a fallback to in case no authority was selected in the primary lottery. 178 | ::: 179 | ###### Definition -def-num- Secondary Slots {#defn-babe-secondary-slots} 180 | :::definition 181 | 182 | **Secondary slots** work alongside primary slot to ensure consistent block production, as described in [Section -sec-num-ref-](sect-block-production#sect-block-production-lottery). The secondary assignee of a block is determined by calculating a specific value, ${i}_{{d}}$, which indicates the index in the authority set ([Definition -def-num-ref-](chap-sync#defn-authority-list)). The corresponding authority in that set has the right to author a secondary block. This calculation is done for every slot in the epoch, ${s}\in{s}{c}_{{n}}$ ([Definition -def-num-ref-](sect-block-production#defn-epoch-duration)). 183 | 184 | $$ 185 | {p}\leftarrow{h}{\left(\text{Enc}_{\text{SC}}{\left({r},{s}\right)}\right)} 186 | $$ 187 | $$ 188 | {i}_{{d}}\leftarrow{p}\text{mod}{A}_{{l}} 189 | $$ 190 | 191 | **where** 192 | - ${r}$ is the Epoch randomness ([Definition -def-num-ref-](sect-block-production#defn-epoch-randomness)). 193 | 194 | - ${s}$ is the slot number ([Definition -def-num-ref-](sect-block-production#defn-epoch-slot)). 195 | 196 | - $\text{Enc}_{\text{SC}}{\left(\ldots\right)}$ encodes its inner value to the corresponding SCALE value. 197 | 198 | - ${h}{\left(\ldots\right)}$ creates a 256-bit Blake2 hash from its inner value. 199 | 200 | - ${A}_{{l}}$ is the lengths of the authority list ([Definition -def-num-ref-](chap-sync#defn-authority-list)). 201 | 202 | If ${i}_{{d}}$ points to the authority, that authority must claim the secondary slot by creating a BABE VRF transcript ([Definition -def-num-ref-](sect-block-production#defn-babe-vrf-transcript)). The resulting values ${d}$ and ${\pi}$ are then used in the Pre-Digest item ([Definition -def-num-ref-](sect-block-production#defn-babe-header)). In the case of secondary slots with plain outputs, respectively the Pre-Digest being of value *2*, the transcript respectively the VRF is skipped. 203 | 204 | ::: 205 | ## -sec-num- Slot Number Calculation {#sect-slot-number-calculation} 206 | 207 | It is imperative for the security of the network that each block producer correctly determines the current slot numbers at a given time by regularly estimating the local clock offset in relation to the network ([Definition -def-num-ref-](sect-block-production#defn-relative-synchronization)). 208 | 209 | :::danger 210 | **The calculation described in this section is still to be implemented and deployed**: For now, each block producer is required to synchronize its local clock using NTP instead. The current slot ${s}$ is then calculated by ${s}={t}_{\text{unix}}/{\mathcal{{T}}}$ where ${\mathcal{{T}}}$ is defined in [Definition -def-num-ref-](sect-block-production#defn-epoch-slot) and ${t}_{\text{unix}}$ is defined in [Definition -def-num-ref-](id-cryptography-encoding#defn-unix-time). That also entails that slot numbers are currently not reset at the beginning of each epoch. 211 | ::: 212 | 213 | Using the median algorithm described in this section, Polkadot achieves synchronization without relying on any external clock source (e.g., through the [NTP](https://en.wikipedia.org/wiki/Network_Time_Protocol) or [GPS](https://en.wikipedia.org/wiki/Global_Positioning_System) protocol). To stay in synchronization, each producer is therefore required to periodically estimate its local clock offset in relation to the rest of the network. 214 | 215 | This estimation depends on the two fixed parameters ${k}$ ([Definition -def-num-ref-](sect-block-production#defn-prunned-best)) and ${s}_{{{c}{q}}}$ ([Definition -def-num-ref-](sect-block-production#defn-chain-quality)). These are chosen based on the results of a [formal security analysis](https://research.web3.foundation/Polkadot/protocols/block-production/Babe#5-security-analysis), currently assuming a ${1}{s}$ clock drift per day and targeting a probability lower than ${0.5}\%$ for an adversary to break BABE in 3 years with resistance against a network delay up to $\frac{{1}}{{3}}$ of the slot time and a BABE constant ([Definition -def-num-ref-](sect-block-production#defn-babe-constant)) of ${c}={0.38}$. 216 | 217 | All validators are then required to run [Median-Algorithm](sect-block-production#algo-median-algorithm) at the beginning of each sync period ([Definition -def-num-ref-](sect-block-production#defn-sync-period)) to update their synchronization using all block arrival times of the previous period. The algorithm should only be run once all the blocks in this period have been finalized, even if only probabilistically ([Definition -def-num-ref-](sect-block-production#defn-prunned-best)). The target slot to which to synchronize should be the first slot in the new sync period. 218 | 219 | ###### Definition -def-num- Slot Offset {#defn-slot-offset} 220 | :::definition 221 | 222 | Let ${s}_{{i}}$ and ${s}_{{j}}$ be two slots belonging to epochs ${\mathcal{{E}}}_{{k}}$ and ${\mathcal{{E}}}_{{l}}$. By **Slot-Offset**${\left({s}_{{i}},{s}_{{j}}\right)}$ we refer to the function whose value is equal to the number of slots between ${s}_{{i}}$ and ${s}_{{j}}$ (counting ${s}_{{j}}$) on the time continuum. As such, we have **Slot-Offset**${\left({s}_{{i}},{s}_{{i}}\right)}={0}$. 223 | ::: 224 | 225 | It is imperative for the security of the network that each block producer correctly determines the current slot numbers at a given time by regularly estimating the local clock offset in relation to the network ([Definition -def-num-ref-](sect-block-production#defn-relative-synchronization)). 226 | 227 | ###### Definition -def-num- Relative Time Synchronization {#defn-relative-synchronization} 228 | :::definition 229 | 230 | The **relative time synchronization** is a tuple of a slot number and a local clock timestamp ${\left({s}_{\text{sync}},{t}_{\text{sync}}\right)}$ describing the last point at which the slot numbers have been synchronized with the local clock. 231 | 232 | ::: 233 | ###### Algorithm -algo-num- Slot Time {#algo-slot-time} 234 | :::algorithm 235 | 240 | 241 | where ${s}$ is the slot number. 242 | ::: 243 | 244 | ###### Algorithm -algo-num- Median Algorithm {#algo-median-algorithm} 245 | :::algorithm 246 | 251 | 252 | where 253 | - ${\mathfrak{{{E}}}}$ is the sync period used for the estimate. 254 | 255 | - ${s}_{\text{sync}}$ is the slot time to estimate. 256 | 257 | - $\text{Slot-Offset}$ is defined in [Slot-Time](sect-block-production#algo-slot-offset). 258 | 259 | - ${\mathcal{{{T}}}}$ is the slot duration defined in [Definition -def-num-ref-](sect-block-production#defn-epoch-slot). 260 | 261 | ::: 262 | ###### Definition -def-num- Pruned Best Chain {#defn-prunned-best} 263 | :::definition 264 | 265 | The **pruned best chain** ${C}^{{{r}^{{k}}}}$ is the longest selected chain ([Definition -def-num-ref-](chap-state#defn-longest-chain)) with the last ${k}$ Blocks pruned. We chose ${k}={140}$. The **last (probabilistic) finalized block** describes the last block in this pruned best chain. 266 | 267 | ::: 268 | ###### Definition -def-num- Chain Quality {#defn-chain-quality} 269 | :::definition 270 | 271 | The **chain quality** ${s}_{{{c}{q}}}$ represents the number of slots that are used to estimate the local clock offset. Currently, it is set to ${s}_{{{c}{q}}}={3000}$. 272 | 273 | The prerequisite for such a calculation is that each producer stores the arrival time of each block ([Definition -def-num-ref-](sect-block-production#defn-block-time)) measured by a clock that is otherwise not adjusted by any external protocol. 274 | 275 | ::: 276 | ###### Definition -def-num- Block Arrival Time {#defn-block-time} 277 | :::definition 278 | 279 | The **block arrival time** of block ${B}$ for node ${j}$ formally represented by ${{T}_{{B}}^{{j}}}$ is the local time of node ${j}$ when node ${j}$ has received block ${B}$ for the first time. If the node ${j}$ itself is the producer of ${B}$, ${{T}_{{B}}^{{j}}}$ is set equal to the time that the block is produced. The index ${j}$ in ${{T}_{{B}}^{{j}}}$ notation may be dropped, and B’s arrival time is referred to by ${T}_{{B}}$ when there is no ambiguity about the underlying node. 280 | 281 | ::: 282 | ###### Definition -def-num- Sync Period {#defn-sync-period} 283 | :::definition 284 | 285 | A is an interval at which each validator (re-)evaluates its local clock offsets. The first sync period ${\mathfrak{{E}}}_{{1}}$ starts just after the genesis block is released. Consequently, each sync period ${\mathfrak{{E}}}_{{i}}$ starts after ${\mathfrak{{E}}}_{{{i}-{1}}}$. The length of the sync period ([Definition -def-num-ref-](sect-block-production#defn-chain-quality)) is equal to ${s}_{{{q}{c}}}$and expressed in the number of slots. 286 | 287 | ::: 288 | ###### Image -img-num- An exemplary result of Median Algorithm in first sync epoch with ${s}_{\text{cq}}={9}$ and ${k}={1}$. {#img-median-algorithm} 289 | 290 | ![](/img/median-algorithm-result.png) 291 | 292 | ## -sec-num- Production Algorithm {#block-production} 293 | 294 | Throughout each epoch, each block producer should run [Invoke-Block-Authoring](sect-block-production#algo-block-production) to produce blocks during the slots it has been awarded during that epoch. The produced block needs to carry the *Pre-Digest* ([Definition -def-num-ref-](sect-block-production#defn-babe-header)) as well as the *block signature* ([Definition -def-num-ref-](sect-block-production#defn-block-signature)) as Pre-Runtime and Seal digest items. 295 | 296 | ###### Definition -def-num- Pre-Digest {#defn-babe-header} 297 | :::definition 298 | 299 | The **Pre-Digest**, or BABE header, ${P}$, is a varying datatype of the following format: 300 | 301 | $$ 302 | {P}={\left\lbrace\begin{matrix}{1}&\rightarrow&{\left({a}_{\text{id}},{s},{d},{\pi}\right)}\\{2}&\rightarrow&{\left({a}_{\text{id}},{s}\right)}\\{3}&\rightarrow&{\left({a}_{\text{id}},{s},{d},{\pi}\right)}\end{matrix}\right.} 303 | $$ 304 | 305 | **where** 306 | - *1* indicates a primary slot with VRF outputs, *2* a secondary slot with plain outputs and *3* a secondary slot with VRF outputs ([Section -sec-num-ref-](sect-block-production#sect-block-production-lottery)). Plain outputs are no longer actively used and only exist for backwards compatibility reasons, respectively to sync old blocks. 307 | 308 | - ${a}_{\text{id}}$ is the unsigned 32-bit integer indicating the index of the authority in the authority set ([Section -sec-num-ref-](chap-sync#sect-authority-set)) who authored the block. 309 | 310 | - ${s}$ is the slot number ([Definition -def-num-ref-](sect-block-production#defn-epoch-slot)). 311 | 312 | - ${d}$ is VRF output ([Block-Production-Lottery](sect-block-production#algo-block-production-lottery) respectively [Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots)). 313 | 314 | - ${\pi}$ is VRF proof ([Block-Production-Lottery](sect-block-production#algo-block-production-lottery) respectively [Definition -def-num-ref-](sect-block-production#defn-babe-secondary-slots)). 315 | 316 | The Pre-Digest must be included as a digest item of Pre-Runtime type in the header digest ([Definition -def-num-ref-](chap-state#defn-digest)) ${H}_{{d}}{\left({B}\right)}$. 317 | ::: 318 | 319 | ###### Algorithm -algo-num- Invoke-Block-Authoring {#algo-block-production} 320 | :::algorithm 321 | 326 | 327 | where $\text{BT}$ is the current block tree, $\text{Block-Production-Lottery}$ is defined in [Block-Production-Lottery](sect-block-production#algo-block-production-lottery) and $\text{Add-Digest-Item}$ appends a digest item to the end of the header digest ${H}_{{d}}{\left({B}\right)}$ ([Definition -def-num-ref-](chap-state#defn-digest)). 328 | ::: 329 | 330 | ###### Definition -def-num- Block Signature {#defn-block-signature} 331 | :::definition 332 | 333 | The **Block Signature** ${S}_{{B}}$ is a signature of the block header hash ([Definition -def-num-ref-](chap-state#defn-block-header-hash)) and defined as 334 | 335 | $$ 336 | \text{Sig}_{{\text{SR25519},{\text{sk}_{{j}}^{{s}}}}}{\left({H}_{{h}}{\left({B}\right)}\right)} 337 | $$ 338 | 339 | ${m}$ should be included in ${H}_{{d}}{\left({B}\right)}$ as the Seal digest item ([Definition -def-num-ref-](chap-state#defn-digest)) of value: 340 | 341 | $$ 342 | {\left({t},\text{id}{\left(\text{BABE}\right)},{m}\right)} 343 | $$ 344 | 345 | in which, ${t}={5}$ is the seal digest identifier and $\text{id}{\left(\text{BABE}\right)}$ is the BABE consensus engine unique identifier ([Definition -def-num-ref-](chap-state#defn-digest)). The Seal digest item is referred to as the **BABE Seal**. 346 | 347 | ::: 348 | ## -sec-num- Epoch Randomness {#sect-epoch-randomness} 349 | 350 | At the beginning of each epoch, ${\mathcal{{E}}}_{{n}}$ the host will receive the randomness seed ${\mathcal{{R}}}_{{{\mathcal{{E}}}_{{{n}+{1}}}}}$ ([Definition -def-num-ref-](sect-block-production#defn-epoch-randomness)) necessary to participate in the block production lottery in the next epoch ${\mathcal{{E}}}_{{{n}+{1}}}$ from the Runtime, through the consensus message ([Definition -def-num-ref-](sect-block-production#defn-consensus-message-babe)) in the digest of the first block. 351 | 352 | ###### Definition -def-num- Randomness Seed {#defn-epoch-randomness} 353 | :::definition 354 | 355 | For epoch ${\mathcal{{E}}}$, there is a 32-byte ${\mathcal{{R}}}_{{{\mathcal{{E}}}}}$ computed based on the previous epochs VRF outputs. For ${\mathcal{{E}}}_{{0}}$ and ${\mathcal{{E}}}_{{1}}$, the randomness seed is provided in the genesis state ([Section -sec-num-ref-](chap-runtime-api#sect-rte-babeapi-epoch)). For any further epochs, the randomness is retrieved from the consensus message ([Definition -def-num-ref-](sect-block-production#defn-consensus-message-babe)). 356 | 357 | ::: 358 | ## -sec-num- Verifying Authorship Right {#sect-verifying-authorship} 359 | 360 | When a Polkadot node receives a produced block, it needs to verify if the block producer was entitled to produce the block in the given slot by running [Verify-Authorship-Right](sect-block-production#algo-verify-authorship-right). [Verify-Slot-Winner](sect-block-production#algo-verify-slot-winner) runs as part of the verification process, when a node is importing a block. 361 | 362 | ###### Algorithm -algo-num- Verify Authorship Right {#algo-verify-authorship-right} 363 | :::algorithm 364 | 369 | 370 | **where** 371 | - $\text{Head}_{{s}}{\left({B}\right)}$ is the header of the block that’s being verified. 372 | 373 | - ${T}_{{B}}$ is ${B}$’s arrival time ([Definition -def-num-ref-](sect-block-production#defn-block-time)). 374 | 375 | - ${H}_{{d}}{\left({B}\right)}$ is the digest sub-component ([Definition -def-num-ref-](chap-state#defn-digest)) of $\text{Head}{\left({B}\right)}$ ([Definition -def-num-ref-](chap-state#defn-block-header)). 376 | 377 | - The Seal ${D}_{{s}}$ is the last element in the digest array ${H}_{{d}}{\left({B}\right)}$ as described in [Definition -def-num-ref-](chap-state#defn-digest). 378 | 379 | - $\text{Seal-Id}$ is the type index showing that a digest item ([Definition -def-num-ref-](chap-state#defn-digest)) of varying type ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-variable-type)) is of type *Seal*. 380 | 381 | - $\text{AuthorityDirectory}^{{{\mathcal{{E}}}_{{c}}}}$ is the set of Authority ID for block producers of epoch ${\mathcal{{E}}}_{{c}}$. 382 | 383 | 1. $\text{AuthorId}$ is the public session key of the block producer. 384 | 385 | - $\text{BT}$ is the pruned block tree ([Definition -def-num-ref-](chap-state#defn-pruned-tree)). 386 | 387 | - $\text{Verify-Slot-Winner}$ is defined in [Verify-Slot-Winner](sect-block-production#algo-verify-slot-winner). 388 | ::: 389 | 390 | ###### Algorithm -algo-num- Verify Slot Winner {#algo-verify-slot-winner} 391 | :::algorithm 392 | 397 | 398 | **where** 399 | 1. $\text{Epoch-Randomness}$ is defined in [Definition -def-num-ref-](sect-block-production#defn-epoch-randomness). 400 | 401 | 2. ${H}_{\text{BABE}}{\left({B}\right)}$ is the BABE header defined in [Definition -def-num-ref-](sect-block-production#defn-babe-header). 402 | 403 | 3. ${\left({o},{p}\right)}$ is the block lottery result for block ${B}$ ([Block-Production-Lottery](sect-block-production#algo-block-production-lottery)), respectively the VRF output ([Definition -def-num-ref-](sect-block-production#defn-babe-vrf-transcript)). 404 | 405 | 4. $\text{Verify-VRF}$ is described in [Section -sec-num-ref-](id-cryptography-encoding#sect-vrf). 406 | 407 | 5. ${T}_{{{\mathcal{{E}}}_{{n}}}}$ is the winning threshold as defined in [Definition -def-num-ref-](sect-block-production#defn-winning-threshold). 408 | ::: 409 | 410 | ## -sec-num- Block Building Process {#sect-block-building} 411 | 412 | The block building process is triggered by [Invoke-Block-Authoring](sect-block-production#algo-block-production) of the consensus engine which in turn runs [Build-Block](sect-block-production#algo-build-block). 413 | 414 | ###### Algorithm -algo-num- Build Block {#algo-build-block} 415 | :::algorithm 416 | 421 | 422 | **where** 423 | - ${C}_{\text{Best}}$ is the chain head at which the block should be constructed ("parent"). 424 | 425 | - ${s}$ is the slot number. 426 | 427 | - $\text{Head}{\left({B}\right)}$ is defined in [Definition -def-num-ref-](chap-state#defn-block-header). 428 | 429 | - $\text{Call-Runtime-Entry}$ is defined in [Definition -def-num-ref-](chap-state#defn-call-into-runtime). 430 | 431 | - $\text{Inherent-Data}$ is defined in [Definition -def-num-ref-](chap-state#defn-inherent-data). 432 | 433 | - $\text{End-Of-Slot}$ indicates the end of the BABE slot as defined [Median-Algorithm](sect-block-production#algo-median-algorithm) respectively [Definition -def-num-ref-](sect-block-production#defn-epoch-slot). 434 | 435 | - $\text{Next-Ready-Extrinsic}$ indicates picking an extrinsic from the extrinsics queue ([Definition -def-num-ref-](chap-state#defn-transaction-queue)). 436 | 437 | - $\text{Block-Is-Full}$ indicates that the maximum block size is being used. 438 | 439 | - $\text{Should-Drop}$ determines based on the result ${R}$ whether the extrinsic should be dropped or remain in the extrinsics queue and scheduled for the next block. The *ApplyExtrinsicResult* ([Definition -def-num-ref-](chap-runtime-api#defn-rte-apply-extrinsic-result)) describes this behavior in more detail. 440 | 441 | - $\text{Drop}$ indicates removing the extrinsic from the extrinsic queue ([Definition -def-num-ref-](chap-state#defn-transaction-queue)). 442 | 443 | - $\text{Add-Seal}$ adds the seal to the block (<<\>\>) before sending it to peers. The seal is removed again before submitting it to the Runtime. 444 | ::: 445 | -------------------------------------------------------------------------------- /docs/sect-lightclient.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Light Clients 3 | --- 4 | 5 | import Pseudocode from '@site/src/components/Pseudocode'; 6 | import warpSyncLightClients from '!!raw-loader!@site/src/algorithms/warpSyncLightClients.tex'; 7 | import queryingStateLightClients from '!!raw-loader!@site/src/algorithms/queryingStateLightClients.tex'; 8 | 9 | ## -sec-num- Requirements for Light Clients {#sect-requirements-lightclient} 10 | 11 | We list the requirements of a Light Client categorized along the three dimensions of Functionality, Efficiency, and Security. 12 | 13 | - **Functional Requirements:** 14 | 15 | 1. Synchronize with full nodes to obtain the latest finalized Block Header [Definition -def-num-ref-](chap-state#defn-block-header), and in turn, the state trie root. 16 | 17 | 2. (Optional) Verify validity of runtime transitions ([Section -sec-num-ref-](chap-state#sect-runtime-interaction)). 18 | 19 | 3. Make queries for data at the latest block height or across a range of blocks. 20 | 21 | 4. Append extrinsics ([Section -sec-num-ref-](chap-state#sect-extrinsics)) to the blockchain via full nodes. 22 | 23 | - **Efficiency Requirements:** 24 | 25 | 1. Efficient bootstrapping and syncing: initializations and update functions of the state have tractable computation and communication complexity and grows at most linearly with the chain size. Generally, the complexity is proportional to the GRANDPA validator set change. 26 | 27 | 2. Querying operations happen by requesting the key-value pair from a full node. 28 | 29 | 3. Further, verifying the validity of responses by the full node is logarithmic in the size of the state. 30 | 31 | - **Security Requirements:** 32 | 33 | 1. Secure bootstrapping and Synchronizing: The probability that an adversarial full node convinces a light client of a forged blockchain state is negligible. 34 | 35 | 2. Secure querying: The probability that an adversary convinces a light client to accept a forged account state is negligible. 36 | 37 | 3. Assure that the submitted extrinsics are appended in a successor block or inform the user in case of failure. 38 | 39 | - **Polkadot Specific Requirements:** 40 | 41 | 1. The client MUST be able to connect to a relay chain using chain state. 42 | 43 | 2. The client MUST be able to retrieve the checkpoint state from a trusted source to speed up initialization. 44 | 45 | 3. The client MUST be able to subscribe/unsubscribe to/from any polkadot-spec-conformant relay chain (Polkadot, Westend, Kusama) 46 | 47 | 4. The client MUST be able to subscribe/unsubscribe to/from parachains that do not use custom protocols or cryptography methods other than those that Polkadot, Westend and Kusama use. 48 | 49 | 5. The client MUST support the following [RPC methods](https://github.com/paritytech/json-rpc-interface-spec): `rpc_methods`, `chainHead_unstable_follow`, `chainHead_unstable_unfollow`, `chainHead_unstable_unpin`, `chainHead_unstable_storage`, `chainHead_unstable_call` `chainHead_unstable_stopCall`. `transaction_unstable_submitAndWatch`, and `transaction_unstable_unwatch` 50 | 51 | 6. The client MUST support the @substrate/connect [connection extension protocol](https://github.com/paritytech/substrate-connect/tree/main/packages/connect-extension-protocol): `ToApplicationError`, `ToApplicationChainReady`, `ToApplicationRpc`, `ToExtensionAddChain`, `ToExtensionAddWellKnownChain`, `ToExtensionRpc`, `ToExtensionRemoveChain`. 52 | 53 | ## -sec-num- Warp Sync for Light Clients {#sect-sync-warp-lightclient} 54 | 55 | Warp sync ([Section -sec-num-ref-](chap-networking#sect-msg-warp-sync)) only downloads the block headers where authority set changes occurred, so-called fragments ([Definition -def-num-ref-](chap-networking#defn-warp-sync-proof)), and by verifying the GRANDPA justifications ([Definition -def-num-ref-](sect-finality#defn-grandpa-justification)). This protocol allows nodes to arrive at the desired state much faster than fast sync. Warp sync is primarily designed for Light Clients. Although, warp sync could be used by full nodes, the sync process may lack information to cater to complete functionality set of full nodes. 56 | 57 | For light clients, it is too expensive to download the state (approx. 550MB) to respond to queries. Rather, the queries are submitted to the Full node, and only the response of the full node is validated using the hash of the state root. Requests for warp sync are performed using the `/dot/sync/warp` *Request-Response* substream, the corresponding network messages are detailed in [Section -sec-num-ref-](chap-networking#sect-protocols-substreams). 58 | 59 | Light clients base their trust in provided snapshots and the ability to slash grandpa votes for equivocation for the period they are syncing via warp sync. Full nodes and above, in contrast, verify each block individually. 60 | 61 | In theory, the `warp sync` process takes the Genesis Block as input and outputs the hash of the state trie root at the latest finalized block. This root hash acts as proof to further validate the responses to queries by the full node. The `warp sync` works by starting from a trusted specified block (e.g., from a snapshot) and verifying the block headers only at the authority set changes. 62 | 63 | Eventually, the light client verifies the finality of the block returned by a full node to ensure that the block is indeed the latest finalized block. This entails two things: 64 | 65 | 1. Check the authenticity of GRANDPA Justifications messages from Genesis to the last finalized block. 66 | 67 | 2. Check the timestamp of the last finalized block to ensure that no other blocks might have been finalized at a later timestamp. 68 | 69 | :::caution 70 | **Long-Range Attack Vulnerabilities**: Warp syncing is particularly vulnerable to what is called long-range attacks. The authorities allowed to finalize blocks can generate multiple proofs of finality for multiple different blocks of the same height. Hence, they can finalize more than one chain at a time. It is possible for two-thirds of the validators that were active at a certain past block N to collude and decide to finalize a different block N', even when N has been finalized for the first time several weeks or months in the past. When a client then warp syncs, it can be tricked to consider this alternative block N' as the finalized one. However, in practice, to mitigate Long-Range Attacks, the starting point of the warp syncing is not too far in the past. How far exactly depends on the logic of the runtime of the chain. For example, in Polkadot, the starting block for the sync should be at max 28 days old to be within the purview of the slashing period for misbehaving nodes. Hence, even though, in theory, warp sync can start from Genesis Block, it is not advised to implement the same in practice. 71 | ::: 72 | 73 | We outline the warp sync process, abstracting out details of verifying the finality and how the full node to sync with is selected. 74 | 75 | ###### Algorithm -algo-num- Warp Sync Light Clients {#algo-warp-sync} 76 | :::algorithm 77 | 82 | 83 | Abstraction of Warp Sync and verification of the latest block’s finality. 84 | 85 | ${SelectFullNode}$: Determines the full node that the light client syncs with. 86 | 87 | ${SyncSithNode}$: Returns the header of the latest finalized block and a list of Grandpa Justifications by the full node. 88 | 89 | ${verifyAuthoritySetChange}$: Verification algorithm which checks the authenticity of the header only at the end of an era where the authority set changes iteratively until reaching the latest era. 90 | 91 | ${verifyFinalty}$: Verifies the finality of the latest block using the Grandpa Justifications messages. 92 | ::: 93 | 94 | The warp syncing process is closely coupled with the state querying procedure used by the light client. We outline the process of querying the state by a light client and validating the response. 95 | 96 | ###### Algorithm -algo-num- Querying State Light Clients {#algo-light-clients-query-state} 97 | :::algorithm 98 | 103 | 104 | Querying State Algorithm. 105 | 106 | ${QueryFullNode}$: Returns the response to the query requested from the Full Node for the query ${q}$ at block height ${h}$. 107 | 108 | ${validityCheck}_{root}$: Predicate that checks the validity of response ${res}$ and associated merkle proof $\pi$ by matching it against the Commit Root Hash ${root}$ obtained as a result of warp sync. 109 | ::: 110 | 111 | ## -sec-num- Runtime Environment for Light Clients {#sect-runtime-environment-lightclient} 112 | 113 | Technically, though a runtime execution environment is not necessary to build a light client, most clients require interacting with the Runtime and the state of the blockchain for integrity checks at the minimum. One can imagine an application scenario like an on-chain light client which only listens to the latest state without ever adding extrinsics. Current implementations of Light Nodes (for e.g., Smoldot) use the wasmtime as its runtime environment to drastically simplify the code. The performance of wasmtime is satisfying enough not to require a native runtime. The details of the runtime API that the environment needs to support can be found in ([Appendix -chap-num-ref-](chap-runtime-api)). 114 | 115 | ## -sec-num- Light Client Messages {#sect-light-msg} 116 | 117 | Light clients are applications that fetch the required data that they need from a Polkadot node with an associated proof to validate the data. This makes it possible to interact with the Polkadot network without requiring to run a full node or having to trust the remote peers. The light client messages make this functionality possible. 118 | 119 | All light client messages are protobuf encoded and are sent over the `/dot/light/2` substream. 120 | 121 | ### -sec-num- Request {#id-request} 122 | 123 | A message with all possible request messages. All messages are sent as part of this message. 124 | 125 | | Type | Id | Description | 126 | |---------------------|-----|------------------| 127 | | `oneof` (`request`) | | The request type | 128 | 129 | Where the `request` can be one of the following fields: 130 | 131 | | Type | Id | Description | 132 | |--------------------------|-----|------------------------------------------------------------------------------------------------------------| 133 | | `RemoteCallRequest` | 1 | A remote call request ([Definition -def-num-ref-](sect-lightclient#sect-light-remote-call-request)) | 134 | | `RemoteReadRequest` | 2 | A remote read request ([Definition -def-num-ref-](sect-lightclient#sect-light-remote-read-request)) | 135 | | `RemoteReadChildRequest` | 4 | A remote read child request ([Definition -def-num-ref-](sect-lightclient#sect-light-remote-read-child-request)) | 136 | 137 | ### -sec-num- Response {#id-response} 138 | 139 | A message with all possible response messages. All messages are sent as part of this message. 140 | 141 | | Type | Id | Description | 142 | |----------------------|-----|-------------------| 143 | | `oneof` (`response`) | | The response type | 144 | 145 | Where the `response` can be one of the following fields: 146 | 147 | | Type | Id | Description | 148 | |----------------------|-----|-------------------------------------------------------------------------------------------------| 149 | | `RemoteCallResponse` | 1 | A remote call response ([Definition -def-num-ref-](sect-lightclient#sect-light-remote-call-response)) | 150 | | `RemoteReadResponse` | 2 | A remote read response ([Definition -def-num-ref-](sect-lightclient#sect-light-remote-read-response)) | 151 | 152 | ### -sec-num- Remote Call Messages {#id-remote-call-messages} 153 | 154 | Execute a call to a contract at the given block. 155 | 156 | ###### Definition -def-num- Remote Call Request {#sect-light-remote-call-request} 157 | :::definition 158 | 159 | Remote call request. 160 | 161 | | Type | Id | Description | 162 | |----------|-----|--------------------------------| 163 | | `bytes` | 2 | Block at which to perform call | 164 | | `string` | 3 | Method name | 165 | | `bytes` | 4 | Call data | 166 | 167 | ::: 168 | ###### Definition -def-num- Remote Call Response {#sect-light-remote-call-response} 169 | :::definition 170 | 171 | Remote call response. 172 | 173 | | Type | Id | Description | 174 | |---------|-----|-----------------------------------------------------------------------------------------------------------------------------------------------------| 175 | | `bytes` | 2 | An *Option* type ([Definition -def-num-ref-](id-cryptography-encoding#defn-option-type)) containing the call proof or *None* if proof generation failed. | 176 | 177 | ::: 178 | ### -sec-num- Remote Read Messages {#id-remote-read-messages} 179 | 180 | Read a storage value at the given block. 181 | 182 | ###### Definition -def-num- Remote Read Request {#sect-light-remote-read-request} 183 | :::definition 184 | 185 | Remote read request. 186 | 187 | | Type | Id | Description | 188 | |------------------|-----|--------------------------------| 189 | | `bytes` | 2 | Block at which to perform call | 190 | | `repeated bytes` | 3 | Storage keys | 191 | 192 | ::: 193 | ###### Definition -def-num- Remote Read Response {#sect-light-remote-read-response} 194 | :::definition 195 | 196 | Remote read response. 197 | 198 | | Type | Id | Description | 199 | |---------|-----|-----------------------------------------------------------------------------------------------------------------------------------------------------| 200 | | `bytes` | 2 | An *Option* type ([Definition -def-num-ref-](id-cryptography-encoding#defn-option-type)) containing the read proof or *None* if proof generation failed. | 201 | 202 | ::: 203 | ### -sec-num- Remote Read Child Messages {#id-remote-read-child-messages} 204 | 205 | Read a child storage value at the given block. 206 | 207 | ###### Definition -def-num- Remote Read Child Request {#sect-light-remote-read-child-request} 208 | :::definition 209 | 210 | Remote read child request. 211 | 212 | | Type | Id | Description | 213 | |---------|-----|------------------------------------------------------------------------| 214 | | `bytes` | 2 | Block at which to perform call | 215 | | `bytes` | 3 | Child storage key, this is relative to the child type storage location | 216 | | `bytes` | 6 | Storage keys | 217 | 218 | The response is the same as for the *Remote Read Request* message, respectively [Definition -def-num-ref-](sect-lightclient#sect-light-remote-read-response). 219 | 220 | ::: 221 | ## -sec-num- Storage for Light Clients {#sect-storage-lightclient} 222 | 223 | The light client requires a persistent storage for saving the state of the blockchain. In addition, it requires efficient Serialization/De-serialization methods to transform SCALE ([Section -sec-num-ref-](id-cryptography-encoding#sect-scale-codec)) encoded network traffic for storing and reading from the persistent storage. 224 | -------------------------------------------------------------------------------- /docs/sect-metadata.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: -chap-num- Metadata 3 | --- 4 | import Metadata from '/static/img/kaitai_render/metadata.svg'; 5 | import MetadataType from '/static/img/kaitai_render/metadata_type.svg'; 6 | import MetadataTypeDefinition from '/static/img/kaitai_render/metadata_type_definition.svg'; 7 | import MetadataTypeFields from '/static/img/kaitai_render/metadata_type_fields.svg'; 8 | import MetadataTypeVariants from '/static/img/kaitai_render/metadata_type_variants.svg'; 9 | import MetadataPallet from '/static/img/kaitai_render/metadata_pallet.svg'; 10 | import PalletStorage from '/static/img/kaitai_render/pallet_storage.svg'; 11 | import StorageDefinition from '/static/img/kaitai_render/storage_definition.svg'; 12 | import PalletConstant from '/static/img/kaitai_render/pallet_constant.svg'; 13 | import MetadataExtrinsic from '/static/img/kaitai_render/metadata_extrinsic.svg'; 14 | 15 | The runtime metadata structure contains all the information necessary on how to interact with the Polkadot runtime. Considering that Polkadot runtimes are upgradable and, therefore, any interfaces are subject to change, the metadata allows developers to structure any extrinsics or storage entries accordingly. 16 | 17 | The metadata of a runtime is provided by a call to `Metadata_metadata` ([Section -sec-num-ref-](chap-runtime-api#sect-rte-metadata-metadata)) and is returned as a scale encoded ([Section -sec-num-ref-](id-cryptography-encoding#sect-scale-codec)) binary blob. How to interpret and decode this data is described in this chapter. 18 | 19 | ## -sec-num- Structure {#sect-rtm-structure} 20 | 21 | The Runtime Metadata is a data structure of the following format: 22 | 23 | $$ 24 | {\left({M},{v}_{{m}},{R},{P},{t}_{{e}},{v}_{{e}},{E},{t}_{{r}}\right)} 25 | $$ 26 | $$ 27 | {R}={\left({r}_{{0}},\ldots,{r}_{{n}}\right)} 28 | $$ 29 | $$ 30 | {P}={\left({p}_{{0}},\ldots,{p}_{{n}}\right)} 31 | $$ 32 | $$ 33 | {E}={\left({e}_{{0}},\ldots,{e}_{{n}}\right)} 34 | $$ 35 | 36 | **where** 37 | - ${M}$ are the first four constant bytes, spelling "meta" in ASCII. 38 | 39 | - ${v}_{{m}}$ is an unsigned 8-bit integer indicating the format version of the metadata structure (currently the value of `14`). 40 | 41 | - ${R}$ is a sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of type definitions ${r}_{{i}}$ ([Definition -def-num-ref-](sect-metadata#defn-rtm-registry-entry)). 42 | 43 | - ${P}$ is a sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of pallet metadata ${p}_{{i}}$ ([Section -sec-num-ref-](sect-metadata#sect-rtm-pallet-metadata)). 44 | 45 | - ${t}_{{e}}$ is the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of the extrinsics. 46 | 47 | - ${v}_{{e}}$ is an unsigned 8-bit integer indicating the format version of the extrinsics (implying a possible breaking change). 48 | 49 | - ${E}$ is a sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of extrinsics metadata ${e}_{{i}}$ ([Definition -def-num-ref-](sect-metadata#defn-rtm-signed-extension-metadata)). 50 | 51 | - ${t}_{{r}}$ is the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of the runtime. 52 | 53 | ###### Image -img-num- Metadata {#img-metadata} 54 | 55 | 56 | 57 | ###### Definition -def-num- Runtime Registry Type Entry {#defn-rtm-registry-entry} 58 | :::definition 59 | 60 | A registry entry contains information about a type in its portable form for serialization. The entry is a data structure of the following format: 61 | 62 | $$ 63 | {r}_{{i}}={\left(\text{id}_{{t}},{p},{T},{D},{c}\right)} 64 | $$ 65 | $$ 66 | {T}={\left({t}_{{0}},\ldots,{t}_{{n}}\right)} 67 | $$ 68 | $$ 69 | {t}_{{i}}={\left({n},{y}\right)} 70 | $$ 71 | 72 | **where** 73 | - $\text{id}_{{t}}$ is a compact integer indicating the identifier of the type. 74 | 75 | - ${p}$ is the path of the type, optional and based on the source file location. Encoded as a sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of strings. 76 | 77 | - ${T}$ is a sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of generic parameters (empty for non-generic types). 78 | 79 | - ${n}$ is the name string of the generic type parameter 80 | 81 | - ${y}$ is a *Option* type containing a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). 82 | 83 | - ${D}$ is the type definition ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-definition)). 84 | 85 | - ${c}$ is the documentation as sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of strings. 86 | 87 | ###### Image -img-num- Metadata Type {#img-metadata-type} 88 | 89 | 90 | ::: 91 | ###### Definition -def-num- Runtime Type Id {#defn-rtm-type-id} 92 | :::definition 93 | 94 | The **runtime type Id** is a compact integer representing the index of the entry ([Definition -def-num-ref-](sect-metadata#defn-rtm-registry-entry)) in ${R},{P}$ or ${E}$ of the runtime metadata structure ([Section -sec-num-ref-](sect-metadata#sect-rtm-structure)), depending on context (starting at ${0}$). 95 | 96 | ::: 97 | ###### Definition -def-num- Type Variant {#defn-rtm-type-definition} 98 | :::definition 99 | 100 | The type definition ${D}$ is a varying datatype ([Definition -def-num-ref-](id-cryptography-encoding#defn-varrying-data-type)) and indicates all the possible types of encodable values a type can have. 101 | 102 | $$ 103 | {D}={\left\lbrace\begin{matrix}{0}&\rightarrow&{C}&\text{composite type (e.g. structure or tuple)}\\{1}&\rightarrow&{V}&\text{variant type}\\{2}&\rightarrow&{s}_{{v}}&\text{sequence type varying length}\\{3}&\rightarrow&{S}&\text{sequence with fixed length}\\{4}&\rightarrow&{T}&\text{tuple type}\\{5}&\rightarrow&{P}&\text{primitive type}\\{6}&\rightarrow&{e}&\text{compact encoded type}\\{7}&\rightarrow&{B}&\text{sequence of bits}\end{matrix}\right.} 104 | $$ 105 | 106 | **where** 107 | - ${C}$ is a sequence of the following format: 108 | 109 | ${C}={\left({{f}_{{0}},}\ldots,{f}_{{n}}\right)}$ 110 | 111 | - ${f_i}$ is a field ([Definition -def-num-ref-](sect-metadata#defn-rtm-field)). 112 | 113 | - ${V}$ is a sequence of the following format: 114 | 115 | ${V}={\left({v}_{{0}},\ldots,{v}_{{n}}\right)}$ 116 | 117 | - ${v}_{{i}}$ is a variant ([Definition -def-num-ref-](sect-metadata#defn-rtm-variant)). 118 | 119 | - ${s}_{{v}}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). 120 | 121 | - ${S}$ is of the following format: 122 | 123 | ${S}={\left({l},{y}\right)}$ 124 | 125 | - ${l}$ is an unsigned 32-bit integer indicating the length 126 | 127 | - ${y}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). 128 | 129 | - ${T}$ is a sequence ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of type Ids ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). 130 | 131 | - ${P}$ is a varying datatype ([Definition -def-num-ref-](id-cryptography-encoding#defn-varrying-data-type)) of the following structure: 132 | 133 | $$ 134 | {P}={\left\lbrace\begin{matrix}{0}&\text{boolean}\\{1}&\text{char}\\{2}&\text{string}\\{3}&\text{unsigned 8-bit integer}\\{4}&\text{unsigned 16-bit integer}\\{5}&\text{unsigned 32-bit integer}\\{6}&\text{unsigned 64-bit integer}\\{7}&\text{unsigned 128-bit integer}\\{8}&\text{unsigned 256-bit integer}\\{9}&\text{signed 8-bit integer}\\{10}&\text{signed 16-bit integer}\\{11}&\text{signed 32-bit integer}\\{12}&\text{signed 64-bit integer}\\{13}&\text{signed 128-bit integer}\\{14}&\text{signed 256-bit integer}\end{matrix}\right.} 135 | $$ 136 | 137 | - ${e}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). 138 | 139 | - ${B}$ is a data structure of the following format: 140 | 141 | ${B}={\left({s},{o}\right)}$ 142 | 143 | - ${s}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) representing the bit store order ([external reference](https://docs.rs/bitvec/latest/bitvec/store/trait.BitStore.html)) 144 | 145 | - ${o}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) the bit order type ([external reference](https://docs.rs/bitvec/latest/bitvec/order/trait.BitOrder.html)). 146 | 147 | ###### Image -img-num- Metadata Type Definition {#img-metadata-type-definition} 148 | 149 | 150 | ::: 151 | 152 | ###### Definition -def-num- Field {#defn-rtm-field} 153 | :::definition 154 | 155 | A field of a data structure of the following format: 156 | 157 | $$ 158 | {{f}_{{i}}=}{\left({n},{y},{y}_{{n}},{C}\right)} 159 | $$ 160 | 161 | **where** 162 | 163 | - ${n}$ is an *Option* type containing the string that indicates the field name. 164 | 165 | - ${y}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). 166 | 167 | - ${y}_{{n}}$ is an *Option* type containing a string that indicates the name of the type as it appears in the source code. 168 | 169 | - ${C}$ is a sequence of varying length containing strings of documentation. 170 | 171 | ###### Image -img-num- Metadata Type Fields {#img-metadata-type-fields} 172 | 173 | 174 | ::: 175 | 176 | ###### Definition -def-num- Variant {#defn-rtm-variant} 177 | :::definition 178 | 179 | A struct variant of the following format: 180 | 181 | $$ 182 | {v}_{{i}}={\left({n},{F},{k},{C}\right)} 183 | $$ 184 | 185 | **where** 186 | 187 | - ${n}$ is a string representing the name of the variant. 188 | 189 | - ${F}$ is a possible empty array of varying length containing field ([Definition -def-num-ref-](sect-metadata#defn-rtm-field)) elements. 190 | 191 | - ${k}$ is an unsigned 8-bit integer indicating the index of the variant. 192 | 193 | - ${C}$ is a sequence of strings containing the documentation. 194 | 195 | ###### Image -img-num- Metadata Type Variants {#img-metadata-type-variants} 196 | 197 | 198 | ::: 199 | ## -sec-num- Pallet Metadata {#sect-rtm-pallet-metadata} 200 | 201 | All the metadata about a pallet, part of the main structure ([Section -sec-num-ref-](sect-metadata#sect-rtm-structure)) and of the following format: 202 | 203 | $$ 204 | {p}_{{i}}={\left({n},{S},{a},{e},{C},{e},{i}\right)} 205 | $$ 206 | 207 | **where** 208 | 209 | - ${n}$ is a string representing the pallet name. 210 | 211 | - ${S}$ is an *Option* type containing the pallet storage metadata ([Definition -def-num-ref-](sect-metadata#defn-rtm-pallet-storage-metadata)). 212 | 213 | - ${a}$ is an *Option* type ([Definition -def-num-ref-](id-cryptography-encoding#defn-option-type)) containing the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of pallet calls. 214 | 215 | - ${e}$ is an *Option* type ([Definition -def-num-ref-](id-cryptography-encoding#defn-option-type)) containing the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of pallet events. 216 | 217 | - ${C}$ is an *Sequence* ([Definition -def-num-ref-](id-cryptography-encoding#defn-scale-list)) of all pallet constant metadata ([Definition -def-num-ref-](sect-metadata#defn-rtm-pallet-constants)). 218 | 219 | - ${e}$ is an *Option* type ([Definition -def-num-ref-](id-cryptography-encoding#defn-option-type)) containing the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of the pallet error. 220 | 221 | - ${i}$ is an unsigned 8-bit integer indicating the index of the pallet, which is used for encoding pallet events and calls. 222 | 223 | ###### Image -img-num- Metadata Pallet {#img-metadata-pallet} 224 | 225 | 226 | 227 | ###### Definition -def-num- Pallet Storage Metadata {#defn-rtm-pallet-storage-metadata} 228 | :::definition 229 | 230 | The metadata about pallets storage. 231 | 232 | $$ 233 | {S}={\left({p},{E}\right)} 234 | $$ 235 | $$ 236 | {E}={\left({e}_{{0}},\ldots,{e}_{{n}}\right)} 237 | $$ 238 | 239 | **where** 240 | 241 | - ${p}$ is the string representing the common prefix used by all storage entries. 242 | 243 | - ${E}$ is an array of varying lengths containing elements of storage entries ([Definition -def-num-ref-](sect-metadata#defn-rtm-storage-entry-metadata)). 244 | 245 | ::: 246 | ###### Definition -def-num- Storage Entry Metadata {#defn-rtm-storage-entry-metadata} 247 | :::definition 248 | 249 | The metadata about a pallets storage entry. 250 | 251 | $$ 252 | {e}_{{i}}={\left({n},{m},{y},{d},{C}\right)} 253 | $$ 254 | $$ 255 | {C}={\left({c}_{{0}},\ldots,{c}_{{n}}\right)} 256 | $$ 257 | 258 | **where** 259 | 260 | - ${n}$ is the string representing the variable name of the storage entry. 261 | 262 | - ${m}$ is an enum type determining the storage entry modifier ([Definition -def-num-ref-](sect-metadata#defn-rtm-storage-entry-modifier)). 263 | 264 | - ${y}$ is the type of the value stored in the entry ([Definition -def-num-ref-](sect-metadata#defn-rtm-storage-entry-type)). 265 | 266 | - ${d}$ is a byte array containing the default value. 267 | 268 | - ${C}$ is an array of varying lengths of strings containing the documentation. 269 | 270 | ###### Image -img-num- Pallet Storage {#img-pallet-storage} 271 | 272 | 273 | ::: 274 | ###### Definition -def-num- Storage Entry Modifier {#defn-rtm-storage-entry-modifier} 275 | ::::definition 276 | 277 | :::info 278 | This might be incorrect and has to be reviewed. 279 | ::: 280 | 281 | The storage entry modifier is a varying datatype ([Definition -def-num-ref-](id-cryptography-encoding#defn-varrying-data-type)) and indicates how the storage entry is returned and how it behaves if the entry is not present. 282 | 283 | $$ 284 | {m}={\left\lbrace\begin{matrix}{0}&\text{optional}\\{1}&\text{default}\end{matrix}\right.} 285 | $$ 286 | 287 | where *0* indicates that the entry returns an *Option* type and therefore *None* if the storage entry is not present. *1* indicates that the entry returns the type ${y}$ with default value ${d}$ (in [Definition -def-num-ref-](sect-metadata#defn-rtm-storage-entry-metadata)) if the entry is not present. 288 | 289 | :::: 290 | ###### Definition -def-num- Storage Entry Type {#defn-rtm-storage-entry-type} 291 | :::definition 292 | 293 | The type of the storage value is a varying datatype ([Definition -def-num-ref-](id-cryptography-encoding#defn-varrying-data-type)) that indicates how the entry is stored. 294 | 295 | $$ 296 | {y}={\left\lbrace\begin{matrix}{0}&\rightarrow&{t}&\text{plain type}\\{1}&\rightarrow&{\left({H},{k},{v}\right)}&\text{storage map}\end{matrix}\right.} 297 | $$ 298 | 299 | where ${t}$, ${k}$ (key) and ${v}$ (value) are all of type Ids ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)). ${H}$ is an array of varying length containing the storage hasher ([Definition -def-num-ref-](sect-metadata#defn-rtm-storage-hasher)). 300 | 301 | ###### Image -img-num- Storage Definition {#img-storage-definition} 302 | 303 | 304 | ::: 305 | 306 | ###### Definition -def-num- Storage Hasher {#defn-rtm-storage-hasher} 307 | :::definition 308 | 309 | The hashing algorithm is used by storage maps. 310 | 311 | $$ 312 | {\left\lbrace\begin{matrix}{0}&\text{128-bit Blake2 hash}\\{1}&\text{256-bit Blake2 hash}\\{2}&\text{Multiple 128-bit Blake2 hashes concatenated}\\{3}&\text{128-bit XX hash}\\{4}&\text{256-bit XX hash}\\{5}&\text{Multiple 64-bit XX hashes concatenated}\\{6}&\text{Identity hashing}\end{matrix}\right.} 313 | $$ 314 | 315 | ::: 316 | ###### Definition -def-num- Pallet Constants {#defn-rtm-pallet-constants} 317 | :::definition 318 | 319 | The metadata about the pallets constants. 320 | 321 | $$ 322 | {c}_{{i}}={\left({n},{y},{v},{C}\right)} 323 | $$ 324 | 325 | **where** 326 | - ${n}$ is a string representing the name of the pallet constant. 327 | 328 | - ${y}$ is the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of the pallet constant. 329 | 330 | - ${v}$ is a byte array containing the value of the constant. 331 | 332 | - ${C}$ is an array of varying lengths containing a string with the documentation. 333 | 334 | ###### Image -img-num- Pallet Constant {#img-pallet-constant} 335 | 336 | 337 | ::: 338 | ## -sec-num- Extrinsic Metadata {#sect-rtm-extrinsic-metadata} 339 | 340 | The metadata about a pallets extrinsics, part of the main structure ([Section -sec-num-ref-](sect-metadata#sect-rtm-structure)) and of the following format: 341 | 342 | ###### Definition -def-num- Signed Extension Metadata {#defn-rtm-signed-extension-metadata} 343 | :::definition 344 | 345 | The metadata about the additional, signed data required to execute an extrinsic. 346 | 347 | $$ 348 | {e}_{{i}}={\left({n},{y},{a}\right)} 349 | $$ 350 | 351 | **where** 352 | - ${n}$ is a string representing the unique signed extension identifier, which may be different from the type name. 353 | 354 | - ${y}$ is a type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of the signed extension, with the data to be included in the extrinsic. 355 | 356 | - ${a}$ is the type Id ([Definition -def-num-ref-](sect-metadata#defn-rtm-type-id)) of the additional signed data, with the data to be included in the signed payload. 357 | 358 | ###### Image -img-num- Metadata Extrinsic {#img-metadata-extrinsic} 359 | 360 | 361 | ::: 362 | -------------------------------------------------------------------------------- /kaitai_structures/block.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: block 3 | imports: 4 | - block_header 5 | - block_body 6 | seq: 7 | - id: header 8 | type: block_header 9 | - id: body 10 | type: block_body -------------------------------------------------------------------------------- /kaitai_structures/block_body.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: block_body 3 | title: BlockBody 4 | endian: le 5 | imports: 6 | - scale 7 | seq: 8 | - id: num_transactions 9 | type: scale::compact_int 10 | - id: transactions 11 | type: transaction 12 | repeat: expr 13 | repeat-expr: num_transactions.value 14 | types: 15 | transaction: 16 | seq: 17 | - id: len_data 18 | type: scale::compact_int 19 | - id: data 20 | size: len_data.value -------------------------------------------------------------------------------- /kaitai_structures/block_header.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: block_header 3 | title: BlockHeader 4 | endian: le 5 | imports: 6 | - scale 7 | - digest 8 | seq: 9 | - id: parent_hash 10 | size: 32 11 | - id: number 12 | type: scale::compact_int 13 | - id: state_root 14 | size: 32 15 | - id: extrinsic_root 16 | size: 32 17 | - id: num_digests 18 | type: scale::compact_int 19 | - id: digests 20 | type: digest 21 | repeat: expr 22 | repeat-expr: num_digests.value -------------------------------------------------------------------------------- /kaitai_structures/digest.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: digest 3 | title: BlockDigest 4 | endian: le 5 | imports: 6 | - scale 7 | seq: 8 | - id: type 9 | type: u1 10 | enum: type_id 11 | - id: value 12 | type: 13 | switch-on: type 14 | cases: 15 | 'type_id::pre_runtime': pre_runtime 16 | 'type_id::post_runtime': post_runtime 17 | 'type_id::seal': seal 18 | 'type_id::runtime_updated': empty 19 | enums: 20 | type_id: 21 | 4: post_runtime 22 | 5: seal 23 | 6: pre_runtime 24 | 8: runtime_updated 25 | types: 26 | pre_runtime: 27 | seq: 28 | - id: engine 29 | type: str 30 | encoding: ASCII 31 | size: 4 32 | - id: payload 33 | type: scale::bytes 34 | post_runtime: 35 | seq: 36 | - id: engine 37 | type: str 38 | encoding: ASCII 39 | size: 4 40 | - id: payload 41 | type: scale::bytes 42 | seal: 43 | seq: 44 | - id: engine 45 | type: str 46 | encoding: ASCII 47 | size: 4 48 | - id: payload 49 | type: scale::bytes 50 | empty: {} -------------------------------------------------------------------------------- /kaitai_structures/metadata.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata 3 | title: Metadata 4 | endian: le 5 | imports: 6 | - scale 7 | - metadata_type 8 | - metadata_pallet 9 | - metadata_extrinsic 10 | seq: 11 | - id: magic 12 | contents: meta 13 | - id: metadata_version 14 | type: u1 15 | 16 | - id: num_types 17 | type: scale::compact_int 18 | - id: types 19 | type: metadata_type 20 | repeat: expr 21 | repeat-expr: num_types.value 22 | 23 | - id: num_pallets 24 | type: scale::compact_int 25 | - id: pallets 26 | type: metadata_pallet 27 | repeat: expr 28 | repeat-expr: num_pallets.value 29 | 30 | - id: extrinsic_type 31 | type: scale::compact_int 32 | - id: extrinsic_version 33 | type: u1 34 | - id: num_extrinsics 35 | type: scale::compact_int 36 | - id: extrinsics 37 | type: metadata_extrinsic 38 | repeat: expr 39 | repeat-expr: num_extrinsics.value 40 | 41 | - id: runtime_type 42 | type: scale::compact_int -------------------------------------------------------------------------------- /kaitai_structures/metadata_extrinsic.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata_extrinsic 3 | title: MetadataExtrinsic 4 | endian: le 5 | imports: 6 | - scale 7 | seq: 8 | - id: name 9 | type: scale::string 10 | - id: type 11 | type: scale::compact_int 12 | - id: additional 13 | type: scale::compact_int -------------------------------------------------------------------------------- /kaitai_structures/metadata_pallet.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata_pallet 3 | title: MetadataPallet 4 | endian: le 5 | imports: 6 | - scale 7 | - pallet_storage 8 | - pallet_constant 9 | seq: 10 | - id: name 11 | type: scale::string 12 | 13 | - id: has_storage 14 | type: u1 15 | - id: storage 16 | type: pallet_storage 17 | if: has_storage != 0 18 | 19 | - id: has_calls 20 | type: u1 21 | - id: calls 22 | type: calls 23 | if: has_calls != 0 24 | 25 | - id: has_events 26 | type: u1 27 | - id: events 28 | type: events 29 | if: has_events != 0 30 | 31 | - id: num_constants 32 | type: scale::compact_int 33 | - id: constants 34 | type: pallet_constant 35 | repeat: expr 36 | repeat-expr: num_constants.value 37 | 38 | - id: has_errors 39 | type: u1 40 | - id: errors 41 | type: errors 42 | if: has_errors != 0 43 | 44 | - id: index 45 | type: u1 46 | types: 47 | calls: 48 | seq: 49 | - id: type 50 | type: scale::compact_int 51 | 52 | events: 53 | seq: 54 | - id: type 55 | type: scale::compact_int 56 | 57 | errors: 58 | seq: 59 | - id: type 60 | type: scale::compact_int -------------------------------------------------------------------------------- /kaitai_structures/metadata_type.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata_type 3 | title: MetadataType 4 | endian: le 5 | imports: 6 | - scale 7 | - metadata_type_definition 8 | seq: 9 | - id: id 10 | type: scale::compact_int 11 | 12 | - id: path 13 | type: scale::string_list 14 | 15 | - id: num_params 16 | type: scale::compact_int 17 | - id: params 18 | repeat: expr 19 | repeat-expr: num_params.value 20 | type: param 21 | 22 | - id: definition 23 | type: metadata_type_definition 24 | 25 | - id: docs 26 | type: scale::string_list 27 | types: 28 | param: 29 | seq: 30 | - id: name 31 | type: scale::string 32 | - id: type 33 | type: scale::maybe_compact_int -------------------------------------------------------------------------------- /kaitai_structures/metadata_type_definition.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata_type_definition 3 | title: MetadataTypeDefinition 4 | endian: le 5 | imports: 6 | - scale 7 | - metadata_type_variants 8 | - metadata_type_fields 9 | seq: 10 | - id: type 11 | type: u1 12 | enum: type 13 | - id: details 14 | type: 15 | switch-on: type 16 | cases: 17 | "type::composite": metadata_type_fields 18 | "type::variant": metadata_type_variants 19 | "type::sequence": sequence 20 | "type::array": array 21 | "type::tuple": tuple 22 | "type::primitive": primitive 23 | "type::compact": compact 24 | "type::bits": bits 25 | enums: 26 | type: 27 | 0: composite 28 | 1: variant 29 | 2: sequence 30 | 3: array 31 | 4: tuple 32 | 5: primitive 33 | 6: compact 34 | 7: bits 35 | types: 36 | sequence: 37 | seq: 38 | - id: type 39 | type: scale::compact_int 40 | 41 | array: 42 | seq: 43 | - id: length 44 | type: u4 45 | - id: type 46 | type: scale::compact_int 47 | 48 | tuple: 49 | seq: 50 | - id: num_types 51 | type: scale::compact_int 52 | - id: types 53 | type: scale::compact_int 54 | repeat: expr 55 | repeat-expr: num_types.value 56 | 57 | primitive: 58 | seq: 59 | - id: id 60 | type: u1 61 | enum: pid 62 | enums: 63 | pid: 64 | 0: bool 65 | 1: char 66 | 2: str 67 | 3: uint8 68 | 4: uint16 69 | 5: uint32 70 | 6: uint64 71 | 7: uint128 72 | 8: uint256 73 | 9: int8 74 | 10: int16 75 | 11: int32 76 | 12: int64 77 | 13: int128 78 | 14: int256 79 | 80 | compact: 81 | seq: 82 | - id: type 83 | type: scale::compact_int 84 | 85 | bits: 86 | seq: 87 | - id: type 88 | type: scale::compact_int 89 | - id: order 90 | type: scale::compact_int -------------------------------------------------------------------------------- /kaitai_structures/metadata_type_fields.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata_type_fields 3 | title: MetadataTypeFields 4 | endian: le 5 | imports: 6 | - scale 7 | seq: 8 | - id: num_fields 9 | type: scale::compact_int 10 | - id: fields 11 | type: field 12 | repeat: expr 13 | repeat-expr: num_fields.value 14 | types: 15 | field: 16 | seq: 17 | - id: name 18 | type: scale::maybe_string 19 | - id: type 20 | type: scale::compact_int 21 | - id: typename 22 | type: scale::maybe_string 23 | - id: docs 24 | type: scale::string_list -------------------------------------------------------------------------------- /kaitai_structures/metadata_type_variants.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: metadata_type_variants 3 | title: MetadataTypeVariants 4 | endian: le 5 | imports: 6 | - scale 7 | - metadata_type_fields 8 | seq: 9 | - id: num_variants 10 | type: scale::compact_int 11 | - id: variants 12 | type: variant 13 | repeat: expr 14 | repeat-expr: num_variants.value 15 | types: 16 | variant: 17 | seq: 18 | - id: name 19 | type: scale::string 20 | - id: composite 21 | type: metadata_type_fields 22 | - id: index 23 | type: u1 24 | - id: docs 25 | type: scale::string_list -------------------------------------------------------------------------------- /kaitai_structures/pallet_constant.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: pallet_constant 3 | title: PalletConstant 4 | endian: le 5 | imports: 6 | - scale 7 | seq: 8 | - id: name 9 | type: scale::string 10 | - id: type 11 | type: scale::compact_int 12 | - id: value 13 | type: scale::bytes 14 | - id: docs 15 | type: scale::string_list 16 | -------------------------------------------------------------------------------- /kaitai_structures/pallet_storage.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: pallet_storage 3 | title: PalletStorage 4 | endian: le 5 | imports: 6 | - scale 7 | - storage_definition 8 | seq: 9 | - id: prefix 10 | type: scale::string 11 | 12 | - id: num_items 13 | type: scale::compact_int 14 | - id: items 15 | type: item 16 | repeat: expr 17 | repeat-expr: num_items.value 18 | types: 19 | item: 20 | seq: 21 | - id: name 22 | type: scale::string 23 | 24 | - id: modifier 25 | type: u1 26 | enum: storage_modifier 27 | 28 | - id: definition 29 | type: storage_definition 30 | 31 | - id: fallback 32 | type: scale::bytes 33 | 34 | - id: docs 35 | type: scale::string_list 36 | enums: 37 | storage_modifier: 38 | 0: optional 39 | 1: default -------------------------------------------------------------------------------- /kaitai_structures/scale.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: scale 3 | title: Simple Concatenated Aggregate Little-Endian 4 | endian: le 5 | bit-endian: be 6 | types: 7 | compact_int: 8 | seq: 9 | - id: data 10 | type: b6 11 | - id: mode 12 | type: b2 13 | - id: extra 14 | size: (mode & 0x1) + ((mode & 0x2) / 0x2 * 3) + ((mode & 0x3) / 0x3 * data) 15 | instances: 16 | value: 17 | value: data + (extra[0] << 6) + (extra[1] << 14) + (extra[2] << 22) + (extra[3] << 30) + (extra[4] << 38) + (extra[5] << 46) + (extra[6] << 54) + (extra[7] << 62) 18 | 19 | maybe_compact_int: 20 | seq: 21 | - id: has_value 22 | type: u1 23 | - id: value 24 | type: compact_int 25 | if: has_value != 0 26 | 27 | bytes: 28 | seq: 29 | - id: length 30 | type: compact_int 31 | - id: value 32 | size: length.value 33 | 34 | string: 35 | seq: 36 | - id: length 37 | type: compact_int 38 | - id: value 39 | type: str 40 | size: length.value 41 | encoding: UTF-8 42 | 43 | maybe_string: 44 | seq: 45 | - id: has_value 46 | type: u1 47 | - id: value 48 | type: string 49 | if: has_value != 0 50 | 51 | string_list: 52 | seq: 53 | - id: num_values 54 | type: compact_int 55 | - id: values 56 | type: string 57 | repeat: expr 58 | repeat-expr: num_values.value 59 | -------------------------------------------------------------------------------- /kaitai_structures/storage_definition.ksy: -------------------------------------------------------------------------------- 1 | meta: 2 | id: storage_definition 3 | title: StorageDefinition 4 | endian: le 5 | imports: 6 | - scale 7 | seq: 8 | - id: type 9 | type: u1 10 | enum: storage_type 11 | - id: details 12 | type: 13 | switch-on: type 14 | cases: 15 | 'storage_type::plain': plain 16 | 'storage_type::map': map 17 | enums: 18 | storage_type: 19 | 0: plain 20 | 1: map 21 | types: 22 | plain: 23 | seq: 24 | - id: type 25 | type: scale::compact_int 26 | map: 27 | seq: 28 | - id: num_hasher 29 | type: scale::compact_int 30 | - id: hasher 31 | type: u1 32 | enum: hasher_type 33 | repeat: expr 34 | repeat-expr: num_hasher.value 35 | 36 | - id: key 37 | type: scale::compact_int 38 | - id: value 39 | type: scale::compact_int 40 | enums: 41 | hasher_type: 42 | 0: blake2_128 43 | 1: blake2_256 44 | 2: blake2_128_128 45 | 3: xxhash_128 46 | 4: xxhash_256 47 | 5: xxhahs_64_64 48 | 6: idhash --------------------------------------------------------------------------------