├── book ├── README.md ├── 99__dev │ ├── 00__index.md │ ├── 02__notes.md │ └── 01__sources.md ├── 02__blockchains │ ├── 00__index.md │ ├── images │ │ └── fork-choice-rules.md │ │ │ ├── uncle-blocks.png │ │ │ ├── lcr-ghost-agree.png │ │ │ └── lcr-ghost-disagree.png │ ├── 06__participation.md │ ├── 01__digital-cash.md │ └── 04__fork-choice-rules.md ├── 04__eth2-overview │ ├── 00__index.md │ ├── 04__development-process.md │ ├── 05__learning-goals.md │ ├── 01__historical-context.md │ └── 03__key-concepts.md ├── 03__eth1 │ ├── 00__index.md │ ├── images │ │ └── proof-of-work │ │ │ ├── lcr.png │ │ │ ├── uncle-blocks.png │ │ │ ├── lcr-ghost-agree.png │ │ │ └── lcr-ghost-disagree.png │ ├── 01__basics.md │ ├── 06__fork-choice-rule.md │ ├── 03__transactions.md │ ├── 05__blocks.md │ ├── 02__state.md │ ├── 08__scaling.md │ └── 07__clients.md ├── 06__building-blocks │ ├── 00__index.md │ ├── 04__committees.md │ ├── 03__ssz.md │ ├── 05__deposit-contract.md │ ├── images │ │ ├── randomness │ │ │ ├── randao-mix.png │ │ │ ├── randao-mix-open.png │ │ │ ├── vdf-structure.png │ │ │ ├── randao-mix-closed.png │ │ │ └── randao-with-vdfs.png │ │ └── bls-signatures │ │ │ └── wholesome-sigs.png │ └── 01__bls-signatures.md ├── 07__beacon-chain │ ├── 00__index.md │ ├── 03__structure.md │ ├── images │ │ └── validator-life-cycle │ │ │ └── validator-life-cycle.png │ ├── 02__validator-duties.md │ └── 01__validator-life-cycle.md ├── 05__proof-of-stake │ ├── images │ │ ├── lmd-ghost │ │ │ ├── lmd-ghost.png │ │ │ ├── lmd-ghost-messages.png │ │ │ └── validator-attacks-ghost.png │ │ ├── casper-ffg │ │ │ └── ffg-finalized-1.png │ │ └── blocks-slots-and-epochs │ │ │ ├── pos-epochs.png │ │ │ ├── pos-height.png │ │ │ ├── pow-height.png │ │ │ ├── ebb-first-slot.png │ │ │ └── ebb-no-first-slot.png │ ├── 00__index.md │ ├── 06__weak-subjectivity.md │ ├── 01__core-concepts.md │ ├── 05__casper-ffg.md │ ├── 02__basic-operation.md │ ├── 04__long-range-attacks.md │ └── 03__fork-choice.md ├── 01__consensus │ ├── images │ │ └── 03__simple-consensus │ │ │ ├── naive1.png │ │ │ ├── naive2.png │ │ │ ├── naive3.png │ │ │ ├── setup.png │ │ │ ├── robust1.png │ │ │ ├── robust2.png │ │ │ ├── robust3.png │ │ │ ├── robust4.png │ │ │ ├── robust5.png │ │ │ ├── robust6.png │ │ │ ├── robust7.png │ │ │ ├── robust8.png │ │ │ └── robust9.png │ ├── 00__index.md │ ├── 02__terminology.md │ ├── 04__crash-faults.md │ ├── 01__origins.md │ ├── 06__applications.md │ ├── 05__byzantine-faults.md │ └── 03__simple-consensus.md └── 00__introduction │ ├── 02__content.md │ ├── 00__foreword.md │ └── 01__motivation.md ├── crowdin.yml ├── index.md ├── .gitignore ├── .vuepress ├── public │ └── _redirects ├── styles │ ├── index.styl │ └── palette.styl ├── enhanceApp.js └── config.js ├── package.json ├── .github └── workflows │ └── build-pdf.yaml ├── LICENSE ├── README.md └── CONTRIBUTING.md /book/README.md: -------------------------------------------------------------------------------- 1 | hello 2 | -------------------------------------------------------------------------------- /book/99__dev/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Miscellany" 3 | --- 4 | -------------------------------------------------------------------------------- /book/02__blockchains/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Blockchains" 3 | --- 4 | -------------------------------------------------------------------------------- /book/04__eth2-overview/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Eth2 Overview" 3 | --- 4 | -------------------------------------------------------------------------------- /book/03__eth1/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "An Introduction to Eth1" 3 | --- 4 | -------------------------------------------------------------------------------- /book/06__building-blocks/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Building Blocks" 3 | --- 4 | -------------------------------------------------------------------------------- /book/07__beacon-chain/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "The Beacon Chain" 3 | --- 4 | -------------------------------------------------------------------------------- /book/99__dev/02__notes.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Notes" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | -------------------------------------------------------------------------------- /book/07__beacon-chain/03__structure.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Structure" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | -------------------------------------------------------------------------------- /crowdin.yml: -------------------------------------------------------------------------------- 1 | files: 2 | - source: /book/**/*.* 3 | translation: '%original_path%/%original_file_name%' 4 | -------------------------------------------------------------------------------- /book/06__building-blocks/04__committees.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Committees" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | -------------------------------------------------------------------------------- /book/06__building-blocks/03__ssz.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Simple Serialize (SSZ)" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | -------------------------------------------------------------------------------- /book/06__building-blocks/05__deposit-contract.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "The Deposit Contract" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | -------------------------------------------------------------------------------- /book/03__eth1/images/proof-of-work/lcr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/03__eth1/images/proof-of-work/lcr.png -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | home: true 3 | tagline: 4 | actionText: reee i want 2 read book now → 5 | actionLink: /book/00__introduction/00__foreword.md 6 | --- 7 | -------------------------------------------------------------------------------- /book/03__eth1/images/proof-of-work/uncle-blocks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/03__eth1/images/proof-of-work/uncle-blocks.png -------------------------------------------------------------------------------- /book/03__eth1/images/proof-of-work/lcr-ghost-agree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/03__eth1/images/proof-of-work/lcr-ghost-agree.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/lmd-ghost/lmd-ghost.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/lmd-ghost/lmd-ghost.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/naive1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/naive1.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/naive2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/naive2.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/naive3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/naive3.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/setup.png -------------------------------------------------------------------------------- /book/03__eth1/images/proof-of-work/lcr-ghost-disagree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/03__eth1/images/proof-of-work/lcr-ghost-disagree.png -------------------------------------------------------------------------------- /book/06__building-blocks/images/randomness/randao-mix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/06__building-blocks/images/randomness/randao-mix.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust1.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust2.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust3.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust4.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust5.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust6.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust7.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust8.png -------------------------------------------------------------------------------- /book/01__consensus/images/03__simple-consensus/robust9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/01__consensus/images/03__simple-consensus/robust9.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/casper-ffg/ffg-finalized-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/casper-ffg/ffg-finalized-1.png -------------------------------------------------------------------------------- /book/06__building-blocks/images/randomness/randao-mix-open.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/06__building-blocks/images/randomness/randao-mix-open.png -------------------------------------------------------------------------------- /book/06__building-blocks/images/randomness/vdf-structure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/06__building-blocks/images/randomness/vdf-structure.png -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | pids 2 | logs 3 | node_modules 4 | npm-debug.log 5 | coverage/ 6 | run 7 | dist 8 | .DS_Store 9 | .nyc_output 10 | .basement 11 | config.local.js 12 | basement_dist 13 | -------------------------------------------------------------------------------- /.vuepress/public/_redirects: -------------------------------------------------------------------------------- 1 | /book/00__introduction/00__forward /book/00__introduction/00__foreword.html 2 | /book/00__introduction/00__forward.html /book/00__introduction/00__foreword.html 3 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/lmd-ghost/lmd-ghost-messages.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/lmd-ghost/lmd-ghost-messages.png -------------------------------------------------------------------------------- /book/06__building-blocks/images/randomness/randao-mix-closed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/06__building-blocks/images/randomness/randao-mix-closed.png -------------------------------------------------------------------------------- /book/06__building-blocks/images/randomness/randao-with-vdfs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/06__building-blocks/images/randomness/randao-with-vdfs.png -------------------------------------------------------------------------------- /.vuepress/styles/index.styl: -------------------------------------------------------------------------------- 1 | /** 2 | * Custom Styles here. 3 | * 4 | * ref：https://v1.vuepress.vuejs.org/config/#index-styl 5 | */ 6 | 7 | .home .hero img 8 | max-width 450px!important 9 | -------------------------------------------------------------------------------- /book/02__blockchains/images/fork-choice-rules.md/uncle-blocks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/02__blockchains/images/fork-choice-rules.md/uncle-blocks.png -------------------------------------------------------------------------------- /book/06__building-blocks/images/bls-signatures/wholesome-sigs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/06__building-blocks/images/bls-signatures/wholesome-sigs.png -------------------------------------------------------------------------------- /book/02__blockchains/images/fork-choice-rules.md/lcr-ghost-agree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/02__blockchains/images/fork-choice-rules.md/lcr-ghost-agree.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/blocks-slots-and-epochs/pos-epochs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/blocks-slots-and-epochs/pos-epochs.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/blocks-slots-and-epochs/pos-height.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/blocks-slots-and-epochs/pos-height.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/blocks-slots-and-epochs/pow-height.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/blocks-slots-and-epochs/pow-height.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/lmd-ghost/validator-attacks-ghost.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/lmd-ghost/validator-attacks-ghost.png -------------------------------------------------------------------------------- /book/02__blockchains/images/fork-choice-rules.md/lcr-ghost-disagree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/02__blockchains/images/fork-choice-rules.md/lcr-ghost-disagree.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/blocks-slots-and-epochs/ebb-first-slot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/blocks-slots-and-epochs/ebb-first-slot.png -------------------------------------------------------------------------------- /book/07__beacon-chain/images/validator-life-cycle/validator-life-cycle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/07__beacon-chain/images/validator-life-cycle/validator-life-cycle.png -------------------------------------------------------------------------------- /book/05__proof-of-stake/images/blocks-slots-and-epochs/ebb-no-first-slot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/smartcontracts/ethereum-book/HEAD/book/05__proof-of-stake/images/blocks-slots-and-epochs/ebb-no-first-slot.png -------------------------------------------------------------------------------- /.vuepress/styles/palette.styl: -------------------------------------------------------------------------------- 1 | /** 2 | * Custom palette here. 3 | * 4 | * ref：https://v1.vuepress.vuejs.org/zh/config/#palette-styl 5 | */ 6 | 7 | $accentColor = #3eaf7c 8 | $textColor = #2c3e50 9 | $borderColor = #eaecef 10 | $codeBgColor = #282c34 11 | -------------------------------------------------------------------------------- /book/07__beacon-chain/02__validator-duties.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Validator Duties" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ::: tip NOTE 8 | I'm not exactly sure where I want to take this section. Perhaps could be used as an opportunity to explain the content of blocks. 9 | ::: 10 | -------------------------------------------------------------------------------- /.vuepress/enhanceApp.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Client app enhancement file. 3 | * 4 | * https://v1.vuepress.vuejs.org/guide/basic-config.html#app-level-enhancements 5 | */ 6 | 7 | export default ({ 8 | Vue, // the version of Vue being used in the VuePress app 9 | options, // the options for the root Vue instance 10 | router, // the router instance for the app 11 | siteData // site metadata 12 | }) => { 13 | // Empty for now 14 | } 15 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "eth2-book", 3 | "version": "0.0.1", 4 | "description": "A book about Ethereum", 5 | "author": "Kelvin Fichter", 6 | "license": "MIT", 7 | "repository": { 8 | "type": "git", 9 | "url": "https://github.com/smartcontracts/eth2-book.git" 10 | }, 11 | "scripts": { 12 | "dev": "vuepress dev .", 13 | "build": "vuepress build .", 14 | "pdf": "vuepress export ." 15 | }, 16 | "devDependencies": { 17 | "@maginapp/vuepress-plugin-katex": "^1.0.11", 18 | "vuepress": "^1.5.3", 19 | "vuepress-plugin-export": "https://github.com/ulivz/vuepress-plugin-export#01418c8" 20 | } 21 | } 22 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Proof of Stake in Eth2" 3 | --- 4 | 5 | Proof of Stake in Eth2 represents a large departure from the Proof of Work rules that governed Eth1. Our deep dive into Eth2 begins with an explanation of the theoretical basis of Proof of Stake. This basis allows us to explore in later chapters the Beacon Chain, Eth2's implementation of a Proof of Stake chain based on the ideas described here. 6 | 7 | Certain mechanisms described within this chapter can be implemented in a variety of ways. We typically present such mechanisms as "black boxes" and leave their implementation details within Eth2 for our chapters regarding the Beacon Chain. Any simplifications made within this chapter serve only to aid explanation and do not attempt to "hand-wave" important conceptual topics. 8 | -------------------------------------------------------------------------------- /.github/workflows/build-pdf.yaml: -------------------------------------------------------------------------------- 1 | name: Generate and upload PDF 2 | 3 | on: 4 | push: 5 | branches: 6 | - main 7 | 8 | jobs: 9 | build-pdf: 10 | runs-on: ubuntu-18.04 11 | 12 | steps: 13 | - uses: actions/checkout@v2 14 | 15 | - name: Setup Node.js 16 | uses: actions/setup-node@v2 17 | with: 18 | node-version: '16.x' 19 | 20 | - name: Setup Java 21 | uses: actions/setup-java@v2 22 | with: 23 | distribution: 'temurin' 24 | java-version: '8' 25 | 26 | - name: Install dependencies 27 | run: yarn 28 | 29 | - name: Build PDF 30 | run: yarn pdf 31 | 32 | - name: Create PDF output file 33 | run: mv 'Kelvin\'s Ethereum Book.pdf' book.pdf 34 | 35 | - name: Upload PDF to IPFS 36 | uses: aquiladev/ipfs-action@v0.2.1-alpha.4 37 | with: 38 | path: './book.pdf' 39 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | (The MIT License) 2 | 3 | Copyright 2019-2022 Kelvin Fichter 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining 6 | a copy of this software and associated documentation files (the 7 | "Software"), to deal in the Software without restriction, including 8 | without limitation the rights to use, copy, modify, merge, publish, 9 | distribute, sublicense, and/or sell copies of the Software, and to 10 | permit persons to whom the Software is furnished to do so, subject to 11 | the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be 14 | included in all copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 20 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 21 | TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 22 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 23 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/06__weak-subjectivity.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Weak Subjectivity" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Casper FFG guarantees that $\smash{\frac{2}{3}}$ of validators have made a decision about the chain that they will not revert later. Whenever a client sees a new finalized checkpoint, they will never accept a fork from before the checkpoint. Clients are effectively immune from long range attacks because they've received a promise from validators in advance. 8 | 9 | However, we face one final, fundamental dilemma. An attacker might not be able to trick existing clients into accepting a conflicting promise. Unfortunately, the client must determine the validity of a promise for themselves. A new client that wasn't online when the first promise was made has no way to tell between the "real" and "fake" promises. 10 | 11 | This dilemma sits at the heart of Proof of Stake. Termed the "weak subjectivity problem," it requires that anyone offline longer than the time for $\smash{\frac{2}{3}}$ of validators to withdraw their funds rely on some external source to determine the "primary" chain. 12 | 13 | ::: tip TODO 14 | Need to finish this section about weak subjectivity. 15 | ::: 16 | -------------------------------------------------------------------------------- /book/01__consensus/00__index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Consensus Mechanisms" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Blockchains (Ethereum included) are ultimately about coming to consensus about the current state of some system (usually a financial system, but not always). It's therefore necessary to understand some of the more fundamental theory behind the protocols that allow computers on a network to come to decisions -- consensus mechanisms. If you truly wish to understand *why* Ethereum is being built in the way it's being built, becoming familiar with traditional consensus mechanism theory is a requirement. This is particularly true in the context of Ethereum's Proof of Stake consensus protocol which returns to more traditional ideas in the field of consensus mechanisms. 8 | 9 | People have been working on consensus mechanisms for decades and there's a mountain of literature about the subject. We've tried to distill this literature into the most important pieces. In this chapter we talk about the key concepts and findings in consensus mechanism research. By the end of this chapter, you should have a solid understanding of what a consensus mechanism attempts to do and the limitations that designers of such mechanisms face. 10 | -------------------------------------------------------------------------------- /book/03__eth1/01__basics.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Basics" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Eth1, or simply Ethereum, is a Proof-of-Work blockchain system that first went into production in 2014. We care about Eth1 for several reasons. Eth1 was the first project to employ many design decisions now standard in many other blockchain systems. These influences carry over to Eth2, the primary subject of this text. Many choices made in the construction of Eth2 are a direct or indirect result of lessons learned from observations of Eth1. It's therefore valuable to understand the core components of Eth1 in order to achieve a greater understanding of Eth2. 8 | 9 | We've already explored the concept of Proof-of-Work in quite some detail in the previous chapter. This chapter instead focuses on the execution layer of Eth1. We examine carefully the manner in which state and changes to that state are represented in Eth1. We explicitly dissect the structure of Eth1 state, transactions, and blocks. We additionally analyze the mechanism that allows Eth1 to host much more complex applications than its predecessors. Finally, we build an understanding of the hurdles Eth1 faces to achieve long-term growth. 10 | 11 | ::: tip TODO 12 | TODO: This became more of an introductory section. Doesn't make much sense to go into more detail here. Probably want to turn this into a "real" introduction. 13 | ::: 14 | -------------------------------------------------------------------------------- /book/03__eth1/06__fork-choice-rule.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Fork Choice Rule" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | We covered the concept of blockchain forks in our previous chapter. In brief summary, forks can occur if nodes produce two or more blocks that reference the same parent block. Nodes must select one of these blocks to consider "canonical," as conflicting blocks may contain mutually exclusive transactions. Nodes can, in theory, make this determination in any manner they'd wish. However, blockchain systems provide a particular set of rules, called a "fork choice rule," that nodes are expected to follow. Fork choice rules are a critical part of any blockchain protocol and ensure that a "main" popular chain reinforces the economic incentives that drive the chain. This section explains Eth1's fork choice rule and clarifies certain misconceptions of its operation. 8 | 9 | Eth1 makes use of the "longest chain rule" discussed earlier in this text. The algorithm, often shortened to the "LCR," dictates that the canonical chain be that with the highest total difficulty. Difficulty in this case refers to the target Proof-of-Work value of each block. Although we've already addressed this subject, one can refer to the following diagram for a brief refresher: 10 | 11 | ::: tip TODO 12 | Diagram 13 | ::: 14 | 15 | Some resources available on the internet suggest, incorrectly, that Eth1 makes use of the alternative GHOST fork choice rule. GHOST, as we know, takes the difficulty of ommer blocks into account during its chain determination. Eth1 does not consider ommers within its fork choice rule, but does address them elsewhere. Producers of ommers referenced in another block are given a "consolation" reward of sorts, currently set at `TODO` ETH, as opposed to the standard `TODO` ETH reward. This reward is designed to minimize the economic cost of ommer blocks to their producers and thereby encourage the existence of small-scale mining operations. 16 | 17 | Eth1 additionally factors ommer production rate into its difficulty adjustment algorithm. When a block contains many ommer references, overall difficulty increases, and vice versa. This has two primary effects on mining as a whole. An uptick in the ommer rate suggests that many nodes are generating blocks at approximately the same time. Increased difficulty helps to decrease these events and, as a result, reduce unnecessary chain forks. Adjustments based on ommer rate simultaneously limit the profitability of attempts to deliberately manipulate the ommer rate for personal gain. 18 | 19 | Luckily, the Eth1 fork choice rule doesn't introduce much novel subject matter. At the close of this section, we've covered almost every core element of Eth1. The remaining sections take a look at the ecosystem around Eth1 and serve to motivate the need for Eth2. Please take a moment to congratulate yourself for making it to this point! We've establish a large majority of the core concepts necessary to deeply understand Eth2. 20 | -------------------------------------------------------------------------------- /book/04__eth2-overview/04__development-process.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Development Process" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Though a large portion of its structure has already been specified, Eth2 is very much still a work in progress. From elements of the platform's application layer to initial implementations of the protocol as a whole, Eth2 leaves plenty of room for contributions by community participants. We've dedicated an entire chapter of this book to cataloging the various components of Eth2 that could benefit from additional attention. We further spend some time detailing the structure and processes behind the Eth2 community. This section acts as a brief summary of those ecosystem properties and of the history of more concrete development work. 8 | 9 | Early work on Eth2 began, in some ways, even before the launch of Eth1. Time and resource constraints limited most of these efforts to a more theoretical domain. Groups within the Ethereum Foundation laid the foundations for Eth1's Proof-of-Stake protocol as early as 2014. Many of these initial inroads served less as serious contenders to Eth1 and more as kindling to light the fire in the collective imaginations of the Ethereum community. With mind share primarily allocated to Eth1, Eth2 remained a long-term vision. 10 | 11 | The Eth2 landscape shifted drastically during 2018. Increased demand on the Eth1 platform once again highlighted the need for improvements in the name of long-term sustainability. Between 2018 and 2019 Eth2 went from a theoretical future to a highly-specified reality. Contributors from countless corners of the Ethereum ecosystem helped to develop a series of concrete design documents that detailed Eth2's proposed operation. By the end of 2019, these documents culminated in the first interactions among seven unique Eth2 client software packages. Though it may have once seemed a farfetched concept, Eth2 is now very much a reality. 12 | 13 | ::: tip TODO 14 | TODO: Insert phases here. 15 | ::: 16 | 17 | Rapid development of Eth2 in the recent past highlights the power of Ethereum's open community structure. Quickly after its launch, Eth1 managed to attract a diverse base of individuals willing to contribute back to the ecosystem. To this day, most Ethereum-related research is carried out in public. Platforms like Ethresearch act as forums in which anyone may read, publish, or give feed back on new ideas. Eth2 development efforts are chronicled in various conference calls held on regular intervals. Almost all Ethereum-related software projects are hosted on platforms like GitHub that actively encourage participation from new contributors. 18 | 19 | We give significantly more detailed overviews of the various Eth2 projects and collaboration platforms in our later chapter exploring the Eth2 ecosystem. However, we've included several sidebars within the following chapters that point readers to relevant projects. Readers are encouraged to look into these sidebars if their contents seem interesting. Ethereum truly makes research and development accessible to an extent unmatched in most project ecosystems. It's likely that any Eth2 project one encounters will be more than happy to receive additional assistance. 20 | -------------------------------------------------------------------------------- /book/01__consensus/02__terminology.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Common Terminology" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | We now direct this chapter into a more detailed analysis of the construction of fault-tolerant distributed computing systems. We begin this study with definitions of certain key concepts used in the specifications of these systems. Exact terminology employed within this book is, whenever possible, faithful to original language of "the literature," and may occasionally rather poorly summarize underlying concepts. We attempt to provide additional context and exposition when we find this to be the case. 8 | 9 | ## Nodes 10 | 11 | Our brief history of fault tolerance thus far has referred in passing to concepts like "computers" and "networks," among others. These are, in practice, highly technical and varied elements. Computers can be constructed in countless ways and still achieve the goal of "computing." One can verify this statement within the span of a brief journey to the nearest electronics store. Physical network connections are similarly diverse in both form and function. It's for these reasons that the field of computer science has developed useful conceptual abstractions that smooth out differences between real components. 12 | 13 | Computers are, unsurprisingly, the foundation of computational systems. However, it's not necessary that the entities interacting within a networked system operate on distinct hardware. Indeed, computer hardware has been capable of running multiple processes in parallel since the mid-20th century. Computer scientists therefore often refer to "nodes" as abstractions of computers on a network. Nodes can be thought of as unique entities on a network that have some computational processing power and access to local memory storage. 14 | 15 | ## Networks 16 | 17 | Communication lines between computers are similarly abstracted to remove any notion of an underlying physical medium. Whether information is transferred via electromagnetic waves, sound waves, or even carrier pigeon makes no difference to our analysis. We instead depict "logical" connections between nodes, over which nodes can send distinct packets of information called "messages." We further assume that some unknown amount of time must elapse between the moment that a message is sent and the moment at which it is received. 18 | 19 | Lines of communication are necessary but not sufficient for collaboration between nodes. Just as humans need languages, nodes must share a vocabulary of sorts in oder to successfully send and receive information. These communication "protocols" allow nodes to understand the meaning of messages on the network. We generally do not define any specific communication protocol in theoretical network studies, though we do assume that one exists. 20 | 21 | ## Protocols 22 | 23 | We also use the term "protocol" to refer to the set of actions nodes in a system are expected to take at any given time. Since node behavior defines the overall properties of a system, it's this protocol that we must now construct. Our primary goal, as previously noted, is to ensure that nodes be able to come to agreement about a particular set and ordering of events. We therefore come to call this construction a "consensus protocol." 24 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Kelvin's Ethereum Book 2 | 3 | I started working on this book on more than two years ago while taking a year-long leave of absence from more traditional work in the crypto space. 4 | The Quilt team over at [ConsenSys](https://consensys.net/) was kind enough to sponsor development of this work for a period of several months. 5 | I also recieved a grant for parts of this book via [MolochDAO](https://molochdao.com/) for which I'm extremely grateful. 6 | Unfortunately, I wasn't able to finish the book before I ran out of money and turned back towards full-time employment (with [Optimism](https://www.optimism.io/), where I still currently work). 7 | 8 | This book was originally titled "The Eth2 Book". 9 | I've since decided that I care more about writing a book about all of the beauty of Ethereum, rather than focusing just on the Eth2 components. 10 | I'm now in the process of reworking the book to be about Ethereum in general, so you'll probably still see lots of references to Eth2 while I clean things up. 11 | 12 | So here we are, with Kelvin's Ethereum Book. 13 | 14 | ## On the general state of the book 15 | 16 | Currently, the book is mostly unpolished and unedited. 17 | Large portions of the skeleton of the book have been filled in, but there are likely to be many missing details, spelling errors, and grammatical mistakes. 18 | My philosophy in writing this book has been a "breadth-first" approach as I find that overall content structure and layout is the most important part of an educational work. 19 | Smaller details, like headings, links, citations, etc., are best to be left toward the end of the project (lest we remove an entire section and waste a significant amount of effort). 20 | 21 | I still feel this is the right way to work on a project of this magnitude, but I acknowledge that it may make parts of the book hard to read in its current state. 22 | I hope that by making the work public I can start to crowd-source a lot of these smaller details. 23 | Anyway, this is all to say that the book is definitely not complete in any sense of the word and I hope you'll bear with me as the kinks get worked out. 24 | 25 | It's also worth noting that Eth2 has changed quite a bit since this project was started. 26 | Some parts of this book (particularly in the later chapters) are likely wholly inaccurate or outdated. 27 | As referenced in the above section regarding the Eth2 rebrand, it's a top priority to review the book for any inaccuracies and to make sure that it matches the narrative around Eth2 as it stands today. 28 | 29 | ## Reading the book 30 | 31 | Now that you've read all of my disclaimers, you can find the latest version of the book at the following link: 32 | 33 | https://eth2.incessant.ink 34 | 35 | Content on that page should update automatically whenever changes are made to this repository. 36 | 37 | ## Contributing 38 | 39 | Any and all contributions to this work are very much appreciated. 40 | As a way of saying thank you to contributors, I will be creating hand-drawn NFTs for everyone who submits suggestions, edits, or other similar contributions (on Optimism, because I'm biased and also because it's way cheaper than sending NFTs on L1 and I'm not made of money). 41 | You'll also be credited as a contributor within the book itself. 42 | 43 | Please see the [Contributing Guide](CONTRIBUTING.md) for more information about the contributing process. 44 | -------------------------------------------------------------------------------- /book/04__eth2-overview/05__learning-goals.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Learning Goals" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | As a final preparation before we dive into technical details, we'll describe our learning strategy and goals. At a high level, our goal for readers is to understand both the overall structure and particular elements of Eth2. The remainder of this text attempts to be complete in its description of Eth2. Such completeness often demands a reasonable level of pedantry, sometimes to the extreme of a software specification. However, we try to provide readers with flexible content pathways when heavy detail is not of particular interest. In general, readers will find that this strategy corresponds to a tendency to construct a "black box" version of a concept before picking at its innards. 8 | 9 | Our separation of component effect and implementation is reflected at an even higher level. We follow in the precedent of previous chapters and construct a theoretical basis for Eth2 before any software-level details. This format is important -- it's much more likely that theory influence practice than vice-versa. Many implementation specifics can only be understood as an attempt to satisfy some theoretical requirement. We additionally hope that this structure will help to separate theory from practice, thereby exposing alternative ways in which some theoretical property can be realized. 10 | 11 | Just as with Eth1, we construct Eth2 first through its consensus mechanism, then its application layer, and finally its low-level client details. This strategy largely follows both the manner in which Eth2 has been designed and the various development phases of the platform's planned release. Eth2 may be extensive, but it is far from dull. Every layer to the system, down even to the networking level, introduces new and interesting ideas and techniques. In combination with our flexible content structure, we hope that this fact will keep content fresh no matter the level of technical detail. 12 | 13 | More concretely, the following chapters largely reflect the "key concepts" described not too long ago. We kick off with a deep dive into Eth2's Proof-of-Stake protocol and the beacon chain that maintains this protocol. This is followed by a shift to the application layer. Shard chains are brought to life by Ewasm and execution environments. The latter two are themselves animated by a series of examples of functional Eth2 applications. We then finally transition into low-level details of topics, like Eth2's networking protocol and light-client designs. After our more technical content, we provide readers with the tools necessary to become more familiar with and contribute to the Eth2 ecosystem. 14 | 15 | Readers are encouraged to approach this text at their own pace. The scale of Eth2 can seem daunting when viewed in its entirety. We have attempted to deliberately structure content as to allow for reading in chunks of variable length. As previously explained, we give explicit options for readers with respect to the level of detail they wish to explore a subject. Readers should feel comfortable approaching a subject at a higher level when given the option to do so. Even so, we clearly identify topics that are crucial and therefore not to be skipped. Those less familiar with Eth2 should most likely approach all content in chronological order. Any reader, however, always has the option to bypass topics with which they already feel sufficiently acquainted. 16 | -------------------------------------------------------------------------------- /book/07__beacon-chain/01__validator-life-cycle.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Validator Life-Cycle" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ::: tip NOTE 8 | Section needs some a stronger vision. 9 | ::: 10 | 11 | ::: tip TODO 12 | Need some sort of intro here, but it largely depends on how we start this chapter. 13 | ::: 14 | 15 | ![Validator life-cycle diagram](./images/validator-life-cycle/validator-life-cycle.png) 16 | 17 | ::: tip TODO 18 | Temporary diagram, needs to be rebuilt before publication. 19 | ::: 20 | 21 | ::: tip TODO 22 | Some sort of fluff about this diagram, just skipping into content for now. 23 | ::: 24 | 25 | ## Joining the System 26 | 27 | The process of becoming an Eth2 validator begins by submitting a deposit of 32 ETH as collateral in case the validator violates any of our slashing rules. Since ETH currently only lives inside Eth1, these deposits are handled by a special "deposit contract" on the Eth1 chain. 28 | 29 | ::: tip NOTE 30 | I don't think this is the right place to talk about the deposit contract in detail, but not sure. 31 | ::: 32 | 33 | Potential validators send a transaction to the deposit contract that contains 32 ETH and certain authentication credentials. Once their deposits have been submitted, users must wait for their deposits to be recognized and processed on Eth2. The Beacon Chain will only process a limited number of deposits within each block, so users may face in implicit waiting period in the event that many others have submitted deposits. 34 | 35 | After their deposits are processed on the Beacon Chain, users are required to wait for an additional period of four epochs before becoming active validators. 36 | 37 | ## Leaving the System 38 | 39 | The process of leaving the Beacon Chain is split into two stages, the "exit" stage followed by the "withdrawal" stage. Any validators in the process of leaving the system is first placed into the "exit" stage, during which these validators no longer participate in consensus but cannot yet withdraw any funds. After an additional waiting period, validators in the "exit" stage enter the "withdrawal" stage and may withdraw their deposits. 40 | 41 | ### Slashed Exits 42 | 43 | Validators caught violating slashing conditions are removed from the validator pool after a period of four epochs. As an additional penalty for their actions, these validators are required to wait for `2^13 epochs`, approximately 36 days, before they are able to withdraw their funds. 44 | 45 | ### Insufficient Balance Exits 46 | 47 | In accordance with our stake leaking mechanism, validators will lose some portion of their deposit (without violating slashing conditions) whenever they fail to produce blocks during their assigned slots. Once a validator's balance drops below a certain threshold, currently defined as 16 ETH, they are automatically exited after a period of four epochs. 48 | 49 | Validators who are exited in this manner are required to wait an additional `2^8 epochs` before they are able to withdraw their funds. This represents the withdrawal delay that gives the network enough time to discover and publish evidence of misbehavior on the part of the validator. During this period, validators can still receive any unpaid rewards or be slashed. If a validator is slashed, they must wait the additional `2^13 epochs` of standard slashed exits. 50 | 51 | ### Voluntary Exits 52 | 53 | Validators who no longer wish to participate in the protocol can choose to submit a voluntary exit. A validator exiting this way must wait for a period of `2^9 epochs` and be processed through the FIFO queue of exits. Just as with insufficient balance exits, validators exiting voluntarily must then wait an additional `2^8 epochs` before withdrawal, during which the validator can receive rewards or be slashed. 54 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/01__core-concepts.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Core Concepts" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Proof of Stake chains, in a nutshell, attempt to replicate the functionality of Proof of Work chains without the reliance on wasteful computational processes. We can make several useful analogies to Proof of Work in order to get a better understanding of the core components of Proof of Stake. 8 | 9 | ## Validators 10 | 11 | As we've previously discussed, Proof of Work chains are extended by **miners** who participate in a "lottery" for the right to produce additional blocks. The probability that any given miner "wins" this lottery is directly proportional to that miner's hash power as a fraction of total hash power. A miner with ten percent of all hash power, for instance, has approximately a ten percent chance to produce the next block. Hash power effectively acts as a proxy measurement for economic investment in the system. 12 | 13 | Proof of Stake chains measure investment *directly* in the form of capital lock-up, instead of *indirectly* through hash power. Users on a Proof of Stake chain may deposit some form of economic value (in the case of Eth2, Ether) in order to receive the right to some influence over the system. Such users are known as **validators**. 14 | 15 | Validators in Proof of Stake can be thought of as replacements for miners in Proof of Work, though their roles are not identical. A validator's purpose within a Proof of Stake system is to perform certain tasks, like block production, that allow the system to continue to operate. Just as miners are rewarded for producing new blocks, validators are rewarded for successfully carrying out any assigned tasks. 16 | 17 | ## Validator Role Selection 18 | 19 | Since Proof of Work chains rely on external investment, they need not keep track of which parties have control of hash power at any given time. However, investment in a Proof of Stake chain is explicitly recorded on the chain itself. As a result, in order to maintain the relationship between investment and influence, Proof of Stake chains assign roles to *known* validators. 20 | 21 | A Proof of Stake chain needs some process that, given the current validator set, assigns validators to perform various tasks. Such a selection process can take various different forms. Any potential process should, though, ensure that validator rewards are proportional to investment. 22 | 23 | For instance, we could trivially guarantee proportional influence with a "round-robin" selection process. A round-robin system would give validators the right to produce a certain number of blocks in a row, based on their percentage of total owned stake. 24 | 25 | ::: tip TODO 26 | Image of round-robin selection. 27 | ::: 28 | 29 | Though simple, this sort of model is vulnerable to some methods of attack. For instance, as validator roles are known far in advance, a malicious actor has more time to orchestrate denial-of-service attacks against specific validators. 30 | 31 | ::: tip TODO 32 | Image of round-robin attacks. 33 | ::: 34 | 35 | We can reduce look-ahead with the introduction of some sort of unpredictability. We explore the specifics of this selection process in Eth2 during our later chapter regarding the Eth2 Beacon Chain. 36 | 37 | ## Slots 38 | 39 | So far, we've discussed this role-selection mechanism in the context of specific *blocks*. However, we quickly run into issues in the case that a validator fails to produce their assigned block. Unless other validators can choose to "skip" this block after some waiting period, the chain will halt indefinitely. 40 | 41 | ::: tip TODO 42 | Image of chain halt due to missed block. 43 | ::: 44 | 45 | Instead of assigning roles for specific *blocks*, a Proof of Stake chain therefore assigns roles for specific *periods of time*, referred to in Eth2 as "slots." Under this model, if a validator fails to produce a block during their assigned slot, other validators can continue to extend the chain. 46 | 47 | ::: tip TODO 48 | Image of no chain halt b/c slots. 49 | ::: 50 | -------------------------------------------------------------------------------- /book/99__dev/01__sources.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Sources" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ## Not Categorized 8 | 9 | - [State Transitions](https://www.youtube.com/watch?v=XIsn8-5Xekc) 10 | - [EVM](https://www.youtube.com/watch?v=OYfy1-5nB64) 11 | 12 | ## Eth2 General 13 | 14 | - [Serenity Design Rationale, The Ethereum Foundation](https://notes.ethereum.org/@vbuterin/rkhCgQteN) 15 | - [Ethereum 2.0 Phases, EthHub](https://docs.ethhub.io/ethereum-roadmap/ethereum-2.0/eth-2.0-phases/) 16 | - [Eth2.0 Specs](https://github.com/ethereum/eth2.0-specs) 17 | 18 | ## Casper FFG 19 | 20 | - [Casper the Friendly Finality Gadget, Vitalik Buterin and Virgil Griffith](https://web.archive.org/web/20191210050307/https://arxiv.org/pdf/1710.09437.pdf) 21 | 22 | ## GHOST 23 | 24 | - [What is Ethereum's Uncle Rate and Why Does It Matter?, ETH Gas Station](https://ethgasstation.info/blog/ethereum-uncle-rate/) 25 | 26 | ## LMD GHOST 27 | 28 | - [Rewriting History: A Brief Introduction to Long Range Attacks, Evangelos Deirmentzoglou](https://web.archive.org/web/20200115212240/https://blog.positive.com/rewriting-history-a-brief-introduction-to-long-range-attacks-54e473acdba9?gi=bd92d364dce3) 29 | - [Long Range Attacks, and a new Fork Choice Rule, Alexander Skidanov](https://web.archive.org/web/20200115204848/https://nearprotocol.com/blog/long-range-attacks-and-a-new-fork-choice-rule/) 30 | - [Decoy-Flip-Flop Attack on LMD GHOST, Ryuya Nakamura](https://web.archive.org/web/20191212052921/https://ethresear.ch/t/decoy-flip-flop-attack-on-lmd-ghost/6001) 31 | 32 | ## Randomness 33 | 34 | - [Two Point Oh: Randomness, Bruno Škvorc](https://web.archive.org/web/20191103205050/https://our.status.im/two-point-oh-randomness/) 35 | - [Two Point Oh: Justification and Finalization](https://our.status.im/two-point-oh-justification-and-finalization/) 36 | - [RANDAO GitHub](https://github.com/randao/randao/tree/25cfefe1adea1aeafce933cd4700cd3f150311c6) 37 | 38 | ## Phase 0 39 | 40 | - [Eth2 Phase 0 Technical Deep Dive](https://www.youtube.com/watch?v=N5DdClfLQfw) 41 | - [Phase 0 For Humans, The Ethereum Foundation](https://notes.ethereum.org/@djrtwo/Bkn3zpwxB?type=view) 42 | - [Ethereum 2.0 Phase 0 --- The Beacon Chain](https://github.com/ethereum/eth2.0-specs/blob/1a450c5a12c2b10bae0400818bc5257418afd2cf/specs/phase0/beacon-chain.md) 43 | 44 | ## Phase 1 45 | 46 | - [Ethereum 2.0 Phase 1 --- Custody Game](https://github.com/ethereum/eth2.0-specs/blob/1a450c5a12c2b10bae0400818bc5257418afd2cf/specs/phase1/custody-game.md) 47 | 48 | ## Phase 2 49 | 50 | - [ETH 2 Phase 2 Wiki](https://hackmd.io/UzysWse1Th240HELswKqVA) 51 | 52 | ## Execution Environments 53 | 54 | - [ETH 2 Phase 2 Wiki: Execution Environments](https://hackmd.io/UzysWse1Th240HELswKqVA?view#Execution-Environments) 55 | 56 | ## BLS 57 | 58 | - [BLS12-381 For The Rest Of Us, Ben Edgington](https://hackmd.io/@benjaminion/bls12-381) 59 | - [Digital Signature](https://en.wikipedia.org/wiki/Digital_signature) 60 | 61 | ## Ewasm 62 | 63 | - [State Execution in Eth 2.0](https://www.youtube.com/watch?v=8H1TCbW0LJQ) 64 | 65 | ## Light Clients 66 | 67 | - [Eth2 Light Clients: How Light is Light?, Cayman Nava](https://hackmd.io/@wemeetagain/SkuswKu_r) 68 | - [Minimal Light Client Design](https://github.com/ethereum/eth2.0-specs/blob/1a450c5a12c2b10bae0400818bc5257418afd2cf/specs/phase1/light-client-sync.md) 69 | 70 | ## Simple Serialize (SSZ) 71 | 72 | - [SimpleSerialize (SSZ)](https://github.com/ethereum/eth2.0-specs/blob/1a450c5a12c2b10bae0400818bc5257418afd2cf/ssz/simple-serialize.md) 73 | - [Eth2 Light Clients: How Light is Light?: SSZ, Cayman Nava](https://hackmd.io/@wemeetagain/SkuswKu_r#SSZ-Simple-Serialize) 74 | 75 | ## Ecosystem 76 | 77 | - [Eth2 Collaboration Culture](https://www.youtube.com/watch?v=ixGeIEgdf4Y) 78 | 79 | ## Sharding 80 | 81 | - [ETH 2 Phase 2 Wiki: Cross Shard Transactions](https://hackmd.io/UzysWse1Th240HELswKqVA?view#Cross-Shard-Transactions) 82 | 83 | ## Fee Markets 84 | 85 | - [ETH 2 Phase 2 Wiki: Fee Markets](https://hackmd.io/UzysWse1Th240HELswKqVA?view#Fee-Markets) 86 | -------------------------------------------------------------------------------- /book/06__building-blocks/01__bls-signatures.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "BLS Signatures" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ## Signatures 101 8 | 9 | Cryptographic signatures, or simply "signatures," are a key component of any blockchain. If you've ever interacted with a smart contract or sent crypto to someone, then you've created a signature! 10 | 11 | Signatures are pretty much what they sound like. Much like physical signatures, cryptographic signatures are meant to act as "stamps" that can be attached to digital messages. Just as physical signatures rely on a "secret" (your chosen method for scribbling your name onto paper), cryptographic signatures rely on a "secret" in the form of a "private key." 12 | 13 | Unlike with physical signatures, however, we don't actually reveal this private key when we create a cryptographic signature. Instead, all private keys have an associated "public key" that can be used to verify that a given signature comes from the corresponding private key, without revealing anything about the private key itself. 14 | 15 | ![Wholesome Signatures](./images/bls-signatures/wholesome-sigs.png) 16 | 17 | ## Signatures in Eth2: A Conundrum 18 | 19 | Validators create attach signatures to their blocks so that other users or validators can easily assert the block's veracity. Validators also attach signatures to other actions that they perform as part of their duties on the Beacon Chain. 20 | 21 | Traditionally, we haven't had much of a problem determining which *specific* signature scheme to use in a blockchain system. Most blockchains today use an algorithm called the "Elliptic Curve Digital Signature Algorithm," or ECDSA for short. It's a bit of a mouthful, so you don't really need to memorize that one. ECDSA signatures are relatively small (65 bytes, less space than the amount of data taken up on your computer by this sentence) and quick to verify (on the order of a few milliseconds). 22 | 23 | Eth2, however, is designed to operate at a scale significantly higher than that of traditional blockchains. We sometimes run into situations where we need to verify hundreds of thousands, of not millions, of signatures within a single epoch. At 65 bytes and two milliseconds per signature, it'd take six megabytes of storage and a full *30 minutes* to verify one million ECDSA signatures. Our Beacon Chain deserves better. 24 | 25 | ## BLS to The Rescue 26 | 27 | Researchers at Stanford, yes, the same ones who came up with VDFs, created a novel signature scheme that makes Eth2 possible. The "Boneh-Lynn-Shacham," or BLS, signatures are, in many ways, just like the traditional signatures used in other blockchains. However, they provide two key properties not found in other signature schemes. 28 | 29 | BLS signatures allow us to combine any number of signatures on any number of messages into a *single* signature with a constant size. Although these signatures are slightly larger than ECDSA signatures (96 bytes instead of 65 bytes), this means that we no longer need megabytes worth of space in blocks dedicated purely to signatures. 30 | 31 | BLS signatures, perhaps more importantly, also allow us to verify any number of signatures on the *same* message in constant time. Once aggregated, one million signatures on the same message can be verified in the about the same amount of time as a single signature on that message. Since validators are often signing the same message (a given block, perhaps), this optimization is extremely useful within the context of Eth2. 32 | 33 | ## Extras 34 | 35 | Signature schemes have three algorithms in combination, each probabilistic polynomial: 36 | 37 | G: key generator, generates a public key and corresponding private key on input 1^n where n is security parameter 38 | 39 | $$G(\text{security_parameter}) \rightarrow (\text{public_key}, \text{private_key})$$ 40 | 41 | S: Returns a signature on inputs pk and string 42 | 43 | $$S(\text{private_key}, \text{message}) \rightarrow \text{signature}$$ 44 | 45 | V: Outputs accepted or rejected on public key, message, signature 46 | 47 | $$V(\text{public_key}, \text{message}, \text{signature}) \rightarrow \{\text{accepted}, \text{rejected}\}$$ 48 | 49 | Correctness: 50 | 51 | $$Pr[ (pk, sk) \leftarrow G(1^n), V(pk, x, S(sk, x)) = \text{accepted}] = 1$$ 52 | 53 | $$Pr[ (pk, sk) \leftarrow G(1^n), (x, t) \leftarrow A^{S(sk, -)}(pk, 1^n), x notin Q, V(pk, x, t) = accepted] < negligible$$ 54 | -------------------------------------------------------------------------------- /book/04__eth2-overview/01__historical-context.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Historical Context" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | We're now approaching our detailed study of the various elements that make up Eth2. Topics discussed within previous chapters will prove useful throughout the entirety of our exploration. Before we dive into technical content, we should take a moment to prepare ourselves. This chapter takes a look at the history, motivations and philosophies behind Eth2. It additionally describes, briefly, the process by which Eth2 has been developed thus far. Finally, this chapter provides readers with some guidance regarding the structure of, and learning strategies for, content covered in the remainder of this work. 8 | 9 | This first section attempts to highlight the various challenges and advancements that served as motivation for the development of Eth2. Fundamental limitations to the scalability of most modern blockchains are clearly significant among other motivators. Systems that scale linearly are unable to satisfy demand at affordable prices without falling into traps of centralization. In the absence of sufficient throughput, blockchains are unlikely to attract the large global audience they target. Existing protocol-level solutions to this problem result in network centralization far out of line with traditional decentralization philosophy. Application-level solutions are throttled by base-layer performance and therefore useful but not sufficient for long-term sustainability. 10 | 11 | Proof-of-Work itself provides several challenges for blockchains, like Eth1, that rely on it to achieve consensus. Proof-of-Work consumes physical resources like electricity at eye-watering rates. Estimates place bitcoin's annual electricity consumption and carbon footprint at levels comparable to the entire countries of Austria and Denmark, respectively. Such excessive waste is clearly damaging to the planet's ecosystem and is not likely to thrive in the current global political environment. Constant hardware upgrades demanded by Proof-of-Work, wasteful themselves, increase the entry cost for new consensus participants. Proposals that attempt to modify a chain's Proof-of-Work algorithm, as to influence the type of hardware necessary for participation, cause seemingly community debate. A shift away from Proof-of-Work is clearly well motivated. 12 | 13 | Core hurdles in scalability and consensus did not go unnoticed by the designers behind Eth1. Many of the aforementioned dilemmas and their impacts on sustainability were explicitly factored into the Eth1 protocol. Vitalik Buterin has noted that limits to throughput were acknowledged from the project's outset. Buterin and others described serious alternatives to Proof-of-Work for Ethereum as early as 2014, more than a year before the system's initial launch date. These considerations were codified by the inclusion of a "difficulty bomb" designed to gradually decrease network performance over time and thereby incentivize the development of new consensus mechanisms. 14 | 15 | Though aware of challenges ahead, the teams that brought Eth1 to fruition can hardly be faulted for their design decisions. Eth1's virtual machine concept was already much more ambitious than earlier blockchain systems. Ethereum's $20m pre-sale cannot be considered insignificant, but pales in comparison to the billions raised by other projects via smart contracts deployed to the platform. Many technical elements of Eth2 simply could not have been constructed without the funding and research enabled by its predecessor. Eth1's strong and vast community of contributors has also without a doubt heavily advanced the state of research in numerous fields of mathematics and computer science. It's only now that Ethereum has become adequately equipped to tackle the various challenges it identified. 16 | 17 | Ethereum's community serves as the final motivator we discuss here, though one can likely identify many more. Eth1's growth has quickly outpaced its capacity for interactions on the network. As countless teams begin to turn some attention to long term sustainability, Eth2 research and development efforts have only accelerated. Ethereum's open community structure attracts individuals with a wide of backgrounds and perspectives. Much research on Eth2 is conducted in public over forums like Ethresearch or on large conference calls held on a regular basis. The ethereum community can very reasonably be attributed with tipping the scale that took Eth2 from a vague idea to a near-future reality in only the recent past. 18 | 19 | ::: tip TODO 20 | Needs conclusion. 21 | ::: 22 | -------------------------------------------------------------------------------- /book/02__blockchains/06__participation.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Network Participation" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Participation in a blockchain network can take many different forms. Readers are by now likely most familiar with the role and operation of nodes that participate in the consensus process. We've also touched on the actions of nodes that produce transactions but do not generate blocks. In practice, blockchain systems will typically make design decisions that allow nodes to operate under various requirements or environmental constraints. Nodes may, for example, be able to disable unnecessary features or even switch into special operational modes for resource-constrained environments. These accessibility improvements often have a significant impact on design decisions. 8 | 9 | Nodes that download and execute all transactions are usually called "full nodes." Miners almost always run full nodes as to quickly process any incoming blocks and speed up their production of new blocks. Full nodes have the important quality of self-sufficiency in that they store all information necessary to ascertain the validity of any given transaction. Many users choose to operate full nodes as to guarantee an up-to-date view of activity on a network. Node software will typically allow users to disable certain features, mining capability for example, if the user deems these features unnecessary. Users can in this way perhaps operate nodes that receive data from the network but do not actively send data to other nodes. 10 | 11 | Some blockchain systems make a distinction between full nodes and so-called "archival nodes." When this is the case, full nodes include the ability to remove, or "prune," information after some specified period of time. Most software that exposes this functionality will only prune data no longer necessary to process new transactions and is therefore considered stale. A UTXO-based system may, for instance, allow nodes to remove spent UTXOs that can, as a result, no longer be used within new transactions. However, as old information can become relevant in the event of a chain fork, software usually defines a large threshold of, say, 1000 blocks that must be appended to a block before its stale data can be pruned. 12 | 13 | Archival nodes then refer to those that do not prune information after a certain period of time. These nodes must often store an order of magnitude more information than non-archival counterparts. Within the context of Eth1, for illustration, full nodes store several hundred gigabytes of data whereas archival nodes store a whooping several terabytes. Archival nodes are commonly used by services like "block explorers" that provide users with the the ability to quickly search or parse blockchain data. These services may additionally elect to modify node software in order to collect useful, but not essential, information in a process called "instrumentation." Modifications of this sort can give much greater insight into a network in exchange for a significant increase in data storage requirements. 14 | 15 | Archival nodes and full nodes can easily expand beyond the reach of resource constrained environments like older laptops and even modern phones. At several hundred gigabytes, full nodes simply demand too much to be feasibly deployed on certain devices. Many blockchain systems make deliberate design decisions to support operation modes that trade off resource requirements for some reduction in security. These "light client" protocols typically aim to consume data at a rate logarithmic to the size of the entire blockchain. Light clients can therefore participate in the network and store only data on the order of a few gigabytes. 16 | 17 | One common approach to light client support requires a distinction between "block header" and "block body" components to a block. Within these systems, a block header represents a concise description of the contents of the block body. Specifically, the header will typically contain the roots of Merkle trees that contain the full list of, for example, transactions within the block body. Light clients can download specific pieces of information within a block by requesting a Merkle proof of inclusion for that particular data point. Light clients can therefore be sure that some information was included within a block at a cost of only the size of the Merkle proof. However, these clients cannot ascertain the validity of a block without access to the block's full contents. Light clients must instead use the Proof-of-Work for a block as a proxy for its validity. 18 | 19 | Distinctions between various types of clients can often be somewhat arbitrary. This is particularly apparent among full nodes that choose to enable or disable different software features. That said, the general definition of full, archival, and light clients can be found in many blockchain systems. New designs introduced in Eth2 only serve to blur any dividing lines further. Eth2 aims to use these changes to improve flexibility and, as a result, expand the platforms from which Eth2 is accessible. 20 | -------------------------------------------------------------------------------- /book/00__introduction/02__content.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Content Overview" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Eth2 is a sprawling technical undertaking with countless moving parts. In creating an educational resource about this subject, we've spent significant time and energy developing a content flow intended to make Eth2 accessible to a wide and diverse audience. Here's a taste of the topics covered within this book. 8 | 9 | ## The Basics of Consensus 10 | 11 | This book begins with an exploration of consensus, the process by which networks of computers can come together to make decisions. Eth2's new Proof-of-Stake consensus protocol is in many ways a return to more traditional ideas in the world of consensus theory. Understanding these core concepts is critical to understanding the limitations in which Ethereum's consensus protocol exists. In *The Basics of Consensus* we take a look at the origins of modern consensus mechanisms. In particular, we're aiming to build up a knowledge of the concept of Byzantine faults and state machines. These two key ideas lay the foundation for Ethereum's development. 12 | 13 | ## The Origins of Blockchains 14 | 15 | The leap from consensus mechanisms to blockchains isn't particularly obvious. In *The Origins of Blockchains* we try to understand how "old school" consensus theory gives rise to the idea of a blockchain when applied to digital finance. We'll see how many of the ideas we now take for granted (like Proof-of-Work and the concepts of "blocks" of transactions) came out of the desire to create effective and open digital economic systems. Although this is a review of concepts you may already feel familiar with, the history of these systems is quite interesting and worth understanding. 16 | 17 | ## Revisiting Eth1 18 | 19 | Our exploration of Eth2 begins with a foray into some of the fundamental elements of Eth1. The concepts and theories of Eth1 are, after all, the original basis of Eth2. One cannot easily begin understand Eth2 without first embarking on an archeological dig of its ancestral components. 20 | 21 | Through the lens of Eth1 we introduce many of the core aspects of modern blockchain protocols. This familiar ground serves as base camp for in-depth study of the models for blocks, transactions, and consensus mechanisms found in almost every blockchain system today. Many of the same fundamentals behind Eth1 can be found playing leading roles in the dance of Eth2. 22 | 23 | The elements of Eth1 also serve to reveal the design's inherent limitations. It is, of course, these very limitations that motivate the development of Eth2. Many of the guiding factors behind Eth2 can be best understood as direct responses to flaws in Eth1. We additionally note some of the various alternative approaches to these problems before we finally embark on our journey into Eth2 proper. 24 | 25 | ## Exploring Eth2 26 | 27 | Our mission is prefaced with a high-level and intuitive introduction to the pillars of Eth2. As such, readers can develop a mental model of the system to serve as reference during our deep-dives into its various components. The intricate elements of Eth2 are much more easily understood when contextualized to the the entirety of the system. In *Exploring Eth2* we take a high-level look at Eth2, try to understand some historical context behind the project, and explain the constraints within which the project is being developed. 28 | 29 | ## Proof of Stake in Ethereum 30 | 31 | Our first true technical rabbit-hole is that of Eth2's Proof of Stake protocol. Perhaps one of the greatest achievements within Eth2, this protocol represents the cumulative effort of countless individuals over the better part of the last decade. We approach this subject with a theoretical mindset, opting to leave implementation details to later chapters. Our study of Proof of Stake attempts to build an intuitive understanding of each of the system's mechanisms as responses to concrete problems. 32 | 33 | ## Eth2 Building Blocks 34 | 35 | *Eth2 Building Blocks* takes a look at the core technical components that make Eth2 possible. *Proof of Stake in Ethereum* introduces many concepts in the abstract, leaving more specific implementations for *Eth2 Building Blocks*. Here we explain, at a low level, how elements like RANDAO, BLS Signatures, and the Deposit Contract work under the hood. 36 | 37 | ## Building the Beacon Chain 38 | 39 | We expand on the theoretical base of Chapter 3 with a detailed exploration of its implementation, the Eth2 Beacon Chain. In *Building the Beacon Chain* we uncover how the conceptual operation of Eth2 is put into practice. We'll be using the building blocks described in *Eth2 Building Blocks* to properly implement the protocol described in *Proof of Stake in Ethereum*. By the end of this chapter, readers will be able to map the various elements of Eth2's Proof of Stake protocol to corresponding logic within the Beacon Chain. 40 | 41 | --- 42 | 43 | ::: tip TODO 44 | There will be more content after this, I just haven't figured out the structure for those chapters yet 45 | ::: 46 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Kelvin's Ethereum Book 2 | 3 | Thank you for looking into contributing to Kelvin's Ethereum Book! <3 4 | 5 | On this page you'll find the most important information about the contributing process. We've tried to make this as straightforward as possible. Thank you again for taking the time to contribute! 6 | 7 | The book is automatically created with [VuePress](https://vuepress.vuejs.org/). 8 | 9 | ## How can I contribute? 10 | 11 | ### Fixing small errors 12 | 13 | One of the easiest ways to contribute to this project is to fix small errors in the book. For these sort of smaller errors, we'd love if you could make a pull request fixing the issue. As long as the fix makes sense (typos are usually pretty obvious), there won't be much back-and-forth and we can just merge the PR. [Here's an example of a great pull request that fixes some typos within the book](https://github.com/smartcontracts/eth2-book/pull/15). 14 | 15 | ### Making subjective suggestions 16 | 17 | When deciding between creating an issue or making a pull request, a good rule of thumb is that if the fix is non-controversial (typos, factual errors, etc.) you should make a PR. For fixes that are more subjective, please [create an issue](https://github.com/smartcontracts/eth2-book/issue) that explains the fix. This allows us to review the suggestion and see if it fits within the content/structure of the book. That way we can avoid situations where you put in a lot of work to create a PR and we end up rejecting it. 18 | 19 | ### Giving general feedback 20 | 21 | This book is meant to be the best possible resource to learn about how Ethereum and Eth2 really work under the hood. We can only make that a reality if people are willing to give feedback about the content/flow/etc. We highly appreciate any sort of feedback. Please feel free to [create an issue](https://github.com/smartcontracts/eth2-book/issue) that includes your feedback so we can try to improve the book. 22 | 23 | ### Translating the book 24 | 25 | We're planning to translate Kelvin's Ethereum Book into as many languages as we can get away with. Translations are being handled automatically via [Crowdin](https://crowdin.com/). If you're interested in helping to translate the book, please join the [Eth2 Book Working Group](https://t.me/+QdbYrQtP0zE3ZDVh) on Telegram. Please note that the book is still a work in progress and only certain parts of the book are ready for translation. 26 | 27 | ### Adding new features 28 | 29 | Kelvin's Ethereum Book is, to an extent, an exploration of the website as a platform for long-form educational content. As a result, Kelvin's Ethereum Book is about 50% book and 50% software project. We love suggestions for new features and capabilities that take the book to the next level. See [here](https://github.com/smartcontracts/eth2-book/pull/7) for a great PR that adds a new feature, for an example. 30 | 31 | ## How to test a contribution 32 | 33 | Kelvin's Ethereum Book can be served locally so that you're able to see and test your changes live. You'll need to install the following: 34 | 35 | - [Node.js](https://nodejs.org/en/) (LTS should be fine, comes with `npm`) 36 | - [yarn](https://classic.yarnpkg.com/en/docs/install) (v1) 37 | 38 | Once you have those installed, you'll need to clone the repository and install its dependencies: 39 | 40 | ``` 41 | git clone https://github.com/smartcontracts/eth2-book.git 42 | cd eth2-book 43 | yarn install 44 | ``` 45 | 46 | Finally, you can serve the book locally with: 47 | 48 | ``` 49 | yarn dev 50 | ``` 51 | 52 | ## Stuff to keep in mind 53 | 54 | ### Only modify the primary English text 55 | 56 | As previously stated, we're trying to translate the book into many different languages. We're using [Crowdin](https://crowdin.com/) to manage the translation process. Since the book is still being written, only certain parts will be ready for translation at any given time. For any sections that have not yet been translated, the original English text will be used. Crowdin handles this duplication process automatically, so there's no need to update these other texts when making a fix. Simply update the English text and the other languages will update automatically. 57 | 58 | ## NFTs for contributors 59 | 60 | As a special thank you, anyone who makes a contribution to this book will receive a hand-drawn [Bookzombie](https://quixotic.io/collection/bookzombies) NFT minted on [Optimism](https://optimism.io). After you make a contribution, you'll be asked for an ETH address. You can (of course) always decline the NFT, it's your choice! 61 | 62 | ### Why NFTs for contributors? 63 | 64 | It's a book about Ethereum, so an NFT seems like a reasonable "thank you" card. I also have fun drawing and creating NFTs. I've found that for many people, the NFTs they receive for contributing will be their first NFT ever, so this is also something of a crypto recruiting mechanism ;-). 65 | 66 | ### Why is it zombie themed? 67 | 68 | Who knows! I was messing around with designs and thought it looked cool. 69 | 70 | ### Why is it on Optimism? 71 | 72 | Optimism is way cheaper than the Ethereum mainnet. I can't really afford to mint NFTs for everyone on mainnet, but I usually only pay about $1-2 per mint NFT on Optimism. 73 | 74 | ### How many Bookzombies can I get? 75 | 76 | I'll send you one Bookzombie per contribution! I'll try to add more cool stuff to your Bookzombie depending on the size and scope of your contribution. 77 | 78 | ### Can I sell my Bookzombie? 79 | 80 | Sure you can! Just don't get mad when Bookzombies become the next Crypto Punks ;-). 81 | -------------------------------------------------------------------------------- /.vuepress/config.js: -------------------------------------------------------------------------------- 1 | const { description } = require('../package') 2 | 3 | module.exports = { 4 | title: 'Kelvin\'s Ethereum Book', 5 | 6 | description: description, 7 | 8 | head: [ 9 | ['meta', { name: 'theme-color', content: '#3eaf7c' }], 10 | ['meta', { name: 'apple-mobile-web-app-capable', content: 'yes' }], 11 | ['meta', { name: 'apple-mobile-web-app-status-bar-style', content: 'black' }], 12 | ], 13 | 14 | locales: { 15 | '/book/': { 16 | lang: 'en-US', 17 | title: 'Kelvin\'s Ethereum Book', 18 | description: description, 19 | } 20 | }, 21 | 22 | themeConfig: { 23 | repo: 'smartcontracts/ethereum-book', 24 | editLinks: true, 25 | docsDir: 'src', 26 | docsBranch: 'main', 27 | editLinkText: 'Edit this page on GitHub', 28 | lastUpdated: false, 29 | algolia: { 30 | appId: '2KUJYE5BP1', 31 | apiKey: 'f07e286e9f7cb070f2cd23fc10cc1517', 32 | indexName: 'eth2-book' 33 | }, 34 | nav: [], 35 | sidebar: { 36 | '/book': [ 37 | { 38 | title: 'Introduction', 39 | collapsable: false, 40 | sidebarDepth: 1, 41 | children: [ 42 | '/book/00__introduction/00__foreword.md', 43 | '/book/00__introduction/01__motivation.md', 44 | '/book/00__introduction/02__content.md', 45 | ] 46 | }, 47 | { 48 | title: 'The Basics of Consensus', 49 | collapsable: false, 50 | sidebarDepth: 1, 51 | children: [ 52 | '/book/01__consensus/00__index.md', 53 | '/book/01__consensus/01__origins.md', 54 | '/book/01__consensus/02__terminology.md', 55 | '/book/01__consensus/03__simple-consensus.md', 56 | '/book/01__consensus/04__crash-faults.md', 57 | '/book/01__consensus/05__byzantine-faults.md', 58 | '/book/01__consensus/06__applications.md', 59 | ] 60 | }, 61 | { 62 | title: 'The Origins of Blockchains', 63 | collapsable: false, 64 | sidebarDepth: 1, 65 | children: [ 66 | '/book/02__blockchains/00__index.md', 67 | '/book/02__blockchains/01__digital-cash.md', 68 | '/book/02__blockchains/02__open-access.md', 69 | '/book/02__blockchains/03__chains-of-blocks.md', 70 | '/book/02__blockchains/04__fork-choice-rules.md', 71 | '/book/02__blockchains/05__applications.md', 72 | '/book/02__blockchains/06__participation.md', 73 | ] 74 | }, 75 | { 76 | title: 'Revisiting Eth1', 77 | collapsable: false, 78 | sidebarDepth: 1, 79 | children: [ 80 | '/book/03__eth1/00__index.md', 81 | '/book/03__eth1/01__basics.md', 82 | '/book/03__eth1/02__state.md', 83 | '/book/03__eth1/03__transactions.md', 84 | '/book/03__eth1/04__evm.md', 85 | '/book/03__eth1/05__blocks.md', 86 | '/book/03__eth1/06__fork-choice-rule.md', 87 | '/book/03__eth1/07__clients.md', 88 | ] 89 | }, 90 | { 91 | title: 'Exploring Eth2', 92 | collapsable: false, 93 | sidebarDepth: 1, 94 | children: [ 95 | '/book/04__eth2-overview/00__index.md', 96 | '/book/04__eth2-overview/01__historical-context.md', 97 | '/book/04__eth2-overview/02__project-philosophy.md', 98 | '/book/04__eth2-overview/03__key-concepts.md', 99 | '/book/04__eth2-overview/04__development-process.md', 100 | '/book/04__eth2-overview/05__learning-goals.md', 101 | ] 102 | }, 103 | { 104 | title: 'Proof of Stake in Ethereum', 105 | collapsable: false, 106 | sidebarDepth: 1, 107 | children: [ 108 | '/book/05__proof-of-stake/00__index.md', 109 | '/book/05__proof-of-stake/01__core-concepts.md', 110 | '/book/05__proof-of-stake/02__basic-operation.md', 111 | '/book/05__proof-of-stake/03__fork-choice.md', 112 | '/book/05__proof-of-stake/04__long-range-attacks.md', 113 | '/book/05__proof-of-stake/05__casper-ffg.md', 114 | '/book/05__proof-of-stake/06__weak-subjectivity.md', 115 | ] 116 | }, 117 | { 118 | title: 'Eth2 Building Blocks', 119 | collapsable: false, 120 | sidebarDepth: 1, 121 | children: [ 122 | '/book/06__building-blocks/00__index.md', 123 | '/book/06__building-blocks/01__bls-signatures.md', 124 | '/book/06__building-blocks/02__randomness.md', 125 | '/book/06__building-blocks/03__ssz.md', 126 | '/book/06__building-blocks/04__committees.md', 127 | '/book/06__building-blocks/05__deposit-contract.md', 128 | ] 129 | }, 130 | { 131 | title: 'Building the Beacon Chain', 132 | collapsable: false, 133 | sidebarDepth: 1, 134 | children: [ 135 | '/book/07__beacon-chain/00__index.md', 136 | '/book/07__beacon-chain/01__validator-life-cycle.md', 137 | '/book/07__beacon-chain/02__validator-duties.md', 138 | '/book/07__beacon-chain/03__structure.md', 139 | ] 140 | }, 141 | ], 142 | } 143 | }, 144 | 145 | plugins: [ 146 | '@vuepress/plugin-back-to-top', 147 | '@vuepress/plugin-medium-zoom', 148 | [ 149 | '@maginapp/vuepress-plugin-katex', 150 | { 151 | delimiters: 'dollars' 152 | }, 153 | ], 154 | [ 155 | 'vuepress-plugin-export', 156 | { 157 | bundles: [{ 158 | filter: /\/book\//, 159 | dest: (siteConfig) => `${siteConfig.title}.pdf`, 160 | }] 161 | } 162 | ] 163 | ] 164 | } 165 | -------------------------------------------------------------------------------- /book/01__consensus/04__crash-faults.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Crash Faults" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | 8 | ::: tip TODO 9 | The Consensus chapter was recently reworked to include a more interactive process of constructing a consensus protocol that slowly handles more and more realistic network conditions. I have not had a chance to update this chapter yet, so it doesn't quite read correctly. I'll try to update this section within the next few days. 10 | ::: 11 | 12 | 13 | Now we'll explore the first interesting problem that arises when trying to create a distributed system: crash faults. When a node suffers a crash fault, we can effectively assume that it ceases to function entirely (at least for the purposes of the protocol). In practice, a crash fault might be something as straightforward as a dead computer or a computer that loses its internet connection to other nodes in the network. We'll later talk a bit more about other types of faults where nodes are still functional but not properly following the prescribed protocol. 14 | 15 | ## Establishing the problem 16 | 17 | Let's briefly re-establish the problem. Say we have a distributed system that consists of a number of nodes (we'll call this number `total_nodes`). Our goal is to develop a protocol that guarantees these nodes will, after some amount of time, come to agreement about a particular ordered list of events. Active nodes must be able to come to agreement even when other nodes might be suffering crash faults. Again, at this point, we're only interested in making our protocol resistant to crash faults and we'll describe mechanisms for dealing with more insidious faults later. How can we make this work? 18 | 19 | For simplicity, let's assume that all nodes on the network are known in advance and that messages are somehow authenticated (perhaps with a cryptographic signature). Since we're assuming that nodes can only suffer from crash faults, we could even skip the need for message authentication (we trust that nodes will either behave honestly or not function at all). Let's also make the assumption that any node is allowed to propose an event, all events are "valid", and we're simply trying to establish some basic ordering. 20 | 21 | All this is simply to establish that: 22 | 23 | 1. Nodes on the network know about all of the other nodes on the network. 24 | 2. Nodes can trust the messages coming from other nodes. 25 | 3. Any node can propose an event. 26 | 27 | ## A basic mechanism 28 | 29 | Given the above assumptions, let's try to construct a consensus mechanism. We've made the assumption that any node can propose an event, so we need some way to figure out the order in which nodes should store these events locally. Although it's possible to come up with many different ways of handling this, one of the easiest methods is simply to assign some particular node to act as the "leader" of the network. 30 | 31 | Our leader node will carry out the following special protocol: 32 | 33 | 1. Listen for events from other nodes and store incoming events in a local FIFO queue. 34 | 2. Whenever the queue is not empty, take the next event (`next_event`) off the queue. 35 | 3. Broadcast a message to the network proposing that `next_event` will be stored as the next event in the chain of events. 36 | 37 | All nodes (including the leader) will then carry out the following: 38 | 39 | 1. Wait to receive a proposal for some `next_event`. 40 | 2. When a proposal is received, broadcast a message that acknowledges receipt of the `next_event`. 41 | 3. Listen for other nodes to send their acknowledgement of the event. 42 | 4. After some threshold number of nodes (`threshold_responses`) broadcast an acknowledgement of the event, store the event in the local version of the canonical chain of events. 43 | 44 | ### Limits of crash-fault resistance 45 | 46 | Though such a system needs to be maximally resistant to faults, there is a theoretical limit to the number of faults we can handle. We would clearly have a problem achieving much of anything if 100% of nodes were to become faulty. Research into this problem found that these crash-fault tolerant systems could sustain up to `n/2 + 1` failures. We often represent this number alternatively through the lens of the total number of nodes necessary given a certain number of faulty nodes, `f`, as `2f + 1`. One intuitive way of understanding this boundary is that if 50% of nodes were allowed to become faulty, then we could split the network into two segments that simply think the other half is faulty and may continue to operate in conflict. 47 | 48 | ### Understanding these limits 49 | 50 | The basis of a crash-fault tolerant system is that a certain threshold of nodes must come to agreement about an operation before the operation is actually executed. One of the simplest methods for reaching this agreement is to hold a vote. Conveniently, since we do not expect malicious behavior from our nodes, we can assign a single node to manage the entire voting process without fear of tampering. This node will send out events to vote on, tally up votes on particular events, and notify nodes of any relevant results. Our only core requirement is that nodes will never execute an operation unless `n/2 + 1` nodes have voted to do so. 51 | 52 | Another way to look at this `n/2 + 1` requirement is to imagine the following scenario. Let's say that `n/2 + 1` nodes voted for event `x` to be `y`. Now assume that at a later time, `n/2 + 1` nodes voted for event `x` to be `z`, a contradiction of the previous vote. In order for this to have happened, `n` nodes must have made `n + 2` votes, meaning at least one node made a contradictory vote. Since nodes cannot make contradictory votes under the crash-fault model, we can safely assume that this could never happen. Therefore, once `n/2 + 1` nodes vote for a particular event, the system will never include a contradictory vote. 53 | 54 | ### Modern applications 55 | 56 | This model of fault-tolerance through voting between machines is still extremely popular today. Services like Apache ZooKeeper and etcd have vastly simplified the process of deploying these systems. Many large organizations use crash-fault tolerant networks to store important information. 57 | -------------------------------------------------------------------------------- /book/03__eth1/03__transactions.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Transaction Model" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | As shown in our analysis of blockchain systems in general, the next component after a system's state model is its transaction structure. Consensus mechanisms come to agreement about a specific ordered set of these transactions. Clients can then determine the current state of a system by passing each transaction, in order, through some specified state transition function. We examine Eth1's state transition function in detail throughout the next section. However, we must first understand the structure and purpose of Eth1's transactions. Here, we explore Eth1's various transaction types, the contents of each transaction, and the basic steps necessary to generate a valid transaction. 8 | 9 | Eth1 represents state as a set of accounts that correspond to, and thereby "own," an ETH balance and some arbitrary storage space. All transactions in Eth1 are sent "from" one account "to" another, both of which must be specified within the transaction itself. Transactions sent from a user account are only valid if they include a signature over the transaction created with the sending account's private key. Transactions may be sent by contract accounts, but only during the course of the execution of some transaction from a user account. These contract account transactions do not include signatures as they can only be created when a contract's code permits. These fields form the basic contents of any transaction. 10 | 11 | ::: tip NOTE 12 | Forgot to add nonce and fee fields to the above. 13 | ::: 14 | 15 | ::: tip NOTE 16 | Change *all* must contain recipient to most w/ exceptions. 17 | ::: 18 | 19 | Eth1 allows for three primary types of transactions. Simple ETH transfers allow one account to send ETH to another. These transactions only include the fields defined above: 20 | 21 | ::: tip TODO 22 | Simple ETH transfer tx structure. 23 | ::: 24 | 25 | Contract deployment transactions make it possible for an account to upload a new contract to Eth1. Contract deployments are identified by the lack of a recipient account address and are the only transactions that do not include the recipient field. Code for the new contract is attached in a "calldata" field. Calldata may additionally contain extra code and input data to be executed as soon as the contract has been deployed. Lastly, contract deployment transactions can include an ETH balance to be transferred to the new contract from the sending account. Altogether, these transactions take the following form: 26 | 27 | ::: tip TODO 28 | Contract deployment tx structure. 29 | ::: 30 | 31 | The third and final transaction type defined by Eth1 is the "contract interaction." This transaction type allows accounts to trigger the code associated with a given contract. Contract interactions contain all basic transaction information, including an optional ETH balance to be sent to the target contract. Like contract deployments, contract interactions contain a calldata field that may be used to provide the contract with relevant input data. Contract code often uses this calldata to determine the exact action the sending account wishes to take. A complete contract interaction transaction appears as follows: 32 | 33 | ::: tip TODO 34 | Contract interaction tx structure. 35 | ::: 36 | 37 | ::: tip NOTE 38 | "Gas" references below to be edited into second paragraph of this section. 39 | ::: 40 | 41 | We've alluded to the fact that transactions must include a fee in the form of "gas." We discuss the exact calculation for this fee in the following section. In a nutshell, gas is a virtual currency used to pay for any transaction. All transactions must contain a base amount of gas (currently 21,000 gas) and a fixed amount of gas depending on the size of calldata in bytes (currently 68 gas per byte). Transactions then must additionally include an amount of gas generally proportional to the resource expenditure necessary for the transaction's execution. So that gas values can remain fixed in the presence of price fluctuations, transactions specify a "gas price," the amount in ETH to be paid per unit gas. This mechanism generally guarantees that nodes who mine a transaction are compensated according to the cost of doing so. 42 | 43 | ::: tip NOTE 44 | Not sure whether to include receipt info (below) here or in next section. 45 | ::: 46 | 47 | Once executed and included within a block, transactions are paired with a "transaction receipt." Transaction receipts act as concise summaries of the impact of a transaction on the system. They include, for instance, a status code that signals whether or not a transaction was completed successfully. Clients will often use information within the receipt to make a determination about any additional actions to take. A full receipt contains the following fields: 48 | 49 | ::: tip TODO 50 | Receipt structure. 51 | ::: 52 | 53 | Most of these fields are likely self explanatory. However, we have yet to cover the concept of "logs." Essentially, a log is a piece of information that can be attached to a transaction receipt by a contract account. Logs are typically employed to relay some useful data about the effect or events of a transaction without directly manipulating the storage of a contract. For instance, a contract may generate a log to efficiently signal that an ETH balance was transferred out of the contract. Logs may alternatively be called events in some circles. 54 | 55 | All logs contain basic identifying information, including the address of the contract that generated the log and references to the block and transaction in which the log was produced. Logs then additionally define a "topic" and some "data." Log topics allow users to identify the particular event represented by the log. Log data provides more detailed information about the topic at hand. For example, if a contract is acting as a marketplace for virtual cats, it may choose to emit a log of the following form whenever a sale is completed: 56 | 57 | ::: tip TODO 58 | Sample log. 59 | ::: 60 | 61 | In the following section, we take a look at Eth1 state transition function. We make use of the concepts discussed in this section and the one preceding it to build a complete picture of Eth1's application layer. Certain details of Eth1's transaction structure do change in Eth2, but the general concepts remain largely in place. Transaction receipts become particularly important in Eth2, as we'll soon come to discover. We're now ready to see how a modern production blockchain system processes transactions that have complex effects. 62 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/05__casper-ffg.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Casper FFG" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | The solution to the long range attack is, effectively, to introduce a new rule that prevents clients from reverting any block created more than `n` blocks ago, where `n` is the amount of time for $\smash{\frac{2}{3}}$ of validators to withdraw their funds. The mechanism within Eth2 that provides this functionality is **Casper FFG**. 8 | 9 | ## Checkpoints 10 | 11 | The first new concept within Casper FFG is the **checkpoint**. Checkpoints are special blocks that are candidates for finalization. We can pick our checkpoint blocks however we please, but Eth2 specifically states that checkpoints occur once every fixed number of blocks. Currently, this happens every 32 slots. We call this period of 32 slots an "epoch." 12 | 13 | ::: tip TODO 14 | Image of checkpoints. 15 | ::: 16 | 17 | When the first slot of an epoch is not empty, this block serves as the checkpoint for that epoch. 18 | 19 | ::: tip TODO 20 | Image of EBB when first slot has block. 21 | ::: 22 | 23 | Since we might miss a slot, it's possible that the first slot of an epoch will be empty. In this case, we use the last known block as the checkpoint block for that epoch. 24 | 25 | ::: tip TODO 26 | Image of EBB when first slot is skipped. 27 | ::: 28 | 29 | We refer to the checkpoint block for an epoch as the "epoch boundary block," or EBB for short. 30 | 31 | ## Supermajority Links and Justified Checkpoints 32 | 33 | Whenever a validator creates an attestation, they must now also include a vote to connect two checkpoint blocks. This vote contains references to two checkpoints that the validator wishes to connect. The purpose of this connection will soon become clear. 34 | 35 | If $\smash{\frac{2}{3}}$ of validators vote to make the same connection between two checkpoints, then we say that there exists a "supermajority link" between the checkpoints. 36 | 37 | We then additionally introduce the concept of a "justified checkpoint." We say that the genesis block is by default a justified checkpoint. Beyond the genesis block, a checkpoint is justified if it is connected to another justified checkpoint by a supermajority link. 38 | 39 | ::: tip TODO 40 | Image of justification 41 | ::: 42 | 43 | This system reveals the purpose of the connections. We are attempting to create an "overlay blockchain," a series of blocks found to be valid by a $\smash{\frac{2}{3}}$ of validators. 44 | 45 | ## Justification and Finalization 46 | 47 | Justification alone is not sufficient for finalization. The reason for this is most easily understood by example. 48 | 49 | Imagine a chain in which we have two potential checkpoint blocks for the same epoch. Such a situation might arise if, for example, network latency delays propagation of the block in the first slot of an epoch. 50 | 51 | ::: tip TODO 52 | Image of our setup. 53 | ::: 54 | 55 | An attacker with $\smash{\frac{1}{3}}$ of stake wants to exploit this situation to prevent any more blocks from being justified. Some portion of honest validators created votes under the assumption that the first block did not exist. Later, when the block became available, other honest validators created votes that conflicted with these first votes. 56 | 57 | ::: tip TODO 58 | Image of two conflicting checkpoints with votes. 59 | ::: 60 | 61 | If we do not allow validators to create conflicting votes, then our attacker can completely prevent more checkpoints from being justified. All they need to do is simply refuse to publish any votes. Since some honest validators are in conflict, it would be impossible to react $\smash{\frac{2}{3}}$ votes on a checkpoint. 62 | 63 | ::: tip TODO 64 | Image showing neither can be justified. 65 | ::: 66 | 67 | Because of cases like this, we must allow validators to create certain votes that contradict themselves. We still do not allow validators to create multiple votes during the same slot, but we permit validators to make two votes with the same source height or where the connections overlap. 68 | 69 | ::: tip TODO 70 | Expand here to detail the conditions we're allowing. 71 | ::: 72 | 73 | This allowance prevents us from halting like we did in the previous scenario. Validators can now simply switch their votes to follow the majority. 74 | 75 | ::: tip TODO 76 | Image where validators switch votes to prevent stall. 77 | ::: 78 | 79 | However, this reveals why justification alone is insufficient for finalization. Once validators can make contradictory votes, multiple conflicting checkpoints can become justified. We need to go slightly further to ensure that conflicting checkpoints cannot be finalized. 80 | 81 | ## Finalization 82 | 83 | We say that a checkpoint is **finalized** if it is justified and connected to its direct parent checkpoint. The logic behind this rule mostly follows from the idea that we want to ensure, without a doubt, that $\smash{\frac{2}{3}}$ of our stake has made a vote that it cannot contradict. 84 | 85 | ::: tip TODO 86 | Really need to expand here. 87 | ::: 88 | 89 | ::: tip TODO 90 | Image for finalization. 91 | ::: 92 | 93 | We previously noted that validators are allowed to make conflicting votes where source height is the same and target height differs, or where the connections intersect. Now we make this explicit and introduce the two casper "commandments." 94 | 95 | 1. A validator cannot make two votes such that $\smash{H(t_1) = H(t_2)}$. 96 | 2. A validator cannot make two votes such that $\smash{H(s_1) < H(s_2) < H(t_2) < H(t_1)}$ 97 | 98 | These two rules do allow for the exceptions previously detailed. However, they ensure some additional properties. The first rule guarantees that we can never have conflicting justified checkpoints at the same height. Therefore, if we do have conflicting checkpoints, one must have a greater height than the other. 99 | 100 | Now, we instate a "fork choice rule" for checkpoints that validators should always follow the justified checkpoint at the greatest height. Again, our first rule guarantees that only one of these will exist. 101 | 102 | ::: tip TODO 103 | Image for fork-choice rule. 104 | ::: 105 | 106 | Since only one exists, if the immediate next checkpoint is justified, then we can be sure that no other conflicting checkpoints can appear later. Rule I directly prevents validators from justifying two child checkpoints, and Rule II prevents any attempts to vote "around" the highest justified checkpoint. 107 | 108 | ::: tip TODO 109 | Image for trying to cheat. 110 | ::: 111 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/02__basic-operation.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Basic Operation" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Many of the design decisions behind Eth2 can be best understood as incremental improvements over a "naive" Proof of Stake protocol. The remainder of this chapter will introduce new concepts as responses to attack vectors against such a naive protocol. By the end of the chapter, our naive system will have evolved into a robust Proof of Stake chain. 8 | 9 | ## Genesis 10 | 11 | Our naive system begins with one initial block, the genesis block, that contains a list of validators and their respective stakes within the system. We break time into a series of slots, each of which are `n` seconds long. Furthermore, we'll initially assume that our validator set does not change, e.g., validators cannot withdraw their funds. Eth2 *does* allow validators to enter or exit the system, but this introduces certain challenges to be addressed later in the chapter. 12 | 13 | ::: tip TODO 14 | Image of basic slot system. 15 | ::: 16 | 17 | ## Role Selection 18 | 19 | For the sake of this initial operational analysis, we construct a role-selection mechanism as follows. Every `M` slots, validator roles are selected randomly for `Slot N` through `Slot N + M`, where `Slot N` is the current slot. 20 | 21 | For simplicity, assume that our random distribution is perfect such that, in the long run, a validator with `X%` of stake produces exactly `X%` of all blocks. This does not entirely reflect the selection process used within Eth2, but is a "good enough" approximation for the moment. 22 | 23 | ::: tip TODO 24 | Image of role-selection system. 25 | ::: 26 | 27 | ## First Steps 28 | 29 | Using this simple selection process, we can begin to create a basic Proof of Stake chain. 30 | 31 | For each slot, our chain assigns a single validator to produce a block and then assigns all other validators to vote on the validity of the block. We say that these other validators are part of the **attestation committee** for this slot. In order to incentivize correct behavior, we reward validators who produce valid blocks or create votes during their assigned slots. 32 | 33 | Under perfect conditions, the validator assigned to each slot will produce a valid block, which the attestation committee will receive before the start of the next slot. Committee members will then check the validity of the block according to some rules and broadcast a vote of validity. 34 | 35 | Assuming we have no forks, the block producer will be rewarded for having produced a block in the canonical chain. Members of the attestation committee are rewarded once their votes are recorded within a future block. We incentivize the inclusion of these attestations by giving block producers an additional reward for each included attestation. 36 | 37 | This process continues, and the chain progresses. 38 | 39 | ::: tip TODO 40 | Image of ideal operation 41 | ::: 42 | 43 | ## Missed Slots 44 | 45 | Of course, in reality, we cannot assume that the chain will operate with such perfection. We might, for example, run into the case that a validator simply chooses not to produce a block during their assigned slot. A software failure may cause a validator to go offline, or a validator may be actively attempting to disrupt the chain. 46 | 47 | In the simple case of a missed slot, members of the corresponding attestation committee have nothing to sign. As a result, neither the block producer or committee members receive a reward for this block. Similarly, in the case that a producer creates an *invalid* block, nodes will automatically reject the block and it can effectively be treated as if no block was produced. 48 | 49 | ::: tip TODO 50 | Image of missed slot. 51 | ::: 52 | 53 | ## Slashing 54 | 55 | We need to introduce some new functionality, however, to handle the case in which a producer creates *two or more* blocks during the same slot. Until now, we hadn't introduced any rules preventing this sort of behavior. We clearly want to patch any holes in our system that allow a validator to so easily create a fork in our chain. 56 | 57 | ::: tip TODO 58 | Image of two blocks during the same slot. 59 | ::: 60 | 61 | Our very first additional mechanism, therefore, is to define a punishment for any validators caught creating more than one block during the same slot. We refer to this sort of punishment as **slashing**. 62 | 63 | A validator is slashed if evidence of misbehavior is included in some later block. In this case, our evidence is the existence of a validator's signature on two distinct blocks with the same slot number. We incentivize the inclusion of this evidence by rewarding the reporting validator with a portion of the penalty. 64 | 65 | Simply punishing the block producer doesn't entirely solve our problem, unfortunately. It's still possible for a validator to create two blocks within the same slot, even though they'll be punished for doing so. A wealthy attacker, for example, might be willing to face the financial penalty simply to see how the network reacts. 66 | 67 | In such an event, the attestation committee is faced with a dilemma. Attestations can only be rewarded if they are included in the history of the chain. If a committee member votes for the "wrong" block, then, from the perspective of the canonical chain, their vote effectively does not exist. As a result, the committee member receives no reward on the canonical chain. The natural and rational choice is, therefore, simply to create votes for both chains. 68 | 69 | ::: tip TODO 70 | Image of nothing-at-stake problem. 71 | ::: 72 | 73 | Since both chains receive votes, both may be equally "valid" with respect to whichever fork-choice rule we develop to handle forks. Assuming that validators are rational, both chains *will* receive the same number of votes. If this continues, our chain will end up as a mess of forks. 74 | 75 | We prevent this behavior by introducing a slashing penalty for any validators who create two votes during the same slot. Our evidence in this case takes the form of a validator's signature on two distinct votes with the same slot number. 76 | 77 | In the case that an attestation committee is faced with this decision, they will naturally want to wait and see which block has received more votes. However, this leads to a situation in which all validators are waiting for votes and, as a result, no validators are publishing votes. 78 | 79 | We circumvent this issue by adding logic to our reward system such that an attestation is more valuable the more quickly it's included in the chain. Validators are therefore forced into making a timely decision. Eth2 designs this reward to halve with each additional slot. 80 | -------------------------------------------------------------------------------- /book/01__consensus/01__origins.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "The Origins of Consensus" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ## When computers became networks 8 | 9 | The origins of modern blockchain systems can be traced back to decades-old research in the field of computer science. This history begins with the advent of networked computing in the second half of the 20th century. Prior to the integration of long-distance communication lines into computing hardware, software systems were effectively limited to individual machines or, in rare circumstances, small local clusters of machines. These systems could rely on the tight proximity of various hardware components to reduce any form of communication latency to a mostly insignificant minimum. 10 | 11 | The historic ARPANET program, established in 1969, would quickly go on to demonstrate both the viability and utility of large-scale computer networks. Computers connected via ARPANET could communicate with one another at previously unimaginable distances and speeds. It would soon be, in an interesting twist of fate, that a pair of computer worms would make apparent the possibility of not only communication but also collaboration between machines on these networks. The Creeper and Reaper worms, quite possibly the very first computer viruses, jumped between various computers connected to ARPANET and automatically duplicated themselves along the way. Although these worms did no damage to their hosts, they sparked the imagination of researchers interested in making use of idle processing power of machines connected to the ARPANET system. 12 | 13 | ## From distributed computing to fault tolerance 14 | 15 | In 1988, the DEC System Research Center launched the first distributed computing project to be coordinated over the internet. Participants received computing tasks from DEC via email and would return their results to DEC upon completion. Other initiatives, including the famous SETI@HOME project of the University of California at Berkeley, soon followed suit and further simplified the process of communicating tasks and results. With access to the computational resources of an extraordinary number of machines, these efforts massively advanced the state of several key fields of mathematics and computer science. 16 | 17 | Simultaneously, other researchers of the time saw potential applications of these networks beyond the pooling of computational resources. Relatively early on in the history of computing, organizations recognized the need for particularly robust computers for use in certain high-value systems. Machines operating in space, for instance, needed to function properly for extended periods of time without any possibility of manual on-board maintenance in the event of an error. Starting in the 1950s, organizations began to develop and utilize so-called "fault-tolerant" computers that had the ability to recover cleanly from significant software and hardware failures. 18 | 19 | Fault-tolerant computing quickly saw wide-spread adoption in critical systems around the world. NASA's CCS fault-tolerant computer found its way into the Voyager spacecraft where, more than forty years later, it continues to operate in the harsh environment of deep space. Military operations, nuclear power plants, and digital financial systems all became natural homes for these error-resistant machines. Fault-tolerant computing was, and continues to be, the foundation of countless computer-based services familiar to us even in the 21st century. 20 | 21 | ## The principle of replication 22 | 23 | At the heart of fault-tolerant computing sits the principle of replication. When some component of a computer fails, it becomes unable to perform its assigned tasks. If, for example, a memory module crashes, then the data stored on that module becomes inaccessible to other components. A computer could recover from such a failure if the information on the module happens to be duplicated somewhere else within the system. Fault-tolerant computers are able to fix or bypass errors in individual components by replicating the functionality of these components onto redundant hardware. 24 | 25 | ## Redundancy at scale 26 | 27 | In his seminal 1978 paper, "Time, Clocks, and the Ordering of Events in a Distributed System," Leslie Lamport, at the time a researcher at Massachusetts Computer Associates, laid the groundwork for an entirely new model of fault-tolerant computing. Instead of redundant hardware within a single computer, Lamport proposed a form of redundancy produced via communication between many different computers connected to a network. 28 | 29 | Lamport demonstrated that this model could be used to accurately replicate "transactional databases" across different machines. Within these databases, each database update is represented as an individual event, or "transaction." The final state of a transactional database is then computed as the result of executing each of these transactions in order. If several computers could come to agreement about the ordering of transactions within such a system, Lamport argued, each computer could accurately reconstruct the final system for itself. 30 | 31 | Lamport further envisioned the possibility that entire computers be replicated the same way. Under the "state machine" model, computers can be represented, in an abstract sense, as entities that store some past information in memory (the "state") and may modify this information according to a given set of operations. Just as with a transactional database, a state machine could be reconstructed if the list of operations it executed were recorded and played back in exactly the same order. Perhaps, therefore, a failure on one computer could be circumvented if execution could seamlessly continue on another. 32 | 33 | ## Some basic design assumptions 34 | 35 | Research into this state machine model of fault tolerance had to be carried out under some set of assumptions about the system itself. Early work was often restricted to systems in which computers were expected to only experience so-called "crash-faults," sometimes also known as "fail-stop faults." Within this framework, computers would either perfectly follow some prescribed protocol or cease to function entirely. Computers on the network would only behave "benevolently" and would not, for instance, attempt to forge messages from other machines. 36 | 37 | Designs operating under this assumption could, as we'll shortly discuss, take advantage of some convenient simplifications. However, this mental framework not only reduced the problem space for researchers, but also generally reflected the needs of many real-world applications. Organizations making use of such systems internally, perhaps to maintain backups of some important database or computing process, could typically ignore the possibility of actively malicious behavior from their own machines. As a result, this approach quickly grew in popularity and has held traction well into the 21st century. 38 | -------------------------------------------------------------------------------- /book/02__blockchains/01__digital-cash.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Digital Cash" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | From the ubiquity of money as a dominant system of social coordination it's no challenge to understand why attempts at digital financial products have been plentiful. Given the inherently social nature of money and finance, it should also come as no surprise that these projects gained significant traction with the launch of the world wide web in 1989. As the dotcom bubble made undeniably clear, the internet-connected universe was enamoured with a wholly digital vision of the future. Digital reality held limitless possibility, but it most certainly helped digital money. 8 | 9 | Beyond the apparent social impact of such a system, already confirmed by the success of other financial conveniences like the ATM, money was ultimately relatively straightforward to construct. Accounts, balances, and transactions translated cleanly into the computer databases necessary to represent finance in cyberspace. Even the ATM required the development of digital banking tools as early as the 1960s. With most of its raw components sitting in plain sight, digital money was simply waiting to be plugged into the internet. 10 | 11 | Within a year of the launch of the world wide web, digital money would have its first taste of sunshine. In 1990, computer scientist David Chaum launched a small company by the name of DigiCash. Based on a protocol Chaum detailed in 1983, DigiCash was truly a computer scientist's ecash. Chaum leveraged his early work regarding blind signatures and built a system that allowed users to securely transfer virtual currency over the web. 12 | 13 | Chaum's system made use of many novel mechanisms we now often take for granted. Transactional integrity was guaranteed by a series of digital signatures used to authorize interactions. Users could even participate without the need to create an account at a traditional bank. Anyone could send and receive money by simply generating a cryptographic keypair. Chaum could even guarantee privacy on every transaction through a few mathematical tricks. DigiCash appeared to be the perfect nu-currency for the digital age. 14 | 15 | As ever, DigiCash came with a DigiCatch. The system was constructed around the existence of a central intermediary. This "bank," though not necessarily actually operated by a major banking institution, was responsible for asserting the validity of each transaction. Chaum's protocol, in a nutshell, flowed as follows: 16 | 17 | 1. A "bank" is responsible for maintaining a database of balances for user accounts. 18 | 2. A "payer" with some balance creates a "note" to be used for payments. 19 | 3. The bank signs the note and debits the payer, as long as the payer has sufficient balance to cover the note. 20 | 4. The payer gives the note to a "payee." 21 | 5. The payee sends the note to the bank for redemption. 22 | 6. The bank verifies the note and credits the payee. 23 | 24 | Chaum's full mechanism further included cryptographic techniques that preserved the privacy of each transaction. DigiCash notes were designed in such a way that the bank was unable to link the notes created by payers to those redeemed by payees. Since the bank would debit the payer when a note was generated, they were simply required to check that the note was valid and had not yet been cashed. 25 | 26 | Although groundbreaking, this system was far from the ideal of trustworthy digital finance. Reliance on a central entity for overall functionality also introduced reliance on the entity to remain honest in its actions. As the veracity of any given note was imparted by its signature, the bank could theoretically manipulate the supply and flow of notes. For instance, if it were so inclined, the bank could issue notes to a payer in vast excess of their actual balance and effectively print a limitless supply of currency. A particularly blatant offender might simply modify account balances outright. Without insight into the balance sheet, DigiCash customers could only rely on external regulation to keep the bank in check. 27 | 28 | In the end, it was market timing, rather than the existence of a single point of trust, that proved to be the ultimate downfall of DigiCash. Chaum was early to a market that was only starting to hit its stride. The company began operations at the birth of the World Wide Web, before Tim Berners-Lee even had a chance to release the world's first web browser in 1991. Furthermore, the US National Science Foundation, which essentially operated the internet "backbone" connections at the time, placed heavy restrictions on commercial activity over their network. As a result, E-commerce expansion was stunted almost entirely until NSF relaxed its position in 1995. Unable to grow and sustain any significant user base, DigiCash slowly burned through its resources over the course of a decade. Finally, in 1998, DigiCash filed for Chapter 11 bankruptcy in the United States. 29 | 30 | Given the failure of DigiCash to gain traction, one might find reason to fault constructions that gave so much power to a single entity. Indeed, several other infamous 90s experiments in digital currency were developed under this model and eventually ceased operations. One may find entertainment in the downfall of Beenz, whose reputation was severely damaged after a software error allowed users to, essentially, print Beenz at will. 31 | 32 | Yet regulation, and not software bugs, ultimately determined who could, or couldn't, do the digital dash to get the digital cash in the new millennium. The UK Financial Services Authority paid Beenz a visit on suspicion that the company was running an "unlicensed bank." Flooz, another dotcom baby, shut down shortly after notification from the FBI that the platform was being used as part of a money laundering scheme. Digital finance was quickly starting to feel the pressure of governmental organizations in the physical realm. 33 | 34 | Not dissuaded by a crackdown, some companies recognized the need to play ball with authorities. Digital finance was clearly still feasible and desirable; online shopping was only getting more popular as the internet continued to grow. Centralized management was not only the simplest model for these systems, but was also often the only one that conformed to existing regulations. Unsurprisingly, digital cash made a resurgence in the decade following the dotcom bust. Big players, some, like PayPal, still recognizable today, managed to develop successful digital cash systems, all under the watchful eye of the world's governmental agencies. 35 | 36 | Of course, the vision of a "decentralized" digital currency, one not directly controlled by global authorities, did not entirely fade out of existence. Computer science theory simply wasn't quite ready to bring that vision to reality. However, small pockets of researchers continued to study this problem and, as we'll explore in the next chapter, eventually found a pathway through it. 37 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/04__long-range-attacks.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Long Range Attacks" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | LMD GHOST effectively blocks a minority chain from outrunning a majority chain. However, our chain has, until now, relied on a fixed validator set. The introduction of a dynamic validator set breaks certain properties of LMD GHOST. 8 | 9 | ## Dynamic Validator Sets 10 | 11 | The purpose of a dynamic validator set is relatively intuitive. Just as with Proof of Work, it seems useful that anyone have the ability to join or leave the system at any time. If the validator set remains fixed, a permanent "nobility" maintains control over the system. Such a "nobility" is in direct violation of many of the core philosophical ideals behind decentralized networks. 12 | 13 | The introduction of a withdrawal system brings with it several challenges for our Proof of Stake chain. A naive method for withdrawals, for instance, might allow validators to exit the system immediately after a withdrawal request is included in the chain. Though convenient, such a system would permit an attacker to wreak havoc on our network. 14 | 15 | Take, for instance, an attacker who creates two blocks in the same slot. We initially addresses this vector with the introduction of slashing penalities. Unfortunately, under certain network conditions, evidence of this behavior may not be immediately included in the blockchain. If an attacker can withdraw their funds before evidence is included, then the attacker faces no financial penalty. 16 | 17 | ::: tip TODO 18 | Image where attacker withdraws because of latency. 19 | ::: 20 | 21 | An attacker could even, for example, execute such an attack simply because the chain randomly selects them to produce multiple blocks in a row. 22 | 23 | ::: tip TODO 24 | Image where attacker withdraws because of random chance. 25 | ::: 26 | 27 | To prevent this sort of behavior, it's necessary to introduce some delay into the withdrawal process. Such a delay should generally be "long enough" that the likelihood of flying under the radar is negligible. The lower bound for this delay can be estimated so that the odds of an attacker with $\smash{\smash{\frac{1}{3}}}$ stake being chosen to produce all blocks before the withdrawal is minimized. 28 | 29 | ## Long Range Attacks 30 | 31 | Although a withdrawal delay ensures that slashable behavior can be punished, the existence of any withdrawal mechanism poses problems for LMD GHOST. Our troubles begin due to the fact that a withdrawn validator faces no consequences for any slashable behavior carried out in the past. Indeed, this is exactly why we initially introduced a withdrawal delay. 32 | 33 | Our delay cannot, however, punish any messages received after the withdrawal is completed that are crafted to look as if they were created in the past. We cannot distinguish between messages actually created in the past, but not seen, and messages constructed to appear as if they were made in the past. Therefore, as long as our delay is finite, an attacker can perform certain slashable actions without fear of being punished. 34 | 35 | ::: tip TODO 36 | Image for information theoretical limitations of slashing with withdrawals. 37 | ::: 38 | 39 | The withdrawal process first removes a validator from the set of active participants. An attacker can only create messages from a time in which they were still active, since newer messages would be rejected by clients. An attacker could, for instance, publish a block that creates a fork some time in the past. 40 | 41 | ::: tip TODO 42 | Image for a useful attack that could be performed. 43 | ::: 44 | 45 | Just as with any other new block, clients would need to factor this new block into their fork-choice rule in order to determine the correct chain to follow. With only a single block and no votes, however, LMD GHOST still clearly favors our primary chain. An attacker needs to go further to cause problems. 46 | 47 | ::: tip TODO 48 | LMD GHOST picks the right chain w/r/t above. 49 | ::: 50 | 51 | ### Long Range Attacks: By Example 52 | 53 | Let's examine one extreme case in which an attacker can halt our chain. Assume that at some time, three validators held, jointly, 100% of stake on the chain. All three validators sign messages in support of some given block. At some time in the future, all three have withdrawn from the chain and are replaced by new validators. 54 | 55 | ::: tip TODO 56 | Image for our setup. 57 | ::: 58 | 59 | Now, by some means, a malicious party has acquired the keys for the original three validators. Perhaps the validators sold their keys for some low price, as they are at no risk of losing any funds. Possibly, an attacker managed to hack the keys as the validators had little reason to continue heavily securing them. No matter the reason, the attacker has managed to relatively cheaply acquire the keys. 60 | 61 | The attacker now uses the keys to create a block that forks the first block supported by all three parties. Then, the attacker publishes this block to other clients. When the clients attempt to decide between the two blocks, they find that LMD GHOST is unable to make a decision. According to the rules of LMD GHOST, both chains have 100% of votes and are equally good. 62 | 63 | ::: tip TODO 64 | Image with the malicious fork. 65 | ::: 66 | 67 | We refer to this exploit as a "long range attack." The attack is effective because the attacker is able to violate the honest majority assumption retroactively. Even if all three validators were faithfully following the protocol before withdrawing their funds, there's no strong incentive to continue doing so once their funds are withdrawn. Our "honesty" assumption is highly dependent on the existence of incentives for correct behavior. 68 | 69 | ### Long Range Attacks: Generalized 70 | 71 | This attack can be generalized beyond the special case in which it was initially presented. For any block in the chain, we can find a specific number of supporting validators from the known validator pool at that block, $\smash{v_s}$. Note that this value is identical to the LMD GHOST score for that block. 72 | 73 | We can also find the number of validators who have withdrawn their funds, $\smash{v_w}$. As withdrawal requests are a form of support for a block, we know that that $\smash{v_w \leq v_s}$. 74 | 75 | In order to execute this attack, an attacker must acquire keys for at least $\smash{v_s + \frac{v_t - v_s}{2}}$ accounts, where $\smash{v_t}$ is the total number of validator accounts. As accounts in $\smash{v_w}$ are already withdrawn, the cost to acquire these accounts decreases. Therefore, as $\smash{v_w}$ approaches $\smash{v_s}$, the total cost to execute this attack also decreases. 76 | 77 | Such an attack, under a majority honesty assumption, becomes possible once $\smash{\frac{2}{3}}$ of the validator set at a given block withdraw their funds. We can determine the minimum distance for a long range attack depending on our withdrawal delay and any limitations on churn. 78 | -------------------------------------------------------------------------------- /book/03__eth1/05__blocks.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Ethereum Blocks" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Blocks, as we know, are collections of transactions that form the history of events within a blockchain system. We make use of blocks, instead of simply chaining together individual transactions, because they reduce the rate of unnecessary chain forks in the presence of high network latency. Any given chain of blocks, connected by each block's reference to its parent, produces some final state as a result of the transactions the chain contains. In this sense, blocks do not contain the state of a system, but provide nodes with the information necessary to compute that state for themselves. In this section, we take a look at the structure and production process of blocks in Eth1. 8 | 9 | As blocks primarily serve to formalize the inclusion of transactions within a blockchain, they themselves introduce little additional complexity to our system. Most complexity is captured within the state and transaction structure. Blocks are simply records of transactions and their effects on Eth1's state. Some content within each block, however, pertains to the process by which blocks are produced. It's therefore both valuable and reasonable to detail the block production process prior to examining low-level block structures. 10 | 11 | All transactions within any block must be initiated by users with access to the private key of some account. In most cases, these transactions are generated by users and then broadcast to the full network of online Eth1 nodes. Once broadcast, transactions remain in a "pool" of other transactions pending for inclusion in a block. This pool does not reside on any single machine, but is instead stored by any node that wishes to access it. Although we often consider there to exist one unified pool, different nodes may see different pools depending on the flow of transaction data through the network. For the sake of most analysis, the mental framework of a global pool is sufficient. 12 | 13 | Nodes that aim to produce, or "mine," blocks must first select transactions for inclusion from the transaction pool. Nodes may choose any of these transactions, but the total gas consumption of transactions within a block is capped by a limit that may only be changed by `1/1024` of the limit of the previous block. Eth1 includes this cap as to avoid unnecessarily large blocks that delay block propagation. Most miners select transactions in order of highest "gas price" to maximize their rewards from transaction fees. 14 | 15 | For each transaction selected from the pool, nodes execute the transaction against their current state. The first transaction selected is executed against the state resulting from the last block known by the node. Each additional transaction then operates on the output state of the previous transaction. Transactions continue to be processed in this manner until the total gas used reaches the specified block gas limit. At this point, the node generates most of the content within the block based on the effect of included transactions. All information attached is detailed later within this section. 16 | 17 | Their blocks mostly complete, nodes must now attempt to find a partial hash collision that makes the block acceptable by other nodes. Eth1 requires that nodes execute a special-purpose hashing algorithm called "Ethash." Ethash uses the information included within a block to randomly select values from a large dataset on the order of several gigabytes in size. The SHA512 hash of these values then produce the final hash attempt result. When a miner increments the nonce field in their block, new values are selected from the dataset. This mechanism forces mining hardware to repeatedly access memory and thereby reduces the effectiveness of custom machines optimized to compute one specific computational task. Ethash has had the effect of significantly extending the period of time during which consumer hardware, namely computer graphics cards, could be profitably employed in the mining process. In theory, this quality expanded the base of people with access to adequate mining devices. 18 | 19 | For a given hash attempt to be a valid Proof-of-Work, it must be at least less than the block's target difficulty. Eth1 computes the target difficulty of a block based primarily on the time difference between the block and its parent. When this difference increases, the difficulty also increases, and vice versa. As a result, general advances in hardware efficiency, leading to a more optimized hashing process, are met with an equal increase in difficulty and, on average, a stable time interval between blocks. Eth1 additionally scales block difficulty based on the current block number and thereby slowly increases difficulty over time. This "difficulty bomb" was originally introduced to provide an incentive for the development of Eth2 in the face of gradually degrading Eth1 network performance. However, the mechanism has been "diffused" by delay on several occasions. Eth1 lastly considers the rate of ommer blocks in its difficulty formula, attempting to maintain a constant rate over time. The complete formula is as follows: 20 | 21 | ::: tip TODO 22 | Something was supposed to go here but I forget what. 23 | ::: 24 | 25 | Once a miner has found a valid Proof-of-Work for their block, they broadcast the block to the network at large. Other nodes can then consider this block when choosing a canonical chain via their fork choice rule. We detail Eth1's fork choice rule in the following section. If a node finds the block to be canonical, it may repeat this block production process for new transactions executed against the final state given by the block. The block production cycle has now been successfully completed. 26 | 27 | Now that we've understood the mechanism by which blocks are generated, we can take a look at Eth1's block structure. Eth1 blocks are composed of two elements, the "block header" and the "block body." The block header serves as a concise description of a block. It includes, among others fields, a block's position in the blockchain, a block's timestamp, and a Proof-of-Work. It additionally contains cryptographic commitments to the contents of the block body. A block's body holds a more complete record of the effect of the block, like the full list of transactions it executes. Let's begin by diving into the header. 28 | 29 | A block's header consists of the following elements: 30 | 31 | ::: tip TODO 32 | Something was supposed to go here but I forget what. 33 | ::: 34 | 35 | A block's body contains: 36 | 37 | ::: tip TODO 38 | Something was supposed to go here but I forget what. 39 | ::: 40 | 41 | Although the exact structure of a block changes in Eth2, many of the same principles found here are quite universal. References to transactions, validation information, and identification details can be found in the blocks of most any blockchain system. Two-part block header and block body distinctions are equally ubiquitous. All of these elements are included within Eth2. Through this examination of a tested and proven Eth1, we've built a strong foundation for our studies of Eth2 to come. 42 | -------------------------------------------------------------------------------- /book/02__blockchains/04__fork-choice-rules.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Fork Choice Rules" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | In the previous section, we mentioned that forks occur when multiple published blocks reference the same previous state. We might face such an event by accident, perhaps if two blocks are published at approximately the same time. However, this could also happen on purpose, as would be the case if a malicious actor were attempting to create a fork in order to "remove" a payment from the chain generally accepted by other users. In either case, it's necessary that there be some mechanism that allows users to determine which fork is the correct or "canonical" one. We refer to such a mechanism as a "fork choice rule." 8 | 9 | Fork choice rules are extremely important for the proper functioning of the underlying blockchain. Fork choice rules determine which chain is to be widely accepted and, therefore, which block producers are to be rewarded. A poorly designed fork choice rule can quite quickly lead to unexpected or undesired behavior on the part of both users and block producers. 10 | 11 | All fork choice rules must have certain properties in order to be maximally useful. In keeping with the notion of openness and accessibility, fork choice rules should be easy to compute. Nodes should be able to select a canonical chain from a set of forks without significant computational effort. Furthermore, many protocol designers feel that fork choice rules should have limited reliance on external parties. Essentially, nodes should be able to execute the rule without needing to confer with some "trusted" central entity or group. 12 | 13 | Perhaps most crucially, fork choice rules must reinforce the economic incentives within the system as a whole. Block producers are typically rewarded with some digital currency transaction contained within the blocks they produce. When a fork choice rule rejects their block, this reward transaction is no longer included within the canonical chain. Generally speaking, if the fork choice rule does not reward the producer who carried out the most computational work, then the direct link between voting power and resource expenditure is broken. 14 | 15 | It's perhaps no surprise then that the earliest fork choice rules simply selected the chain with the most total work. 16 | 17 | ::: tip TODO 18 | LCR explainer 19 | ::: 20 | 21 | The longest chain rule has the effect that a competing fork can only become canonical once its total difficulty overtakes that of any other chain. Difficulty essentially acts as a proxy for resource expenditure, so we can say that the chain chosen by the LCR generally follows whichever parties control, jointly, more than half the total current hash power. The probability of producing a valid block increases with the amount of hash power dedicated to doing so. A group of miners with more than half total hash power will always be slightly more likely than any other group to find a valid block. Over time, this ensures with increasing probability that a minority chain cannot overtake a majority one. 22 | 23 | One significant shortcoming of the longest chain rule is that it favors smaller groups with more hash power per member over larger groups with the same total hash power. This stems from the fact that a group working on many different blocks on the same chain can only produce a single block to be factored in by the LCR. The total rate of block production between the two groups will be approximately equal, as the first generates fewer blocks with higher frequency and the second many blocks with lower frequency. However, a larger percentage of these blocks become ommers and are thereby ignored by the LCR. The LCR effectively weighs the "winning" block as if only its producer, and not any others in the group, had performed any work to extend the target chain. 24 | 25 | The alternative GHOST fork choice rule attempts to address this problem by accounting for the existence of ommer blocks. When computing a score for each block, GHOST sums the difficulty of all chains that stem from the block and not only the heaviest one. This modification to the LCR guarantees that a majority group will always outpace a minority one, even if the majority is poorly coordinated. In general, GHOST has a positive effect on the extent of decentralization among the mining population. The basic algorithm behind GHOST is: 26 | 27 | 1. First, we start at the genesis block. 28 | 2. Next, we look at all of the available forks from the current block, if any. 29 | 3. If there are no more blocks in the chain, we select the current block as the head of our chain and stop our search. 30 | 4. If we only have one potential chain (no forks), then we move onto the next block. 31 | 5. If we have more than one potential chain (a fork), then we move on to the first block of the fork with the most *total* blocks, including uncle blocks. 32 | 6. Head back to step (2) with our current block. 33 | 34 | GHOST is additionally useful when block production rate is closer to network latency. For instance, this would be the case when network latency and block production rate are both on the order of 5-15 seconds. Lower ratio between the two metrics leads to an increase in the probability that a miner fails to see a new block and instead begins to work on a block stemming from the same parent. If the miner later chooses to switch their efforts to the new block, any work carried out is essentially now wasted. GHOST ensures that the miner can still influence the canonical chain by continuing work on their initial block. GHOST is typically accompanied by a reward system that compensates the producers of ommer bocks in line with the probability of such an event. 35 | 36 | ## GHOST vs. LCR 37 | 38 | GHOST often agrees with the longest-chain rule: 39 | 40 | ![GHOST and LCR Agree](/book/02__blockchains/images/fork-choice-rules/lcr-ghost-agree.png) 41 | 42 | Here, the LCR picks `Block F` because it's part of the longest chain (six blocks in total). Our GHOST rules really only apply at `Block B`, where we have our first fork. The chain after `Block C'` has a total of three blocks, whereas the chain after `Block C` has a total of four blocks. GHOST therefore moves onto `Block C` and doesn't find any other forks until stopping at `Block F`. 43 | 44 | However, in some cases, GHOST will disagree with the longest-chain rule: 45 | 46 | ![GHOST and LCR Disagree](/book/02__blockchains/images/fork-choice-rules/lcr-ghost-disagree.png) 47 | 48 | In this chain, the LCR picks `Block G` because it's part of the longest chain (seven blocks). GHOST diverges from the LCR after `Block B`. Although the chain following `Block C` is longer at five blocks, there are a total of *six* blocks in the fork starting at `Block C'` when we count uncle blocks. 49 | 50 | GHOST and the LCR have a lot in common, but clearly have certain key differences. Although GHOST consumes more information, it also introduces complexities for implementation. The LCR has seen significantly more use within existing blockchains running in production today. Our study of GHOST will, however, prove valuable when we later analyze its modification and adoption in Eth2. 51 | -------------------------------------------------------------------------------- /book/03__eth1/02__state.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "State Model" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | The manner in which a blockchain represents activities has a massive impact on the type of activities it can support. We saw in our analysis of the UTXO model that UTXOs are well suited for relatively simple value-transfer systems. However, they were much less appropriate for applications that required some form of autonomous "agent" to handle transfers. Without the ability to maintain some state, UTXOs could essentially only be used to control the flow of individual transactions. 8 | 9 | Ethereum forever changed the blockchain landscape with the introduction of the account-based state model. State in Ethereum is represented as a mapping between account identifiers and information about those accounts. Essentially, Ethereum is a database where accounts are stored and can be modified. This mapping is called the Ethereum "world state." On an abstract level, it looks something like this: 10 | 11 | ::: tip TODO 12 | TODO: World state diagram. 13 | ::: 14 | 15 | Given the ID, or address, of a particular account, the world state returns some information associated with the account. This information is called the "account state." The account state for all accounts, for instance, contains a "balance" field that represents the account's current balance of a currency known as Ether, or ETH. ETH can be transferred between accounts. ETH is also used as the reward for block producers and the fee users must pay for any transactions on the system. 16 | 17 | Eth1 allows for two types of accounts. The first is the "user account," which is controlled by a cryptographic private key. The address of a user account is derived from the public key associated with its private key and is therefore guaranteed to be unique. A user account can make a transaction, perhaps an ETH balance transfer, by signing the transaction with the private key. Besides a balance, user account state includes a field for a "nonce," which counts the total number of transactions sent from that account. A user account takes the following structure within Ethereum: 18 | 19 | ```json 20 | { 21 | "nonce": , 22 | "balance": 23 | } 24 | ``` 25 | 26 | Ethereum also allows for accounts that are controlled by some code and not by a private key. These "contract accounts" can send and receive transactions like any other. We examine the structure and behavior of these accounts in the next section. Contract account state also contains a `balance` field. Unlike user accounts, however, they contain an additional `codeHash` field that stores the hash of the code that defines the contract's behavior. Furthermore, contracts have an associated `nonce`, which counts the total number of contracts deployed by that contract. 27 | 28 | Contract accounts also contain a final field called the `storageRoot`. This is a compact reference to the information that the contract wishes to store for later. User accounts are not allowed to store information in this manner. We use the word "root" because the actual information is stored in a special data structure called a "Merkle-Patricia Trie." A Merkle-Patricia Trie is much like a Merkle Tree, except it allows for key-value access. Since this structure appears in multiple places throughout Ethereum, we explain it thoroughly below. 29 | 30 | The structure of a contract account in Ethereum is as follows: 31 | 32 | ```json 33 | { 34 | "nonce": , 35 | "balance": , 36 | "codeHash": "", 37 | "storageRoot": "" 38 | } 39 | ``` 40 | 41 | In practice, we use the above structure for both user accounts and contract accounts. As user accounts have no `codeHash` or `storageRoot`, the entries for these fields are set to certain default values. The `storageRoot` for a user account is simply the empty string, whereas the `codeHash` for a user account is the `keccak256` hash of the empty string. For instance, a user account may contain the following state: 42 | 43 | ```json 44 | { 45 | "nonce": 1, 46 | "balance": 10000, 47 | "codeHash": "0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470", 48 | "storageRoot": "" 49 | } 50 | ``` 51 | 52 | We now briefly turn our attention to the Merkle-Patricia Trie, a data structure frequently used in the world of Ethereum. The Merkle-Patricia Trie allows us to store key-value pairs within a traditional Merkle Tree structure. The "trie" component of the name stems from a special tree data structure called a "radix trie," in which branches represent some key. At each node, there are some children each associated with a given key. For instance, the root of such a trie might look like this: 53 | 54 | ::: tip TODO 55 | First diagram. 56 | ::: 57 | 58 | We can add values to these nodes so the trie takes the form: 59 | 60 | ::: tip TODO 61 | Second diagram. 62 | ::: 63 | 64 | Using the key `H` we get a value of `10`. With `L`, we get `5`. We can add more nodes to make the trie more interesting: 65 | 66 | ::: tip TODO 67 | Third diagram. 68 | ::: 69 | 70 | Within this trie, the value at `HEY` is `7`, while the value at `HI` is `9`. These values are all arbitrary, but they demonstrate how such a trie allows us to store key-value pairs. Ethereum's trie is constructed so that keys are hex values of a fixed length (32 bytes). Each node in the trie has 16 children, corresponding to the hex digits `0-F`. This looks something like this: 71 | 72 | ::: tip TODO 73 | Fourth diagram. 74 | ::: 75 | 76 | Again, each node here has a value. Given the hex string `67FDA`, we would go to the sixth child of the root, the seventh child of that child, and so on. The value at our final node is the result of our lookup. 77 | 78 | ::: tip TODO 79 | Fifth diagram. 80 | ::: 81 | 82 | We can then use this trie as a Merkle Tree. Starting from the bottom, we hash the value of children together and this becomes the value of their parents. We repeat this process until we get to the root node. 83 | 84 | ::: tip TODO 85 | TODO: Sixth diagram. 86 | ::: 87 | 88 | The hashed value at this root node is then the Merkle root of the tree. Since this is a Merkle tree, we can prove inclusion of a specific leaf with a Merkle proof. Unlike a binary Merkle Tree, our proof might need up to 15 siblings for each level in the tree. Ethereum has some efficiency improvements for this problem in practice when the tree is sparse. 89 | 90 | ::: tip TODO 91 | TODO: Seventh diagram. 92 | ::: 93 | 94 | This sort of Merkle-Patricia Trie is used to represent storage values for contracts. All variables and stored values are given a unique 32 byte identifier. One can later look up the value of these variables through the trie. The full account state for a contract account therefore looks like this: 95 | 96 | ::: tip TODO 97 | TODO: Diagram. 98 | ::: 99 | 100 | The world state itself is also stored in a Merkle-Patricia Trie, except the keys are account addresses. Combining this with the account state, we have our final image of state in Ethereum: 101 | 102 | ::: tip TODO 103 | TODO: Diagram. 104 | ::: 105 | -------------------------------------------------------------------------------- /book/05__proof-of-stake/03__fork-choice.md: -------------------------------------------------------------------------------- 1 | --- 2 | path: "/chapters/proof-of-stake/fork-choice" 3 | title: "Fork Choice" 4 | status: "2" 5 | --- 6 | 7 | # {{ $frontmatter.title }} 8 | 9 | Since forks are always a possibility in any blockchain, we need to extend our basic Proof of Stake chain with a fork-choice rule that allows validators to find the "correct" chain. 10 | 11 | In the case of a one-block fork, we can simply say that validators should follow whichever fork has more votes. 12 | 13 | ::: tip TODO 14 | Image for one-block fork. 15 | ::: 16 | 17 | As ever, this decision becomes more complex when deciding between chains more than one block in length. 18 | 19 | ## An Old Friend: The Longest Chain Rule 20 | 21 | We could consider, for instance, using a version of the Longest Chain Rule modified to fit our Proof of Stake environment. Instead of following the chain with the highest total difficulty, we follow the chain with the most total votes, weighted by stake. 22 | 23 | Our algorithm would take the form: 24 | 25 | ::: tip TODO 26 | Add LCR algorithm here. 27 | ::: 28 | 29 | Such an algorithm would be quite effective at resolving a fork such as the one shown below, in which we can clearly see a "winner": 30 | 31 | ::: tip TODO 32 | Image with lopsided forks. 33 | ::: 34 | 35 | Unfortunately, the realities of a blockchain network leave this fork-choice rule open to attack. 36 | 37 | ### High-Latency Attacks 38 | 39 | Imagine a scenario, for instance, in which a malicious validator controls $\smash{\frac{1}{3}}$ of the stake on our network. This attacker wants to create a competing chain that somehow overcomes the primary chain. 40 | 41 | If the primary chain is operating smoothly, then the Longest Chain Rule prevents the attacker from getting very far. Even if the attacker refuses to publish blocks on the primary chain, thereby leaving an empty slot every third block on average, the attacker can only create one block for every two created on the primary chain. Furthermore, these blocks will only have half of the votes, on average, of primary chain blocks. 42 | 43 | ::: tip TODO 44 | Image with attacker's chain in ideal conditions. 45 | ::: 46 | 47 | The Longest Chain Rule starts to show weakness, however, in the presence of particularly high network latency. With sufficient latency on the primary chain, we may run into situations in which a validator does not receive the previous block before the validator's assigned slot begins. In this case, the validator will build their block on the last known block. Once the missing block becomes available, we end up with a small chain fork. 48 | 49 | ::: tip TODO 50 | Image with attacker's chain and small latency fork. 51 | ::: 52 | 53 | An attacker can use this latency to their advantage. As latency increases, the primary chain extends in length more and more slowly. If latency reaches four times the slot length, then the primary chain will, on average, only extend by one block (with $\smash{\frac{2}{3}}$ votes) for every two blocks (with $\smash{\frac{1}{3}}$ votes each) on the attacker's chain. 54 | 55 | ::: tip TODO 56 | Image with attacker's chain and large latency fork. 57 | ::: 58 | 59 | At any higher latency, the attacker's chain will quickly begin to outpace the primary chain, even though the attacker only controls $\smash{\frac{1}{3}}$ of the total stake. The Longest Chain Rule doesn't seem to accurately capture the will of the majority as latency increases. 60 | 61 | ::: tip TODO 62 | Image with attacker's chain and massive latency fork, LCR picks attacker. 63 | ::: 64 | 65 | ## The New Kid on the Block: LMD GHOST 66 | 67 | Eth2 introduces a fork choice rule called "Latest Message Driven GHOST", LMD GHOST for short. Based on the GHOST algorithm described in our overview of Eth1, LMD GHOST patches the holes in the Longest Chain Rule by acknowledging votes on uncle blocks. 68 | 69 | LMD GHOST, like GHOST, consider votes within uncle blocks to a chain as votes for the chain itself. Furthermore, in order to avoid double-counting votes, LMD GHOST only takes into account the *latest messages* from each validator. In combination, these two rules give the following algorithm: 70 | 71 | ::: tip TODO 72 | LMD GHOST algorithm. 73 | ::: 74 | 75 | LMD GHOST often agrees with the Longest Chain Rule. 76 | 77 | ::: tip TODO 78 | Image LMD GHOST agrees with LCR. 79 | ::: 80 | 81 | However, LMD GHOST addresses the attack vector for the Longest Chain Rule because a minority chain can never receive more votes than a majority chain. The attacker's chain from our previous example can never receive more than $\smash{\frac{1}{3}}$ votes, no matter the length of the chain. 82 | 83 | ::: tip TODO 84 | Image for attacker failing under LMD GHOST. 85 | ::: 86 | 87 | Additionally, the attacker is unable to force the rule to favor their chain in the case that the primary chain has stopped producing blocks altogether. 88 | 89 | ::: tip TODO 90 | Image for attacker failing even when other chain is offline. 91 | ::: 92 | 93 | ### Saved Message Attacks 94 | 95 | So far, our chain has slashing rules that prevent validators from creating multiple votes during the same slot. Furthermore, LMD GHOST ensures that votes for the same chain created by the same validator in different slots won't be counted twice. We have not, however, prevented validators from making votes for *different* chains during different slots. 96 | 97 | An attacker with sufficient stake could abuse this fact to cause a prolonged fork. An attacker could "save" votes by refusing to publish them during previous slots and then later "release" these votes to manipulate the fork decisions of other validators. 98 | 99 | For instance, take the following situation in which an attacker with $\smash{\frac{1}{3}}$ stake has not published votes for several slots in a row. 100 | 101 | ::: tip TODO 102 | Image for saved messages. 103 | ::: 104 | 105 | At this point, the attacker has accumulated $\smash{\frac{1}{3}}$ votes for each of the missed slots. The attacker then waits for a fork in the chain, perhaps due to network latency, and publishes $\smash{\frac{1}{3}}$ votes in favor of one of the two blocks. 106 | 107 | ::: tip TODO 108 | Image for publication of initial votes. 109 | ::: 110 | 111 | Next, the attacker waits until they see other validators publish enough votes that this first block almost has a majority. At this point, the attacker uses another set of saved up votes to vote for the other forked. Since LMD GHOST considers latest messages, it will remove the attacker's original votes and only consider the latest ones. Now, LMD GHOST swaps and suddenly favors the other block. 112 | 113 | ::: tip TODO 114 | Image for publication of second votes. 115 | ::: 116 | 117 | Since LMD GHOST favors the second block, validators who have not yet voted will begin to submit votes for the second block. Our attacker can continue to execute this "flip-flop" attack for many blocks in a row, forcing the chain to split between two potential forks. 118 | 119 | Eth2 addresses this problem by refusing to accept any votes older than a certain number of slots. This prevents any attacker from saving up a significant number of votes. 120 | -------------------------------------------------------------------------------- /book/03__eth1/08__scaling.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Scaling Ethereum" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ::: tip NOTE 8 | Not very happy with this draft yet. Needs a clearer vision. 9 | ::: 10 | 11 | Ethereum blocks are created roughly every 15 seconds and contain, on average, somewhere between 50 and 150 transactions. Other comparable blockchain systems share a similarly sluggish pace in comparison to modern centralized payment providers. However, this stems not from any lack of demand on the part of users; Ethereum currently has approximately 60,000 transactions pending for inclusion within the blockchain. Such a discrepancy in supply and demand suggests a more fundamental limitation on these blockchain systems. 12 | 13 | Ethereum's limited throughput is actually a result of caps set by miners on the total amount of gas that can be expended within their blocks. Certain blockchains, like Bitcoin, instead set fixed caps to the size of all blocks on the network. This fact seems to suggest the reasonable conclusion that these systems should increase throughput by simply removing any sort of limit on gas expenditure within blocks. We can become more familiar with the fundamental problems of blockchain scaling by exploring the impact of this sort of change in an imaginary blockchain. 14 | 15 | We'll begin by defining two variables for our analysis. `state_size` represents the total size of the state within our imaginary chain. `participation_cost` represents the average cost for a user to participate in the consensus process of our chain. Like in most blockchains, users that participate in consensus must execute all transactions within each block for themselves. 16 | 17 | From this last property, we can begin to see our dilemma. If we double the size of each block in the chain, we've doubled `state_size`. However, users now also need to execute roughly twice as many transactions, so we've also doubled `participation_cost`. For this reason, block size increases are considered a form of *linear scaling*, in that they increase `state_size` at the same rate as `participation_cost`. 18 | 19 | Increases to `participation_cost` are, generally, bad for a blockchain system. Much of the decentralized nature of Ethereum and other similar blockchains relies on the ability for a wide variety of users to easily participate in the system. This diversity and accessibility increases the robustness of the protocol to centralization or collusion between different parties. It is, quite simply, harder to attack a network when its participants are culturally, geographically, and socially diverse. 20 | 21 | So, as `participation_cost` increases, we very quickly begin to face issues with centralization. Clearly, a higher entry cost naturally reduces the number of individuals who can afford to participate in the network. Given the rampant wealth inequality of modern capitalism, even small increases to `participation_cost` might exclude large swaths of potential participants. 22 | 23 | Our concerns don't end there, unfortunately. If `participation_cost` grow to a sufficient level, users may no longer find it feasible to participate from a home computer and may be forced to rely on already-centralized cloud infrastructure. Such a scenario would give cloud providers significant control over the network simply by nature of having control over the physical hardware running its nodes. 24 | 25 | This relationship between participation costs and centralization forms the basis of the scalability dilemma in modern blockchain systems. Linear increases `state_size` and `participation_cost` are ineffective long-term solutions. Instead, blockchains seek to find constructions that allow for *superlinear* scaling, in which `state_size` increases result in a disproportionately small increase to `participation_cost`. 26 | 27 | In order for `state_size` to increase more rapidly than `participation_cost`, it *must* be true that some users are not executing all transactions. Indeed, the basis for all superlinear scaling is that, under certain conditions, users may safely ignore a given transaction entirely. 28 | 29 | ## Layer 2 Scaling Strategies 30 | 31 | Several strategies have been developed toward this goal of superlinear scaling. One overarching class of strategies, layer 2 scaling mechanisms, focuses on the introduction of blockchain *applications* that increase total transaction throughput. Generally, these applications take advantage of situations in which groups of users are making many transactions amongst themselves. 32 | 33 | Payment channels are one such layer 2 scaling mechanism used to simplify the process of making payments within a very small group, typically only two people. Users can "enter" into a payment channel with other users by depositing funds into an application on an existing blockchain. These deposits represent the initial balances within the channel, which can then be modified by agreement of all parties in the channel. 34 | 35 | Since all parties are agreeing to all balance changes, there's no need to publish all of these changes publicly. Instead, the users simply need to publish the *final* set of balances in order to "close" the channel. Users could potentially carry out an unlimited number of transactions privately before settling back onto the blockchain. Since these intermediate transactions are never published, `participation_cost` for the primary blockchain does not increase. 36 | 37 | ```text 38 | TODO: Is it worth going this deeply into L2? Should we talk about plasma and rollups? 39 | ``` 40 | 41 | ## Layer 1 Scaling Strategies 42 | 43 | Layer 2 scaling mechanisms are inherently limited by the blockchains they're deployed to. These mechanisms can often be quite complex by themselves, and often need to introduce additional complexities to address limitations of their host chains. As a result, we simultaneously explore layer 1 scaling opportunities, another class of mechanisms involving modifications to the underlying blockchain itself. 44 | 45 | The previously discussed block size increase would fall under the category of layer 1 scaling, as it involves a modification to the base layer of the system. Certain blockchain networks utilize this philosophy heavily and simply require that users bear a very high participation cost. However, we found this sort of change inadequate as it only scaled the system linearly. 46 | 47 | ```text 48 | TODO: How much detail should we add about sharding? 49 | ``` 50 | 51 | ## An Aside: Combining Layer 1 and Layer 2 52 | 53 | Because layer 1 changes impact the base protocol, they can often be difficult to implement. Large client updates are complex to organize and can be contentious in some cases. Particularly, changes to miner rewards or economics have often faced strong backlash. In general, layer 2 mechanisms attempt to increase the throughput of a blockchain system without introducing changes at the base layer. By avoiding contentious changes, layer 2 approaches can innovate much more quickly than their layer 1 counterparts. 54 | 55 | However, these mechanisms are always limited by the underlying blockchain. Layer 1 improvements generally increase the number of available options on layer 2. These two classes of improvements therefore tend to feed into one another; both layer 1 and layer 2 have their place in the scaling ecosystem. 56 | -------------------------------------------------------------------------------- /book/00__introduction/00__foreword.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Foreword" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ## A bit of history on the book 8 | 9 | *Kelvin's Ethereum Book*, previously *The Eth2 Book*, has been a work in progress for several years. I originally started work on this project in late 2019 while taking a year off from more traditional employment in the crypto space. At the time, this project was being kept alive with funding from the Quilt team over at [ConsenSys](https://consensys.net/) and later with a grant from [MolochDAO](https://molochdao.com/). I'm forever grateful to both groups for taking a chance on this effort. 10 | 11 | Work on this project stalled in 2020 when some things changed in my personal life and I mostly ran out of money. I ended up taking a full-time job as a software engineer (and/or benevolent chaos monkey) at [Optimism](https://optimism.io) which, at the time, was a tiny startup that needed a lot of attention. I no longer had the time and energy to commit to a project like this, and the project mostly went abandoned. 12 | 13 | Still, I felt a strong responsibility to eventually return to the project when I was finally able to do so. It's been about two years now, and Optimism is finally at a place where I feel like I can start to build things outside of work again. When thinking about what side-projects to tackle next, *Kelvin's Ethereum Book* was the obvious first choice. Writing a book is *hard*. I've always hoped I would be able to say that I'd been able to write a book. So work on *Kelvin's Ethereum Book* begins again. 14 | 15 | ## What this book is about 16 | 17 | *Kelvin's Ethereum Book* is a love letter to Ethereum. It's my meditation on Ethereum as a technical object. I probably won't have time to write about all of the applications that have proliferated on top of Ethereum, but we'll see. My goal with this book is to create a complete end-to-end explainer of Ethereum. I want to explore every nook and cranny, every minute detail. 18 | 19 | ## On the general state of the book 20 | 21 | Currently, the book is mostly unpolished and unedited. Large portions of the skeleton of the book have been filled in, but there are likely to be many missing details, spelling errors, and grammatical mistakes. My philosophy in writing this book has been a "breadth-first" approach as I find that overall content structure and layout is the most important part of an educational work. Smaller details, like headings, links, citations, etc., are best to be left toward the end of the project (lest we remove an entire section and waste a significant amount of effort). 22 | 23 | I still feel this is the right way to work on a project of this magnitude, but I acknowledge that it may make parts of the book hard to read in its current state. I hope that by making the work public I can start to crowd-source a lot of these smaller details. Anyway, this is all to say that the book is definitely not complete in any sense of the word and I hope you'll bear with me as the kinks get worked out. 24 | 25 | ## How this book is being written 26 | 27 | I originally wrote this book alone (inside of a private repository). I spent most of my time for a period of about six months sitting in front of my computer staring at this book. I wrote most of my first drafts on paper -- I have a stack of dozens of sheets of hand-written drafts sitting somewhere in my house. I thought this was the only way to write, but then again I'd never written a book before. 28 | 29 | I found that this approach to writing was somewhat lonely. With a subject as complicated as Eth2, it's important to get lots of eyes on your content. Much of the information related to Eth2 is still sitting in the heads of just a few people (although the extent to which this is true has been reduced significantly since I first started writing the book). I also found that there was little pressure to keep pushing on the project when things got difficult. I figured that I needed a new way to write this content on my second attempt. 30 | 31 | As a result, I'm now writing all of this content out in the open. I plan to make this my primary project outside of Optimism for the foreseeable future. You can view the entire [source for this book on GitHub](https://github.com/smartcontracts/ethereum-book). You're more that welcome to make edits to this content by submitting pull requests. You should be able to find links to edit this content at the bottom of each page. 32 | 33 | ## Contributing 34 | 35 | Any and all contributions to this work are very much appreciated. I would love for this to be a collaborative effort. As a way of saying thank you to contributors, I will be creating hand-drawn NFTs for everyone who submits suggestions, edits, or other similar contributions (on Optimism, because I'm biased and also because it's way cheaper than sending NFTs on L1 and I'm not made of money). You'll also be credited as a contributor within the book itself. 36 | 37 | As far as the contributing process goes, here are my initial thoughts: 38 | 39 | - For small typos and other similar minor corrections, please make a [pull request](https://github.com/smartcontracts/eth2-book/pulls) that includes the change. I'll usually merge these without any back-and-forth, assuming the correction is correct. 40 | - For larger suggestions (like comments about inaccuracies, content structure suggestions, etc.), please start by creating an [issue](https://github.com/smartcontracts/eth2-book/issues) so that I can review the change before a pull request gets made. I may have thoughts on the subject and I don't want to waste your time with a pull request until we've been able to come to consensus about the validity of a larger change. 41 | 42 | ## A quick note on the Eth2 rebrand 43 | 44 | Long after work on this project started, [the Ethereum community rebranded "Eth2" to refer to the new consensus layer](https://blog.ethereum.org/2022/01/24/the-great-eth2-renaming/) instead of using Eth2 as the name for the entire system itself. I have no strong opinions on the change, but this does now mean that it's necessary to go back and edit large parts of this work. Conveniently, this is a great excuse for going over the entire book to do some edits. I think this is probably an important first task to start working on now that the book is public. 45 | 46 | You will likely see parts of this book change quite significantly as I attempt to reframe things to refer to the ["Ethereum Upgrades"](https://ethereum.org/en/upgrades/) and try to limit usage of the term "Eth2". Of all the things that were subject to change when I started working on this project, a new name for the whole thing wasn't something I really expected. 47 | 48 | ## Acknowledgements 49 | 50 | First of all, I'd like to thank the Quilt team over at [ConsenSys](https://consensys.net/) for taking a chance on this book in 2019. I was very particular about my writing style and I appreciate all of the patience I received with reviews and feedback from the Quilt team. I'd like to thank [Will Villanueva](https://twitter.com/wjvill), [lightclients](https://twitter.com/lightclients), and [Ben Edgington](https://twitter.com/benjaminion_xyz) for acting as editors and reviewers for the project in 2019 and 2020. 51 | 52 | I'd also like to thank [MolochDAO](https://molochdao.com/) for giving me a generous grant in 2020 when the project moved away from funding via ConsenSys. I felt that I hadn't lived up to the grant when the project came to a halt, but I hope that I can pay that grant forward by finally turning this into a useful resource for the Ethereum community as a whole. 53 | 54 | This list is not complete and will be updated heavily as time goes on. I'll do my best to make sure that everyone who's contributed to this effort is reflected here. For now, enjoy the book! 55 | -------------------------------------------------------------------------------- /book/03__eth1/07__clients.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Client Types" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Blockchain clients can access and participate in the system in various ways and to differing degrees. We previously explored the general concepts of full, archival, and light clients. Many users choose to operate full nodes that process all transactions but prune components of the system state after it's not longer likely to be useful. Some others run archival nodes that do not prune state, primarily for the purpose of searching and analyzing blockchain data. Others still maintain light clients that can store significantly less information at the cost of some hit to security. Within this section, we take a look at some of the techniques used in Eth1 to support these different operational modes. 8 | 9 | Full nodes are the most common form of client amongst those that download and execute all new transactions. It's valuable to understand how these nodes implement state pruning in order to reduce storage requirements. Specifics can vary between different client software packages, but many high-level goals are universal. Recall that the Eth1 world state takes the form of a mapping from account addresses to account state structures. Contract accounts in particular include a "storage" field within this structure. Each block updates the world state in some way. We're most interested in finding and "forgetting" elements of the world state that exist in one block but are removed in the next. 10 | 11 | One way in which such a removal from the world state can occur is through the storage deletion operation available to contract accounts. Contract storage takes the form of a key-value database accessible only to the contract itself. Contracts may occasionally decide to delete the value associated with some key when that value is no longer useful. Eth1's state tree would be modified by this action such that the particular tree node corresponding to this key-value pair no longer exists: 12 | 13 | ::: tip TODO 14 | Deletion operation impact diagram. 15 | ::: 16 | 17 | Although the original node has been dropped from the trie, clients should generally hold on to the node for the possibility of a chain fork. If a fork lacking the deletion operation were to become canonical, the original node would once again be relevant: 18 | 19 | ::: tip TODO 20 | Effect of a fork diagram. 21 | ::: 22 | 23 | It's for this reason that client software typically defines a large threshold of blocks that must be appended to any given block before state pruning can be considered. Thresholds often fall on the order of several thousand blocks. If some particular node in the state trie was removed and not later re-inserted before the threshold was crossed, the node can relatively safely be deleted in its entirety. This strategy only puts full nodes at risk in the event of a fork spanning a number of blocks greater than the threshold (for context, the largest fork in Eth1's history spanned `TODO: ??` blocks). If this were ever to happen, full nodes would be forced to rely on the relatively small population of archival nodes to retrieve any relevant historical data. 24 | 25 | Eth1 archival nodes are somewhat resource intensive and therefore out of reach of many network participants. At the time of writing, these nodes store on the order of several terabytes of data. Some services, like block explorers, take nodes a step further and implement custom logging, or "instrumentation," features. These additions serve to record useful information often related to the inner details of transaction execution. For instance, popular block explorer EtherScan captures the complete list of EVM instructions carried out during each contract interaction. Users of the service can then step through and carefully analyze the effects of any given transaction. Though beneficial, this information is not necessary for normal network participation and can greatly increase cost and storage requirements. 26 | 27 | Eth1 natively supports light client modes for the sake of resource-constrained environments. As is typical, light clients follow slightly different protocols when carrying out different actions. We'll first take a look at the process by which a light client can ascertain the state of a given account. Much of Eth1's support for this behavior comes from its distinction between block headers and block bodies. Block bodies include complete block information, such as the full list of executed transactions. Block headers, on the other hand, contain identifying information about a block and, crucially, the roots of Merkle trees generated from the contents of the body. 28 | 29 | The root of the world state trie can be found in every block header. Light clients can download headers at the relatively small cost of 508 bytes per header. Clients can then make use of this root to find the state of an account at some block by following a simple procedure. Given that the state trie is designed to be easily traversable, light clients can request specific nodes from the trie one-by-one until they reach their target node. The following diagram depicts this process: 30 | 31 | ::: tip TODO 32 | Diagram showing requests necessary to get state element. 33 | ::: 34 | 35 | Light clients can employ a very similar protocol to check for the inclusion of a transaction within a block. Transaction receipts already contain the index of a transaction in the full list of transactions in a block. Light clients can therefore simply request nodes from a block's transaction trie until they find their desired transaction: 36 | 37 | ::: tip TODO 38 | Diagram showing requests necessary to get transaction. 39 | ::: 40 | 41 | ::: tip TODO 42 | Need to figure out where to first bring up Bloom filters, probably should be explained before this. 43 | ::: 44 | 45 | Light clients can also watch for specific transaction logs with some additional effort. Block headers contain a Bloom filter populated by the logs of all transactions in the corresponding block. Light clients can first use this filter to find blocks that may include relevant logs. For each matching block, the client must then download the full list of transaction receipts to check whether they do, in fact, contain relevant logs. This process can be visualized as follows: 46 | 47 | ::: tip TODO 48 | Diagram showing light client log process. 49 | ::: 50 | 51 | Light client protocols allow users to retrieve and verify data from the system at a significantly decreased storage cost. However, without the full block data, these clients cannot be certain that the block is entirely valid. Light clients would not, for instance, necessarily be aware of the presence of a transaction that was not properly executed. Light clients instead typically rely on presence of later blocks to judge the authenticity of a block at hand. Under standard network conditions, a majority of hash power should be dedicated to the extension of a valid chain. Light clients can therefore somewhat safely assume a block to be valid after sufficiently many blocks have been produced to follow it. Of course, this metric is less secure than full validation of every block. 52 | 53 | Client flexibility continues to be improved in Eth2. Light client support is particularly expanded with the addition of new tree structures. Eth2 light clients will be able to drill down into significant more detailed elements of a block before having to download chunks of data. These changes aim to bring Ethereum to a much wider range of platforms, perhaps even opening the door to efficient light client implementations designed as applications on other blockchain systems. We explore these new possibilities in greater detail within our later chapter on Eth2 light client protocols. Many of the techniques applied to client constructions in Eth2 will come in handy not only in our discussion of Eth2, but in study of nearly any blockchain system. 54 | -------------------------------------------------------------------------------- /book/01__consensus/06__applications.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Stateful Applications" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Before we move on to our analysis of more modern fault tolerant systems, we should become familiar with the ways in which these systems represent applications. As we know, BFT systems provide us with a set and ordering of events, or "transactions." These transactions alone are not very useful unless we attach some sort of meaning to the information they contain. However, once we do this, we can effectively represent any type of application that we can imagine. The process of designing any such application requires the definition of only three components, the state, the transaction, and the state transition function. 8 | 9 | The "state" of an application refers to the stored information that details the universe of that application at some point in time. The term isn't particularly descriptive, possibly a reflection of the vast variety of structures and values that the state of an application can take. For any potential use-case it's important that the state is carefully constructed as to encapsulate fully the set of desired functionalities. Let's now attempt to understand this concept by example. 10 | 11 | Although the idea of a "transaction" may sound strongly tied to finance, we use the term without any direct financial connotation. To cement this point, our example application will attempt to avoid references to finance. Perhaps we'd like to build an application that keeps track of the number of cats currently residing in the author's home. A basic structure for the state of such an application may take the following form: 12 | 13 | ```json 14 | { 15 | "number_of_cats": 16 | } 17 | ``` 18 | 19 | We can further extend this state to add more detail to the application. In addition to a simple `number_of_cats` field, we could introduce a mapping between the names of each cat and some information about them. Here, we'll include the age of each cat and the number of objects they've broken over the last year. 20 | 21 | ```json 22 | { 23 | "number_of_cats": , 24 | "cats": { 25 | "": { 26 | "age": , 27 | "objects_broken": 28 | } 29 | } 30 | } 31 | ``` 32 | 33 | When we first initialize this state, we'll have an empty application with no cats. This is relatively boring, so we now need to introduce some mechanism through which the state can be mutated. The first piece of this puzzle is the transaction. Transactions are messages that communicate some desired change to the state of the application. We can develop different transaction "types" that correspond to changes to different parts of the state. 34 | 35 | For instance, we can construct a format for a particular transaction type that adds a new cat to our home: 36 | 37 | ```json 38 | { 39 | "transaction_type": "New Kitty", 40 | "cat": { 41 | "name": "", 42 | "age": , 43 | "objects_broken": 44 | } 45 | } 46 | ``` 47 | 48 | We can then additionally define a type of transaction that modifies some trait about a cat in our home: 49 | 50 | ```json 51 | { 52 | "transaction_type": "Update Kitty", 53 | "cat": "", 54 | "field": "", 55 | "change": 56 | } 57 | ``` 58 | 59 | As you may be able to infer from these examples, transactions tell an application about the changes a user is attempting to make. Our first transaction type includes all of the information necessary to add a new cat to the state. Our second transaction instead only includes the specific field we want to mutate, and by how much. Essentially, these are descriptive representations of the "things happening" within our little application. 60 | 61 | In order to make these transactions useful, however, we must define some logic that details exactly how a given transaction will impact the state of the system. We refer to this logic as the "state transition function" of the application. When given some initial state and a transaction, the state transition function will return a new state. We often depict this function mathematically as: 62 | 63 | ``` 64 | state_transition_function = (old_state, transaction) => new_state 65 | ``` 66 | 67 | Within our specific application, the state transition function should be able to recognize and handle the two transaction types previously defined. When it sees a "New Kitty" transaction, it should add the cat to the state and increment the total number of cats. When it sees an "Update Kitty" transaction, it should mutate the state according to the specified update. An implementation of this logic in Python might resemble the following: 68 | 69 | ```python 70 | def state_transition_function(old_state, transaction): 71 | new_state = old_state 72 | 73 | if transaction["transaction_type"] is "New Kitty": 74 | cat = transaction["cat"] 75 | 76 | new_state[cat["name"]] = { 77 | "age": cat["age"], 78 | "objects_broken": cat["objects_broken"] 79 | } 80 | 81 | new_state["number_of_cats"] = old_state["number_of_cats"] + 1 82 | 83 | if transaction["transaction_type"] is "Update Kitty": 84 | cat = transaction["cat"] 85 | field = transaction["field"] 86 | change = transaction["change"] 87 | 88 | new_state[cat][field] = old_state[cat][field] + change 89 | 90 | return new_state 91 | ``` 92 | 93 | We've defined our three components, the state, the transaction, and the state transition function. We've now represented the entirety of our application. We can run through a dynamic example of the application as a series of transactions operating on some initial state. We're assuming here that the application begins in an "empty" state: 94 | 95 | ```json 96 | { 97 | "number_of_cats": 0, 98 | "cats": {} 99 | } 100 | ``` 101 | 102 | Next, we introduce a series of transactions: 103 | 104 | ```json 105 | { 106 | "transaction_type": "New Kitty", 107 | "cat": { 108 | "name": "Dexter", 109 | "age": 5, 110 | "objects_broken": 10 111 | } 112 | }, 113 | { 114 | "transaction_type": "New Kitty", 115 | "cat": { 116 | "name": "Nico", 117 | "age": 1, 118 | "objects_broken": 0 119 | } 120 | }, 121 | { 122 | "transaction_type": "Update Kitty", 123 | "cat": "Nico", 124 | "field": "objects_broken", 125 | "change": 20 126 | } 127 | { 128 | "transaction_type": "New Kitty", 129 | "cat": { 130 | "name": "Moony", 131 | "age": 0, 132 | "objects_broken": 0 133 | } 134 | } 135 | ``` 136 | 137 | After running each of these transactions through our state transition function in order, we can determine the final state of our system: 138 | 139 | ```json 140 | { 141 | "number_of_cats": 3, 142 | "cats": { 143 | "Dexter": { 144 | "age": 5, 145 | "objects_broken": 10 146 | }, 147 | "Nico": { 148 | "age": 1, 149 | "objects_broken": 20 150 | }, 151 | "Moony": { 152 | "age": 0, 153 | "objects_broken": 0 154 | } 155 | } 156 | } 157 | ``` 158 | 159 | Although this example has been rather simplistic, it demonstrates the basic process of constructing an application for transaction-based systems. These same fundamental principles are applicable to almost any application one can dream up. The state of a tic-tac-toe board, for instance, might contain the current positions of "X"s and "O"s on the board, and its transactions would update the board. Entire virtual computers can be represented under this model, as we'll soon discover in our discussion of Ethereum. In the following chapter, we'll come to understand how these ideas have been applied to the next generation of fault tolerant systems. 160 | -------------------------------------------------------------------------------- /book/00__introduction/01__motivation.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Motivation" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | ## Why this book exists 8 | 9 | *Kelvin's Ethereum Book* is, perhaps unsurprisingly, an attempt to solidify the state of written educational content regarding Eth2. Knowledge of Eth2 has, until now, been largely stored in the collective brain bank of researchers and developers focused on the project. Deep understanding of Eth2 is unlikely to come under these circumstances without access to key individuals. Existing published content additionally tends to fall too far on the extremes of simplicity or complexity. 10 | 11 | This reality is unfortunate given the current timeline and importance of Eth2. A transition to a better Ethereum -- one that addresses many of the shortcomings of Eth1 -- is clearly necessary for the project's long-term prospects. Further given that development of Eth2 has accelerated quickly within the past several years and [The Merge](https://ethereum.org/en/upgrades/merge/) is now on the horizon, it's time for an approachable resource that can help readers understand and prepare for the changes to come. 12 | 13 | ## Why you should care about Eth2 14 | 15 | Whether you're an application developer, researcher, or simply somehow invested in the Ethereum ecosystem, it's extremely valuable to build a comprehensive understanding of Eth2. Eth2 shares many similarities with its predecessor, but simultaneously introduces new quirks that impact the structure and operation of Ethereum-based applications. Developers may need to rework certain aspects of their applications. Researchers may discover new tools that allow them to design novel constructions. End-users may wish to understand how these changes will impact functionality and user experiences. 16 | No matter who you are, there's something valuable to learn within these pages. 17 | 18 | Eth2 also provides a unique opportunity for you (yes you!) to have an impact on the future of Ethereum. Although much of the project's initial direction is well defined, many design decisions, particularly on the application layer, are still open for debate. Eth2 research and development is community-driven, and therefore accessible to anyone seeking to provide input. This book aims, in part, to connect readers to projects or research efforts they may find interesting. We include pipelines that help readers become familiar with the process of entering and contributing to these projects. We strongly encourage interested readers to explore these pipelines and the possibilities they may hold. 19 | 20 | ## What you can learn from Eth2 21 | 22 | Eth2 additionally presents an opportunity for motivated readers to become deeply acquainted with the fundamental elements of any blockchain system. The Eth2 consensus mechanism has largely been designed from the ground up in response to lessons learned throughout the lifespan of Eth1. Its transition from "Proof-of-Work" to "Proof-of-Stake" brings Eth2 closer to older, more established constructions in the field of computer science. Through the lens of Eth2, readers can understand what really makes a blockchain tick and to relate the novel components of Eth2 back to the inner workings of more established designs. Readers will, as a result, not only comprehend Eth2, but also build an intuitive understanding of blockchain principles that can be applied in countless other contexts. 23 | 24 | In line with the method by which Eth2 has been developed, this book revisits subjects with which readers may already be familiar. We kick off our study with an examination of the roots of the modern blockchain in the field of distributed systems. We then take a look at how these elements give rise to the blockchain in the face of a shifting problem space. Core blockchain components -- Proof-of-Work, transactions, and even the concept of blocks themselves -- are reconstructed not as inevitabilities but as potential solutions to concrete problems. In this manner the reader may begin to imagine alternative approaches to these same problems. We are all capable of exploring novel solutions once we've developed a complete picture of our constraints. 25 | 26 | ## What you need to know 27 | 28 | *Kelvin's Ethereum Book* generally attempts to avoid assumptions of existing background knowledge. Of course, this expands the potential audience for the work but, perhaps more importantly, this methodology is also intended to deconstruct common misconceptions. Blockchain systems are the emergent result of relatively self-contained building blocks. We systematically develop bases of knowledge that aim to provide readers with all of the tools necessary to fully grasp subsequent topics. Certain subjects not necessary to understand in order to continue our study are left to external resources for interested readers. All other key information can be found contained entirely within this work. 29 | 30 | This book intends to be approachable by readers with any level of relevant background knowledge. We attempt to detail basic concepts in clear, complete language without trending too far towards unnecessary simplicity or complexity. The subjects covered in this text are undeniably cut from a "technical" cloth. Still, we aim to make technical topics accessible to any potential reader. The linear layout of the book, based on our system of building blocks, allows those with background knowledge to enter the work at whichever point they deem most valuable. That said, even weathered blockchain researchers may find a new perspective from content covered in early chapters. 31 | 32 | ## How this book is structured 33 | 34 | To the end of maximal utility, *Kelvin's Ethereum Book* is structured as something between a traditional printed book and a more modern digital wiki. Content is laid out as to be consumable both in isolation or in series with other sections. Readers will find sidebars within each section that point to topics previously covered that serve as required background knowledge for the subject at hand. Those interested in specific topics may derive value from revisiting linked background sections. We additionally provide frequent links and references to earlier sections as one might expect from the wiki format. Readers on the web will also find a search bar at the top of each page and have the option to view certain figures in an interactive or animated form. 35 | 36 | ## How to get the most out of this book 37 | 38 | *Kelvin's Ethereum Book* is meant to be something like a "choose your own adventure" book. Although it contains all of the relevant information necessary to understand Ethereum from the ground up, you should feel free to skip any sections that you already feel comfortable with. For instance, if you're intimately familiar with Ethereum you may wish to skip directly to the Eth2 Overview chapter of this book. 39 | 40 | There's no need to approach this book linearly but it can still be beneficial to do so. We've tried to take a fresh look at many of the core blockchain concepts that you may take for granted. Even if you're an expert in everything crypto, you might still find something brand new or unexpected. Of course, the ultimate path of your journey is up to you! 41 | 42 | ## One last note 43 | 44 | As we come to the close of this introduction, we'll briefly restate the primary goals of this work. *Kelvin's Ethereum Book*, at its core, aims to help readers build a strong understanding of block Eth2 and blockchain systems in general. We tackle this task by developing bases of knowledge and revisiting certain subjects that some may take for granted. We simultaneously provide readers with relevant links to external reference material and clear pathways to become involved with various projects. 45 | 46 | At the end of the day, *Kelvin's Ethereum Book* looks to help readers gather the tools necessary to become independent and well-equipped thinkers within the Ethereum space. It's through the free exchange of information that we can construct a better and stronger Ethereum community. We hope that you enjoy your journey! 47 | -------------------------------------------------------------------------------- /book/01__consensus/05__byzantine-faults.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Byzantine Faults" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | Although crash-fault tolerant systems are heavily used in production applications, they sometimes prove insufficient for the task at hand. Smaller projects may be able to assume that nodes will always behave according to protocol. However, once a system becomes highly valuable, the attack landscape shifts. Perfect protocol implementations may prevent unintended node behavior, but they do not prevent an attacker from gaining access to the machines on which nodes are running. Even with strong security practices, almost any system could be compromised, at least to some extent, by a malicious adversary. 8 | 9 | It's possible to use simple crash-fault tolerance in the face of this problem, though this introduces less-than-ideal scenarios. For instance, a system could recover from an attack by reverting to a previous state. Of course, this relies on the existence of backups not yet compromised by the attacker and additionally results in a potentially massive service disruption. Research effort was, as a result, expended in the search for protocols that could provide native resistance to the presence of malicious nodes. This problem proved difficult enough to spur the development of an entirely new field of research. 10 | 11 | The problem space for systems robust to actively malicious nodes was summarily defined in a famous 1982 paper entitled, "The Byzantine Generals Problem." Within this work, Robert Shostak, Marshall Pease, and none other than the aforementioned Leslie Lamport present an entertaining abstraction of the problem in the context of a Byzantine army surrounding a large city. The author's rendition of the underlying elements in such a network became so strongly connected to the space that resulting protocols came to be known as "Byzantine fault tolerant." 12 | 13 | "The Byzantine Generals Problem" describes a fictional scenario in which a Byzantine army is attempting to execute a military maneuver around a vast city. The army is broken out into divisions, each of which is controlled by a general. Some of these generals are "loyal" and will always follow their prescribed protocol, while others are "traitors" and will attempt to actively manipulate the system to the greatest extent possible. The paper attempts to develop a protocol that allows loyal generals to agree on a unified battle plan, even in the presence of traitors. 14 | 15 | The authors give a few more constraints to further flesh out the problem. Generals communicate with one another via messenger, so we can clearly expect some time to elapse between send and receipt. One of the generals is assigned as the "commanding general" and is responsible for sending out initial orders. All other generals are referred to as "lieutenant generals." Any general, whether a lieutenant general or the commanding general, may be a traitor. Given this additional context, the exact problem statement from the paper is: 16 | 17 | A commanding general must send an order to his `n - 1` lieutenant generals such that: 18 | 19 | 1. All loyal lieutenants obey the same order. 20 | 2. If the commanding general is loyal, then every loyal lieutenant obeys the order he sends. 21 | 22 | Under this more challenging model, traitors can, for instance, lie about the orders they received in an attempt to trick loyal generals into acting on conflicting orders. Unfortunately, this reduces the number of faults the system can sustain in comparison to crash-fault tolerant systems. The authors demonstrate that with three generals, a single traitor can cause a total failure. Let's take a look at a sketch for this proof. 23 | 24 | A three-general army has one commanding general and two lieutenant generals, here depicted as `c`, `l1`, and `l2`. If our traitor is a lieutenant general, then the loyal lieutenant may receive conflicting orders: 25 | 26 | However, the loyal general may see the same exact set of messages if the commander is the traitor: 27 | 28 | Since `l1` sees the same messages in both cases, there's no way for `l1` to determine which of the generals is a traitor. This holds, without less of generality, for `l2`. As a result, the loyal generals are unable to find one another and come to any agreement about the correct order to execute. 29 | 30 | The paper later shows that if there are `n` traitors, there must be at least `2n + 1` loyal generals for the system to function properly. This requirement stems from the lack of a trusted "coordinator" available to make generals aware of their own orders and orders given to others. Previously, in our discussion of crash-fault tolerance, we assigned a node to manage the voting process, something quite similar to the commanding general in this new scenario. However, as the commanding general may be a traitor, loyal lieutenants cannot simply trust the commander's messages. 31 | 32 | Instead, the lieutenants must communicate directly with other generals in order to make a decision. The paper describes two solutions that both rely on this sort of direct communication between generals. For now, we'll assume that a general may communicate directly with any other general, i.e., the generals and their communication lines form a completely connected graph. Each solution effectively requires that, upon receiving an order from the commander, lieutenant generals share their orders with all other generals. 33 | 34 | The difference between the two presented solutions lies in the use of cryptographic signatures in the second. These signatures make it impossible for a traitor to forge a message from another general. Most modern systems use this technique, so we'll explore the basic elements of this solution now. We follow a two-step process in which the commander first distributes orders to lieutenants, and lieutenants then share their received orders with all other generals. 35 | 36 | If the commanding general is loyal, this protocol can relatively easily find a common order for loyal generals. Since traitors cannot forge an alternative order, the loyal lieutenants will only see a single order given by the commander. With only one possible order, the lieutenants will end up executing the same action. 37 | 38 | If the commander is a traitor, more formal reasoning is necessary to see the correctness of the system. It's possible that a general sees two conflicting orders from the commander. This general now knows that the commander is a traitor, but must still come to agreement with all other loyal lieutenants. We handle this scenario with a deterministic function that defines how loyal lieutenants should act when given a list of possible actions. We can therefore still execute a common order as long as we can guarantee that all loyal generals receive the same list of possible orders. 39 | 40 | A traitorous commanding general could coordinate with other traitors to attempt to give loyal generals different sets of orders. For instance, the traitors could choose to only send the conflicting order to half of the loyal generals. Without further communication, the loyal generals would not act in unison. We can provide a simple solution by requiring that loyal generals relay conflicting orders to all other generals. After this second round of communication, all loyal generals are guaranteed to share the same information. The paper shows that we can achieve the same result if loyal generals only relay orders with less than `f` votes, where `f` is the maximum number of traitors that can be safely handled by the system. 41 | 42 | One way to understand the `3f + 1` general requirement for `f` traitors is again through the lens of a "threshold" for carrying out an action. In the context of crash-faults, we needed a threshold of `n - f` votes in order to make a decision. Since crashed nodes could not vote, we could derive that all of the `n - f` votes came from properly functioning nodes. When the faulty nodes, in this case our traitorous generals, can actually contribute to the vote, we must increase the threshold to effectively discount the possibility of malicious votes. 43 | -------------------------------------------------------------------------------- /book/04__eth2-overview/03__key-concepts.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Key Concepts" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | We'll now run through a brief summary of the various core theoretical components that define Eth2 as a novel and unique system. This section acts somewhat like an introduction and overview of the remaining chapters of this text. Certain topics, such as Eth2's networking layer, are given their own chapters but not discussed here as they form less of Eth2's identity. We try to remain relatively brief in these summaries given that significantly more detailed explorations are to follow. We've attempted to provide readers of this section with a bird's-eye view of Eth2 so that one can more easily understand individual components as parts of a whole. 8 | 9 | ## Proof-of-Stake 10 | 11 | Eth2 moves away from Proof-of-Work consensus mechanisms and instead introduces a "Proof-of-Stake" protocol. At a high level, Proof-of-Stake mechanisms judge the relative "voting power" of each participant not by their ability to expend physical resources, but by their ownership of digital tokens. Token balances of consensus participants are tracked within the blockchain itself. This class of constructions differs fundamentally from Proof-of-Work systems in that active participants are explicitly identified as such. This is not to say, however, that users are unable to enter or exit the set of consensus participants. Many Proof-of-Stake designs support movement of this sort given that users meet certain requirement, perhaps a minimum token deposit to act as the user's "stake." 12 | 13 | The addition of an internal list of participants gives Proof-of-Stake some interesting advantages over Proof-of-Work. Most obvious is the elimination of resource waste central to Proof-of-Work mechanisms. A minimized carbon footprint is clearly desirable in the context of modern events. Explicit balances also give protocol designers the ability to carefully construct rewards and, crucially, penalties for certain behaviors. The power to penalize malicious activity makes it possible for Proof-of-Stake mechanisms to significantly increase the cost of an attack and as a result reduce the cost to adequately secure the system. Lower barriers to entry that naturally stem from the elimination of expensive and short-lived mining hardware only further boosts participation and security. 14 | 15 | Token balances carry additional secondary benefits for Proof-of-Stake. One such benefit of great value is the newfound relevance of traditional BFT systems research. Much of the divide between early blockchain protocols and more established BFT mechanisms can be attributed to the desire to remove lists of participants. By including this feature without sacrificing user mobility, Proof-of-Stake systems can reap most of the benefits of newer and older designs. Research carried out in the realm of BFT consensus can now be more directly applied to the Ethereum ecosystem. Crucially, proofs of correctness established more than three decades ago carry over to Eth2 with little additional effort. 16 | 17 | Like any significant change, Proof-of-Stake brings with it new territory, some familiar and some not. Core principles discussed in our examination of BFT mechanisms essentially underpin Proof-of-Stake. However, we must also introduce new concepts in order to adapt these earlier principles to a blockchain setting. Terminology is introduced or modified as to better reflect the functionality of Proof-of-Stake. Consensus participant activity differs from that in Proof-of-Work, though its final purpose remains relatively the same at the outset. Even so, Proof-of-Stake shares many more similarities than differences with the various consensus mechanisms discussed thus far. 18 | 19 | ## The Beacon Chain 20 | 21 | Eth2 takes a large step away from its colleagues with regard to the structure of the blockchain itself. Whereas most blockchain systems execute consensus logic and application logic within the same blocks. Eth2 splits these responsibilities across different chains. Any activity relating to the Eth2 consensus process is carried out on the so-called "beacon chain." Some researchers describe this chain as Eth2's "system chain," a space dedicated entirely to ensuring the proper operation of all other activities in the network. The beacon chain gives Eth2 a cleaner separation between the very distinct functionality required by the consensus process and the applications that process supports. 22 | 23 | It's difficult to provide analogies for this design in other blockchain systems, primarily because no major blockchain has yet to make a similar separation. We can partly ascribe this fact to the ubiquity of Proof-of-Work. Without the need to maintain a list of active consensus participants, Proof-of-Work is only required to attach a minimal amount of consensus-related information to each block. Most of this information consists of the actual "proof of work" that validates a block along with an identifier for the block's creator. Proof-of-Stake, on the other hand, carries out significantly more consensus-level logic within the blockchain itself. For instance, Eth2 must keep track of users who wish to enter or exit the consensus process. Given this abundance of on-chain activity, it's simply easier to handle consensus in a separate space. 24 | 25 | ## Shard Chains 26 | 27 | The beacon chain manages consensus logic and "shard chains," sometimes called "shards," manage application logic. Eth2 is one of the first blockchain systems to dedicate not one but many different functionally-identical chains to applications. Each shard chain is, in some respects, analogous in purpose to the current Eth1 chain. Users deploy and interact with applications on these shard chains. Shards act independently of one another and, as a result, vastly expand total transaction throughput in comparison to Eth1. Through some clever mathematical tricks, Eth2 can guarantee the validity of all shards without requiring consensus participants to execute every transaction. In effect this "sharded" system opens the door to highly scalable blockchain constructions without a corresponding shift towards centralization. 28 | 29 | Sharding provides users with additional benefits over other scalability approaches. Users of a single shard chain can ascertain the validity of their shard without downloading or processing any other shard. Therefore, even though total system throughput has greatly increased, user storage and processing requirements are mostly fixed. Individuals or applications can even interact with those on other shards via "cross-shard transactions." These special transactions are made possible by a continuous process of "cross-linking," in which shards data is regularly inserted into the beacon chain in the form of a cryptographic commitment. Though they introduce many improvements and possibilities, shard chains are fundamentally akin the primary chain of most modern blockchains. 30 | 31 | ## Ewasm 32 | 33 | Revamped application chains in Eth2 are accompanied by a new and improved model for executing application logic. Eth1's EVM was designed at a point in time when few adequate alternatives existed. The EVM's custom-built nature was crucial to flexible application behavior without exposing many potential security risks. However, this same nature introduced quirks and complexities that all but prevented the use of widely used programming languages and tools. Instead, an ecosystem of Ethereum-specific languages and tooling was, and continues to be, built out over the course of Eth1's lifetime. It'll be sorely missed, or perhaps not so sorely by some, but it's time to move away from the EVM in Eth2. 34 | 35 | The EVM is responsible for managing two closely related but fundamentally different activities. Most of this responsibility falls on acting as a virtual machine -- executing programs and returning any relevant output results. This functionality is not specific to the EVM. Of course, our phones, laptops, and other computerized machines must also be capable of executing programs and returning results. The EVM is distinct from these computers in that it is also responsible for natively understanding the structure of Eth1's state. This is unusual as most computers have no built-in knowledge of the applications they execute. It's much more frequently the case that the data structures of an application are managed by the application itself. 36 | 37 | It should once again be noted that and the designers of the EVM can hardly be faulted for this decision. Few other options were available for consideration and, at the end of the day, the final product worked. Fortunately, we now have mature alternatives that can resolve this quirk. Eth2 takes the step of decoupling program execution and state representation. In place of the EVM, Eth2 introduces "Ewasm" a slightly modified version of the web assembly, or "wasm," virtual machine. Originally designed as a way for developers to build web applications in languages other than JavaScript, wasm is a powerful and, importantly, light-weight virtual machine. Wasm must be modified to fit the context of Ethereum, but wide-spread tech industry support invariably extends to Ewasm. 38 | 39 | ## Execution Environments 40 | 41 | Ewasm manages program execution in Eth2, but has no native concept of "accounts" or "transactions." Instead, these concepts are shifted onto a layer called the "execution environment." Unlike Eth1, Eth2 allows multiple execution environments to operate at the same time. Each environment defines its own representation of state and the way in which that state can be manipulated by applications within that environment. This model essentially converts the fixed structure of state in Eth1 into a more flexible middleware layer. Execution environments would, for instance, make it possible for UTXO- and account-based applications to coexist and even interact. 42 | 43 | At the time of writing, the exact process by which new execution environments will be introduced to Eth2 is unclear. It seems most likely that the platform will launch with several well-vetted environments but eventually allow any user to deploy an environment for a large fee. Given that the mechanism is intended to spur innovation in state representation, Eth2 will almost certainly make deployments openly accessible at some point in time. However, environments of questionable quality may hamper the platform's initial release. Moving into production with a few environments that have faced sufficient scrutiny should provide short-term utility and security without limiting the potential for future improvement. 44 | -------------------------------------------------------------------------------- /book/01__consensus/03__simple-consensus.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Simple Consensus" 3 | --- 4 | 5 | # {{ $frontmatter.title }} 6 | 7 | In this section, we'll work to develop a simple consensus mechanism that can properly function under perfect conditions. After we've created this foundation, we'll introduce new problems and attack vectors and update our mechanism accordingly. For now, though, it's best to start with a simple core that can begin to explain the basics of consensus. 8 | 9 | ## Restating the goal 10 | 11 | Let's briefly restate the goal of our mini consensus mechanism. We have a series of nodes (computers) that are interacting with one another over a network. We want to develop a system in which these nodes can talk to one another and come to an agreement (consensus) about the ordering of some set of "events". For now, let's not worry about what these events represent and simply assume that we want to come to agreement over their ordering. 12 | 13 | ## Initial setup 14 | 15 | Let's start simple and try to create a consensus protocol that works for a network with only three nodes. Each of these nodes are connected to every other node. We're going to make two key assumptions that will have a large impact on our the type of protocols we can construct: 16 | 17 | 1. Nodes will always perfectly follow the prescribed protocol (no faults). 18 | 2. It takes some small random amount of time for messages to travel across the entire network. 19 | 20 |
21 | 22 |
23 | 24 | ## A naive protocol 25 | 26 | Let's start by describing a naive protocol and then examine why the protocol is broken. One of the simplest protocols we could construct would be to have almost no protocol at all. So, let's build a minimal protocol: 27 | 28 | 1. Every node on the network is allowed to broadcast events whenever they want. 29 | 2. Whenever a node receives an event, they add that event to their canonical list of events. 30 | 31 | ### A working example 32 | 33 | Let's first look at an example where this protocol seems to work perfectly well. First, Node #1 is going to broadcast a Event #1 to the other nodes on the network. 34 | 35 |
36 | 37 |
38 | 39 | Great! Event #1 was received successfully by Nodes #2 and #3. All of our nodes have the same canonical list of events and our consensus protocol appears to be working correctly. Now Node #2 is going to broadcast Event #2. 40 | 41 |
42 | 43 |
44 | 45 | Looks like we're still good here. Nodes #1 and #3 both received Event #2, and all of the nodes on the network still share the same canonical list of events. 46 | 47 | ### A broken example 48 | 49 | It looks like our basic consensus mechanism is working just fine. So why isn't this good enough? Let's take a look at a slightly trickier example that breaks our consensus mechanism. Previously we looked at examples where only one node was sending a message at a time. Unfortunately, things get messy when multiple nodes are sending messages at once. Let's suppose that Node #1 is sending Event #1 at the same time that Node #2 is sending Event #2. 50 | 51 |
52 | 53 |
54 | 55 | Now things are really confusing. Node #1 decided to store Event #1 first and then broadcast it to the rest of the network. Node #1 then received Event #2 and stored after Event #2. However, from the perspective of Node #2 this whole thing happened in reverse. Node #3 is also in trouble since the order in which these events are received depends entirely on the random communication delay between the two nodes. We've fallen out of consensus! 😭 56 | 57 | ## A robust protocol 58 | 59 | Looks like our simple protocol is broken. A little bit of random chance (two nodes happening to send messages at the same time) and the reality of our basic assumptions (network delay) has put a hole in our protocol. We'll need to come up with something a bit better if we want to maintain consensus in all network conditions allowed by our initial assumptions. 60 | 61 | ### Selecting a leader 62 | 63 | Although you could probably come up with other ways to solve the problem we just discovered, we'll be using one of the simplest approaches. We're going to select one of our nodes to act as the "leader" of our consensus protocol. Our leader is going to be responsible for progressing the state of the system. 64 | 65 | We're going to tweak our original protocol as follows: 66 | 67 | 1. Every node can send event to the leader whenever they want. 68 | 2. The leader will store events in its canonical list in the order they were received. 69 | 3. The leader will re-broadcast these events in the order they were received. 70 | 4. The leader will wait longer than the network delay between sending out events. 71 | 5. Nodes only store events in their canonical list when received from the leader. 72 | 73 |
74 | 75 |
76 | 77 | ### Another simple example 78 | 79 | Let's look at how this protocol works in practice. Our naive protocol worked perfectly well when only one node was sending messages at a time, so let's make sure our new protocol can handle the same situation. Here we'll have Node #3 first send Event #1 over to the leader. 80 | 81 |
82 | 83 |
84 | 85 | The leader has Event #1 stored in its local list, but none of the other nodes have stored anything. Now, Node #2 is going to send Event #2 to the leader. 86 | 87 |
88 | 89 |
90 | 91 | Just like before, the leader stored Event #2 in its own canonical list but none of the other nodes have done anything. Now here's where the magic happens. The leader is going to tell both of the nodes to commit some events to their canonical list in the order they were received. First, they'll be told to commit Event #1. 92 | 93 |
94 | 95 |
96 | 97 | So far, so good. Both nodes have Event #1 stored locally. Next they'll be told to commit Event #2. 98 | 99 |
100 | 101 |
102 | 103 | Perfect! All three of our nodes have the same canonical list of events. Of course, this isn't too impressive since our naive protocol could do this too. But it's nice to know that this mechanism actually works. 104 | 105 | ### Staying undefeated 106 | 107 | Now it's time to put our protocol to the same test that our naive mechanism failed. We're going to have several nodes sending messages at the same time. Here, Node #3 is going to sent Event #1 to the leader at the same time that Node #2 sends Event #2 to the leader. 108 | 109 |
110 | 111 |
112 | 113 | At this point, it's unclear whether Event #1 or Event #2 got to the leader first. However, we'll show in a second that the order of the incoming events doesn't have an impact on the correctness of the protocol. So, for the moment, we'll simply assume that Event #1 came first and Event #2 came second. 114 | 115 |
116 | 117 |
118 | 119 | Now our leader will continue to follow the protocol. Based on the order in which these events came in, the leader will ask each other node to commit these events to their canonical list of events. In our current example, this means that the leader will ask all other nodes to commit Event #1 to their canonical list. 120 | 121 |
122 | 123 |
124 | 125 | Great! Everything worked as expected. We'll repeat the same process for Event #2... 126 | 127 |
128 | 129 |
130 | 131 | And, of course, everything worked out just fine. All of our nodes share the same canonical list, which means that our consensus mechanism worked. There might be a problem here if the leader quickly asked nodes to commit both Event #1 and Event #2 (because of our random network latency). However, our protocol tells the leader to hold off from sending a message until we're sure that the previous message would've been received by all other nodes on the network. 132 | 133 | It's quite easy to imagine that this same protocol would work perfectly well if the leader had received Event #2 before Event #1. The only difference in the final picture would be that all nodes would share the list `[Event #2, Event #1]` instead of `[Event #1, Event #2]`. We still have consensus, even though the end result is different. 134 | 135 | ### Making it official 136 | 137 | The robust mechanism we just constructed should work perfectly under the conditions we laid out. No matter how many nodes exist on the network, and no matter how many events these nodes are sending at the same time, the existence of a single leader will always guarantee that the network comes to consensus. In an effort to be more exact, let's briefly sketch out a proof that this does actually work. 138 | 139 | Let's restate the basic protocol. The leader will receive events randomly from the network. The leader adds these events to its personal canonical list as they come in, so the leader is definitely in consensus with itself. The leader will, one by one, tell the network to commit each event in its canonical list (in order). When other nodes receive an event from the leader, they will commit that event to their canonical list. 140 | 141 | Since there's a maximum network delay (a maximum amount of time that it takes a message to propagate across the network), the leader can be sure that the leader's events are received by all other nodes within this `network_delay`. Additionally, since all nodes are perfect and do not suffer from faults, the leader knows that after `network_delay` all nodes have correctly committed the event. The leader must simply wait for longer than `network_delay` to send the next event to the network to guarantee that events will always come in one particular order. Super simple! 142 | 143 | ## Recapping our work 144 | 145 | So far we've created a simple consensus protocol that functions under very ideal network conditions. Of course, this protocol really does only work under the very tight conditions that we've defined in this section. If we want to be able to handle more realistic scenarios, we'll need to introduce some more features to our consensus protocol. In the next section, we'll introduce the idea of **crash faults** where nodes cease to function entirely and start to cause some chaos on the network. 146 | --------------------------------------------------------------------------------