├── .github └── workflows │ ├── deploy.yml │ └── test.yml ├── .gitignore ├── README.md ├── book.toml ├── fix-citation-author.sh ├── fix-latex.sh ├── install-deps.sh ├── mermaid-init.js ├── mermaid.min.js └── src ├── SUMMARY.md ├── architecture.md ├── architecture ├── bytecode-circuit.md ├── ecdsa-circuit.md ├── evm-circuit.md ├── evm-circuit │ ├── multi-step.md │ ├── multi-step_diagram.png │ └── opcode-fetching.md ├── evm-circuit_internal-call.png ├── evm-circuit_step-transition.png ├── keccak-circuit.md ├── mpt-circuit.md ├── state-circuit.md └── tx-circuit.md ├── architecture_diagram.png ├── architecture_diagram2.png ├── design.md ├── design ├── random-linear-combinaion.md ├── random-linear-combinaion │ └── full-runnable-code.md ├── recursion.md ├── recursion_aggregation-serial.png ├── reversible-write-reversion.md ├── reversible-write-reversion2.md ├── state-write-reversion2_call-depth.png ├── state-write-reversion_reversion-nested.png └── state-write-reversion_reversion-simple.png └── introduction.md /.github/workflows/deploy.yml: -------------------------------------------------------------------------------- 1 | name: Deploy 2 | on: 3 | push: 4 | branches: 5 | - main 6 | env: 7 | MDBOOK_URL: "https://github.com/rust-lang/mdBook/releases/download/v0.4.15/mdbook-v0.4.15-x86_64-unknown-linux-gnu.tar.gz" 8 | MDBOOK_TOC_URL: "https://github.com/badboy/mdbook-toc/releases/download/0.8.0/mdbook-toc-0.8.0-x86_64-unknown-linux-gnu.tar.gz" 9 | # MDBOOK_KATEX_URL: "https://github.com/lzanini/mdbook-katex/releases/download/v0.2.10/mdbook-katex-v0.2.10-x86_64-unknown-linux-gnu.tar.gz" 10 | # MDBOOK_KATEX_URL: "https://github.com/drmingdrmer/mdbook-katex/releases/download/v0.2.17/mdbook-katex-v0.2.17-linux.zip" 11 | MDBOOK_MERMAID_URL: "https://github.com/badboy/mdbook-mermaid/releases/download/v0.10.0/mdbook-mermaid-v0.10.0-x86_64-unknown-linux-gnu.tar.gz" 12 | 13 | jobs: 14 | deploy: 15 | runs-on: ubuntu-latest 16 | steps: 17 | - uses: actions/checkout@v2 18 | with: 19 | fetch-depth: 0 20 | # - name: Install rust toolchain 21 | # uses: actions-rs/toolchain@v1 22 | # with: 23 | # toolchain: stable 24 | # - name: Cargo cache 25 | # uses: actions/cache@v2 26 | # with: 27 | # path: | 28 | # ~/.cargo/ 29 | # key: cargo-cache 30 | # - name: Install mdbook and plugins 31 | # run: | 32 | # ./install-deps.sh 33 | # echo $HOME/.cargo/bin >> $GITHUB_PATH 34 | - name: Install mdbook and plugins 35 | run: | 36 | mkdir mdbook 37 | curl -sSL ${MDBOOK_URL} | tar -xz --directory=./mdbook 38 | # curl -sSL ${MDBOOK_KATEX_URL} | tar -xz --directory=./mdbook 39 | # mv ./mdbook/target/x86_64-unknown-linux-gnu/release/mdbook-katex ./mdbook 40 | # rmdir ./mdbook/target/x86_64-unknown-linux-gnu/release/ 41 | # curl -sSL ${MDBOOK_KATEX_URL} -o mdbook-katex.zip 42 | # unzip mdbook-katex.zip -d ./mdbook 43 | curl -sSL ${MDBOOK_TOC_URL} | tar -xz --directory=./mdbook 44 | curl -sSL ${MDBOOK_MERMAID_URL} | tar -xz --directory=./mdbook 45 | echo `pwd`/mdbook >> $GITHUB_PATH 46 | - name: Deploy GitHub Pages 47 | run: | 48 | mdbook build 49 | git worktree add gh-pages gh-pages 50 | git config user.name "Deploy from CI" 51 | git config user.email "" 52 | cd gh-pages 53 | # Delete the ref to avoid keeping history. 54 | git update-ref -d refs/heads/gh-pages 55 | rm -rf * 56 | rm -rf .github 57 | rm -rf .gitignore 58 | mv ../book/* . 59 | git add . 60 | git commit -m "Deploy $GITHUB_SHA to gh-pages" 61 | git push --force 62 | -------------------------------------------------------------------------------- /.github/workflows/test.yml: -------------------------------------------------------------------------------- 1 | name: Test 2 | on: 3 | pull_request: 4 | branches: [ main ] 5 | 6 | env: 7 | MDBOOK_URL: "https://github.com/rust-lang/mdBook/releases/download/v0.4.15/mdbook-v0.4.15-x86_64-unknown-linux-gnu.tar.gz" 8 | MDBOOK_TOC_URL: "https://github.com/badboy/mdbook-toc/releases/download/0.8.0/mdbook-toc-0.8.0-x86_64-unknown-linux-gnu.tar.gz" 9 | MDBOOK_MERMAID_URL: "https://github.com/badboy/mdbook-mermaid/releases/download/v0.10.0/mdbook-mermaid-v0.10.0-x86_64-unknown-linux-gnu.tar.gz" 10 | 11 | jobs: 12 | build: 13 | runs-on: ubuntu-latest 14 | steps: 15 | - uses: actions/checkout@v2 16 | with: 17 | fetch-depth: 0 18 | - name: Install mdbook and plugins 19 | run: | 20 | mkdir mdbook 21 | curl -sSL ${MDBOOK_URL} | tar -xz --directory=./mdbook 22 | curl -sSL ${MDBOOK_TOC_URL} | tar -xz --directory=./mdbook 23 | curl -sSL ${MDBOOK_MERMAID_URL} | tar -xz --directory=./mdbook 24 | echo `pwd`/mdbook >> $GITHUB_PATH 25 | - name: Build mdbook 26 | run: | 27 | mdbook build 28 | 29 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | book 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Archiving notice 2 | 3 | This repository is no longer maintained. For details, please refer to the README on [zkevm-circuits](https://github.com/privacy-scaling-explorations/zkevm-circuits/blob/main/README.md) 4 | 5 | --- 6 | 7 | # [Deprecated] zkEVM (Community Edition) Documentation 8 | 9 | This is the documentation of the design and specification of the zkEVM 10 | community edition. 11 | 12 | This documentation is written in markdown and organized into an 13 | [mdbook](https://github.com/rust-lang/mdBook) which can be [viewed 14 | here](https://privacy-scaling-explorations.github.io/zkevm-docs/). 15 | 16 | # Setup 17 | 18 | First install mdbook and the enabled extensions: 19 | ```sh 20 | cargo install mdbook 21 | cargo install mdbook-mermaid 22 | cargo install mdbook-toc 23 | ``` 24 | 25 | Now the mdbook can be built and served locally at [localhost:3000](http://localhost:3000): 26 | ```sh 27 | mdbook serve 28 | ``` 29 | -------------------------------------------------------------------------------- /book.toml: -------------------------------------------------------------------------------- 1 | [book] 2 | authors = ["The zkEVM community"] 3 | language = "en" 4 | multilingual = false 5 | src = "src" 6 | title = "zkEVM (Community Edition) Documentation" 7 | 8 | [preprocessor] 9 | 10 | # [preprocessor.katex] 11 | # throw-on-error = true 12 | # trust = false 13 | 14 | [preprocessor.toc] 15 | command = "mdbook-toc" 16 | renderer = ["html"] 17 | max-level = 4 18 | 19 | [preprocessor.mermaid] 20 | command = "mdbook-mermaid" 21 | 22 | [output] 23 | 24 | # [output.katex] 25 | 26 | [output.html] 27 | mathjax-support = true 28 | additional-js = ["mermaid.min.js", "mermaid-init.js"] 29 | -------------------------------------------------------------------------------- /fix-citation-author.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Helper script to fix citation from HackMD style to regular markdown style 4 | 5 | sed -r -i'' 's/> \[name=([^]]*)\]/>\n> \*\*\1\*\*/g' $(find src -name "*.md") 6 | -------------------------------------------------------------------------------- /fix-latex.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Helper script to fix how MathJax LaTeX works in mdbook VS how it works in markdown. 4 | 5 | sed -r -i'' 's/ \\\\\\hline/ \\\\\\\\\\hline/g' $(find src -name "*.md") 6 | sed -r -i'' 's/\$([^ ]+)\$/\\\\(\1\\\\)/g' $(find src -name "*.md") 7 | -------------------------------------------------------------------------------- /install-deps.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -ex 3 | 4 | cargo install mdbook 5 | cargo install mdbook-mermaid 6 | cargo install mdbook-toc 7 | # cargo install --git "https://github.com/lzanini/mdbook-katex" 8 | -------------------------------------------------------------------------------- /mermaid-init.js: -------------------------------------------------------------------------------- 1 | mermaid.initialize({startOnLoad:true}); 2 | -------------------------------------------------------------------------------- /src/SUMMARY.md: -------------------------------------------------------------------------------- 1 | # Summary 2 | 3 | - [Introduction](./introduction.md) 4 | - [Circuit Architecture](./architecture.md) 5 | - [EVM Circuit](./architecture/evm-circuit.md) 6 | - [Opcode Fetching](./architecture/evm-circuit/opcode-fetching.md) 7 | - [Multi-Step Implementation](./architecture/evm-circuit/multi-step.md) 8 | - [Tx Circuit](./architecture/tx-circuit.md) 9 | - [State Circuit](./architecture/state-circuit.md) 10 | - [Bytecode Circuit](./architecture/bytecode-circuit.md) 11 | - [ECDSA Circuit](./architecture/ecdsa-circuit.md) 12 | - [Keccak Circuit](./architecture/keccak-circuit.md) 13 | - [Merkle Patricia Tree Circuit](./architecture/mpt-circuit.md) 14 | - [Design Notes](./design.md) 15 | - [Random Linear Combination](./design/random-linear-combinaion.md) 16 | - [Full Runnable Code](./design/random-linear-combinaion/full-runnable-code.md) 17 | - [Recursion](./design/recursion.md) 18 | - [Reversible Write Reversion Note 1](./design/reversible-write-reversion.md) 19 | - [Reversible Write Reversion Note 2](./design/reversible-write-reversion2.md) 20 | -------------------------------------------------------------------------------- /src/architecture.md: -------------------------------------------------------------------------------- 1 | # Architecture 2 | 3 | 4 | 5 | # Concepts 6 | 7 | ## Architecture diagram 8 | 9 | Each circuit is layouted to be capable to build their own custom constraints. When circuits encounter some expensive operations, they can outsource the effort to other circuits through the usage of lookup arguments. 10 | The relationship between circuits looks like: 11 | 12 | ![](./architecture_diagram2.png) 13 | 14 | List of circuits and tables they generate/verify: 15 | 16 | | Circuit | Table | 17 | | --- | --- | 18 | | [EVM Circuit](./architecture/evm-circuit.md) | | 19 | | [Bytecode Circuit](./architecture/bytecode-circuit.md) | [Bytecode Table](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/tables.md#bytecode_table) | 20 | | [State Circuit](./architecture/state-circuit.md) | [Rw Table](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/tables.md#rw_table) | 21 | | Block Circuit | [Block Table](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/tables.md#block_table) | 22 | | [Tx Circuit](./architecture/tx-circuit.md) | [Tx Table](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/tables.md#tx_table) | 23 | | [MPT Circuit](./architecture/mpt-circuit.md) | MPT Table | 24 | | [Keccak Circuit](./architecture/keccak-circuit.md) | Keccak Table | 25 | | [ECDSA Circuit](./architecture/ecdsa-circuit.md) | ECDSA Table | 26 | 27 | In the end, the circuits would be assembled depending on their dimension and the desired capacity. For example, we can just combine 2 different circuits by using different columns, or stack them using same columns with extra selectors. 28 | 29 | In order to reduce the time required to build a proof of a full block and to 30 | simplify the verification step, an aggregation circuit is being build so that condenses the 31 | verification of each sub-circuit proofs shown in the diagram. See [Design 32 | Notes, Recursion](./design/recursion.md) for details on the recursion strategy 33 | used in the aggregation circuit. 34 | 35 | ## Circuit as a lookup table 36 | 37 | In halo2, the lookup is flexible to be configured. Anything able to be turned into `Expression` could be used as `item: Tuple[int, ...]` or `table: Set[Tuple[int, ...]]` in lookup. Enabling `assert item in table`. The `Expression` includes `Constant`, `Fixed`, `Advice` or `Instance` column at arbitrary rotation. 38 | 39 | The motivation to have multiple circuits as lookup tables is that EVM contains many circuit unfriendly operations like random read-write data access, "wrong" field operation (ECDSA on secp256k1), traditional hash functions like `keccak256`, etc... And many of them accept variable lenght input. 40 | 41 | These expensive operations make it hard to design an EVM circuit to verify computation traces because each step could possibly contain some of the operations mentioned above. So we tried to separate these expensive operations into single-purpose circuits which have a more friendly layout, and use them via lookups to communicate it's input and output, Outsourcing the effort. 42 | 43 | The reason input-output lookups could be used to outsource the effort is that we know the that the lookup-ed table is configured with constraints to verify the input and output are in some relationship. For example, we let Bytecode circuit to hold a set of tuple `(code_hash, index, opcode)`, and each `code_hash` is verified to be the keccak256 digest of opcodes it contains, then in EVM circuit we can load `opcode` with `(code_hash, program_counter)` by looking up the Bytecode table. 44 | 45 | However, there are some properties we can't ensure only with lookups (which ultimately only prove that the contents of all the lookups are a subset of a table). We want to constraint that the amount of all (looked-up) `item`s should be equal to the size of `table`, which is required by the EVM circuit and State circuit to prevent extra malicious writes in the `table`. In such case (the set of looked up items define the table exactly), we need some extra constraint to ensure the relationship is correct. A naive approach is to count all `item` in State circuit (which in the end is the size of the `table`) and constraint it to be equal to the value counted in the EVM circuit. 46 | 47 | 48 | ## EVM word encoding 49 | 50 | See [Design Notes, Random Linear Combination](./design/random-linear-combinaion.md) 51 | 52 | - [Word encoding spec](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/word-encoding.md) 53 | 54 | # Custom types 55 | 56 | # Constants 57 | 58 | | Name | Value | Description | 59 | | -------------------- | ------------ | ------------------------------- | 60 | | `MAX_MEMORY_ADDRESS` | `2**40 - 1` | max memory address allowed [^1] | 61 | | `MAX_GAS` | `2**64 - 1` | max gas allowed | 62 | | `MAX_ETHER` | `2**256 - 1` | max value of ether allowed [^2] | 63 | 64 | 65 | [^1]: The explicit max memory address in EVM is actually `32 * (2**32 - 1)`, which is the one that doesn't make memory expansion gas cost overflow `u64`. In our case, memory address is allowed to be 5 bytes, but will constrain the memory expansion gas cost to fit `u64` in success case. 66 | 67 | [^2]: I didn't find a explicit upper bound on value of ether (for `balance` or `gas_price`) in yellow paper, but handling unbounded big integer seems unrealistic in circuit, so with `u256` as a hard bound seems reasonable. 68 | -------------------------------------------------------------------------------- /src/architecture/bytecode-circuit.md: -------------------------------------------------------------------------------- 1 | # Bytecode Circuit 2 | 3 | Bytecode circuit iterates over contract bytecodes to verify each bytecode has valid hash. 4 | 5 | It serves as a lookup table for EVM circuit to do random access of any index of bytecode. 6 | 7 | # Implementation 8 | 9 | - [spec](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/bytecode-proof.md) 10 | - [python](https://github.com/appliedzkp/zkevm-specs/blob/master/src/zkevm_specs/bytecode.py) 11 | - [circuit](https://github.com/appliedzkp/zkevm-circuits/tree/main/zkevm-circuits/src/bytecode_circuit) 12 | -------------------------------------------------------------------------------- /src/architecture/ecdsa-circuit.md: -------------------------------------------------------------------------------- 1 | # ECDSA Cicruit 2 | 3 | ECDSA circuit iterates over signatures to verify recovered public key from signature is valid, or verify signature is invalid. 4 | 5 | It serves as a lookup table for EVM and Tx circuit to do public key recover. 6 | 7 | -------------------------------------------------------------------------------- /src/architecture/evm-circuit.md: -------------------------------------------------------------------------------- 1 | # EVM Circuit 2 | 3 | 4 | 5 | # Introduction 6 | 7 | EVM circuit iterates over transactions included in the proof to verify that each execution step of a transaction is valid. Basically the scale of a step is the same as in the EVM, so usually we handle one opcode per step, except those opcodes like `SHA3` or `CALLDATACOPY` that operate on variable size of memory, which would require multiple "virtual" steps. 8 | 9 | > The scale of a step somehow could be different depends on the approach, an extreme case is to implement a VM with reduced instruction set (like TinyRAM) to emulate EVM, which would have a much smaller step, but not sure how it compares to current approach. 10 | > 11 | > **han** 12 | 13 | To verify if a step is valid, we first enumerate all possible execution results of a step in the EVM including success and error cases, and then build a custom constraint to verify that the step transition is correct for each execution result. 14 | 15 | For each step, we constrain it to enable one of the execution results, and specially, to constrain the first step to enable `BEGIN_TX`, which then repeats the step to verify the full execution trace. Also each step is given access to next step to propagate the tracking information, by putting constraints like `assert next.program_counter == curr.program_counter + 1`. 16 | 17 | # Concepts 18 | 19 | ## Execution result 20 | 21 | It's intuitive to have each opcode as a branch in step. However, EVM has so rich opcodes that some of them are very similar like `{ADD,SUB}`, `{PUSH*}`, `{DUP*}` and `{SWAP*}` that seem to be handled by almost identical constraint with small tweak (to swap a value or automatically done due to linearity), it seems we could reduce our effort to only implement it once to handle multiple opcodes in single branch. 22 | 23 | In addition, an EVM state transition could also contain serveral kinds of error cases, we also need to take them into consideration to be equivalent to EVM. It would be annoying for each opcode branch to handle their own error cases since it needs to halt the step and return the execution context to caller. 24 | 25 | Fortunately, most error cases are easy to verify with some pre-built lookup table even they could happen to many opcodes, only some tough errors like out of gas due to dynamic gas usage need to be verified one by one. So we further unroll all kinds of error cases as kinds of execution result. 26 | 27 | So we can enumerate [all possible execution results](https://github.com/appliedzkp/zkevm-specs/blob/83ad4ed571e3ada7c18a411075574110dfc5ae5a/src/zkevm_specs/evm/execution_result/execution_result.py#L4) and turn EVM circuit into a finite state machine like: 28 | 29 | ```mermaid 30 | flowchart LR 31 | BEGIN[.] --> BeginTx; 32 | 33 | BeginTx --> |no code| EndTx; 34 | BeginTx --> |has code| EVMExecStates; 35 | EVMExecStates --> EVMExecStates; 36 | EVMExecStates --> EndTx; 37 | 38 | EndTx --> BeginTx; 39 | 40 | EndTx --> EndBlock; 41 | EndBlock --> EndBlock; 42 | EndBlock --> END[.]; 43 | ``` 44 | ```mermaid 45 | flowchart LR 46 | subgraph A [EVMExecStates] 47 | BEGIN2[.] --> SuccessStep; 48 | BEGIN2[.] --> ReturnStep; 49 | SuccessStep --> SuccessStep; 50 | SuccessStep --> ReturnStep; 51 | ReturnStep --> |not is_root| SuccessStep; 52 | ReturnStep --> |not is_root| ReturnStep; 53 | ReturnStep --> |is_root| END2[.]; 54 | end 55 | ``` 56 | 57 | - **BeginTx**: 58 | - Beginning of a transaction. 59 | - **EVMExecStates** = [ SuccessStep | ReturnStep ] 60 | - **SuccessStep** = [ ExecStep | ExecMetaStep | ExecSubStep ] 61 | - Set of states that suceed and continue the execution within the call. 62 | - **ReturnStep** = [ ExplicitReturn | Error ] 63 | - Set of states that halt the execution of a call and return to the caller 64 | or go to the next tx. 65 | - **ExecStep**: 66 | - 1-1 mapping with a GethExecStep for opcodes that map to a single gadget 67 | with a single step. Example: `ADD`, `MUL`, `DIV`, `CREATE2`. 68 | - **ExecMetaStep**: 69 | - N-1 mapping with a GethExecStep for opcodes that share the same gadget 70 | (due to similarity) with a single step. For example `{ADD, SUB}`, 71 | `{PUSH*}`, `{DUP*}` and `{SWAP*}`. 72 | A good example on how these are grouped is the `StackOnlyOpcode` struct. 73 | - **ExecSubStep**: 74 | - 1-N mapping with a GethExecStep for opcodes that deal with dynamic size 75 | arrays for which multiple steps are generated. 76 | - `CALLDATACOPY` -> CopyToMemory 77 | - `RETURNDATACOPY` -> TODO 78 | - `CODECOPY` -> TODO 79 | - `EXTCODECOPY` -> IN PROGRESS 80 | - `SHA3` -> IN PROGRESS 81 | - `LOGN` -> CopyToLog 82 | - **ExplicitReturn**: 83 | - 1-1 mapping with a GethExecStep for opcodes that return from a call 84 | without exception. 85 | - **Error** = [ ErrorEnoughGas | ErrorOutOfGas ] 86 | - Set of states that are associated with exceptions caused by opcodes. 87 | - **ErrorEnoughGas**: 88 | - Set of error states that are unrelated to out of gas. Example: 89 | `InvalidOpcode`, `StackOverflow`, `InvalidJump`. 90 | - **ErrorOutOfGas**: 91 | - Set of error states for opcodes that run out of gas. For each opcode 92 | (sometimes group of opcodes) that has dynamic memory gas usage, there is 93 | a specific **ErrorOutOfGas** error state. 94 | - **EndTx** 95 | - End of a transaction. 96 | - **EndBlock** 97 | - End of a block (serves also as padding for the rest of the state step slots) 98 | 99 | 100 | > In the current implementation, we ask the opcode implementer to also implement error cases, which seems to be a redundant effort. 101 | > But by doing this, they can focus more on opcode's success case. Also error cases are usually easier to verify, so I think it also reduces the overall implementation complexity. 102 | > 103 | > **han** 104 | 105 | ## Random access data 106 | 107 | In EVM, the interpreter has the ability to do any random access to data like block context, account balance, stack and memory in current scope, etc... Some of these access are read-write and others are read-only. 108 | 109 | In EVM circuit, we leverage the concept [Circuit as a lookup table](#Circuit-as-a-lookup-table) to duplicate these random data access to other circuits in a different layout and verify that they are consistent and valid. After these random data access are verified, we can use them just as if they were only tables. [Here](https://github.com/appliedzkp/zkevm-specs/blob/83ad4ed571/src/zkevm_specs/evm/table.py#L108) are the tables currently used in the EVM circuit. 110 | 111 | For read-write access data, EVM circuit looks up State circuit with a sequentially `rw_counter` (read-write counter) to make sure the read-write access is chronological. It also uses a flag `is_write` to check data consistency between different write access. 112 | 113 | For read-only access data, EVM circuit looks-up Bytecode circuit, Tx circuit and Call circuit directly. 114 | 115 | ## Reversible write reversion 116 | 117 | In EVM, reversible writes can be reverted if any call fails. There are many kinds of reversible writes, a complete list can be found [here](https://github.com/ethereum/go-ethereum/blob/master/core/state/journal.go#L87-L141). 118 | 119 | In EVM circuit, each call is attached with a flag (`is_persistent`) to know if it succeeds or not. So ideally, we only need to do reversion on these kinds of reversible writes which affect future execution before reversion: 120 | 121 | - `TxAccessListAccount` 122 | - `TxAccessListStorageSlot` 123 | - `AccountNonce` 124 | - `AccountBalance` 125 | - `AccountCodeHash` 126 | - `AccountStorage` 127 | 128 | On some others we don't need to do reversion because they don't affect future execution before reversion, we only write them when `is_persistent` is `1`: 129 | 130 | - `TxRefund` 131 | - `AccountDestructed` 132 | 133 | > Another tag is `TxLog`, which also doesn't affect future execution. It should be explained where to write such record to after we decide where to build receipt trie. 134 | > 135 | > **han** 136 | 137 | To enable reversible write reversion, we need some meta information of a call: 138 | 139 | 1. `is_persistent` - To know if we need reversion or not. 140 | 2. `rw_counter_end_of_reversion` - To know at which point in the future we should revert. 141 | 3. `reversible_write_counter` - To know how many reversible writes we have done until now. 142 | 143 | Then at each reversible write, we first check if `is_persistent` is `0`, if so we do an extra reversible write at `rw_counter_end_of_reversion - reversible_write_counter` with the old value, which reverts the reversible write in a reverse order. 144 | 145 | For more notes on reversible write reversion see: 146 | - [Design Notes, Reversible Write Reversion Note 1](../design/reversible-write-reversion.md) 147 | - [Design Notes, Reversible Write Reversion Note 2](../design/reversible-write-reversion2.md) 148 | 149 | ## Opcode fetching 150 | 151 | In EVM circuit, there are 3 kinds of opcode source for execution or copy: 152 | 153 | 1. Contract interaction: 154 | Opcode is lookup from contract bytecode in Bytecode circuit by tuple `(code_hash, index, opcode)` 155 | 2. Contract creation in root call: 156 | Opcode is lookup from tx calldata in Tx circuit by tuple `(tx_id, TxTableTag.Calldata, index, opcode)` 157 | 3. Contract creation in internal call: 158 | Opcode is lookup from caller's memory in State circuit by tuple `(rw_counter, False, caller_id, index, opcode)` 159 | 160 | Before we fetch opcode from any source, it checks if the index is in the given range, if not, it follows the behavior of current EVM to implicitly returning `0`. 161 | 162 | ## Internal call 163 | 164 | EVM supports internal call triggered by opcodes. In EVM circuit, the opcodes (like `CALL` or `CREATE`) that trigger internal call, will: 165 | - Save their own `call_state` into State circuit. 166 | - Setup next call's context. 167 | - Initialize next step's `call_state` to start a new environment. 168 | 169 | Then the opcodes (like `RETURN` or `REVERT`) and error cases that halt, will restore caller's `call_state` and set it back to next step. 170 | 171 | For a simple `CALL` example with illustration (many details are hided for simplicity): 172 | 173 | ![](./evm-circuit_internal-call.png) 174 | 175 | # Constraints 176 | 177 | ## `main` 178 | 179 | ==TODO== Explain each execution result 180 | 181 | # Implementation 182 | 183 | - [spec](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/evm-proof.md) 184 | - [python](https://github.com/appliedzkp/zkevm-specs/tree/master/src/zkevm_specs/evm) 185 | - [circuit](https://github.com/appliedzkp/zkevm-circuits/tree/main/zkevm-circuits/src/evm_circuit) 186 | -------------------------------------------------------------------------------- /src/architecture/evm-circuit/multi-step.md: -------------------------------------------------------------------------------- 1 | 8 | 9 | # Multi-Step Implementation 10 | 11 | 12 | 13 | # Introduction 14 | 15 | In EVM, there are serveral opcodes moving dynamic-length bytes around between different sources, here is a complete list: 16 | 17 |
18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 |
TypeOpcodeSourceDestination
1CODECOPY
EXTCODECOPY		bytecode[code_hash]calls[call_id].memory
CALLDATACOPY and is_roottxs[tx_id].calldatacalls[call_id].memory
CALLDATACOPY and not_rootcalls[caller_call_id].memorycalls[call_id].memory
2RETURNDATACOPYcalls[callee_call_id].memorycalls[call_id].memory
3RETURN and is_create
CREATE
CREATE2		calls[call_id].memorybytecode[code_hash]
SHA3calls[call_id].memoryTBD
4RETURN and not_createcalls[call_id].memorycalls[caller_call_id].memory
REVERT and not_createcalls[call_id].memorycalls[caller_call_id].memory
74 |
75 | 76 | With illustration: 77 | 78 | ![](./multi-step_diagram.png) 79 | 80 | There could be classified to be 4 types: 81 | 82 | 1. `* -> memory (padding)` 83 | - Including: 84 | - `CALLDATACOPY` 85 | - `CODECOPY` 86 | - `EXTCODECOPY` 87 | - Copy from calldata or code to current memory. 88 | - Memory gets filled with 0's when copied out of source's range, in other words, source is padded with 0's to the range. 89 | 2. `* -> memory (no padding)` 90 | - Including `RETURNDATACOPY`. 91 | - Similar to Type 1, but the range is explicitly checked to be in source's range, otherwise the EVM halts with exception. So no padding issue. 92 | 3. `memory -> * (no range capped)` 93 | - Including: 94 | - `RETURN` when `is_create` 95 | - `CREATE` 96 | - `CREATE2` 97 | - `SHA3` 98 | - Copy from current memory to destination. 99 | - No padding issue since memory is always expanded implicitly due to lazy initialization. 100 | 4. `memory -> * (range capped)` 101 | - Including: 102 | - `RETURN` when `not_create` 103 | - `REVERT` when `not_create` 104 | - Similar to Type 3, but the range is capped by caller's assignment. 105 | 106 | ## Approaches 107 | 108 | ### Approach #1 - Given access to previous step 109 | 110 | Take `CALLDATALOAD` as an example, in the [approach](https://github.com/appliedzkp/zkevm-specs/blob/2864c3f0f6cb905b8548da9cde76fea13a42085f/src/zkevm_specs/evm/execution_result/calldatacopy.py) by @icemelon, it requires access to previous step to infer what's the state of current step, to know if the step is the first step, we check 111 | 112 | 1. `curr.opcode == CALLDATALOAD` 113 | 2. `prev.execution_state != CALLDATALOAD or prev.finished is True` 114 | 115 | And it transit the `StepState` at the last step, which is inferred from if the bytes left to copy is less then a step's amount. 116 | 117 | ### Approach #2 - Introduce internal `ExecutionState` 118 | 119 | This approach introduce internal `ExecutionState` with extra constraint of `ExecutionState` transition, and the inputs will be passed by constraint from previous step. The new `ExecutionState` are: 120 | 121 | - `CopyMemoryToMemory` 122 | - Can only transited from: 123 | - `RETURN` 124 | - `REVERT` 125 | - `CALLDATACOPY` 126 | - `RETURNDATACOPY` 127 | - Inputs: 128 | - `src_call_id` - id of source call (to be read) 129 | - `dst_call_id` - id of destination call (to be written) 130 | - `src_end` - end of source, it returns `0` when indexing out of this. 131 | - `src_offset` - memory offset of source call 132 | - `dst_offset` - memory offset of destination call 133 | - `bytes_left` - how many bytes left to copy 134 | - Note: 135 | - The `src_end` is only used by `CALLDATACOPY` since only it needs padding. 136 | - `CopyTxCalldataToMemory` 137 | - Can only transited from `CALLDATACOPY` 138 | - Inputs: 139 | - `tx_id` - id of current tx 140 | - `src_end` - end of source, it returns `0` when indexing out of this 141 | - `src_offset` - calldata offset of tx 142 | - `dst_offset` - memory offset of current call 143 | - `bytes_left` - how many bytes left to copy 144 | 145 | - `CopyBytecodeToMemory` 146 | - Can only transited from: 147 | - `CODECOPY` 148 | - `EXTCODECOPY` 149 | - Inputs: 150 | - `code_hash` - hash of bytecode 151 | - `src_end` - end of source, it returns `0` when indexing out of this 152 | - `src_offset` - calldata offset of tx 153 | - `dst_offset` - memory offset of current call 154 | - `bytes_left` - how many bytes left to copy 155 | 156 | - `CopyMemoryToBytecode` 157 | - Can only transited from: 158 | - `CREATE` - copy init code 159 | - `CREATE2` - copy init code 160 | - `RETURN` - copy deployment code 161 | - Inputs: 162 | - `code_hash` - hash of bytecode 163 | - `src_offset` - calldata offset of tx 164 | - `dst_offset` - memory offset of current call 165 | - `bytes_left` - how many bytes left to copy 166 | - Note 167 | - This differs from `CopyBytecodeToMemory` in that it doesn't have padding. 168 | 169 | > If we can have a better way to further generalize these inner `ExecutionState`, we can have less redundant implementation. 170 | > 171 | > **han** 172 | 173 | And they do the bytes copy with range check specified by trigger `ExecutionState`. 174 | 175 | Also these internal `ExecutionState`s always propagate `StepState`s as the same value, since the transition is already done by the trigger of `ExecutionState`. 176 | 177 | Take `CALL` then `CALLDATALOAD` as an example: 178 | 179 | - Caller executes `CALL` with stack values (naming referenced from [`instruction.go#L668`](https://github.com/ethereum/go-ethereum/blob/master/core/vm/instructions.go#L668)): 180 | - `inOffset = 32` 181 | - `inSize = 32` 182 | - Callee executes `CALLDATALOAD` with stack values (naming referenced from [`instruction.go#L301-L303`](https://github.com/ethereum/go-ethereum/blob/master/core/vm/instructions.go#L301-L303)): 183 | - `memOffset = 0` 184 | - `dataOffset = 64` 185 | - `length = 32` 186 | - The first step of `CopyMemoryToMemory` will receive inputs: 187 | - `src_call_id = caller_call_id` 188 | - `dst_call_id = callee_call_id` 189 | - `src_end = inOffset + inSize = 64` 190 | - `src_offset = inOffset + dataOffset = 96` 191 | - `dst_offset = memOffset = 0` 192 | - `bytes_left = length = 32` 193 | 194 | Then, in every step we check if `src_offset < src_end`, if not, we need to disable the source lookup and fill zeros into destination. Then add the `*_offset` by the amount of bytes we process at a step, and subtract `bytes_left` also by it, then propagate them to next step. 195 | 196 | ## Conclusion 197 | 198 | Comparison between the 2 approaches: 199 | 200 | - Approach #1 201 | - Pros 202 | - No additional `ExecutionState` 203 | - Cons 204 | - Each multi-step opcodes will have at least 3 extra nested branches: 205 | - `is_first` - If the step is the first 206 | - `not_first` - If the step is n-th step 207 | - `is_final` - If the step is final 208 | - Approach #2 209 | - Pros 210 | - Each multi-step opcodes only need to prepare the inputs of those inner `ExecutionState` and do the correct `StepState` transition. 211 | - Only 2 nested branches: 212 | - `not_final` - If the step is n-th step 213 | - `is_final` - If the step is final 214 | - Cons 215 | - Additional `ExecutionState` 216 | 217 | In the context of current implementation, approach #2 seems easier to implement due to the separation of complexity, and also less prover effort. 218 | 219 | In the context of re-designed EVM circuit (re-use instruction instead of building giant custom gates), it seems no difference on prover effort between the 2 approaches, but approach #2 seems better because it extracts the nested branch and should reduce the usage of rows. 220 | 221 | -------------------------------------------------------------------------------- /src/architecture/evm-circuit/multi-step_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/architecture/evm-circuit/multi-step_diagram.png -------------------------------------------------------------------------------- /src/architecture/evm-circuit/opcode-fetching.md: -------------------------------------------------------------------------------- 1 | # Opcode Fetching 2 | 3 | 4 | 5 | # Introduction 6 | 7 | For opcode fetching, we might have 3 sources in different situation: 8 | 9 | 1. When contract interaction, we lookup `bytecode_table` to read bytecode. 10 | 2. When contract creation in root call, we lookup `tx_table` to read transaction's calldata. 11 | 3. When contract creation in internal call, we lookup `rw_table` to read caller's memory. 12 | 13 | Also we need to handle 2 kinds of annoying EVM features: 14 | 15 | 1. Implicit `STOP` returning if fetching out of range. 16 | 2. For `JUMP*`, we need to verify: 17 | 1. destination is a `JUMPDEST` 18 | 2. destination is not a data section of `PUSH*` 19 | 20 | Since for each step `program_counter` only changes in 3 situation: 21 | 22 | ```python 23 | if opcode in [JUMP, JUMPI]: 24 | program_counter = dest 25 | elif opcode in range(PUSH1, PUSH1 + 32): 26 | program_counter += opcode - PUSH1 + 1 27 | else: 28 | program_counter += 1 29 | ``` 30 | 31 | For all opcodes except for `JUMP*` and `PUSH*`, we only need to worry about first issue, and we can solve it by checking if `bytecode_length <= program_counter` then detect such case. 32 | 33 | For `PUSH*` we can do the lookup only when `program_counter + x < bytecode_length` and simulate the "implicit `0`". (Other opcodes like `CALLDATALOAD`, `CALLDATACOPY`, `CODECOPY`, `EXTCODECOPY` also encounter such "implicit `0`" problem, and we need to handle them carefully). 34 | 35 | However, for `JUMP*` we need one more trick to handle, especially for the **issue 2.2.**, which seems not possible to check if we don't scan through all opcodes from the beginning to the end. 36 | 37 | Focus on solving the **issue 2.2.**, my thought went through 2 steps: 38 | 39 | ## Step #1 - `is_code` Annotation 40 | 41 | If the opcode is layouted to be adjacent like the `bytecode_table` or `tx_table`, we can annotate each row with `push_data_rindex` and `is_code`: 42 | 43 | > `push_data_rindex` means push data's reverse index, which starts from `1` instead of `0`. 44 | > 45 | > **han** 46 | 47 | $$ 48 | \begin{array}{|c|c|} 49 | \hline 50 | \texttt{{bytecode_hash,tx_id}} & \texttt{index} & \texttt{opcode} & \texttt{push_data_rindex} & \texttt{is_code} & \text{note} \\\\\hline 51 | \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 52 | \texttt{0xff} & \texttt{0} & \texttt{PUSH1} & \texttt{0} & \texttt{1} \\\\\hline 53 | \texttt{0xff} & \texttt{1} & \texttt{0xef} & \texttt{1} & \texttt{0} \\\\\hline 54 | \texttt{0xff} & \texttt{2} & \texttt{0xee} & \texttt{0} & \texttt{1} \\\\\hline 55 | \texttt{0xff} & \texttt{3} & \texttt{PUSH2} & \texttt{0} & \texttt{1} \\\\\hline 56 | \texttt{0xff} & \texttt{4} & \texttt{PUSH1} & \texttt{2} & \texttt{0} & \text{is not code} \\\\\hline 57 | \texttt{0xff} & \texttt{5} & \texttt{PUSH1} & \texttt{1} & \texttt{0} & \text{is not code} \\\\\hline 58 | \texttt{0xff} & \texttt{6} & \texttt{JUMPDEST} & \texttt{0} & \texttt{1} & \text{is code!} \\\\\hline 59 | \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 60 | \end{array} 61 | $$ 62 | 63 | The constraint would be like: 64 | 65 | ```python 66 | class Row: 67 | code_hash_or_tx_id: int 68 | index: int 69 | opcode: int 70 | push_data_rindex: int 71 | is_code: int 72 | 73 | def constraint(prev: Row, curr: Row, is_first_row: bool): 74 | same_source = curr.code_hash_or_tx_id == prev.code_hash_or_tx_id 75 | 76 | assert curr.is_code == is_zero(curr.push_data_rindex) 77 | 78 | if is_first_row or same_source: 79 | assert curr.push_data_rindex == 0 80 | else: 81 | if prev.is_code: 82 | if (prev.opcode - PUSH1) in range(32): 83 | assert curr.push_data_rindex == prev.opcode - PUSH1 + 1 84 | else: 85 | assert curr.push_data_rindex == 0 86 | else: 87 | assert curr.push_data_rindex == prev.push_data_rindex - 1 88 | ``` 89 | 90 | And when handling `JUMP*` we can check `is_code` for verification. 91 | 92 | However, the memory in the State circuit it's layouted to be `memory_address` and then `rw_counter`, which we can't select at some specific point to do such analysis. So this approach seems not work on all situations. 93 | 94 | ## Step #2 - Explicitly copy memory to bytecode_table 95 | 96 | It seems inevitable to copy the memory to `bytecode_table` since the `CREATE*` needs it to know the `bytecode_hash`. So maybe we can abuse such constraint to also copy the creation bytecode to the `bytecode_table`. Althought the hash of it means nothing, we still can use it as a unique identifier to index out the opcode. 97 | 98 | Then we can define an internal multi-step execution result `COPY_MEMORY_TO_BYTECODE` which can only transit from `CREATE*` or `RETURN`, and copy the memory from offset with length to the `bytecode_table`. 99 | 100 | Although it costs many steps to copy the creation code, it makes the opcode fetching source become simpler with only `bytecode_table` and `tx_table`. The issue of memory's unfriendly layout is also gone, **issue 2.2.** is then resolved. 101 | 102 | > Memory copy on creation code seems terrible since a prover can reuse the same large chunk of memory to call multiple times of `CREATE*`, and we always need to copy them, which might cost many steps. 103 | > We need some benchmark to see if a block contains full of such `CREATE*` to know how much gas we can verify in a block, then know if it's aligned to current gas cost model or not, and decide whether to further optimize it. 104 | > 105 | > **han** 106 | 107 | ## Random Thought 108 | 109 | ### Memory copy optimization 110 | 111 | When it comes to "memory copy", it means in EVM circuit we lookup both `rw_table` and `bytecode_table` to make sure the chunk of memory indeed exists in the latter table. However, EVM circuit doesn't have a friendly layout to do such operation (it costs many expressions to achieve so). 112 | 113 | If we want to further optimize "memory copy" in respect to the concern highlighted in [Step #2](#step-2---explicitly-copy-memory-to-bytecode_table), since we know the memory to be copied is in chunk, and in `bytecode_table` it also exists in chunk, then we seem to let Bytecode circuit to do such operation with correct `rw_counter`, and in EVM circuit we only need to "trigger" such operation. We can add extra selector columns to enable it like: 114 | 115 | $$ 116 | \begin{array}{|c|c|} 117 | \hline 118 | \texttt{call_id} & \texttt{memory_offset} & \texttt{rw_counter} & \texttt{bytecode_hash} & \texttt{bytecode_length} & \texttt{index} & \texttt{opcode} \\\\\hline 119 | \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 120 | \texttt{3} & \texttt{64} & \texttt{38} & \texttt{0xff} & \texttt{4} & \texttt{0} & \texttt{PUSH1} \\\\\hline 121 | \texttt{3} & \texttt{64} & \texttt{39} & \texttt{0xff} & \texttt{4} & \texttt{1} & \texttt{0x00} \\\\\hline 122 | \texttt{3} & \texttt{64} & \texttt{40} & \texttt{0xff} & \texttt{4} & \texttt{2} & \texttt{DUP1} \\\\\hline 123 | \texttt{3} & \texttt{64} & \texttt{41} & \texttt{0xff} & \texttt{4} & \texttt{3} & \texttt{RETURN} \\\\\hline 124 | \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 125 | \end{array} 126 | $$ 127 | 128 | > `bytecode_length` is required no matter we adopt this or not. It's ignored previously for simplicity 129 | > 130 | > **han** 131 | 132 | Then the constraint in Bytecode circuit might look like: 133 | 134 | ```python 135 | class Row: 136 | call_id: int 137 | memory_offset: int 138 | rw_counter: int 139 | 140 | bytecode_hash: int 141 | bytecode_length: int 142 | index: int 143 | opcode: int 144 | 145 | def copy_memory_constraint(prev: Row, curr: Row, is_first_row: bool): 146 | same_source = curr.bytecode_hash == prev.bytecode_hash 147 | 148 | if same_source: 149 | assert curr.call_id == prev.call_id 150 | assert curr.memory_offset == prev.memory_offset 151 | assert curr.rw_counter == prev.rw_counter + 1 152 | 153 | if curr.call_id is not 0: 154 | assert ( 155 | curr.rw_counter, # rw_counter 156 | False, # is_write 157 | Memory, # tag 158 | curr.call_id, # call_id 159 | curr.memory_offset + curr.index, # memory_address 160 | curr.opcode, # byte 161 | 0, 162 | 0, 163 | ) in rw_table 164 | ``` 165 | 166 | And in EVM circuit we only needs to make sure the first row of such series exist, then transit the `rw_counter` by `bytecode_length` to next step. 167 | 168 | ### Memory copy generalizaiton 169 | 170 | For opcodes like `PUSH*`, `CALLDATALOAD`, `CALLDATACOPY`, `CODECOPY`, `EXTCODECOPY` we need to copy bytecode to memory and it seems that we can reuse the `COPY_MEMORY_TO_BYTECODE`, with a small tweak to change the `is_write` to memory to `True`. 171 | 172 | ### Tx calldata copy 173 | 174 | Since we already copy memory, why not also copy the calldata part of `tx_table` to `bytecode_table`? We can use the same trick as in [Memory copy optimization](#Memory-copy-optimization) to make sure tx calldata is copied to `bytecode_table`. Then we only have a single source to do opcode fetching, which simplifies a lot of things. 175 | 176 | > The only concern is, will this cost much on `bytecode_table`'s capacity? We still need actual benchmark to see if it's adoptable. 177 | > 178 | > **han** 179 | 180 | > I think so it would be better to maintain only one byte_code_table for all related using if it is feasible, calldata copy of contract creation seems double the table size 181 | > 182 | > **dream** 183 | 184 | -------------------------------------------------------------------------------- /src/architecture/evm-circuit_internal-call.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/architecture/evm-circuit_internal-call.png -------------------------------------------------------------------------------- /src/architecture/evm-circuit_step-transition.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/architecture/evm-circuit_step-transition.png -------------------------------------------------------------------------------- /src/architecture/keccak-circuit.md: -------------------------------------------------------------------------------- 1 | # Keccak Circuit 2 | 3 | Keccak256 circuit iterates over hashes to verify each hash is valid. 4 | 5 | It serves as a lookup table for EVM, Bytecode, Tx and MPT circuit to do hash. 6 | -------------------------------------------------------------------------------- /src/architecture/mpt-circuit.md: -------------------------------------------------------------------------------- 1 | # Merkle Patricia Trie Circuit 2 | 3 | MPT circuit (Merkle Patricia Trie circuit) iterates over merkle patricia trie transition to verify each update is valid. 4 | 5 | It serves as a lookup table for State and Tx circuit to do merkle patricia trie update. 6 | -------------------------------------------------------------------------------- /src/architecture/state-circuit.md: -------------------------------------------------------------------------------- 1 | # State Circuit 2 | 3 | 4 | 5 | # Introduction 6 | 7 | The State circuit iterates over random read-write access records of EVM circuit to verify that each piece of data is consistent between different writes. It also verifies the state merkle patricia trie root hash corresponds to a valid transition from old to new one incrementally, where both are from public input. 8 | 9 | To verify if data is consistent, it first verifies that all access records are grouped by their unique identifier and sorted by order of access. Then verifies that the records between writes are consistent. It also verifies that data is in the correct format. 10 | 11 | It serves as a lookup table for EVM circuit to do consistent random read-write access. 12 | 13 | To prevent any malicious insertion of access record, we also verify the amount of random read-write access records in State circuit is equal to the amount in EVM circuit (the final value of `rw_counter`). 14 | 15 | # Concepts 16 | 17 | ## Read-write unit grouping 18 | 19 | The first thing to ensure data is consistent between different writes is to give each data an unique identifier, then group data chunks by the unique identifier. And finally, then sort them by order of access `rw_counter`. 20 | 21 | Here are all kinds of data with their unique identifier: 22 | 23 | 24 | | Tag | Unique Index | Values | 25 | | ------------------------- | ---------------------------------------- | ------------------------------------- | 26 | | `TxAccessListAccount` | `(tx_id, account_address)` | `(is_warm, is_warm_prev)` | 27 | | `TxAccessListAccountStorage` | `(tx_id, account_address, storage_slot)` | `(is_warm, is_warm_prev)` | 28 | | `TxRefund` | `(tx_id)` | `(value, value_prev)` | 29 | | `Account` | `(account_address, field_tag)` | `(value, value_prev)` | 30 | | `AccountStorage` | `(account_address, storage_slot)` | `(value, value_prev)` | 31 | | `AccountDestructed` | `(account_address)` | `(is_destructed, is_destructed_prev)` | 32 | | `CallContext` | `(call_id, field_tag)` | `(value)` | 33 | | `Stack` | `(call_id, stack_address)` | `(value)` | 34 | | `Memory` | `(call_id, memory_address)` | `(byte)` | 35 | 36 | Different tags have different constraints on their grouping and values. 37 | 38 | Most tags also keep the previous value `*_prev` for convenience, which helps reduce the lookup when EVM circuit is performing a write with a `diff` to the current value, or performing a write with a reversion. 39 | 40 | ## Lazy initialization 41 | 42 | EVM's memory expands implicitly, for example, when the memory is empty and it enounters a `mload(32)`, EVM first expands to memory size to `64`, and then loads the bytes just initialized to push to the stack, which is always a `0`. 43 | 44 | The implicit expansion behavior makes even the simple `MLOAD` and `MSTORE` complicated in EVM circuit, so we have a trick to outsource the effort to State circuit by constraining the first record of each memory unit to be a write or have value `0`. It saves the variable amount of effort to expand memory and ignore those never used memory, only used memory addresses will be initlized with `0` so as lazy initialization. 45 | 46 | > This concept is also used in another case: the opcode `SELFDESTRUCT` also has ability to update the variable amount of data. It resets the `balance`, `nonce`, `code_hash`, and every `storage_slot` even if it's not used in the step. So for each state under account, we can add a `revision_id` handle such case, see [Design Notes, Reversible Write Reversion Note2, SELFDESTRUCT](./reversible-write-reversion2.md#selfdestruct) for details. 47 | > ==TODO== Convert this into an issue for discussion 48 | > 49 | > **han** 50 | 51 | ## Trie opening and incrementally update 52 | 53 | # Constraints 54 | 55 | ## `main` 56 | 57 | ==TODO== Explain each tag 58 | 59 | 71 | 72 | # Implementation 73 | 74 | - [spec](https://github.com/appliedzkp/zkevm-specs/blob/master/specs/state-proof.md) 75 | - [python](https://github.com/appliedzkp/zkevm-specs/blob/master/src/zkevm_specs/state.py) 76 | - [circuit](https://github.com/appliedzkp/zkevm-circuits/blob/main/zkevm-circuits/src/state_circuit.rs) 77 | -------------------------------------------------------------------------------- /src/architecture/tx-circuit.md: -------------------------------------------------------------------------------- 1 | # Tx Circuit 2 | 3 | Tx circuit iterates over transactions included in proof to verify each transaction has valid signature. It also verifies the built transaction merkle patricia trie has same root hash as public input. 4 | 5 | Main part of Tx circuit will be instance columns whose evaluation values are built by verifier directly. See the [issue](https://github.com/appliedzkp/zkevm-circuits/issues/122) for more details. 6 | 7 | To verify if a transaction has valid signature, it hashes the RLP encoding of transaction and recover the address of signer with signature, then verifies the signer address is correct. 8 | 9 | It serves as a lookup table for EVM circuit to do random access of any field of transaction. 10 | 11 | To prevent any skip of transactions, we verify that the amount of transactions in the Tx circuit is equal to the amount that verified in EVM circuit. 12 | -------------------------------------------------------------------------------- /src/architecture_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/architecture_diagram.png -------------------------------------------------------------------------------- /src/architecture_diagram2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/architecture_diagram2.png -------------------------------------------------------------------------------- /src/design.md: -------------------------------------------------------------------------------- 1 | # Design Notes 2 | 3 | Here are some collected documents on design notes of the zkEVM. 4 | -------------------------------------------------------------------------------- /src/design/random-linear-combinaion.md: -------------------------------------------------------------------------------- 1 | # Random Linear Combination 2 | 3 | 4 | 5 | ## Introduction 6 | 7 | In the circuit, in order to reduce the number of constraints, we use the random linear combination as a cheap hash function on range-checked bytes for two scenarios: 8 | 9 | 1. Encode 32-bytes word (256-bits) in 254-bits field 10 | 2. Accumulate (or fit/encode) arbitrary-length bytes in 254-bits field 11 | 12 | On the first scenario, it allows us to store an EVM word in a single witness value, without worrying about the fact that a word doesn't fit in the field. Most of the time we move these random linear combination word directly from here to there, and only when we need to perform arithmetic or bitwise operation we will decode the word into bytes (with range check on each byte) to do the actual operation. 13 | 14 | Alternatively we could also store an EVM word in 2 witnes values, representing hi-part and lo-part; but it makes us need to move 2 witnes valuess around for each word. Note that the constraints optimizations obtained by using the random linear combination have not been properly analized yet. 15 | 16 | On the second scenario, it allows us to easily do RLP encoding for a transaction or a merkle (hexary) patricia trie node in a fixed amount of witnesses, without worrying about the fact that RLP encoded bytes could have arbitrary and unlimited length (for MPT node it has a max length, but for tx it doesn't). Each accumulated witness will be further decompress/decomposite/decode into serveral bytes and fed to `keccak256` as input. 17 | 18 | > It would be really nice if we can further ask `keccak256` to accept a accumulated witness and the amount of bytes it contains as inputs. 19 | > 20 | > **han** 21 | 22 | ## Concern on randomness 23 | 24 | The way randomness is derived for random linear combination is important: if done improperly, a malicious prover could find a collision to make a seemigly incorrect witness pass the verification (allowing minting Ether from thin air). 25 | 26 | Here are 2 approaches trying to derive a reasonable randomness to mitigate the risk. 27 | 28 | ### 1. Randomness from committed polynomials with an extra round 29 | 30 | Assuming we could separate all random linear combined witnesses to different polynomials in our constraint system, we can: 31 | 1. Commit polynomials except those for random linear combined witnesses 32 | 2. Derive the randomness from commitments as public input 33 | 3. Continue the proving process. 34 | 35 | ### 2. Randomness from all public inputs of circuit 36 | 37 | > Update: We should just follow traditional Fiat-Shamir (approach 1), to always commit and generate challenge. Assuming EVM state transition is deterministic is not working for malicious prover. 38 | 39 | The public inputs of circuit at least contains: 40 | 41 | - Transactions raw data (input) 42 | - Old state trie root (input) 43 | - New state trie root (output) 44 | 45 | Regardless of the fact that the new state trie root could be an incorrect one (in the case of an attack), since the state trie root implies all the bytes it contains (including transaction raw data), if we derive the randomness from all of them, the malicious prover needs to first decide what's the new (incorrect) state trie root and then find the collisions with input and output. This somehow limits the possible collision pairs because the input and output are also fixed. 46 | 47 | ## A minimal deterministic system using random linear combination 48 | 49 | The following example shows how the random linear combination is used to compare equality of words using a single witness value. 50 | 51 | Suppose a deterministic virtual machine consists of 2 opcodes `PUSH32` and `ADD`, and the VM runs as a pure function `run` as described: 52 | 53 | ### Pseudo code 54 | 55 | ```python 56 | def randomness_approach_2(bytecode: list, claimed_output: list) -> int: 57 | return int.from_bytes(keccak256(bytecode + claimed_output), 'little') % FP 58 | 59 | def run(bytecode: list, claimed_output: list): 60 | """ 61 | run takes bytecode to execute and treat the top of stack as output in the end. 62 | """ 63 | 64 | # Derive randomness 65 | r = randomness_approach_2(bytecode, claimed_output) 66 | 67 | # Despite an EVM word is 256-bit which is larger then field size, we store it 68 | # as random linear combination in stack. Top value is in the end of the list. 69 | stack = [] 70 | 71 | program_counter = 0 72 | while program_counter < len(bytecode): 73 | opcode = bytecode[program_counter] 74 | 75 | # PUSH32 76 | if opcode == 0x00: 77 | # Read next 32 bytes as an EVM word from bytecode 78 | bytes = bytecode[program_counter+1:program_counter+33] 79 | # Push random linear combination of the EVM word to stack 80 | stack.append(random_linear_combine(bytes, r)) 81 | program_counter += 33 82 | # ADD 83 | elif opcode == 0x01: 84 | # Pop 2 random linear combination EVM word from stack 85 | a, b = stack.pop(), stack.pop() 86 | # Decompress them into little-endian bytes 87 | bytes_a, bytes_b = rlc_to_bytes[a], rlc_to_bytes[b] 88 | # Add them together 89 | bytes_c = add_bytes(bytes_a, bytes_b) 90 | # Push result as random linear combination to stack 91 | stack.append(random_linear_combine(bytes_c, r)) 92 | program_counter += 1 93 | else: 94 | raise ValueError("invalid opcode") 95 | 96 | assert rlc_to_bytes[stack.pop()] == claimed_output, "unsatisfied" 97 | ``` 98 | 99 | ### [Full runnable code](./random-linear-combinaion/full-runnable-code.md) 100 | 101 | All the random linear combination or decompression will be constraint in PLONK constraint system, where the randomness is fed as public input. 102 | 103 | The randomness is derived from both input and output (fed to keccak256), which corresponds to [approach 2](#2-Randomness-from-all-public-inputs-of-circuit). Although it uses raw value in bytes instead of hashed value, but assuming the keacck256 and the merkle (hexary) patricia trie in Ethereum are collision resistant, it should be no big differece between the two cases. 104 | 105 | The issue at least reduces to: **Whether a malicious prover can find collisions between stack push and pop, after it decides the input and output**. 106 | 107 | -------------------------------------------------------------------------------- /src/design/random-linear-combinaion/full-runnable-code.md: -------------------------------------------------------------------------------- 1 | # Random Linear Combination full runnable code 2 | 3 | ```python 4 | from Crypto.Hash import keccak 5 | from Crypto.Random.random import randrange 6 | 7 | 8 | def keccak256(data: list) -> list: 9 | return list(keccak.new(digest_bits=256).update(bytes(data)).digest()) 10 | 11 | 12 | # BN254 scalar field size. 13 | FP = 21888242871839275222246405745257275088548364400416034343698204186575808495617 14 | 15 | 16 | def fp_add(a: int, b: int) -> int: return (a + b) % FP 17 | def fp_mul(a: int, b: int) -> int: return (a * b) % FP 18 | 19 | 20 | # rlc_to_bytes records the original bytes of a random linear combination. 21 | # In circuit we ask prover the provide bytes and verify all bytes are in range 22 | # and the random linear combination matches. 23 | rlc_to_bytes = dict() 24 | 25 | 26 | def random_linear_combine(bytes: list, r: int) -> int: 27 | """ 28 | random_linear_combine returns bytes[0] + r*bytes[1] + ... + (r**31)*bytes[31]. 29 | """ 30 | 31 | rlc = 0 32 | for byte in reversed(bytes): 33 | assert 0 <= byte < 256 34 | rlc = fp_add(fp_mul(rlc, r), byte) 35 | 36 | rlc_to_bytes[rlc] = bytes 37 | 38 | return rlc 39 | 40 | 41 | def add_bytes(lhs: list, rhs: list) -> list: 42 | """ 43 | add_bytes adds 2 little-endian bytes value modulus 2**256 and returns result 44 | as bytes also in little-endian. 45 | """ 46 | 47 | result = ( 48 | int.from_bytes(lhs, 'little') + 49 | int.from_bytes(rhs, 'little') 50 | ) % 2**256 51 | 52 | return list(result.to_bytes(32, 'little')) 53 | 54 | 55 | def randomness_approach_2(bytecode: list, claimed_output: list) -> int: 56 | return int.from_bytes(keccak256(bytecode + claimed_output), 'little') % FP 57 | 58 | 59 | def run(bytecode: list, claimed_output: list): 60 | """ 61 | run takes bytecode to execute and treat the top of stack as output in the end. 62 | """ 63 | 64 | # Derive randomness 65 | r = randomness_approach_2(bytecode, claimed_output) 66 | 67 | # Despite an EVM word is 256-bit which is larger then field size, we store it 68 | # as random linear combination in stack. Top value is in the end of the list. 69 | stack = [] 70 | 71 | program_counter = 0 72 | while program_counter < len(bytecode): 73 | opcode = bytecode[program_counter] 74 | 75 | # PUSH32 76 | if opcode == 0x00: 77 | # Read next 32 bytes as an EVM word from bytecode 78 | bytes = bytecode[program_counter+1:program_counter+33] 79 | # Push random linear combination of the EVM word to stack 80 | stack.append(random_linear_combine(bytes, r)) 81 | program_counter += 33 82 | # ADD 83 | elif opcode == 0x01: 84 | # Pop 2 random linear combination EVM word from stack 85 | a, b = stack.pop(), stack.pop() 86 | # Decompress them into little-endian bytes 87 | bytes_a, bytes_b = rlc_to_bytes[a], rlc_to_bytes[b] 88 | # Add them together 89 | bytes_c = add_bytes(bytes_a, bytes_b) 90 | # Push result as random linear combination to stack 91 | stack.append(random_linear_combine(bytes_c, r)) 92 | program_counter += 1 93 | else: 94 | raise ValueError("invalid opcode") 95 | 96 | assert rlc_to_bytes[stack.pop()] == claimed_output, "unsatisfied" 97 | 98 | 99 | def test_run(): 100 | a, b = randrange(0, 2**256), randrange(0, 2**256) 101 | c = (a + b) % 2**256 102 | run( 103 | bytecode=[ 104 | 0x00, *a.to_bytes(32, 'little'), # PUSH32 a 105 | 0x00, *b.to_bytes(32, 'little'), # PUSH32 b 106 | 0x01 # ADD 107 | ], 108 | claimed_output=list(c.to_bytes(32, 'little')), 109 | ) 110 | 111 | 112 | test_run() 113 | ``` 114 | 115 | 116 | -------------------------------------------------------------------------------- /src/design/recursion.md: -------------------------------------------------------------------------------- 1 | # Recursion 2 | 3 | 4 | 5 | ## Projects 6 | 7 | - 8 | - 9 | - 10 | - 11 | 12 | ### Aztec barretenberg 13 | 14 | #### Transcript for inner cirucit 15 | 16 | The rollup circuit uses blake2s to generate the challenge, but with compressed input by pedersen hash [^barretenberg-transcript]. The inputs are serialized into a list of 31 bytes field elements as the input of pedersen hash. 17 | 18 | Also the rollup circuit use 128-bits challenge to reduce MSM work, then most of scalars are 128-bits, only those scalars multiplied with some other stuff (e.g. \\(\zeta\omega^i\\) or batch evaluation) would need 254-bits MSM. 19 | 20 | [^barretenberg-transcript]: 21 | 22 | #### Public inputs of inner circuit 23 | 24 | Barretenberg handles the public inputs along with permutation argument [^barretenberg-permutation-widget]. Its constraint system implicitly copies \\(m\\) public inputs \\(p\\) to first \\(m\\) witness of the first column, and because the public inputs' contribution \\(\frac{1}{\delta}\\) to the grand product is easy to compute, so it doesn't need to evaluate public inputs poly at \\(\zeta\\). Instead, it verifies grand product's final value is the contribution \\(\delta\\). 25 | 26 | $$ 27 | \frac{1}{\delta} = \prod_{i\in[m]}\left(\frac{p_i + \beta\omega^i + \gamma}{p_i + \beta k_{p}\omega^i + \gamma}\right) 28 | $$ 29 | 30 | > Not sure if this is cheaper than calculating evaluation of public inputs, but it's really intersting. 31 | > 32 | > **han** 33 | 34 | The actual verifier doesn't calculate the \\(\frac{1}{\delta}\\) for inner circuit, it just concatenates all the public inputs and let rollup circuit to calculate \\(\frac{1}{\delta}\\) for each inner circuit. 35 | 36 | [^barretenberg-permutation-widget]: Explaination:
Used in permutation widget: 37 | 38 | ## Calculations 39 | 40 | ### Interpolation 41 | 42 | #### Barycentric formula [^barycentric-formula] 43 | 44 | Using 45 | 46 | $$ 47 | \begin{aligned} 48 | & \ell(x) = (x-x_0)(x-x_1)\cdots(x-x_k) \\ 49 | & \ell^\prime(x_j) = \frac{d\ell(x)}{dx}\rvert_{x=x_j} = \prod_{i=0,i\ne j}^k(x_j-x_i) 50 | \end{aligned} 51 | $$ 52 | 53 | We can rewrite Lagrange basis as 54 | 55 | $$ 56 | \ell_j(x) = \frac{\ell(x)}{\ell^\prime(x_j)(x-x_j)} 57 | $$ 58 | 59 | Or by defining the _barycentric weights_ 60 | 61 | $$ 62 | w_j = \frac{1}{\ell^\prime(x_j)} 63 | $$ 64 | 65 | Then given evaluations $\{y_i\}_{i\in[k]} = \{f(x_i)\}_{i\in[k]}$ of polynomial \\(f\\), the interpolation of \\(f\\) now can be evaluated as 66 | 67 | $$ 68 | f(x) = \ell(x) \sum_{j=0}^k\frac{w_j}{x-x_j}y_j 69 | $$ 70 | 71 | Which, if the weights \\(w_j\\) can be pre-computed, requires only \\(\mathcal{O}(k)\\) as opposed to \\(\mathcal{O}(k^2)\\) for evaluating the Lagrange basis \\(\ell_j(x)\\) individually. 72 | 73 | We can furthers simplify it by considering the barycentric interpolation of the constant function \\(g(x) = 1\\) 74 | 75 | $$ 76 | g(x) = 1 = \ell(x) \sum_{j=0}^k\frac{w_j}{x-x_j} 77 | $$ 78 | 79 | Divigin \\(f(x)\\) by \\(g(x) = 1\\) doesn't modify the interpolation, yet yields 80 | 81 | $$ 82 | f(x) = \frac{\sum_{j=0}^k\frac{w_j}{x-x_j}y_j}{\sum_{j=0}^k\frac{w_j}{x-x_j}} 83 | $$ 84 | 85 | Then we don't even need to evaluate \\(\ell(x)\\). 86 | 87 | [^barycentric-formula]:
88 | 89 | #### Interpolation of rotation set 90 | 91 | Defining rotation set \\(R\\) contains \\(k\\) different points \\(\{\zeta_1\omega_j\}_{j\in[k]}\\), where \\(\zeta_1\\) is the challenge opening point, and \\(\omega_i\\) are the generator to some power (a.k.a rotation). 92 | 93 | > Not sure if all arguments of halo2 in the future still have the same opening base \\(\zeta_1\\) for all queries to multiopen. If not, this approach might not work. An implementation can be found [here](https://github.com/han0110/halo2/blob/feature%2Fbarycentric-for-r/halo2_proofs/src/poly/multiopen/shplonk/verifier.rs#L60-L99). 94 | > 95 | > **han** 96 | 97 | In SHPLONK, the verifier needs to calculate interpolation \\(r(\zeta_{2})\\) from rotation set \\(R\\) and their claimed evaluation value \\(\{y_j\}_{j\in[k]}\\). 98 | 99 | One of the largest benefit of barycentric formula is pre-computed barycentric weight. Although \\(R\\) contains different values in each proof, we can still pre-compute the normalized barycentric weight without \\(\zeta_1\\), to gain the benefit. 100 | 101 | $$ 102 | w_j = \frac{1}{\prod_{i=0,i\ne j}^k(\omega_j-\omega_i)} 103 | $$ 104 | 105 | And for each rotation set, the actual work is 106 | 107 | $$ 108 | w_j^\prime = \frac{w_j}{\zeta_1^{k-1} * (\zeta_2 - \zeta_1\omega_j)} 109 | $$ 110 | 111 | Also each rotation set might contain more than one polynomials, for each polynomial, its work is 112 | 113 | $$ 114 | r(\zeta_2) = \frac{\sum_{j=0}^kw_j^\prime y_j}{\sum_{j=0}^kw_j^\prime} 115 | $$ 116 | 117 | Where the denominator for one set only needs to be calcuated once. 118 | 119 | #### Interpolation of public inputs 120 | 121 | All lagrange basis could be rotated from the $\ell_0(X) = \frac{X^n-1}{n(X-1)}$ 122 | 123 | $$ 124 | \ell_i(X) = \ell_0(X\omega^{-i}) = \frac{X^n-1}{n(X\omega^{-i}-1)} 125 | $$ 126 | 127 | Given \\(m\\) public inputs \\(p\\), the interpolation at \\(\zeta\\) is 128 | 129 | $$ 130 | p(\zeta) = \sum_{i\in[m]}p_i\ell_i(\zeta) = \frac{\zeta^n-1}{n}\sum_{i\in[m]}\frac{p_i}{(\zeta\omega^{-i}) - 1} 131 | $$ 132 | 133 | Note that \\(\frac{\zeta^n-1}{n}\\) needs \\(\log_2(n)\\) squaring of \\(\zeta\\) with a substraction and a division. 134 | 135 | And each extra public input costs about 4 operations (mul + sub + div + add). 136 | 137 | ## Random thoughts 138 | 139 | ### Use multi-layer recursive proof for ZKEVM 140 | 141 | In our case, we have split ZKEVM into different pieces, with some dependency relation in between. 142 | 143 | Fortunately, the dependency relation currently is always being like: One verified circuit serves itself as a lookup table for another. For example, once State circuit is verified to satasify its own relation, then we can synthesize its columns to be a lookup table for EVM circuit to do random access. 144 | 145 | Serving self as a lookup table only needs to pass a single check, that is the table commitment(s) (random linearly combined or not). And the difference between fixed table and such verified private table is: The former is built when setup so it is already trusted, the latter is verified for each proof instance and is trusted as long as the relation is well-defined. 146 | 147 | So, if a single recursive circuit can't aggregate all different circuits in ZKEVM, we can incrementally aggregate them, and only expose the verified private table's commitment(s) as public input, for next proofs' aggregation. 148 | 149 | > If we can have some kinds of "global randomness", we can do vector table's random linear combination first, then the extra public input for exposing verified private table is just a single group element (4 values in rns). 150 | > 151 | > **han** 152 | 153 | Illustration of serially aggregating State circuit and EVM circuit: 154 | 155 | ![](./recursion_aggregation-serial.png) 156 | 157 | 158 | 1. Accumulated proof which contains LHS and RHS of pairing
159 | 2. RW Table contains 10 commitments which are verified to open to desired synthesized expressions 160 | 161 | -------------------------------------------------------------------------------- /src/design/recursion_aggregation-serial.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/design/recursion_aggregation-serial.png -------------------------------------------------------------------------------- /src/design/reversible-write-reversion.md: -------------------------------------------------------------------------------- 1 | # Reversible Write Reversion 2 | 3 | Reversible write reversion might be the most tricky part to explain of the EVM circuit. This note aims to illustrate how the current approach works with some diagrams, and collect all the other approaches for comparison. 4 | 5 | ## Revert or not 6 | 7 | With full execution trace of a block, if we iterate over it once, we can know if each call (including create) is successful or not, and then determine which reversible writes are persistent, and which are not. 8 | 9 | So each call could be annotated with 2 tags: 10 | 11 | - `is_success` - If this call ends with `STOP` or `RETURN` 12 | - `is_persistent` - If this call and all its caller have `is_success == true` 13 | 14 | Only reversible writes inside a call with `is_persistent == true` will be applied to the new state. Reversible writes in a call with `is_persistent == false` will be reverted at the closest call that has `is_success == false`. 15 | 16 | ## Current approach 17 | 18 | Since the only requirement of a read/write access is `rw_counter` uniqueness, we are not restricted to only do read/writes with sequential `rw_counter` in a step, instead we can do read/write with any `rw_counter`, as long as we don't break the `rw_counter` uniqueness requirement. 19 | 20 | We ask the prover to tell us each call's information including: 21 | 22 | - `is_success` - Described above 23 | - `is_persistent` - Described above 24 | - `rw_counter_end_of_reversion` - The `rw_counter` at the end of reversion of the call if it has `is_persistent == false` 25 | 26 | In EVM circuit we track the value `reversible_write_counter` to count how many reversible writes have been made so far. This value is initialized at `0` of each call. 27 | 28 | With `is_persistent`, `rw_counter_end_of_reversion` and `reversible_write_counter`, we can do the reversible write with its corresponding reversion at the same step, because we know at which point it should happen. The pseudo code of reversible write looks like: 29 | 30 | ```python 31 | rw_table.lookup( 32 | rw_counter=rw_counter, 33 | reversible_write=reversible_write, # write to new value 34 | ) 35 | 36 | if not is_persistent: 37 | rw_table.lookup( 38 | rw_counter=rw_counter_end_of_reversion - reversible_write_counter, 39 | reversible_write=reversible_write.reverted(), # write to previous value 40 | ) 41 | 42 | rw_counter += 1 43 | reversible_write_counter += 1 44 | ``` 45 | 46 | Note that the we are increasing the `reversible_write_counter`, so the reversions are applied in reverse order in `rw_table`, which is exactly the behavior we want. 47 | 48 | Another important check is to ensure `rw_counter_end_of_reversion` is correct. At the step that does the reversions, we check if there is a gap of `rw_counter` to the next step, where the gap size is exactly the `reversible_write_counter`. And in the end of the gap, the `rw_counter` is exactly `rw_counter_end_of_reversion`. The pseudo code looks like: 49 | 50 | ```python 51 | if not is_persistent: 52 | assert rw_counter_end_of_reversion == rw_counter + reversible_write_counter 53 | rw_counter = call.rw_counter_end_of_reversion + 1 54 | ``` 55 | 56 | With illustration: 57 | 58 | ![](./state-write-reversion_reversion-simple.png) 59 | 60 | The step that does the reversions is also responsible for reversions of its successful callees. Note that each callee's `reversible_write_counter` is initialized at `0`. To make sure they can do the reversion at correct `rw_counter`, we need to propagate the `rw_counter_end_of_reversion` to be itself minus the current accumulated `reversible_write_counter`. The pseudo code looks like: 61 | 62 | ```python 63 | if not is_persistent and callee_is_success: 64 | assert callee_rw_counter_end_of_reversion \ 65 | == rw_counter_end_of_reversion - reversible_write_counter 66 | ``` 67 | 68 | ![](./state-write-reversion_reversion-nested.png) 69 | 70 | At the end of successful callee, we accumulate the `reversible_write_counter` to its caller. 71 | 72 | ### Adjustment for `SELFDESTRUCT` 73 | 74 | See [Design Notes, Reversible Write Reversion Note2, SELFDESTRUCT](./reversible-write-reversion2.md#selfdestruct) 75 | 76 | ## Other approaches 77 | 78 | ### With `revision_id` 79 | 80 | TODO 81 | -------------------------------------------------------------------------------- /src/design/reversible-write-reversion2.md: -------------------------------------------------------------------------------- 1 | # Reversible Write Reversion Note 2 2 | 3 | # ZKEVM - State Circuit Extension - `StateDB` 4 | 5 | ## Reversion 6 | 7 | In EVM, there are multiple kinds of `StateDB` updates that could be reverted when any internal call fails. 8 | 9 | - `tx_access_list_account` - `(tx_id, address) -> accessed` 10 | - `tx_access_list_storage_slot` - `(tx_id, address, storage_slot) -> accessed` 11 | - `account_nonce` - `address -> nonce` 12 | - `account_balance` - `address -> balance` 13 | - `account_code_hash` - `address -> code_hash` 14 | - `account_storage` - `(address, storage_slot) -> storage` 15 | 16 | The complete list can be found [here](https://github.com/ethereum/go-ethereum/blob/master/core/state/journal.go#L87-L141). For `tx_refund`, `tx_log`, `account_destructed` we don't need to write and revert because those state changes don't affect future execution, so we only write them when `is_persistent=1`. 17 | 18 | ### Visualization 19 | 20 | ![](./state-write-reversion2_call-depth.png) 21 | 22 | - Black arrow represents the time, which is composed by points of sequential `rw_counter`. 23 | - Red circle represents the revert section. 24 | 25 | The actions that write to the `StateDB` inside the red box will also revert themselves in the revert section (red circle), but in reverse order. 26 | 27 | Each call needs to know its `rw_counter_end_of_revert_section` to revert with the correct `rw_counter`. If callee is a success call but in some red box (`is_persistent=0`), we need to copy caller's `rw_counter_end_of_revert_section` and `reversible_write_counter` to callee's. 28 | 29 | ## `SELFDESTRUCT` 30 | 31 | The opcode `SELFDESTRUCT` sets the flag `is_destructed` of the account, but before that transaction ends, the account can still be executed, receive ether, and access storage as usual. The flag `is_destructed` takes effect only after a transaction ends. 32 | 33 | In particular, the state trie gets finalized after each transaction, and only when state trie gets finalized the account is actually deleted. After the transaction with `SELFDESTRUCT` is finalized, any further transaction treats the account as an empty account. 34 | 35 | > So if some contract executed `SELFDESTRUCT` but then receive some ether, those ether will vanish into thin air after the transaction is finalized. Soooo weird. 36 | > 37 | > **han** 38 | 39 | The `SELFDESTRUCT` is a powerful opcode that makes many state changes at the same time including: 40 | 41 | - `account_nonce` 42 | - `account_balance` 43 | - `account_code_hash` 44 | - all slots of `account_storage` 45 | 46 | The first 3 values are relatively easy to handle in circuit: we could track an extra `selfdestruct_counter` and `rw_counter_end_of_tx` and set them to empty value at `rw_counter_end_of_tx - selfdestruct_counter`, which is just how we handle reverts. 47 | 48 | However, the `account_storage` is tricky because we don't track the storage trie and update it after each transaction, instead we only track each used slot in storage trie and update the storage trie after the whole block. 49 | 50 | ### Workaround for consistency check 51 | 52 | It seems that we need to annotate each account with a `revision_id`. The `revision_id` increases only when `is_destructed` is set and `tx_id` changes. With the different `revision_id`s we can reset the values in State circuit for `nonce`, `balance`, `code_hash`, and each `storage` just like we initialize the memroy. 53 | 54 | So `address -> is_destructed` becomes `(tx_id, address) -> (revision_id, is_destructed)`. 55 | 56 | Then we add an extra `revision_id` to `nonce`, `balance`, `code_hash` and `storage`. For `nonce`, `balance` and `code_hash` we group them by `(address, revision_id) -> {nonce,balance,code_hash}`, for `storage` we group them by `(address, storage_slot, revision_id) -> storage_value`. 57 | 58 | Here is an example of `account_balance` with `revision_id`: 59 | 60 | $$ 61 | \begin{array}{|c|c|} 62 | \hline 63 | \texttt{address} & \texttt{revision_id} & \texttt{rwc} & \texttt{balance} & \texttt{balance_prev} & \texttt{is_write} & \text{note} \\\\\hline 64 | \color{#aaa}{\texttt{0xfd}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 65 | \texttt{0xfe} & \texttt{1} & \color{#aaa}{\texttt{x}} & \texttt{10} & \color{#aaa}{\texttt{x}} & \texttt{1} & \text{open from trie} \\\\\hline 66 | \texttt{0xfe} & \texttt{1} & \texttt{23} & \texttt{20} & \texttt{10} & \texttt{1} \\\\\hline 67 | \texttt{0xfe} & \texttt{1} & \texttt{45} & \texttt{20} & \texttt{20} & \texttt{0} \\\\\hline 68 | \texttt{0xfe} & \texttt{1} & \texttt{60} & \texttt{0} & \texttt{20} & \texttt{1} \\\\\hline 69 | \texttt{0xfe} & \color{#f00}{\texttt{1}} & \texttt{63} & \texttt{5} & \texttt{0} & \texttt{1} \\\\\hline 70 | \texttt{0xfe} & \color{#f00}{\texttt{2}} & \color{#aaa}{\texttt{x}} & \color{#f00}{\texttt{0}} & \color{#aaa}{\texttt{x}} & \texttt{1} & \text{reset} \\\\\hline 71 | \texttt{0xfe} & \texttt{2} & \texttt{72} & \texttt{0} & \texttt{0} & \texttt{0} \\\\\hline 72 | \color{#aaa}{\texttt{0xff}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 73 | \end{array} 74 | $$ 75 | 76 | Note that after contract selfdestructs, it can still receive ether, but the ether will vanish into thin air after transaction gets finalized. The reset is like the lazy initlization of memory, **the value is set to `0` when `revision_id` is different**. 77 | 78 | Here is how we increase the `revision_id`: 79 | 80 | $$ 81 | \begin{array}{|c|c|} 82 | \hline 83 | \texttt{address} & \texttt{tx_id} & \texttt{rwc} & \texttt{revision_id} & \texttt{is_destructed} & \texttt{is_destructed_prev} & \texttt{is_write} & \text{note} \\\\\hline 84 | \color{#aaa}{\texttt{0xfd}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 85 | \texttt{0xff} & \texttt{1} & \color{#aaa}{\texttt{x}} & \texttt{1} & \texttt{0} & \color{#aaa}{\texttt{x}} & \texttt{1} & \text{init} \\\\\hline 86 | \texttt{0xff} & \texttt{1} & \texttt{11} & \texttt{1} & \texttt{0} & \texttt{0} & \texttt{0} \\\\\hline 87 | \texttt{0xff} & \texttt{1} & \texttt{17} & \texttt{1} & \texttt{1} & \texttt{0} & \texttt{1} & \text{self destruct} \\\\\hline 88 | \texttt{0xff} & \color{#f00}{\texttt{1}} & \texttt{29} & \texttt{1} & \color{#f00}{\texttt{1}} & \texttt{1} & \texttt{1} & \text{self destruct again} \\\\\hline 89 | \texttt{0xff} & \color{#f00}{\texttt{2}} & \color{#aaa}{\texttt{x}} & \color{#f00}{\texttt{2}} & \texttt{0} & \color{#aaa}{\texttt{x}} & \texttt{1} & \text{increase} \\\\\hline 90 | \texttt{0xff} & \texttt{2} & \texttt{40} & \texttt{2} & \texttt{0} & \texttt{0} & \texttt{0} \\\\\hline 91 | \texttt{0xff} & \texttt{3} & \color{#aaa}{\texttt{x}} & \texttt{2} & \texttt{0} & \color{#aaa}{\texttt{x}} & \texttt{1} & \text{no increase} \\\\\hline 92 | \color{#aaa}{\texttt{0xff}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} & \color{#aaa}{\texttt{-}} \\\\\hline 93 | \end{array} 94 | $$ 95 | 96 | Because self destruct only takes effect after the transaction, we **increase the `revision_id` only when `tx_id` is different and `is_destructed` is set**. 97 | 98 | ### Workaround for trie update 99 | 100 | The State circuit not only checks consistency, it also triggers the update of the storage tries and state trie. 101 | 102 | Originally, some part of State circuit would assign the first row value and collect the last row value of each account's `nonce`, `balance`, `code_hash` as well as the first & last used slots of storage, then update the state trie. 103 | 104 | With `revision_id`, it needs to peek the final `revision_id` first, and collect the last row value with the `revision_id` to make sure all values are actually reset. 105 | 106 | ## Reference 107 | 108 | - [`journal.go`](https://github.com/ethereum/go-ethereum/blob/master/core/state/journal.go) 109 | - [Pragmatic destruction of `SELFDESTRUCT`](https://hackmd.io/@vbuterin/selfdestruct#SELFDESTRUCT-is-the-only-opcode-that-breaks-important-invariants) 110 | -------------------------------------------------------------------------------- /src/design/state-write-reversion2_call-depth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/design/state-write-reversion2_call-depth.png -------------------------------------------------------------------------------- /src/design/state-write-reversion_reversion-nested.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/design/state-write-reversion_reversion-nested.png -------------------------------------------------------------------------------- /src/design/state-write-reversion_reversion-simple.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/privacy-scaling-explorations/zkevm-docs/e6ab9a73632fbdaabaa8f44db797a9955a5adf0f/src/design/state-write-reversion_reversion-simple.png -------------------------------------------------------------------------------- /src/introduction.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | The zkEVM aims to specify and implement a solution to validate Ethereum blocks 4 | via zero knowledge proofs. The project aims to achieve 100% compatibility with 5 | the Ethereum's EVM. It's an open-source project that is contributed and owned 6 | by the community. Check out the contributors at 7 | [here](https://github.com/appliedzkp/zkevm-circuits/graphs/contributors) and 8 | [here](https://github.com/appliedzkp/zkevm-specs/graphs/contributors). 9 | 10 | This book contains general documentation of the project. 11 | 12 | The project currently has two goals: 13 | 14 | ## zkRollup 15 | 16 | Build a solution that allows deploying a layer 2 network that is compatible 17 | with the Ethereum ecosystem (by following the Ethereum specification) and 18 | submits zero knowledge proofs of correctly constructed new blocks to a layer 1 19 | smart contract which validates such proofs (and acts as a consensus layer). 20 | 21 | The usage of zero knowledge proofs to validate blocks allows clients to 22 | validate transactions quicker than it takes to process them, offering benefits 23 | in scalability. 24 | 25 | ## Validity proofs 26 | 27 | Build a solution that allows generating zero knowledge proofs of blocks from an 28 | existing Ethereum network (such as mainnet), and publish them in a smart 29 | contract in the same network. 30 | 31 | The usage of zero knowledge proofs to validate blocks allows light clients to 32 | quickly synchronize many blocks with low resource consumption, while 33 | guaranteeing the correctness of the blocks without needing trust on external 34 | parties. 35 | 36 | # Status 37 | 38 | The zkEVM project is not yet complete, so you may find parts that are not yet 39 | implemented, incomplete, or don't have a specification. At the same time, 40 | other parts which are already implemented may be changed in the future. 41 | 42 | # Links 43 | 44 | - [Implementation](https://github.com/appliedzkp/zkevm-circuits) 45 | - [Specification](https://github.com/appliedzkp/zkevm-specs) 46 | 47 | --------------------------------------------------------------------------------