├── .gitignore ├── CN ├── 00_geth.md ├── 01_account_state.md ├── 02_state_management_statedb.md ├── 03_state_management_stateTrie.md ├── 04_transaction.md ├── 05_block_blockchain.md ├── 06_mining_hash_gpu.md ├── 07_p2p_net_node_sync.md ├── 10_state_data_optimization.md ├── 11_leveldb_in_practice.md ├── 12_caching.md ├── 13_evm.md ├── 14_signer.md ├── 15_rpc.md ├── 20_bft_consensus.md ├── 21_rollup.md ├── 22_ads.md ├── 23_bloom_filter.md ├── 24_turing_halting.md ├── 25_lsm_tree.md ├── 26_txn_concurrency.md ├── 30_geth_private_network.md ├── 31_solidity_in_practice.md ├── 32_oracle.md ├── 33_query.md ├── 41_system_tunning.md ├── 42_developer_view.md ├── 43_metrics.md ├── 44_golang_ethereum.md └── 45_eth2.md ├── LICENSE ├── README.md ├── example ├── account │ └── main.go ├── deploy │ ├── DeployContract.go │ ├── SendTransaction.go │ └── contracts │ │ └── storage.go ├── private_chain │ ├── genesis.json │ └── init.sh ├── signature │ └── main.go └── workflow │ └── main.go └── figs ├── 01 ├── account_storage.png └── remix.png ├── 02 └── tx_execu_flow.png └── 04 └── tx_exec_calls.png /.gitignore: -------------------------------------------------------------------------------- 1 | # Local temporary file 2 | .DS_Store 3 | .vscode 4 | 5 | # Temp File 6 | _draft -------------------------------------------------------------------------------- /CN/00_geth.md: -------------------------------------------------------------------------------- 1 | # 00_万物的起点：Geth Start \ 2 | 3 | ## 什么是 Geth？ 4 | 5 | Geth 是基于 Go 语言开发以太坊的客户端，它实现了 Ethereum 协议（黄皮书）中所有需要的实现的功能模块，包括状态管理，挖矿，P2P 网络通信，密码学，数据库，EVM 解释器等。我们可以通过启动 Geth 来运行一个 Ethereum 的节点。go-ethereum 是包含了 Geth 在内的一个代码库，它包含了 Geth，以及编译 Geth 所需要的其他代码。在本系列中，我们会深入 go-ethereum 代码库，从 High-level 的 API 接口出发，沿着 Ethereum 主 Workflow，逐一的理解 Ethereum 具体实现的细节。 6 | 7 | 为了方便区分，在接下来的文章中，我们用 Geth 来表示 Geth 的客户端程序，用 go-ethereum(geth) 来表示 go-ethereum 的代码库。 8 | 9 | ### go-ethereum Codebase 结构 10 | 11 | 为了更好的从整体工作流的角度来理解 Ethereum，根据主要的业务功能，我们将 go-ethereum 划分成如下几个模块来分析。 12 | 13 | - Geth Client 模块 14 | - Core 数据结构模块 15 | - State Management 模块 16 | - StateDB 模块 17 | - Trie 模块 18 | - State Optimization (Pruning) 19 | - Mining 模块 20 | - EVM 模块 21 | - P2P 网络模块 22 | - 节点数据同步 23 | - ... 24 | 25 | 目前，go-ethereum 项目的主要目录结构如下所示： 26 | 27 | ```html 28 | cmd/ ethereum 相关的 Command-line 程序。该目录下的每个子目录都包含一个可运行的 main.go。 29 | |── clef/ Ethereum 官方推出的 Account 管理程序。 30 | |── geth/ Geth 的本体。 31 | core/ 以太坊核心模块，包括核心数据结构，statedb，EVM 等算法实现 32 | |── rawdb/ db 相关函数的高层封装（在 ethdb 和更底层的 leveldb 之上的封装） 33 | |── state/ 34 | ├──statedb.go StateDB 结构用于存储所有的与 Merkle trie 相关的存储，包括一些循环 state 结构 35 | |── types/ 包括 Block 在内的以太坊核心数据结构 36 | |── block.go 以太坊 block 37 | |── bloom9.go 一个 Bloom Filter 的实现 38 | |── transaction.go 以太坊 transaction 的数据结构与实现 39 | |── transaction_signing.go 用于对 transaction 进行签名的函数的实现 40 | |── receipt.go 以太坊收据的实现，用于说明以太坊交易的结果 41 | |── vm/ 42 | |── genesis.go 创世区块相关的函数，在每个 geth 初始化的都需要调用这个模块 43 | |── tx_pool.go Ethereum Transaction Pool 的实现 44 | consensus/ 45 | |── consensus.go 共识相关的参数设定，包括 Block Reward 的数量 46 | console/ 47 | |── bridge.go 48 | |── console.go Geth Web3 控制台的入口 49 | ethdb/ Ethereum 本地存储的相关实现，包括 leveldb 的调用 50 | |── leveldb/ Go-Ethereum 使用的与 Bitcoin Core version 一样的 Leveldb 作为本机存储用的数据库 51 | miner/ 52 | |── miner.go 矿工模块的实现。 53 | |── worker.go 真正的 block generation 的实现实现，包括打包 transaction，计算合法的 Block 54 | p2p/ Ethereum 的 P2P 模块 55 | |── params Ethereum 的一些参数的配置，例如：bootnode 的 enode 地址 56 | |── bootnodes.go bootnode 的 enode 地址 like: aws 的一些节点，azure 的一些节点，Ethereum Foundation 的节点和 Rinkeby 测试网的节点 57 | rlp/ RLP 的 Encode 与 Decode 的相关 58 | rpc/ Ethereum RPC 客户端的实现 59 | les/ Ethereum light client 的实现 60 | trie/ Ethereum 中至关重要的数据结构 Merkle Patrica Trie(MPT) 的实现 61 | |── committer.go Trie 向 Memory Database 提交数据的工具函数。 62 | |── database.go Memory Database，是 Trie 数据和 Disk Database 提交的中间层。同时还实现了 Trie 剪枝的功能。**非常重要** 63 | |── node.go MPT 中的节点的定义以及相关的函数。 64 | |── secure_trie.go 基于 Trie 的封装的 Trie 结构。与 trie 中的函数功能相同，不过 secure_trie 中的 key 是经过 hashKey() 函数 hash 过的，无法通过路径获得原始的 key 值 65 | |── stack_trie.go Block 中使用的 Transaction/Receipt Trie 的实现 66 | |── trie.go MPT 具体功能的函数实现 67 | ``` 68 | 69 | ## Geth Start 70 | 71 | ### 前奏：Geth Console 72 | 73 | 当我们想要部署一个 Ethereum 节点的时候，最直接的方式就是下载官方提供的发行版的 geth 程序。Geth 是一个基于 CLI 的应用，启动 Geth 和调用 Geth 的功能性 API 需要使用对应的指令来操作。Geth 提供了一个相对友好的 console 来方便用户调用各种指令。当我第一次阅读 Ethereum 的文档的时候，我曾经有过这样的疑问，为什么 Geth 是由 Go 语言编写的，但是在官方文档中的 Web3 的 API 却是基于 Javascript 的调用？ 74 | 75 | 这是因为 Geth 内置了一个 Javascript 的解释器：*Goja* (interpreter)，来作为用户与 Geth 交互的 CLI Console。我们可以在`console/console.go`中找到它的定义。 76 | 77 | 78 | 79 | ```go 80 | // Console is a JavaScript interpreted runtime environment. It is a fully fledged 81 | // JavaScript console attached to a running node via an external or in-process RPC 82 | // client. 83 | type Console struct { 84 | client *rpc.Client // RPC client to execute Ethereum requests through 85 | jsre *jsre.JSRE // JavaScript runtime environment running the interpreter 86 | prompt string // Input prompt prefix string 87 | prompter prompt.UserPrompter // Input prompter to allow interactive user feedback 88 | histPath string // Absolute path to the console scrollback history 89 | history []string // Scroll history maintained by the console 90 | printer io.Writer // Output writer to serialize any display strings to 91 | } 92 | ``` 93 | 94 | ### 启动 95 | 96 | 了解 Ethereum，我们首先要了解 Ethereum 客户端 Geth 是怎么运行的。 97 | 98 | 99 | 100 | Geth 程序的启动点位于`cmd/geth/main.go/main()`函数处，如下所示。 101 | 102 | ```go 103 | func main() { 104 | if err := app.Run(os.Args); err != nil { 105 | fmt.Fprintln(os.Stderr, err) 106 | os.Exit(1) 107 | } 108 | } 109 | ``` 110 | 111 | 我们可以看到`main()`函数非常的简短，其主要功能就是启动一个解析 command line 命令的工具：`gopkg.in/urfave/cli.v1`。我们会发现在 cli app 初始化的时候会调用`app.Action = geth`，来调用`geth()`函数。`geth()`函数就是用于启动 Ethereum 节点的顶层函数，其代码如下所示。 112 | 113 | ```go 114 | func geth(ctx *cli.Context) error { 115 | if args := ctx.Args(); len(args) > 0 { 116 | return fmt.Errorf("invalid command: %q", args[0]) 117 | } 118 | 119 | prepare(ctx) 120 | stack, backend := makeFullNode(ctx) 121 | defer stack.Close() 122 | 123 | startNode(ctx, stack, backend, false) 124 | stack.Wait() 125 | return nil 126 | } 127 | ``` 128 | 129 | 在`geth()`函数，我们可以看到有三个比较重要的函数调用，分别是：`prepare()`，`makeFullNode()`，以及`startNode()`。 130 | 131 | `prepare()` 函数的实现就在当前的`main.go`文件中。它主要用于设置一些节点初始化需要的配置。比如，我们在节点启动时看到的这句话：*Starting Geth on Ethereum mainnet...* 就是在`prepare()`函数中被打印出来的。 132 | 133 | `makeFullNode()`函数的实现位于`cmd/geth/config.go`文件中。它会将 Geth 启动时的命令的上下文加载到配置中，并生成`stack`和`backend`这两个实例。其中`stack`是一个 Node 类型的实例，它是通过`makeFullNode()`函数调用`makeConfigNode()`函数来生成。Node 是 Geth 生命周期中最顶级的实例，它的开启和关闭与 Geth 的启动和关闭直接对应。关于 Node 类型的定义位于`node/node.go`文件中。 134 | 135 | `backend`实例是指的是具体 Ethereum Client 的功能性实例。它是一个 Ethereum 类型的实例，负责提供更为具体的以太坊的功能性 Service，比如管理 Blockchain，共识算法等具体模块。它根据上下文的配置信息在调用`utils.RegisterEthService()`函数生成。在`utils.RegisterEthService()`函数中，首先会根据当前的 config 来判断需要生成的 Ethereum backend 的类型，是 light node backend 还是 full node backend。我们可以在`eth/backend/new()`函数和`les/client.go/new()`中找到这两种 Ethereum backend 的实例是如何初始化的。Ethereum backend 的实例定义了一些更底层的配置，比如 chainid，链使用的共识算法的类型等。这两种后端服务的一个典型的区别是 light node backend 不能启动 Mining 服务。在`utils.RegisterEthService()`函数的最后，调用了`Nodes.RegisterAPIs()`函数，将刚刚生成的 backend 实例注册到`stack`实例中。 136 | 137 | ```go 138 | eth := &Ethereum{ 139 | config: config, 140 | merger: merger, 141 | chainDb: chainDb, 142 | eventMux: stack.EventMux(), 143 | accountManager: stack.AccountManager(), 144 | engine: ethconfig.CreateConsensusEngine(stack, chainConfig, ðashConfig, config.Miner.Notify, config.Miner.Noverify, chainDb), 145 | closeBloomHandler: make(chan struct{}), 146 | networkID: config.NetworkId, 147 | gasPrice: config.Miner.GasPrice, 148 | etherbase: config.Miner.Etherbase, 149 | bloomRequests: make(chan chan *bloombits.Retrieval), 150 | bloomIndexer: core.NewBloomIndexer(chainDb, params.BloomBitsBlocks, params.BloomConfirms), 151 | p2pServer: stack.Server(), 152 | shutdownTracker: shutdowncheck.NewShutdownTracker(chainDb), 153 | } 154 | ``` 155 | 156 | `startNode()`函数的作用是正式的启动一个 Ethereum Node。它通过调用`utils.StartNode()`函数来触发`Node.Start()`函数来启动`Stack`实例（Node）。在`Node.Start()`函数中，会遍历`Node.lifecycles`中注册的后端实例，并在启动它们。此外，在`startNode()`函数中，还是调用了`unlockAccounts()`函数，并将解锁的钱包注册到`stack`中，以及通过`stack.Attach()`函数创建了与 local Geth 交互的 RPClient 模块。 157 | 158 | 在`geth()`函数的最后，函数通过执行`stack.Wait()`，使得主线程进入了阻塞状态，其他的功能模块的服务被分散到其他的子协程中进行维护。 159 | 160 | ### Node 161 | 162 | 正如我们前面提到的，Node 类型在 Geth 的生命周期性中属于顶级实例，它负责作为与外部通信的外部接口，比如管理 rpc server，http server，Web Socket，以及 P2P Server 外部接口。同时，Node 中维护了节点运行所需要的后端的实例和服务 (`lifecycles []Lifecycle`)。 163 | 164 | ```go 165 | // Node is a container on which services can be registered. 166 | type Node struct { 167 | eventmux *event.TypeMux 168 | config *Config 169 | accman *accounts.Manager 170 | log log.Logger 171 | keyDir string // key store directory 172 | keyDirTemp bool // If true, key directory will be removed by Stop 173 | dirLock fileutil.Releaser // prevents concurrent use of instance directory 174 | stop chan struct{} // Channel to wait for termination notifications 175 | server *p2p.Server // Currently running P2P networking layer 176 | startStopLock sync.Mutex // Start/Stop are protected by an additional lock 177 | state int // Tracks state of node lifecycle 178 | 179 | lock sync.Mutex 180 | lifecycles []Lifecycle // All registered backends, services, and auxiliary services that have a lifecycle 181 | rpcAPIs []rpc.API // List of APIs currently provided by the node 182 | http *httpServer // 183 | ws *httpServer // 184 | httpAuth *httpServer // 185 | wsAuth *httpServer // 186 | ipc *ipcServer // Stores information about the ipc http server 187 | inprocHandler *rpc.Server // In-process RPC request handler to process the API requests 188 | 189 | databases map[*closeTrackingDB]struct{} // All open databases 190 | } 191 | ``` 192 | 193 | #### Node 的关闭 194 | 195 | 在前面我们提到，整个程序的主线程因为调用了`stack.Wait()`而进入了阻塞状态。我们可以看到 Node 结构中声明了一个叫做`stop`的 channel。由于这个 Channel 一直没有被赋值，所以整个 Geth 的主进程才进入了阻塞状态，持续并发的执行其他的业务协程。 196 | 197 | ```go 198 | // Wait blocks until the node is closed. 199 | func (n *Node) Wait() { 200 | <-n.stop 201 | } 202 | ``` 203 | 204 | 当`n.stop`这个 Channel 被赋予值的时候，Geth 函数就会停止当前的阻塞状态，并开始执行相应的一系列的资源释放的操作。这个地方的写法还是非常有意思的，值得我们参考。我们为读者编写了一个简单的示例：如何使用 Channel 来管理 Go 程序的生命周期。 205 | 206 | 值得注意的是，在目前的 go-ethereum 的 codebase 中，并没有直接通过给`stop`这个 channel 赋值方式来结束主进程的阻塞状态，而是使用一种更简洁粗暴的方式：调用 close 函数直接关闭 Channel。我们可以在`node.doClose()`找到相关的实现。`close`是 go 语言的原生函数，用于关闭 Channel 时使用。 207 | 208 | ```go 209 | // doClose releases resources acquired by New(), collecting errors. 210 | func (n *Node) doClose(errs []error) error { 211 | // Close databases. This needs the lock because it needs to 212 | // synchronize with OpenDatabase*. 213 | n.lock.Lock() 214 | n.state = closedState 215 | errs = append(errs, n.closeDatabases()...) 216 | n.lock.Unlock() 217 | 218 | if err := n.accman.Close(); err != nil { 219 | errs = append(errs, err) 220 | } 221 | if n.keyDirTemp { 222 | if err := os.RemoveAll(n.keyDir); err != nil { 223 | errs = append(errs, err) 224 | } 225 | } 226 | 227 | // Release instance directory lock. 228 | n.closeDataDir() 229 | 230 | // Unblock n.Wait. 231 | close(n.stop) 232 | 233 | // Report any errors that might have occurred. 234 | switch len(errs) { 235 | case 0: 236 | return nil 237 | case 1: 238 | return errs[0] 239 | default: 240 | return fmt.Errorf("%v", errs) 241 | } 242 | } 243 | ``` 244 | 245 | ### Ethereum API Backend 246 | 247 | 我们可以在`eth/backend.go`中找到`Ethereum`这个结构体的定义。这个结构体包含的成员变量以及接收的方法实现了一个 Ethereum full node 所需要的全部功能和数据结构。我们可以在下面的代码定义中看到，Ethereum 结构体中包含了`TxPool`，`Blockchain`，`consensus.Engine`，`miner`等最核心的几个数据结构作为成员变量，我们会在后面的章节中详细的讲述这些核心数据结构的主要功能，以及它们的实现的方法。 248 | 249 | ```go 250 | // Ethereum implements the Ethereum full node service. 251 | type Ethereum struct { 252 | config *ethconfig.Config 253 | 254 | // Handlers 255 | txPool *core.TxPool 256 | blockchain *core.BlockChain 257 | handler *handler 258 | ethDialCandidates enode.Iterator 259 | snapDialCandidates enode.Iterator 260 | merger *consensus.Merger 261 | 262 | // DB interfaces 263 | chainDb ethdb.Database // Block chain database 264 | 265 | eventMux *event.TypeMux 266 | engine consensus.Engine 267 | accountManager *accounts.Manager 268 | 269 | bloomRequests chan chan *bloombits.Retrieval // Channel receiving bloom data retrieval requests 270 | bloomIndexer *core.ChainIndexer // Bloom indexer operating during block imports 271 | closeBloomHandler chan struct{} 272 | 273 | APIBackend *EthAPIBackend 274 | 275 | miner *miner.Miner 276 | gasPrice *big.Int 277 | etherbase common.Address 278 | 279 | networkID uint64 280 | netRPCService *ethapi.PublicNetAPI 281 | 282 | p2pServer *p2p.Server 283 | 284 | lock sync.RWMutex // Protects the variadic fields (e.g. gas price and etherbase) 285 | 286 | shutdownTracker *shutdowncheck.ShutdownTracker // Tracks if and when the node has shutdown ungracefully 287 | } 288 | 289 | ``` 290 | 291 | 节点启动和停止 Mining 的就是通过调用`Ethereum.StartMining()`和`Ethereum.StopMining()`实现的。设置 Mining 的收益账户是通过调用`Ethereum.SetEtherbase()`实现的。 292 | 293 | ```go 294 | // StartMining starts the miner with the given number of CPU threads. If mining 295 | // is already running, this method adjust the number of threads allowed to use 296 | // and updates the minimum price required by the transaction pool. 297 | func (s *Ethereum) StartMining(threads int) error { 298 | ... 299 | // If the miner was not running, initialize it 300 | if !s.IsMining() { 301 | ... 302 | // Start Mining 303 | go s.miner.Start(eb) 304 | } 305 | return nil 306 | } 307 | ``` 308 | 309 | 这里我们额外关注一下`handler`这个成员变量。`handler`的定义在`eth/handler.go`中。 310 | 311 | 我们从从宏观角度来看，一个节点的主工作流需要：1. 从网络中获取/同步 Transaction 和 Block 的数据 2. 将网络中获取到 Block 添加到 Blockchain 中。而`handler`就维护了 backend 中同步/请求数据的实例，比如`downloader.Downloader`，`fetcher.TxFetcher`。关于这些成员变量的具体实现，我们会在后续的文章中详细介绍。 312 | 313 | ```go 314 | type handler struct { 315 | networkID uint64 316 | forkFilter forkid.Filter // Fork ID filter, constant across the lifetime of the node 317 | 318 | snapSync uint32 // Flag whether snap sync is enabled (gets disabled if we already have blocks) 319 | acceptTxs uint32 // Flag whether we're considered synchronised (enables transaction processing) 320 | 321 | checkpointNumber uint64 // Block number for the sync progress validator to cross reference 322 | checkpointHash common.Hash // Block hash for the sync progress validator to cross reference 323 | 324 | database ethdb.Database 325 | txpool txPool 326 | chain *core.BlockChain 327 | maxPeers int 328 | 329 | downloader *downloader.Downloader 330 | blockFetcher *fetcher.BlockFetcher 331 | txFetcher *fetcher.TxFetcher 332 | peers *peerSet 333 | merger *consensus.Merger 334 | 335 | eventMux *event.TypeMux 336 | txsCh chan core.NewTxsEvent 337 | txsSub event.Subscription 338 | minedBlockSub *event.TypeMuxSubscription 339 | 340 | peerRequiredBlocks map[uint64]common.Hash 341 | 342 | // channels for fetcher, syncer, txsyncLoop 343 | quitSync chan struct{} 344 | 345 | chainSync *chainSyncer 346 | wg sync.WaitGroup 347 | peerWG sync.WaitGroup 348 | } 349 | ``` 350 | 351 | 这样，我们就介绍了 Geth 及其所需要的基本模块是如何启动的和关闭的。我们在接下来将视角转入到各个模块中，从更细粒度的角度深入 Ethereum 的实现。 352 | 353 | ### Related Terms 354 | 355 | - Geth 356 | - go-ethereum (geth) 357 | 358 | ### Appendix 359 | 360 | 这里补充一个 Go 语言的语法知识：**类型断言**。在`Ethereum.StartMining()`函数中，出现了`if c, ok := s.engine.(*clique.Clique); ok`的写法。这中写法是 Golang 中的语法糖，称为类型断言。具体的语法是`value, ok := element.(T)`，它的含义是如果`element`是`T`类型的话，那么 ok 等于`True`, `value`等于`element`的值。在`if c, ok := s.engine.(*clique.Clique); ok`语句中，就是在判断`s.engine`的是否为`*clique.Clique`类型。 361 | 362 | ```go 363 | var cli *clique.Clique 364 | if c, ok := s.engine.(*clique.Clique); ok { 365 | cli = c 366 | } else if cl, ok := s.engine.(*beacon.Beacon); ok { 367 | if c, ok := cl.InnerEngine().(*clique.Clique); ok { 368 | cli = c 369 | } 370 | } 371 | ``` 372 | -------------------------------------------------------------------------------- /CN/01_account_state.md: -------------------------------------------------------------------------------- 1 | # Account and Contract 2 | 3 | ## 概述 4 | 5 | 我们常常听到这样一个说法，"Ethereum 和 Bitcoin 最大的不同之一是二者使用管理链上数据的模型不同。其中，而 Bitcoin 是基于 UTXO 模型的 Blockchain/Ledger 系统，Ethereum 是基于 Account 模型的系统"。那么，这个另辟蹊径的 Account 模型究竟不同在何处呢？在本文中我们来探索一下以太坊中的基本数据单元 (Metadata) 之一的 Account。 6 | 7 | 简单的来说，Ethereum 依赖于一种*基于交易的状态机模型* (Transaction-based State Machine)。其中，状态 (State) 表示了某一实例 (instance) 在*某一时刻*下的值 (value)。在以太坊中，State 对应的基本数据结构，称为 StateObject。当 StateObject 的值发生了变化时，我们称为*状态转移*。在 Ethereum 的运行模型中，StateObject 所包含的数据会因为 Transaction 的执行引发数据更新/删除/创建，引发状态转移，我们说：StateObject 的状态从当前的 State 转移到另一个 State。 8 | 9 | 在 Ethereum 中，承载 StateObject 的具体实例就是 Ethereum 中的 Account。通常，我们提到的 State 具体指的就是 Account 在某个时刻的包含的数据的值。 10 | 11 | - Account --> StateObject 12 | - State --> The value/data of the Account 13 | 14 | 总的来说，Account （账户）是参与链上交易 (Transaction) 的基本角色，是 Ethereum 状态机模型中的基本单位，承担了链上交易的发起者以及接收者的角色。目前，在以太坊中，有两种类型的 Account，分别是外部账户 (EOA) 以及合约账户 (Contract)。 15 | 16 | ### EOA 17 | 18 | 外部账户 (EOA) 是由用户直接控制的账户，负责签名并发起交易 (Transaction)。用户通过 Account 的私钥来保证对账户数据的控制权。 19 | 20 | 合约账户 (Contract)，简称为合约，是由外部账户通过 Transaction 创建。合约账户，保存了**不可篡改的图灵完备的代码段**，以及保存一些**持久化的数据**。这些代码段使用专用语言书写 (Like: Solidity)，并通常提供一些对外部访问 API 接口函数。这些 API 接口可以通过 Transaction，或者通过本地/第三方提供的 RPC 服务来调用。这种模式构成了目前的 DApp 生态圈的基础。 21 | 22 | 通常，合约中的函数用于计算以及查询或修改合约中的持久化数据。我们经常看到这样的描述"**一旦被记录到区块链上数据不可被修改**，或者**不可篡改的智能合约**"。现在我们知道这种描述是不准确。针对一个链上的智能合约，不可修改/篡改的部分是合约中的代码段，或说是合约中的*函数逻辑*/*代码逻辑是*不可以被修改/篡改的。而链上合约中的持久化的数据是可以通过调用代码段中的函数进行数据操作的 (CURD)，包括修改和删除，具体取决于合约函数中的代码逻辑。 23 | 24 | 根据*合约中函数是否会修改合约中持久化的变量*，合约中的函数可以分为两种，只读函数和写函数。 25 | 如果用户**只**希望查询某些合约中的持久化数据，而不对数据进行修改的话，那么用户只需要调用相关的只读函数。调用只读函数不需要通过构造一个 Transaction 来查询数据。用户可以通过直接调用本地数据或者第三方提供的数据，来调用对应的函数。如果用户需要对合约中的数据进行更新，那么他就要构造一个 Transaction 来请求合约中相对应的鞋函数。注意，当用户通过构造 Transaction 的方式来调用合约中的函数时，每个 Transaction 只能调用一个合约中的一个 API 函数。 26 | 27 | 对于如何编写合约，以及 Ethereum 如何解析 Transaction 并调用对应的合约中 API 的，我们会在后面的文章中详细的进行解析。 28 | 29 | ## StateObject, Account, Contract 30 | 31 | 在实际代码中，这两种 Account 都是由`stateObject`这一数据结构定义的。`stateObject`的相关代码位于* core/state/state_object.go *文件中，隶属于* package state*。我们摘录了`stateObject`的结构代码，如下所示。通过下面的代码，我们可以观察到，`stateObject`是由小写字母开头。根据 go 语言的特性，我们可以知道这个结构主要用于 package 内部数据操作，并不对外暴露。 32 | 33 | ```go 34 | type stateObject struct { 35 | address common.Address 36 | addrHash common.Hash // hash of ethereum address of the account 37 | data types.StateAccount 38 | db *StateDB 39 | dbErr error 40 | 41 | // Write caches. 42 | trie Trie // storage trie, which becomes non-nil on first access 43 | code Code // contract bytecode, which gets set when code is loaded 44 | 45 | // 这里的 Storage 是一个 map[common.Hash]common.Hash 46 | originStorage Storage // Storage cache of original entries to dedup rewrites, reset for every transaction 47 | pendingStorage Storage // Storage entries that need to be flushed to disk, at the end of an entire block 48 | dirtyStorage Storage // Storage entries that have been modified in the current transaction execution 49 | fakeStorage Storage // Fake storage which constructed by caller for debugging purpose. 50 | 51 | // Cache flags. 52 | // When an object is marked suicided it will be delete from the trie 53 | // during the "update" phase of the state transition. 54 | dirtyCode bool // true if the code was updated 55 | suicided bool 56 | deleted bool 57 | } 58 | ``` 59 | 60 | ### Address 61 | 62 | 在`stateObject`这一结构体中，开头的两个成员变量为`address`以及 address 的哈希值`addrHash`。`address`是 common.Address 类型，`addrHash`是 common.Hash 类型，它们分别对应了一个 20 字节长度的 byte 数组和一个 32 字节长度的 byte 数组。关于这两种数据类型的定义如下所示。 63 | 64 | ```go 65 | // Lengths of hashes and addresses in bytes. 66 | const ( 67 | // HashLength is the expected length of the hash 68 | HashLength = 32 69 | // AddressLength is the expected length of the address 70 | AddressLength = 20 71 | ) 72 | // Address represents the 20 byte address of an Ethereum account. 73 | type Address [AddressLength]byte 74 | // Hash represents the 32 byte Keccak256 hash of arbitrary data. 75 | type Hash [HashLength]byte 76 | ``` 77 | 78 | 在 Ethereum 中，每个 Account 都拥有独一无二的地址。Address 作为每个 Account 的身份信息，类似于现实生活中的身份证，它与用户信息时刻绑定而且不能被修改。 79 | 80 | ### data and StateAccount 81 | 82 | 继续向下探索，我们会遇到成员变量 data，它是一个`types.StateAccount`类型的变量。在上面的分析中我们提到，`stateObject`这种类型只对 Package State 这个内部使用。所以相应的，Package State 也为外部 Package API 提供了与 Account 相关的数据类型"State Account"。在上面的代码中我们就可以看到，"State Account"对应了 State Object 中"data Account"成员变量。State Account 的具体数据结构的被定义在"core/types/state_account.go"文件中 (~~在之前的版本中 Account 的代码位于 core/account.go~~)，其定义如下所示。 83 | 84 | ```go 85 | // Account is the Ethereum consensus representation of accounts. 86 | // These objects are stored in the main account trie. 87 | type StateAccount struct { 88 | Nonce uint64 89 | Balance *big.Int 90 | Root common.Hash // merkle root of the storage trie 91 | CodeHash []byte 92 | } 93 | ``` 94 | 95 | 其中的包含四个变量为： 96 | 97 | - Nonce 表示该账户发送的交易序号，随着账户发送的交易数量的增加而单调增加。每次发送一个交易，Nonce 的值就会加 1。 98 | - Balance 表示该账户的余额。这里的余额指的是链上的 Global/Native Token Ether。 99 | - Root 表示当前账户的下 Storage 层的 Merkle Patricia Trie 的 Root。EOA 账户这个部分为空值。 100 | - CodeHash 是该账户的 Contract 代码的哈希值。EOA 账户这个部分为空值。 101 | 102 | ### db 103 | 104 | 上述的几个成员变量基本覆盖了 Account 主工作流相关的全部成员变量。那么我们继续向下看，会遇到`db`和`dbErr`这两个成员变量。db 这个变量保存了一个 StateDB 类型的指针。这是为了方便调用 StateDB 相关的 API 对 Account 所对应的 stateObject 进行操作。StateDB 本质上是 Ethereum 用于管理 stateObject 信息的而抽象出来的内存数据库。所有的 Account 数据的更新，检索都会使用 StateDB 提供的 API。关于 StateDB 的具体实现，功能，以及如何与更底层 (leveldb) 进行结合的，我们会在之后的文章中进行详细描述。 105 | 106 | ### Cache 107 | 108 | 对于剩下的成员变量，它们的主要用于内存 Cache。trie 用于保存 Contract 中的持久化存储的数据，code 用于缓存 contract 中的代码段到内存中，它是一个 byte 数组。剩下的四个 Storage 字段主要在执行 Transaction 的时候缓存 Contract 合约修改的持久化数据，比如 dirtyStorage 就用于缓存在 Block 被 Finalize 之前，Transaction 所修改的合约中的持久化存储数据。对于外部账户，由于没有代码字段，所以对应 stateObject 对象中的 code 字段，以及四个 Storage 类型的字段对应的变量的值都为空 (originStorage, pendingStorage, dirtyStorage, fakeStorage)。 109 | 110 | 从调用关系上看，这四个缓存变量的调用关系是 originStorage --> dirtyStorage--> pendingStorage。关于 Contract 的 Storage 层的详细信息，我们会在后面部分进行详细的描述。 111 | 112 | ## 深入 Account 113 | 114 | ### Private Key & Public Kay & Address 115 | 116 | #### 账户安全的问题 117 | 118 | 我们经常会在各种科技网站，自媒体上听到这样的说法，"用户在区块链系统中保存的 Cryptocurrency/Token，除了用户自己，不存在一个中心化的第三方可以不经过用户的允许转走你的财富"。这个说法基本是正确的。目前，用户账户里的由链级别定义 Crypto，或者称为原生货币 (Native Token)，比如 Ether，Bitcoin，BNB(Only in BSC)，是没办法被第三方在不被批准的情况下转走的。这是因为链级别上的所有数据的修改都要经过用户私钥 (Private Key) 签名的 Transaction。只要用户保管好自己账户的私钥 (Private Key)，保证其没有被第三方知晓，就没有人可以转走你链上的财富。 119 | 120 | 我们说上述说法是基本正确，而不是完全正确的原因有两个。首先，用户的链上数据安全是基于当前 Ethereum 使用的密码学工具足够保证：不存在第三方可以在**有限的时间**内在**不知道用户私钥的前提**下获取到用户的私钥信息来伪造签名交易。当然这个安全保证前提是当今 Ethereum 使用的密码学工具的强度足够大，没有计算机可以在有限的时间内 hack 出用户的私钥信息。在量子计算机出现之前，目前 Ethereum 和其他 Blockchain 使用的密码学工具的强度都是足够安全的。这也是为什么很多新的区块链项目在研究抗量子计算机密码体系的原因。第二点原因是，当今很多的所谓的 Crypto/Token 并不是链级别的数据，而是在链上合约中存储的数据，比如 ERC-20 Token 和 NFT 对应的 ERC-721 的 Token。由于这部分的 Token 都是基于合约代码生成和维护的，所以这部分 Token 的安全依赖于合约本身的安全。如果合约本身的代码是有问题的，存在后门或者漏洞，比如存在给第三方任意提取其他账户下 Token 的漏洞，那么即使用户的私钥信息没有泄漏，合约中的 Token 仍然可以被第三方获取到。由于合约的代码段在链上是不可修改的，合约代码的安全性是极其重要的。所以，有很多研究人员，技术团队在进行合约审计方面的工作，来保证上传的合约代码是安全的。此外随着 Layer-2 技术和一些跨链技术的发展，用户持有的“Token”，在很多情况下不是我们上面提到的安全的 Naive Token，而是 ERC-20 甚至只是其他合约中的简单数值记录。这种类型的资产的安全性是低于 layer-1 上的 Native Token 的。用户在持有这类资产的时候需要小心。这里我们推荐阅读 Jay Freeman 所分析的关于一个热门 Layer-2 系统 Optimism 上的由于非 Naive Token 造成的 [任意提取漏洞](https://www.saurik.com/optimism.html)。 121 | 122 | #### Account Generation 123 | 124 | 下面我们简单讲述，在 Ethereum 中一个账户的私钥和地址是如何产生的。 125 | 126 | - 首先我们通过随机得到一个长度 64 位 account 的私钥。这个私钥就是平时需要用户激活钱包时需要的记录，一旦这个私钥暴露了，钱包也将不再安全。 127 | - 64 个 16 进制位，256bit，32 字节 128 | `var AlicePrivateKey = "289c2857d4598e37fb9647507e47a309d6133539bf21a8b9cb6df88fd5232032"` 129 | 130 | - 在得到私钥后，我们使用私钥来计算公钥和 account 的地址。基于上述私钥，我们使用 ECDSA 算法，选择 spec256k1 曲线进行计算。通过将私钥带入到所选择的椭圆曲线中，计算出点的坐标即是公钥。以太坊和比特币使用了同样的 spec256k1 曲线，在实际的代码中，我们也可以看到在 crypto 中，go-Ethereum 直接调用了比特币的代码。 131 | `ecdsaSK, err := crypto.ToECDSA(privateKey)` 132 | 133 | - 对私钥进行椭圆加密之后，我们可以得到 64bytes 的数，它是由两个 32bytes 的数构成，这两个数代表了 spec256k1 曲线上某个点的 XY 值。 134 | `ecdsaPK := ecdsaSK.PublicKey` 135 | - 以太坊的地址，是基于上述公钥 (ecdsaSK.PublicKey) 的 [Keccak-256 算法] 之后的后 20 个字节，并且用 0x 开头。 136 | - Keccak-256 是 SHA-3（Secure Hash Algorithm 3）标准下的一种哈希算法 137 | `addr := crypto.PubkeyToAddress(ecdsaSK.PublicKey)` 138 | 139 | #### Signature & Verification 140 | 141 | - Hash（m,R）*X +R = S* P 142 | - P 是椭圆曲线函数的基点 (base point) 可以理解为一个 P 是一个在曲线 C 上的一个 order 为 n 的加法循环群的生成元。n 为质数。 143 | - R = r * P (r 是个随机数，并不告知 verifier) 144 | - 以太坊签名校验的核心思想是：首先基于上面得到的 ECDSA 下的私钥 ecdsaSK 对数据 msg 进行签名 (sign) 得到 msgSig. 145 | `sig, err := crypto.Sign(msg[:], ecdsaSK)` 146 | `msgSig := decodeHex(hex.EncodeToString(sig))` 147 | 148 | - 然后基于 msg 和 msgSig 可以反推出来签名的公钥（用于生成账户地址的公钥 ecdsaPK）。 149 | `recoveredPub, err := crypto.Ecrecover(msg[:],msgSig)` 150 | - 通过反推出来的公钥得到发送者的地址，并与当前 txn 的发送者在 ECDSA 下的 pk 进行对比。 151 | `crypto.VerifySignature(testPk, msg[:], msgSig[:len(msgSig)-1])` 152 | - 这套体系的安全性保证在于，即使知道了公钥 ecdsaPk/ecdsaSK.PublicKey 也难以推测出 ecdsaSK 以及生成他的 privateKey。 153 | 154 | #### ECDSA & spec256k1 曲线 155 | 156 | - Elliptic curve point multiplication 157 | - Point addition P + Q = R 158 | - Point doubling P + P = 2P 159 | - y^2 = x^3 +7 160 | - Based Point P 是在椭圆曲线上的群的生成元 161 | - x 次 computation on Based Point 得到 X 点，x 为私钥，X 为公钥。x 由 Account Private Key 得出。 162 | - 在 ECC 中的+号不是四则运算中的加法，而是定义椭圆曲线 C 上的新的二元运算 (Point Multiplication)。他代表了过两点 P 和 Q 的直线与椭圆曲线 C 的交点 R‘关于 X 轴对称的点 R。因为 C 是关于 X 轴对称的所以关于 X 对称的点也都在椭圆曲线上。 163 | 164 | ## 深入 Contract 165 | 166 | - 这部分的示例代码位于：[[example/signature](example/signature)] 中。 167 | 168 | ### Contract Storage （合约存储） 169 | 170 | [在文章的开头](#general Background) 我们提到，在外部账户对应的 stateObject 结构体的实例中，有四个 Storage 类型的变量是空值。那显然的，这四个变量是为 Contract 类型的账户准备的。 171 | 172 | 在* state_object.go *文件的开头部分 (41 行左右），我们可以找到 Storage 类型的定义。具体如下所示。 173 | 174 | ```go 175 | type Storage map[common.Hash]common.Hash 176 | ``` 177 | 178 | 我们可以看到，*Storage *是一个 key 和 value 都是`common.Hash`类型的 map 结构。common.Hash 类型，则对应了一个长度为 32bytes 的 byte 类型数组。这个类型在 go-ethereum 中被大量使用，通常用于表示 32 字节长度的数据，比如 Keccak256 函数的哈希值。在之后的旅程中，我们也会经常看到它的身影，它的定义在 common.type.go 文件中。 179 | 180 | ```go 181 | // HashLength is the expected length of the hash 182 | HashLength = 32 183 | // Hash represents the 32 byte Keccak256 hash of arbitrary data. 184 | type Hash [HashLength]byte 185 | ``` 186 | 187 | 从功能层面讲，外部账户 (EOA) 与合约账户 (Contract) 不同的点在于，外部账户并没有维护自己的代码 (codeHash) 以及额外的 Storage 层。相比与外部账户，合约账户额外保存了一个存储层 (Storage) 用于存储合约代码中持久化的变量的数据。在上文中我们提到，StateObject 中的声明的四个 Storage 类型的变量，就是作为 Contract Storage 层的内存缓存。 188 | 189 | 在 Ethereum 中，每个合约都维护了自己的*独立*的 Storage 空间，我们称为 Storage 层。Storage 层的基本组成单元称为槽 (Slot)，若干个 Slot 按照* Stack *的方式集合在一起构造成了 Storage 层。每个 Slot 的大小是 256 bits，也就是最多保存 32 bytes 的数据。作为基本的存储单元，Slot 管理的方式与内存或者 HDD 中的基本单元的管理方式类似，通过地址索引的方式被上层函数访问。Slot 的地址索引的长度同样是 32 bytes(256 bits)，寻址空间从 0x0000000000000000000000000000000000000000000000000000000000000000 到 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF。因此，每个 Contract 的 Storage 层最多可以保存$2^{256} - 1$个 Slot。也就说在理论状态下，一个 Contract 可以最多保存$(2^{256} - 1)$ bytes 的数据，这是个相当大的数字。Contract 同样使用 MPT 来管理 Storage 层的 Slot。值得注意的是，Storage 层的数据并不会被打包进入 Block 中。唯一与 Chain 内数据相关的是，Storage Trie 的根数据被保存在 StateAccount 结构体中的 Root 变量中（它是一个 32bytes 长的 byte 数组）。当某个 Contract 的 Storage 层的数据发生变化时，根据骨牌效应，向上传导到 World State Root 的值发生变化，从而影响到 Chain 数据。目前，Storage 层的数据读取和修改是在执行相关 Transaction 的时候，通过 EVM 调用两个专用的指令* OpSload *和* OpSstore *触发。 190 | 191 | 我们知道目前 Ethereum 中的大部分合约都通过 Solidity 语言编写。Solidity 做为强类型的图灵完备的语言，支持多种类型的变量。总的来说，根据变量的长度性质，Ethereum 中的持久化的变量可以分为定长的变量和不定长度的变量两种。定长的变量有常见的单变量类型，比如 uint256。不定长的变量包括了由若干单变量组成的 Array，以及 KV 形式的 Map 类型。 192 | 193 | 根据上面的介绍，我们了解到对 Contract Storage 层的访问是通过 Slot 的地址来进行的。请读者先思考下面的几个问题： 194 | 195 | - **如何给定一个包含若干持久化存储变量的 Solidity 的合约，EVM 是怎么给其包含的变量分配存储空间的呢？** 196 | - 怎么保证 Contract Storage 的一致性读写的？（怎么保证每个合约的验证者和执行者都能获取到相同的数据？) 197 | 198 | 我们将通过下面的一些实例来展示，在 Ethereum 中，Contract 是如何保存持久化变量的，以及保证所有的参与者都能一致性读写的 Contract 中的数据的。 199 | 200 | ### Contract Storage Example One 201 | 202 | 我们使用一个简单的合约来展示 Contract Storage 层的逻辑，合约代码如下所示。在本例中，我们使用了一个叫做"Storage"合约，其中定义了了三个持久化 uint256 类型的变量分别是 number, number1, 以及 number2。同时，我们定义一个 stores 函数给这个三个变量进行赋值。 203 | 204 | ```solidity 205 | // SPDX-License-Identifier: GPL-3.0 206 | 207 | pragma solidity >=0.7.0 <0.9.0; 208 | 209 | /** 210 | * @title Storage 211 | * @dev Store & retrieve value in a variable 212 | */ 213 | contract Storage { 214 | 215 | uint256 number; 216 | uint256 number1; 217 | uint256 number2; 218 | 219 | function stores(uint256 num) public { 220 | number = num; 221 | number1 = num + 1; 222 | number2 = num + 2; 223 | } 224 | 225 | function get_number() public view returns (uint256){ 226 | return number; 227 | } 228 | 229 | function get_number1() public view returns (uint256){ 230 | return number1; 231 | } 232 | 233 | function get_number2() public view returns (uint256){ 234 | return number2; 235 | } 236 | } 237 | ``` 238 | 239 | 我们使用 [Remix](https://remix.ethereum.org/) 来在本地部署这个合约，并构造一个调用 stores(1) 函数的 Transaction，同时使用 Remix debugger 来 Storage 层的变化。在 Transaction 生效之后，合约中三个变量的值将被分别赋给 1，2，3。此时，我们观察 Storage 层会发现，存储层增加了三个 Storage Object。这三个 Storage Object 对应了三个 Slot。所以在本例中，合约增加了三个 Slots 来存储数据。我们可以发现每个 Storage Object 由三个字段组成，分别是一个 32 bytes 的 key 字段和 32 bytess 的 value 字段，以及外层的一个 32 bytes 的字段。这三个字段在下面的例子中都表现为 64 位的 16 进制数 (32 Bytes)。 240 | 241 | 下面我们来逐个解释一下这个三个值的实际意义。首先我们观察内部的 Key-Value 对，可以发现下面三个 Storage Object 中 key 的值其实是从 0 开始的递增整数，分别是 0，1，2。它代表了当前 Slot 的地址索引值，或者说该 Slot 在 Storage 层对应的绝对位置 (Position)。比如，key 的值为 0 时，它代表整个 Storage 层中的第 1 个 Slot，或者说在 1 号位置的 Slot，当 key 等于 1 时代表 Storage 层中的第 2 个 Slot，以此类推。每个 Storage Object 中的 value 变量，存储了合约中三个变量的值 (1,2,3)。而 Storage Object 外层的值由等于 Storage Object 的 key 的值的 sha3 的哈希值。比如，下面例子中的第一个 Storage Object 的外层索引值"0x290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563" 是通过 keccak256(0) 计算出的值，代表了第一个 Slot position 的 Sha3 的哈希，而"0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6" 对应了是 keccak(1) 的值。我们在 [示例代码](../example/account/main.go) 中展示了如何计算的过程。 242 | 243 | ```json 244 | { 245 | "0x290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563": { 246 | "key": "0x0000000000000000000000000000000000000000000000000000000000000000", 247 | "value": "0x0000000000000000000000000000000000000000000000000000000000000001" 248 | }, 249 | "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6": { 250 | "key": "0x0000000000000000000000000000000000000000000000000000000000000001", 251 | "value": "0x0000000000000000000000000000000000000000000000000000000000000002" 252 | }, 253 | "0x405787fa12a823e0f2b7631cc41b3ba8828b3321ca811111fa75cd3aa3bb5ace": { 254 | "key": "0x0000000000000000000000000000000000000000000000000000000000000002", 255 | "value": "0x0000000000000000000000000000000000000000000000000000000000000003" 256 | } 257 | } 258 | ``` 259 | 260 | 读者可能以及发现了，在这个 Storage Object 中，外层的索引值其实与 Key 值的关系是一一对应的，或者说这两个键值本质上都是关于 Slot 位置的唯一索引。这里我们简单讲述一下这两个值在使用上的区别。Key 值代表了 Slot 在 Storage 层的 Position，这个值用于会作为 stateObject.go/getState() 以及 setState() 函数的参数，用于定位 Slot。如果我们继续深入上面的两个函数，我们就会发现，当内存中不存在该 Slot 的缓存时，geth 就会尝试从更底层的数据库中来获取这个 Slot 的值。而 Storage 在更底层的数据，是由 Secure Trie 来维护的，Secure Trie 中的 Key 值都是需要 Hash 的。所以在 Secure Trie 层我们查询/修改需要的键值就是外层的 hash 值。具体的关于 Secure Trie 的描述可以参考 [Trie](10_tire_statedb.md) 这一章节。总结下来，在上层函数 (stateObject) 调用中使用的键值是 Slot 的 Position，在下层的函数 (Trie) 调用中使用的键值是 Slot 的 Position 的哈希值。 261 | 262 | ```go 263 | func (t *SecureTrie) TryGet(key []byte) ([]byte, error) { 264 | // Secure Trie 中查询的例子 265 | // 这里的 key 还是 Slot 的 Position 266 | // 但是在更下层的 Call 更下层的函数的时候使用了这个 Key 的 hash 值作为查询使用的键值。 267 | return t.trie.TryGet(t.hashKey(key)) 268 | } 269 | ``` 270 | 271 | ### Account Storage Example Two 272 | 273 | 下面我们来看另外的一个例子。在这个例子中，我们调整一下合约中变量的声明顺序，从 (number，number1，number2) 调整为 (number 2, number 1, number)。合约代码如下所示。 274 | 275 | ```solidity 276 | // SPDX-License-Identifier: GPL-3.0 277 | 278 | pragma solidity >=0.7.0 <0.9.0; 279 | 280 | /** 281 | * @title Storage 282 | * @dev Store & retrieve value in a variable 283 | */ 284 | contract Storage { 285 | 286 | uint256 number2; 287 | uint256 number; 288 | 289 | function stores(uint256 num) public { 290 | number = num; 291 | number1 = num + 1; 292 | number2 = num + 2; 293 | } 294 | 295 | function get_number() public view returns (uint256){ 296 | return number; 297 | } 298 | 299 | function get_number1() public view returns (uint256){ 300 | return number1; 301 | } 302 | 303 | function get_number2() public view returns (uint256){ 304 | return number2; 305 | } 306 | } 307 | ``` 308 | 309 | 同样我们还是构造 Transaction 来调用合约中的 stores 函数。此时我们可以在 Storage 层观察到不一样的结果。我们发现 number2 这个变量的值被存储在了第一个 Slot 中（Key:"0x0000000000000000000000000000000000000000000000000000000000000000"），而 number 这个变量的值北存储在了第三个 Slot 中 (Key:"0x0000000000000000000000000000000000000000000000000000000000000002")。 310 | 311 | ```json 312 | { 313 | "0x290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563": { 314 | "key": "0x0000000000000000000000000000000000000000000000000000000000000000", 315 | "value": "0x0000000000000000000000000000000000000000000000000000000000000003" 316 | }, 317 | "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6": { 318 | "key": "0x0000000000000000000000000000000000000000000000000000000000000001", 319 | "value": "0x0000000000000000000000000000000000000000000000000000000000000002" 320 | }, 321 | "0x405787fa12a823e0f2b7631cc41b3ba8828b3321ca811111fa75cd3aa3bb5ace": { 322 | "key": "0x0000000000000000000000000000000000000000000000000000000000000002", 323 | "value": "0x0000000000000000000000000000000000000000000000000000000000000001" 324 | } 325 | } 326 | ``` 327 | 328 | 这个例子可以说明，在 Ethereum 中，变量对应的存储层的 Slot，是按照其在在合约中的声明顺序，从第一个 Slot（position：0）开始分配的。 329 | 330 | ### Account Storage Example Three 331 | 332 | 我们再考虑另一种情况：声明的三个变量，但只对其中的两个变量进行赋值。具体的来说，我们按照 number，number1，和 number2 的顺序声明三个 uint256 变量。但是，在函数 stores 中只对 number1 和 number2 进行赋值操作。合约代码如下所示。 333 | 334 | ```solidity 335 | // SPDX-License-Identifier: GPL-3.0 336 | 337 | pragma solidity >=0.7.0 <0.9.0; 338 | 339 | /** 340 | * @title Storage 341 | * @dev Store & retrieve value in a variable 342 | */ 343 | contract Storage { 344 | uint256 number; 345 | uint256 number1; 346 | uint256 number2; 347 | 348 | function stores(uint256 num) public { 349 | number1 = num + 1; 350 | number2 = num + 2; 351 | } 352 | 353 | function get_number() public view returns (uint256){ 354 | return number; 355 | } 356 | 357 | function get_number1() public view returns (uint256){ 358 | return number1; 359 | } 360 | 361 | function get_number2() public view returns (uint256){ 362 | return number2; 363 | } 364 | } 365 | ``` 366 | 367 | 基于上述合约，我们构造 transaction 并调用 stores 函数，输入参数 1，将 number1 和 number2 的值修改为 2，和 3。在 transaction 执行完成后，我们可以观察到 Storage 层 Slot 的结果如下所示。 368 | 369 | ```json 370 | { 371 | "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6": { 372 | "key": "0x0000000000000000000000000000000000000000000000000000000000000001", 373 | "value": "0x0000000000000000000000000000000000000000000000000000000000000002" 374 | }, 375 | "0x405787fa12a823e0f2b7631cc41b3ba8828b3321ca811111fa75cd3aa3bb5ace": { 376 | "key": "0x0000000000000000000000000000000000000000000000000000000000000002", 377 | "value": "0x0000000000000000000000000000000000000000000000000000000000000003" 378 | } 379 | } 380 | ``` 381 | 382 | 我们可以观察到，stores 函数调用的结果只对在合约的 Storage 层中位置在 1 和 2 位置的两个 Slot 进行了赋值。值得注意的是，在本例中，对于 Slot 的赋值是从 1 号位置 Slot 的开始，而不是 0 号 Slot。这说明对于固定长度的变量，其值的所占用的 Slot 的位置在 Contract 初始化开始的时候就已经分配的。即使变量只是被声明还没有真正的赋值，保存其值所需要的 Slot 也已经被 EVM 分配完毕。而不是在第一次进行变量赋值的时候，进行再对变量所需要的的 Slot 进行分配。 383 | 384 | ![Remix Debugger](../figs/01/remix.png) 385 | 386 | ### Account Storage Example Four 387 | 388 | 在 Solidity 中，有一类特殊的变量类型** Address**，通常用于表示账户的地址信息。例如在 ERC-20 合约中，用户拥有的 token 信息是被存储在一个 (address->uint) 的 map 结构中。在这个 map 中，key 就是 Address 类型的，它表示了用户实际的 address。目前 Address 的大小为 160bits(20bytes)，并不足以填满一整个 Slot。因此当 Address 作为 value 单独存储在的时候，它并不会排他的独占用一个 Slot。我们使用下面的例子来说明。 389 | 390 | 在下面的示例中，我们声明了三个变量，分别是 number(uint256)，addr(address)，以及 isTrue(bool)。我们知道，在以太坊中 Address 类型变量的长度是 20 bytes，所以一个 Address 类型的变量是没办法填满整个的 Slot(32 bytes) 的。同时，布尔类型在以太坊中只需要一个 bit(0 or 1) 的空间。因此，我们构造 transaction 并调用函数 storeaddr 来给这三个变量赋值，函数的 input 参数是一个 uint256 的值，一个 address 类型的值，分别为{1, “0xb6186d3a3D32232BB21E87A33a4E176853a49d12”}。 391 | 392 | ```solidity 393 | // SPDX-License-Identifier: GPL-3.0 394 | 395 | pragma solidity >=0.7.0 <0.9.0; 396 | 397 | /** 398 | * @title Storage 399 | * @dev Store & retrieve value in a variable 400 | */ 401 | contract Storage { 402 | 403 | uint256 number; 404 | address addr; 405 | bool isTrue; 406 | 407 | function stores(uint256 num) public { 408 | // number1 = num + 1; 409 | // number2 = num + 2; 410 | } 411 | 412 | function storeaddr(uint256 num, address a) public { 413 | number = num; 414 | addr = a; 415 | isTure = true; 416 | } 417 | 418 | function get_number() public view returns (uint256){ 419 | return number; 420 | } 421 | 422 | } 423 | ``` 424 | 425 | Transaction 的运行后 Storage 层的结果如下面的 Json 所示。我们可以观察到，在本例中 Contract 声明了三个变量，但是在 Storage 层只调用了两个 Slot。第一个 Slot 用于保存了 uint256 的值，而在第二个 Slot 中 (Key:0x0000000000000000000000000000000000000000000000000000000000000001) 保存了 addr 和 isTrue 的值。这里需要注意，虽然这种将两个小于 32 bytes 长的变量合并到一个 Slot 的做法节省了物理空间，但是也同样带来读写放大的问题。因为在 Geth 中，读操作最小的读的单位都是按照 32bytes 来进行的。在本例中，即使我们只需要读取 isTrue 或者 addr 这两个变量的值，在具体的函数调用中，我们仍然需要将对应的 Slot 先读取到内存中。同样的，如果我们想修改这两个变量的值，同样需要对整个的 Slot 进行重写。这无疑增加了额外的开销。所以在 Ethereum 使用 32 bytes 的变量，在某些情况下消耗的 Gas 反而比更小长度类型的变量要小（例如 unit8)。这也是为什么 Ethereum 官方也建议使用长度为 32 bytes 变量的原因。 426 | 427 | // Todo Gas cost? here or in EVM Section 428 | 429 | ```json 430 | { 431 | "0x290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563": { 432 | "key": "0x0000000000000000000000000000000000000000000000000000000000000000", 433 | "value": "0x0000000000000000000000000000000000000000000000000000000000000001" 434 | }, 435 | "0xb10e2d527612073b26eecdfd717e6a320cf44b4afac2b0732d9fcbe2b7fa0cf6": { 436 | "key": "0x0000000000000000000000000000000000000000000000000000000000000001", 437 | "value": "0x000000000000000000000001b6186d3a3d32232bb21e87a33a4e176853a49d12" 438 | } 439 | } 440 | ``` 441 | 442 | ### Account Storage Example Five 443 | 444 | 对于变长数组和 Map 结构的变量存储分配则相对的复杂。虽然 Map 本身就是 key-value 的结构，但是在 Storage 层并不直接使用 map 中 key 的值或者 key 的值的 sha3 哈希值来作为 Storage 分配的 Slot 的索引值。目前，Geth 首先会使用 map 中元素的 key 的值和当前 Map 变量声明位置对应的 slot 的值进行拼接，再使用拼接后的值的 keccak256 哈希值作为 Slot 的位置索引 (Position)。我们在下面的例子中展示了 Geth 是如何处理 map 这种变长的数据结构的。在下面的合约中，我们声明了一个定长的 uint256 类型的对象 number，和一个 [string=>uint256] 类型的 Map 对象。 445 | 446 | 447 | 448 | ```solidity 449 | // SPDX-License-Identifier: GPL-3.0 450 | 451 | pragma solidity >=0.7.0 <0.9.0; 452 | 453 | /** 454 | * @title Storage 455 | * @dev Store & retrieve value in a variable 456 | */ 457 | contract Storage { 458 | 459 | uint256 number; 460 | 461 | mapping(string => uint256) balances; 462 | 463 | function set_balance(uint256 num) public { 464 | number = num; 465 | balances["hsy"] = num; 466 | balances["lei"] = num + 1; 467 | } 468 | 469 | function get_number() public view returns (uint256){ 470 | return number; 471 | } 472 | 473 | } 474 | ``` 475 | 476 | 我们构造一个 Transaction 来调用`set_balance`函数。在 Transaction 执行之后的 Storage 层的结果如下面的 Json 所示。我们发现，对于定长的变量 number 占据了第一个 Slot 的空间 (Position:0x0000000000000000000000000000000000000000000000000000000000000000)。但是对于 Map 类型变量 balances，它包含的两个数据并没有按照变量定义的物理顺序来定义 Slot。此外，我们观察到存储这两个值的 Slot 的 key，也并不是这两个字在 mapping 中 key 的直接 hash。正如我们在上段中提到的那样，Geth 会使用 Map 中元素的的 key 值与当前 Map 被分配的 slot 的位置进行拼接，之后对拼接之后对值进行使用 keccak256 函数求得哈希值，来最终得到 map 中元素最终的存储位置。比如在本例中，按照变量定义的顺序，balances 这个 Map 变量会被分配到第二个 Slot，对应的 Slot Position 是 1。因此，balances 中的 kv 对分配到的 slot 的位置就是，keccak(key, 1)，这里是一个特殊的拼接操作。 477 | 478 | ```json 479 | { 480 | "0x290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563": { 481 | "key": "0x0000000000000000000000000000000000000000000000000000000000000000", 482 | "value": "0x0000000000000000000000000000000000000000000000000000000000000001" 483 | }, 484 | "0xa601d8e9cd2719ca27765dc16042655548d1ac3600a53ffc06b4a06a12b7c65c": { 485 | "key": "0xbaded3bf529b04b554de2e4ee0f5702613335896b4041c50a5555b2d5e279f91", 486 | "value": "0x0000000000000000000000000000000000000000000000000000000000000001" 487 | }, 488 | "0x53ac6681d92653b13055d2e265b672e2db2b2a19407afb633928597f144edbb0": { 489 | "key": "0x56a8a0d158d59e2fd9317c46c65b1e902ed92f726ecfe82c06c33c015e8e6682", 490 | "value": "0x0000000000000000000000000000000000000000000000000000000000000002" 491 | } 492 | } 493 | ``` 494 | 495 | 为了验证上面的说法，我们使用 go 语言编写了一段代码，来调用相关的库来验证一下上面的结论。对于 balances["hsy"]，它被分配的 Slot 的位置可以由下面的代码求得。读者可以阅读/使用 [示例代码](../example/account/main.go) 进行尝试。这里的 k1 是一个整形实数，代表了 Slot 的在 storage 层的位置 (Position)。 496 | 497 | ```go 498 | k1 := solsha3.SoliditySHA3([]byte("hsy"), solsha3.Uint256(big.NewInt(int64(1)))) 499 | fmt.Printf("Test the Solidity Map storage Key1: 0x%x\n", k1) 500 | ``` 501 | 502 | // TODO: The wallet part. Not sure in this section or another section. 503 | 504 | ## Wallet 505 | 506 | - KeyStore 507 | - Private Key 508 | - 助记词 509 | 510 | ## Reference 511 | 512 | - 513 | -------------------------------------------------------------------------------- /CN/02_state_management_statedb.md: -------------------------------------------------------------------------------- 1 | # State Management (1) : StateDB 2 | 3 | ## General 4 | 5 | 在本文中，我们剖析一下Ethereum State 管理模块中最重要的几个数据结构，StateDB, Trie，Secure Trie，以及StackTrie。我们讲通过分析Ethereum中的主workflow的方式来深入理解这三个数据结构的使用场景，以及设计上的不同。 6 | 7 | 首先，StateDB是这三个数据结构中最高层的封装，它是直接提供了与StateObject (Account，Contract)相关的CURD的接口给其他的模块，比如： 8 | 9 | - Mining 模块，执行新Blockchain中的交易形成新的world state。 10 | - Block同步模块，执行新Blockchain中的交易形成新的world state，与header中的state root进行比较验证。 11 | - EVM中的两个与Contract中的持久化存储相关的两个opcode, sStore, sSload. 12 | 13 | ## StateDB 14 | 15 | 我们可以在genesis block创建的相关代码中，找到直接相关的例子。 16 | 17 | ```go 18 | statedb.Commit(false) 19 | statedb.Database().TrieDB().Commit(root, true, nil) 20 | ``` 21 | 22 | 具体World State的更新顺序是: 23 | 24 | ```mermaid 25 | flowchart LR 26 | StateDB --> Memory_Trie_Database --> LevelDB 27 | ``` 28 | 29 | StateDB 调用Commit的时候并没有同时触发TrieDB的Commit。 30 | 31 | 在Block被插入到Blockchain的这个Workflow中，stateDB的commit首先在`writeBlockWithState`函数中被调用了。之后`writeBlockWithState`函数会判断GC的状态来决定在本次调用中，是否需要向Disk Database写入数据。 32 | 33 | ### From Memory to Disk 34 | 35 | 当新的Block被添加到Blockchain时，State的数据并不一会立即被写入到Disk Database中。在`writeBlockWithState`函数中，函数会判断gc条件，只有满足一定的条件，才会在此刻调用TrieDB中的Cap或者Commit函数将数据写入Disk Database中。 36 | -------------------------------------------------------------------------------- /CN/03_state_management_stateTrie.md: -------------------------------------------------------------------------------- 1 | # State Management (2) : World State Trie and Storage Trie 2 | 3 | ## Trie in brief 4 | 5 | Trie结构是Ethereum中用于管理数据的基本数据结构，它被广泛的运用在Ethereum里的多个模块中，包括管理全局的World State Trie，管理Contract中持久化存储Key-Value 对的Storage Trie，以及每个Block中的Transaction Trie 和 Receipt Trie。 6 | 7 | 在以太坊的体系中，广义上的Trie的指的是Merkel Patricia Trie(MPT)这种数据结构。在实际的实现中，根据业务功能的不同，在go-ethereum中一共实现了三种不同的MPT的instance，分别是，`Trie`，`Secure Trie`以及`Stack Trie`. 8 | 9 | 10 | 11 | 从调用关系上看`Trie`是最底层的核心结构，它用于之间负责StateObject数据的保存，以及提供相应的CURD函数。它的定义在trie/trie.go文件中。 12 | 13 | Secure Trie结构本质上是对Trie的一层封装。它具体的CURD操作的实现都是通过`Trie`中定义的函数来执行的。它的定义在`trie/secure_trie.go`文件中。目前StateDB中的使用的Trie是经过封装之后的Secure Trie。这个Trie也就是我们常说的World State Trie，它是唯一的一个全局Trie结构。与Trie不同的是，Secure Trie要求新加入的Key-Value pair中的Key的数据都是Sha过的。这是为了方式恶意的构造Key来增加MPT的高度。 14 | 15 | ```go 16 | type SecureTrie struct { 17 | trie Trie 18 | hashKeyBuf [common.HashLength]byte 19 | secKeyCache map[string][]byte 20 | secKeyCacheOwner *SecureTrie // Pointer to self, replace the key cache on mismatch 21 | } 22 | ``` 23 | 24 | 不管是Secure Trie还是Trie，他们的创建的前提是:更下层的db的实例已经创建成功了，否则就会报错。 25 | 26 | 值得注意的是一个关键函数Prove的实现，并不在这两个Trie的定义文件中，而是位于`trie/proof.go`文件中。 27 | 28 | ## Trie Operations 29 | 30 | ### Read Operation 31 | 32 | ### Insert 33 | 34 | ```go 35 | func (t *Trie) insert(n node, prefix, key []byte, value node) (bool, node, error) { 36 | fmt.Println("Out n:", &n) 37 | if len(key) == 0 { 38 | if v, ok := n.(valueNode); ok { 39 | return !bytes.Equal(v, value.(valueNode)), value, nil 40 | } 41 | return true, value, nil 42 | } 43 | switch n := n.(type) { 44 | case *shortNode: 45 | matchlen := prefixLen(key, n.Key) 46 | // If the whole key matches, keep this short node as is 47 | if matchlen == len(n.Key) { 48 | dirty, nn, err := t.insert(n.Val, append(prefix, key[:matchlen]...), key[matchlen:], value) 49 | if !dirty || err != nil { 50 | return false, n, err 51 | } 52 | return true, &shortNode{n.Key, nn, t.newFlag()}, nil 53 | } 54 | // Otherwise branch out at the index where they differ. 55 | branch := &fullNode{flags: t.newFlag()} 56 | var err error 57 | _, branch.Children[n.Key[matchlen]], err = t.insert(nil, append(prefix, n.Key[:matchlen+1]...), n.Key[matchlen+1:], n.Val) 58 | if err != nil { 59 | return false, nil, err 60 | } 61 | _, branch.Children[key[matchlen]], err = t.insert(nil, append(prefix, key[:matchlen+1]...), key[matchlen+1:], value) 62 | if err != nil { 63 | return false, nil, err 64 | } 65 | // Replace this shortNode with the branch if it occurs at index 0. 66 | if matchlen == 0 { 67 | return true, branch, nil 68 | } 69 | // Otherwise, replace it with a short node leading up to the branch. 70 | return true, &shortNode{key[:matchlen], branch, t.newFlag()}, nil 71 | 72 | case *fullNode: 73 | dirty, nn, err := t.insert(n.Children[key[0]], append(prefix, key[0]), key[1:], value) 74 | if !dirty || err != nil { 75 | return false, n, err 76 | } 77 | n = n.copy() 78 | n.flags = t.newFlag() 79 | n.Children[key[0]] = nn 80 | return true, n, nil 81 | 82 | case nil: 83 | return true, &shortNode{key, value, t.newFlag()}, nil 84 | 85 | case hashNode: 86 | // We've hit a part of the trie that isn't loaded yet. Load 87 | // the node and insert into it. This leaves all child nodes on 88 | // the path to the value in the trie. 89 | rn, err := t.resolveHash(n, prefix) 90 | if err != nil { 91 | return false, nil, err 92 | } 93 | dirty, nn, err := t.insert(rn, prefix, key, value) 94 | if !dirty || err != nil { 95 | return false, rn, err 96 | } 97 | return true, nn, nil 98 | 99 | default: 100 | panic(fmt.Sprintf("%T: invalid node: %v", n, n)) 101 | } 102 | } 103 | ``` 104 | 105 | 这里有一个关于go语言的知识。我们可以观察到insert函数的第一个参数是一个变量名为n的node类型的变量。有趣的是，在switch语句中我们看到了一个这样的写法. 106 | 107 | ```go 108 | switch n := n.(type) 109 | ``` 110 | 111 | 显然语句两端的*n*的含义并不相同。这种写法在go中是合法的。 112 | 113 | ### Update 114 | 115 | ### Delete 116 | 117 | ### Finalize And Commit and Commit to Disk 118 | 119 | - 在leveldb中保存的是Trie中的节点。 120 | - 121 | 122 | 在Geth中，Trie并不是实时更新的，而是依赖于Committer和Database两个额外的辅助单位。 123 | 124 | ```mermaid 125 | flowchart LR 126 | Trie.Commit --> Committer.Commit --> trie/Database.insert 127 | ``` 128 | 129 | 我们可以看到Trie的Commit并不会真的对Disk Database的值进行修改。 130 | 131 | Trie真正更新到Disk Database的，是依赖于`trie/Database.Commit`函数的调用。我们可以在诸多函数中找到这个函数的调用比如。 132 | 133 | ```go 134 | func GenerateChain(config *params.ChainConfig, parent *types.Block, engine consensus.Engine, db ethdb.Database, n int, gen func(int, *BlockGen)) ([]*types.Block, []types.Receipts) { 135 | ... 136 | // Write state changes to db 137 | root, err := statedb.Commit(config.IsEIP158(b.header.Number)) 138 | if err != nil { 139 | panic(fmt.Sprintf("state write error: %v", err)) 140 | } 141 | if err := statedb.Database().TrieDB().Commit(root, false, nil); err != nil { 142 | panic(fmt.Sprintf("trie write error: %v", err)) 143 | } 144 | ... 145 | } 146 | ``` 147 | 148 | ## StackTrie 149 | 150 | StackTrie虽然也是MPT结构，但是它与另外的两个Trie最大的不同在于，其主要作用不是用于存储数据，而是用于给一组数据生成证明。比如，在Block中的Transaction Hash以及Receipt Hash都是基于StackTrie生成的。这里我们使用一个更直观的例子。这个部分的代码位于*core/block_validator.go*中。在block_validator中定义了一系列验证用的函数, 比如`ValidateBody`和 `ValidateState`函数。我们选取了这两个函数的其中一部分，如下所示。为了验证Block的合法性，ValidateBody和ValidateState函数分别在本地基于Block中提供的数据来构造Transaction和Receipt的哈希来与Header中的TxHash与ReceiptHash。我们可以发现，函数`types.DeriveSha`需要一个`TrieHasher`类型的参数。但是在具体调用的时候，却传入了了一个`trie.NewStackTrie`类型的变量。这是因为StackTrie实现了TrieHasher接口所需要的三个函数，所以这种调用是合法的。我们可以在*core/types/hashing.go*中找到TrieHasher的定义。这里`DeriveSha`不断的向StackTrie中添加数据，并最终返回StackTrie的Root哈希值。 151 | 152 | 同时，我们可以发现，在调用DeriveSha函数的时候，我们每次都会new一个新的StackTrie作为参数。这也反映出了，StackTrie的主要作用就是生成验证用的Proof，而不是存储数据。 153 | 154 | ```golang 155 | func (v *BlockValidator) ValidateBody(block *types.Block) error { 156 | ... 157 | if hash := types.DeriveSha(block.Transactions(), trie.NewStackTrie(nil)); hash != header.TxHash { 158 | return fmt.Errorf("transaction root hash mismatch: have %x, want %x", hash, header.TxHash) 159 | } 160 | ... 161 | } 162 | ``` 163 | 164 | ```golang 165 | func (v *BlockValidator) ValidateState(block *types.Block, statedb *state.StateDB, receipts types.Receipts, usedGas uint64) error { 166 | ... 167 | // Tre receipt Trie's root (R = (Tr [[H1, R1], ... [Hn, Rn]])) 168 | receiptSha := types.DeriveSha(receipts, trie.NewStackTrie(nil)) 169 | if receiptSha != header.ReceiptHash { 170 | return fmt.Errorf("invalid receipt root hash (remote: %x local: %x)", header.ReceiptHash, receiptSha) 171 | } 172 | ... 173 | } 174 | 175 | ``` 176 | 177 | ### FQA 178 | 179 | #### State Trie的更新是什么时候发生的？ 180 | 181 | State Trie的更新，通常是指的是基于State Trie中节点值的变化从而重新计算State Trie的Root的Hash值的过程。目前这一过程是通过调用StateDB中的`IntermediateRoot`函数来完成的。 182 | 183 | 我们从三个粒度层面来看待State Trie更新的问题。 184 | 185 | - Block 层。 186 | 在一个新的Block Insert到Blockchain的过程中，State Trie可能会发生多次的更新。比如，在每次Transaction被执行之后，`IntermediateRoot`函数都会被调用。同时，更新后的 State Trie的Root值，会被写入到Transaction对应的Receipt中。请注意，在调用`IntermediateRoot`函数时，更新后的值在此时并没有被立刻写入到Disk Database中。此时的State Trie Root只是基于内存中的数据计算出来的。真正的Trie数据写盘，需要等到`trieDB.Commit`函数的执行。 187 | - Transaction 层。 188 | 如上面提到的，在每次Transaction执行完成后，系统都会调用一次StateDB的`IntermediateRoot`函数，来更新State Trie。并且会将更新后的Trie的Root Hash写入到该Transaction对应的Receipt中。这里提一下关于`IntermediateRoot`函数细节。在IntermediateRoot`函数调用时，会首先更新被修改的Contract的Storage Trie的Root。 189 | - Instruction 层。 190 | 执行Contract的Instruction，并不会直接的引发State Trie的更新。比如，我们知道，EVM指令`OpSstore`会修改Contract中的持久化存储。这个指令调用了StateDB中的`SetState`函数，并最终调用了对应的StateObject中的`setState`函数。StateObject中的`setState` 函数并没有直接对Contract的Storage Trie进行更新，而是将修改的存储对象保存在了StateObject中的*dirtyStorage* 中(*dirtyStorage*是用于缓存Storage Slot数据的Key-Value Map). Storage Trie的更新是由更上层的函数调用所触发的，比如`IntermediateRoot`函数，以及`StateDB.Commit`函数。 191 | 192 | ## Reference 193 | 194 | - [1] 195 | -------------------------------------------------------------------------------- /CN/04_transaction.md: -------------------------------------------------------------------------------- 1 | # Transaction: 一个Transaction的生老病死 2 | 3 | ## 概述 4 | 5 | 我们知道，Ethereum的基本模型是基于交易的状态机模型(Transaction-based State Machine)。在[Account章节](./01_account.md)我们简述了一下Account/Contract的基本数据结构。本章我们就来探索一下，Ethereum中的一个基本数据结构Transaction。在本文中，我们提到的交易指的是在Ethereum Layer-1层面上构造的交易，以太坊生态中的Layer-2中的交易不在我们的讨论中。 6 | 7 | 首先，Transaction是Ethereum执行数据操作的媒介，它主要起到下面的几个作用: 8 | 9 | 1. 在Layer-1网络上的Account之间进行Native Token的转账。 10 | 2. 创建新的Contract。 11 | 3. 调用Contract中会修改目标Contract中持久化数据或者间接修改其他Account/Contract数据的函数。 12 | 13 | 这里我们对Transaction的功能性的细节再进行一些额外的补充。首先，Transaction只能创建Contract账户，而不能用于创建外部账户(EOA)。第二，如果调用Contract中的只读函数，是不需要构造Transaction的。相对的，所有参与Account/Contract数据修改的操作都需要通过Transaction来进行。第三，广义上的Transaction只能由外部账户(EOA)构建。Contract是没有办法显式构造Layer-1层面的交易的。在某些合约函数的执行过程中，Contract在可以通过构造internal transaction来与其他的合约进行交互，但是这种Internal transaction与我们提到的Layer-1层面的交易有所不同，我们会在之后的章节介绍。 14 | 15 | ## LegacyTx & AccessListTX & DynamicFeeTx 16 | 17 | 下面我们根据源代码中的定义来了解一下Transaction的数据结构。Transaction结构体的定义位于*core/types/transaction.go*中。Transaction的结构体如下所示。 18 | 19 | ```go 20 | type Transaction struct { 21 | inner TxData // Consensus contents of a transaction 22 | time time.Time // Time first seen locally (spam avoidance) 23 | 24 | // caches 25 | hash atomic.Value 26 | size atomic.Value 27 | from atomic.Value 28 | } 29 | ``` 30 | 31 | 从代码定义中我们可以看到，`Transaction`的结构体是非常简单的，它只包含了五个变量分别是, `TxData`类型的inner，`Time`类型的time，以及三个`atomic.Value`类型的hash，size，以及from。这里我们需要重点关注一下`inner`这个变量。目前与Transaction直接相关的数据都由这个变量来维护。 32 | 33 | 目前，`TxData`类型是一个接口，它的定义如下面的代码所示。 34 | 35 | ```go 36 | type TxData interface { 37 | txType() byte // returns the type ID 38 | copy() TxData // creates a deep copy and initializes all fields 39 | 40 | chainID() *big.Int 41 | accessList() AccessList 42 | data() []byte 43 | gas() uint64 44 | gasPrice() *big.Int 45 | gasTipCap() *big.Int 46 | gasFeeCap() *big.Int 47 | value() *big.Int 48 | nonce() uint64 49 | to() *common.Address 50 | 51 | rawSignatureValues() (v, r, s *big.Int) 52 | setSignatureValues(chainID, v, r, s *big.Int) 53 | } 54 | ``` 55 | 56 | 这里注意，在目前版本的geth中(1.10.*)，根据[EIP-2718][EIP2718]的设计，原来的TxData现在被声明成了一个interface，而不是定义了具体的结构。这样的设计好处在于，后续版本的更新中可以对Transaction类型进行更加灵活的修改。目前，在Ethereum中定义了三种类型的Transaction来实现TxData这个接口。按照时间上的定义顺序来说，这三种类型的Transaction分别是，LegacyT，AccessListTx，TxDynamicFeeTx。LegacyTx顾名思义，是原始的Ethereum的Transaction设计，目前市面上大部分早年关于Ethereum Transaction结构的文档实际上都是在描述LegacyTx的结构。而AccessListTX是基于EIP-2930(Berlin分叉)的Transaction。DynamicFeeTx是[EIP-1559](https://eips.ethereum.org/EIPS/eip-1559)(伦敦分叉)生效之后的默认的Transaction。 57 | 58 | (PS:目前Ethereum的黄皮书只更新到了Berlin分叉的内容，还没有添加London分叉的更新, 2022.3.10) 59 | 60 | ### LegacyTx 61 | 62 | ```go 63 | type LegacyTx struct { 64 | Nonce uint64 // nonce of sender account 65 | GasPrice *big.Int // wei per gas 66 | Gas uint64 // gas limit 67 | To *common.Address `rlp:"nil"` // nil means contract creation 68 | Value *big.Int // wei amount 69 | Data []byte // contract invocation input data 70 | V, R, S *big.Int // signature values 71 | } 72 | ``` 73 | 74 | ### AccessListTX 75 | 76 | ```go 77 | type AccessListTx struct { 78 | ChainID *big.Int // destination chain ID 79 | Nonce uint64 // nonce of sender account 80 | GasPrice *big.Int // wei per gas 81 | Gas uint64 // gas limit 82 | To *common.Address `rlp:"nil"` // nil means contract creation 83 | Value *big.Int // wei amount 84 | Data []byte // contract invocation input data 85 | AccessList AccessList // EIP-2930 access list 86 | V, R, S *big.Int // signature values 87 | } 88 | ``` 89 | 90 | ### DynamicFeeTx 91 | 92 | 如果我们观察DynamicFeeTx就会发现，DynamicFeeTx的定义其实就是在LegacyTx/AccessListTX的定义的基础上额外的增加了GasTipCap与GasFeeCap这两个字段。 93 | 94 | ```go 95 | type DynamicFeeTx struct { 96 | ChainID *big.Int 97 | Nonce uint64 98 | GasTipCap *big.Int // a.k.a. maxPriorityFeePerGas 99 | GasFeeCap *big.Int // a.k.a. maxFeePerGas 100 | Gas uint64 101 | To *common.Address `rlp:"nil"` // nil means contract creation 102 | Value *big.Int 103 | Data []byte 104 | AccessList AccessList 105 | 106 | // Signature values 107 | V *big.Int `json:"v" gencodec:"required"` 108 | R *big.Int `json:"r" gencodec:"required"` 109 | S *big.Int `json:"s" gencodec:"required"` 110 | } 111 | ``` 112 | 113 | ## Transaction的执行流程 114 | 115 | Transaction的执行主要在发生在两个Workflow中: 116 | 117 | 1. Miner在打包新的Block时。此时Miner会按Block中Transaction的打包顺序来执行其中的Transaction。 118 | 2. 其他节点添加Block到Blockchain时。当节点从网络中监听并获取到新的Block时，它们会执行Block中的Transaction，来更新本地的State Trie的 Root，并与Block Header中的State Trie Root进行比较，来验证Block的合法性。 119 | 120 | 一条Transaction执行，可能会涉及到多个Account/Contract的值的变化，最终造成一个或多个Account的State的发生转移。在Byzantium分叉之前的Geth版本中，在每个Transaction执行之后，都会计算一个当前的State Trie Root，并写入到对应的Transaction Receipt中。这符合以太坊黄皮书中的原始设计。即交易是使得Ethereum状态机发生状态状态转移的最细粒度单位。读者们可能已经来开产生疑惑了，“每个Transaction都会重算一个State Trie Root”的方式岂不是会带来大量的计算(重算一次一个MPT Path上的所有Node)和读写开销(新生成的MPT Node是很有可能最终被持久化到LevelDB中的)？结论是显然的。因此在Byzantium分叉之后，在一个Block的验证周期中只会计算一次的State Root。我们仍然可以在`state_processor.go`找寻到早年代码的痕迹。最终，一个Block中所有Transaction执行的结果使得World State发生状态转移。下面我们就来根据geth代码库中的调用关系，从Miner的视角来探索一个Transaction的生命周期。 121 | 122 | ### Native Token Transferring Transaction 123 | 124 | ### Transaction修改Contract的持久化存储的 125 | 126 | 在Ethereum中，当Miner开始构造新的区块的时候，首先会启动*miner/worker.go*的 `mainLoop()`函数。具体的函数如下所示。 127 | 128 | ```go 129 | func (w *worker) mainLoop() { 130 | .... 131 | // 设置接受该区块中挖矿奖励的账户地址 132 | coinbase := w.coinbase 133 | w.mu.RUnlock() 134 | 135 | txs := make(map[common.Address]types.Transactions) 136 | for _, tx := range ev.Txs { 137 | acc, _ := types.Sender(w.current.signer, tx) 138 | txs[acc] = append(txs[acc], tx) 139 | } 140 | // 这里看到，通过NewTransactionsByPriceAndNonce获取一部分的Tx并打包 141 | txset := types.NewTransactionsByPriceAndNonce(w.current.signer, txs, w.current.header.BaseFee) 142 | tcount := w.current.tcount 143 | //提交打包任务 144 | w.commitTransactions(txset, coinbase, nil) 145 | .... 146 | } 147 | ``` 148 | 149 | 在Mining新区块前，Worker首先需要决定，哪些Transaction会被打包到新的Block中。这里选取Transaction其实经历了两个步骤。首先，`txs`变量保存了从Transaction Pool中拿去到的合法的，以及准备好被打包的交易。这里举一个例子，来说明什么是**准备好被打包的交易**，比如Alice先后发了新三个交易到网络中，对应的Nonce分别是100和101，102。假如Miner只收到了100和102号交易。那么对于此刻的Transaction Pool来说Nonce 100的交易就是**准备好被打包的交易**，交易Nonce 是102需要等待Nonce 101的交易被确认之后才能提交。 150 | 151 | 在Worker会从Transaction Pool中拿出若干的transaction, 赋值给*txs*之后, 然后调用`NewTransactionsByPriceAndNonce`函数按照Gas Price和Nonce对*txs*进行排序，并将结果赋值给*txset*。此外在Worker的实例中，还存在`fillTransactions`函数，为了未来定制化的给Transaction的执行顺序进行排序。 152 | 153 | 在拿到*txset*之后，mainLoop函数会调用`commitTransactions`函数，正式进入Mining新区块的流程。`commitTransactions`函数如下所示。 154 | 155 | ```go 156 | func (w *worker) commitTransactions(txs *types.TransactionsByPriceAndNonce, coinbase common.Address, interrupt *int32) bool { 157 | .... 158 | 159 | // 首先给Block设置最大可以使用的Gas的上限 160 | gasLimit := w.current.header.GasLimit 161 | if w.current.gasPool == nil { 162 | w.current.gasPool = new(core.GasPool).AddGas(gasLimit) 163 | // 函数的主体是一个For循环 164 | for{ 165 | ..... 166 | // params.TxGas表示了transaction 需要的最少的Gas的数量 167 | // w.current.gasPool.Gas()可以获取当前block剩余可以用的Gas的Quota，如果剩余的Gas足以开启一个新的Tx，那么循环结束 168 | if w.current.gasPool.Gas() < params.TxGas { 169 | log.Trace("Not enough gas for further transactions", "have", w.current.gasPool, "want", params.TxGas)break 170 | } 171 | .... 172 | tx := txs.Peek() 173 | if tx == nil { 174 | break 175 | } 176 | .... 177 | // 提交单条Transaction 进行验证 178 | logs, err := w.commitTransaction(tx, coinbase) 179 | .... 180 | } 181 | } 182 | ``` 183 | 184 | `commitTransactions`函数的主体是一个for循环，每次获取结构体切片头部的txs.Peek()的transaction，并作为参数调用函数miner/worker.go的`commitTransaction()`。`commitTransaction()`函数如下所示。 185 | 186 | ```go 187 | func (w *worker) commitTransaction(tx *types.Transaction, coinbase common.Address) ([]*types.Log, error){ 188 | // 在每次commitTransaction执行前都要记录当前StateDB的Snapshot,一旦交易执行失败则基于这个Snapshot进行回滚。 189 | // TODO StateDB如何进行快照(Snapshot)和回滚的 190 | snap := w.current.state.Snapshot() 191 | // 调用执行Transaction的函数 192 | receipt, err := core.ApplyTransaction(w.chainConfig, w.chain, &coinbase, w.current.gasPool, w.current.state, w.current.header, tx, &w.current.header.GasUsed, *w.chain.GetVMConfig()) 193 | .... 194 | } 195 | ``` 196 | 197 | Blockchain系统中的Transaction和DBMS中的Transaction一样，要么完成要么失败。所以在调用执行Transaction的函数前，首先记录了一下当前world state的Snapshot，用于交易失败时回滚操作。之后调用core/state_processor.go/ApplyTransaction()函数。 198 | 199 | ```go 200 | func ApplyTransaction(config *params.ChainConfig, bc ChainContext, author *common.Address, gp *GasPool, statedb *state.StateDB, header *types.Header, tx *types.Transaction, usedGas *uint64, cfg vm.Config) (*types.Receipt, error) { 201 | // 将Transaction 转化为Message的形式 202 | msg, err := tx.AsMessage(types.MakeSigner(config, header.Number), header.BaseFee) 203 | if err != nil { 204 | return nil, err 205 | } 206 | // Create a new context to be used in the EVM environment 207 | blockContext := NewEVMBlockContext(header, bc, author) 208 | vmenv := vm.NewEVM(blockContext, vm.TxContext{}, statedb, config, cfg) 209 | // 调用执行Contract的函数 210 | return applyTransaction(msg, config, bc, author, gp, statedb, header.Number, header.Hash(), tx, usedGas, vmenv) 211 | } 212 | ``` 213 | 214 | 在 ApplyTransaction()函数中首先Transaction会被转换成Message的形式。在执行每一个Transaction的时候，都会生成一个新的EVM来执行。之后调用core/state_processor.go/applyTransaction()函数来执行Message。 215 | 216 | ```go 217 | func applyTransaction(msg types.Message, config *params.ChainConfig, bc ChainContext, author *common.Address, gp *GasPool, statedb *state.StateDB, blockNumber *big.Int, blockHash common.Hash, tx *types.Transaction, usedGas *uint64, evm *vm.EVM) (*types.Receipt, error) { 218 | .... 219 | // Apply the transaction to the current state (included in the env). 220 | result, err := ApplyMessage(evm, msg, gp) 221 | .... 222 | 223 | } 224 | ``` 225 | 226 | 之后调用core/state_transition.go/ApplyMessage()函数。 227 | 228 | ```go 229 | func ApplyMessage(evm *vm.EVM, msg Message, gp *GasPool) (*ExecutionResult, error) { 230 | return NewStateTransition(evm, msg, gp).TransitionDb() 231 | } 232 | ``` 233 | 234 | 之后调用core/state_transition.go/TransitionDb()函数。 235 | 236 | ```go 237 | func (st *StateTransition) TransitionDb() (*ExecutionResult, error) { 238 | .... 239 | ret, st.gas, vmerr = st.evm.Call(sender, st.to(), st.data, st.gas, st.value) 240 | .... 241 | } 242 | ``` 243 | 244 | 之后调用core/vm/evm.go/Call()函数。 245 | 246 | ```go 247 | func (evm *EVM) Call(caller ContractRef, addr common.Address, input []byte, gas uint64, value *big.Int) (ret []byte, leftOverGas uint64, err error) { 248 | .... 249 | // Execute the contract 250 | ret, err = evm.interpreter.Run(contract, input, false) 251 | .... 252 | } 253 | ``` 254 | 255 | 之后调用core/vm/interpreter.go/Run()函数。 256 | 257 | ```go 258 | // Run loops and evaluates the contract's code with the given input data and returns 259 | // the return byte-slice and an error if one occurred. 260 | func (in *EVMInterpreter) Run(contract *Contract, input []byte, readOnly bool) (ret []byte, err error) { 261 | .... 262 | cost = operation.constantGas // For tracing 263 | // UseGas 函数：当前剩余的gas quota减去input 参数。 264 | // 剩余的gas 小于input直接返回false 265 | // 否则当前的gas quota减去input并返回true 266 | if !contract.UseGas(operation.constantGas) { 267 | return nil, ErrOutOfGas 268 | } 269 | .... 270 | // execute the operation 271 | res, err = operation.execute(&pc, in, callContext) 272 | .... 273 | 274 | } 275 | ``` 276 | 277 | 在更细粒度的层面，每个opcode循环调用core/vm/jump_table.go中的execute函数。这里值得一提的是，获取Contract中每条Operate的方式，是从Contact中的code数组中按照第n个拿取。 278 | 279 | ```go 280 | // GetOp returns the n'th element in the contract's byte array 281 | func (c *Contract) GetOp(n uint64) OpCode { 282 | return OpCode(c.GetByte(n)) 283 | } 284 | 285 | // GetByte returns the n'th byte in the contract's byte array 286 | func (c *Contract) GetByte(n uint64) byte { 287 | if n < uint64(len(c.Code)) { 288 | return c.Code[n] 289 | } 290 | 291 | return 0 292 | } 293 | ``` 294 | 295 | OPCODE的具体实现代码位于core/vm/instructor.go文件中。比如，对Contract中持久化数据修改的OPSSTORE指令的实现位于opStore()函数中。而opStore的函数的具体操作又是调用了StateDB中的SetState函数，将Go-ethereum中的几个主要的模块串联了起来。 296 | 297 | ```go 298 | func opSstore(pc *uint64, interpreter *EVMInterpreter, scope *ScopeContext) ([]byte, error) { 299 | loc := scope.Stack.pop() 300 | val := scope.Stack.pop() 301 | //根据指令跟地址来修改StateDB中某一存储位置的值。 302 | interpreter.evm.StateDB.SetState(scope.Contract.Address(),loc.Bytes32(), val.Bytes32()) 303 | return nil, nil 304 | } 305 | 306 | //core/state/stateDB 307 | func (s *StateDB) SetState(addr common.Address, key, value common.Hash) { 308 | stateObject := s.GetOrNewStateObject(addr) 309 | if stateObject != nil { 310 | stateObject.SetState(s.db, key, value) 311 | } 312 | } 313 | 314 | 对于一条调用合约函数的交易，其中必然会存在修改StateDB的操作。通过上述的函数调用关系，我们就完成了在一个新区块的形成过程中，Transaction如何修改StateDB的Workflow。 315 | 316 | ```mermaid 317 | flowchart LR 318 | commitTransactions --> commitTransaction --> ApplyTransaction --> applyTransaction --> ApplyMessage --> TransactionDB --> Call --> Run --> opSstore --> StateDB --> StateObject --> Key-Value-Trie 319 | ``` 320 | 321 | ![Transaction Execution Flow](../figs/02/tx_execu_flow.png) 322 | 323 | ![Transaction Execution stack Flow](../figs/04/tx_exec_calls.png) 324 | 325 | 326 | ## [Validator] 验证节点是如何执行Transaction来更新World State 327 | 328 | 而对于不参与Mining的节点，他们执行Block中Transaction的入口是在core/blockchain.go中的InsertChain()函数。InsertChain函数通过调用内部函数insertChain，对调用中的core/state_processor.go中的Process()函数。Process函数的核心在于循环遍历Block中的Transaction，调用上述的applyTransaction函数。从这里开始更底层的调用关系就与Mining Workflow中的调用关系相同。 329 | 330 | ```go 331 | // Process processes the state changes according to the Ethereum rules by running 332 | // the transaction messages using the statedb and applying any rewards to both 333 | // the processor (coinbase) and any included uncles. 334 | // 335 | // Process returns the receipts and logs accumulated during the process and 336 | // returns the amount of gas that was used in the process. If any of the 337 | // transactions failed to execute due to insufficient gas it will return an error. 338 | func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg vm.Config) (types.Receipts, []*types.Log, uint64, error) { 339 | var ( 340 | receipts types.Receipts 341 | usedGas = new(uint64) 342 | header = block.Header() 343 | blockHash = block.Hash() 344 | blockNumber = block.Number() 345 | allLogs []*types.Log 346 | gp = new(GasPool).AddGas(block.GasLimit()) 347 | ) 348 | // Mutate the block and state according to any hard-fork specs 349 | if p.config.DAOForkSupport && p.config.DAOForkBlock != nil && p.config.DAOForkBlock.Cmp(block.Number()) == 0 { 350 | misc.ApplyDAOHardFork(statedb) 351 | } 352 | blockContext := NewEVMBlockContext(header, p.bc, nil) 353 | vmenv := vm.NewEVM(blockContext, vm.TxContext{}, statedb, p.config, cfg) 354 | // Iterate over and process the individual transactions 355 | for i, tx := range block.Transactions() { 356 | msg, err := tx.AsMessage(types.MakeSigner(p.config, header.Number), header.BaseFee) 357 | if err != nil { 358 | return nil, nil, 0, fmt.Errorf("could not apply tx %d [%v]: %w", i, tx.Hash().Hex(), err) 359 | } 360 | statedb.Prepare(tx.Hash(), i) 361 | //核心: 与Mining中Commit Transaction不同，Process在外部循环Block中的Transaction并单条执行。 362 | receipt, err := applyTransaction(msg, p.config, p.bc, nil, gp, statedb, blockNumber, blockHash, tx, usedGas, vmenv) 363 | if err != nil { 364 | return nil, nil, 0, fmt.Errorf("could not apply tx %d [%v]: %w", i, tx.Hash().Hex(), err) 365 | } 366 | receipts = append(receipts, receipt) 367 | allLogs = append(allLogs, receipt.Logs...) 368 | } 369 | // Finalize the block, applying any consensus engine specific extras (e.g. block rewards) 370 | p.engine.Finalize(p.bc, header, statedb, block.Transactions(), block.Uncles()) 371 | 372 | return receipts, allLogs, *usedGas, nil 373 | } 374 | ``` 375 | 376 | ## Background of State-based Blockchain 377 | 378 | - State-based Blockchain 的数据主要由两部分的数据管理模块组成：World State 和 Blockchain。 379 | - State Object是系统中基于K-V结构的基础数据元素。在Ethereum中，State Object是Account。 380 | - World State表示了System中所有State Object的最新值的一个Snapshot，。 381 | - Blockchain是以块为单位的数据结构，每个块中包含了若干Transaction。Blockchain 可以被视为历史交易数据的组合。 382 | - Transaction是Blockchain System中与承载数据更新的载体。通过Transaction，State Object从当前状态切换到另一个状态。 383 | - World State的更新是以Block为单位的。 384 | 385 | 1. 386 | 2. 387 | 388 | [EIP2718]: https://eips.ethereum.org/EIPS/eip-2718 389 | 390 | ## Read Transaction from Database 391 | 392 | 当我们想要通过Transaction的Hash查询一个Transaction具体的数据的时候，上层的API会调用`eth/api_backend.go`中的`GetTransaction()`函数，并最终调用了`core/rawdb/accessors_indexes.go`中的`ReadTransaction()`函数来查询。 393 | 394 | ```go 395 | func (b *EthAPIBackend) GetTransaction(ctx context.Context, txHash common.Hash) (*types.Transaction, common.Hash, uint64, uint64, error) { 396 | tx, blockHash, blockNumber, index := rawdb.ReadTransaction(b.eth.ChainDb(), txHash) 397 | return tx, blockHash, blockNumber, index, nil 398 | } 399 | ``` 400 | 401 | 这里值得注意的是，在读取Transaction的时候，`ReadTransaction()`函数首先获取了保存该Transaction的函数block body，并循环遍历该Block Body中获取到对应的Transaction。这是因为，虽然Transaction是作为一个基本的数据结构(Transaction Hash可以保证Transaction的唯一性)，但是在写入数据库的时候就是被按照Block Body的形式被整个的打包写入到Database中的。具体的可以查看`core/rawdb/accesssor_chain.go`中的`WriteBlock()`和`WriteBody()`函数。 402 | 403 | ```go 404 | func ReadTransaction(db ethdb.Reader, hash common.Hash) (*types.Transaction, common.Hash, uint64, uint64) { 405 | blockNumber := ReadTxLookupEntry(db, hash) 406 | if blockNumber == nil { 407 | return nil, common.Hash{}, 0, 0 408 | } 409 | blockHash := ReadCanonicalHash(db, *blockNumber) 410 | if blockHash == (common.Hash{}) { 411 | return nil, common.Hash{}, 0, 0 412 | } 413 | body := ReadBody(db, blockHash, *blockNumber) 414 | if body == nil { 415 | log.Error("Transaction referenced missing", "number", *blockNumber, "hash", blockHash) 416 | return nil, common.Hash{}, 0, 0 417 | } 418 | for txIndex, tx := range body.Transactions { 419 | if tx.Hash() == hash { 420 | return tx, blockHash, *blockNumber, uint64(txIndex) 421 | } 422 | } 423 | log.Error("Transaction not found", "number", *blockNumber, "hash", blockHash, "txhash", hash) 424 | return nil, common.Hash{}, 0, 0 425 | } 426 | ``` 427 | 428 | ## Terms 429 | 430 | - Externally Owned Account(EOA) 外部账户 -------------------------------------------------------------------------------- /CN/05_block_blockchain.md: -------------------------------------------------------------------------------- 1 | # From Block to Blockchain 2 | 3 | ## Block in General 4 | 5 | ### Date Structure 6 | 7 | ```go 8 | type Block struct { 9 | header *Header 10 | uncles []*Header 11 | transactions Transactions 12 | hash atomic.Value 13 | size atomic.Value 14 | td *big.Int 15 | ReceivedAt time.Time 16 | ReceivedFrom interface{} 17 | } 18 | ``` 19 | 20 | ```go 21 | type Header struct { 22 | ParentHash common.Hash `json:"parentHash" gencodec:"required"` 23 | UncleHash common.Hash `json:"sha3Uncles" gencodec:"required"` 24 | Coinbase common.Address `json:"miner" gencodec:"required"` 25 | Root common.Hash `json:"stateRoot" gencodec:"required"` 26 | TxHash common.Hash `json:"transactionsRoot" gencodec:"required"` 27 | ReceiptHash common.Hash `json:"receiptsRoot" gencodec:"required"` 28 | Bloom Bloom `json:"logsBloom" gencodec:"required"` 29 | Difficulty *big.Int `json:"difficulty" gencodec:"required"` 30 | Number *big.Int `json:"number" gencodec:"required"` 31 | GasLimit uint64 `json:"gasLimit" gencodec:"required"` 32 | GasUsed uint64 `json:"gasUsed" gencodec:"required"` 33 | Time uint64 `json:"timestamp" gencodec:"required"` 34 | Extra []byte `json:"extraData" gencodec:"required"` 35 | MixDigest common.Hash `json:"mixHash"` 36 | Nonce BlockNonce `json:"nonce"` 37 | // BaseFee was added by EIP-1559 and is ignored in legacy headers. 38 | BaseFee *big.Int `json:"baseFeePerGas" rlp:"optional"` 39 | } 40 | ``` 41 | 42 | ## Blockchain 43 | 44 | ### Date Structure 45 | 46 | ```golang 47 | type BlockChain struct { 48 | chainConfig *params.ChainConfig // Chain & network configuration 49 | cacheConfig *CacheConfig // Cache configuration for pruning 50 | 51 | db ethdb.Database // Low level persistent database to store final content in 52 | snaps *snapshot.Tree // Snapshot tree for fast trie leaf access 53 | triegc *prque.Prque // Priority queue mapping block numbers to tries to gc 54 | gcproc time.Duration // Accumulates canonical block processing for trie dumping 55 | 56 | // txLookupLimit is the maximum number of blocks from head whose tx indices 57 | // are reserved: 58 | // * 0: means no limit and regenerate any missing indexes 59 | // * N: means N block limit [HEAD-N+1, HEAD] and delete extra indexes 60 | // * nil: disable tx reindexer/deleter, but still index new blocks 61 | txLookupLimit uint64 62 | 63 | hc *HeaderChain 64 | rmLogsFeed event.Feed 65 | chainFeed event.Feed 66 | chainSideFeed event.Feed 67 | chainHeadFeed event.Feed 68 | logsFeed event.Feed 69 | blockProcFeed event.Feed 70 | scope event.SubscriptionScope 71 | genesisBlock *types.Block 72 | 73 | // This mutex synchronizes chain write operations. 74 | // Readers don't need to take it, they can just read the database. 75 | chainmu *syncx.ClosableMutex 76 | 77 | currentBlock atomic.Value // Current head of the block chain 78 | currentFastBlock atomic.Value // Current head of the fast-sync chain (may be above the block chain!) 79 | 80 | stateCache state.Database // State database to reuse between imports (contains state cache) 81 | bodyCache *lru.Cache // Cache for the most recent block bodies 82 | bodyRLPCache *lru.Cache // Cache for the most recent block bodies in RLP encoded format 83 | receiptsCache *lru.Cache // Cache for the most recent receipts per block 84 | blockCache *lru.Cache // Cache for the most recent entire blocks 85 | txLookupCache *lru.Cache // Cache for the most recent transaction lookup data. 86 | futureBlocks *lru.Cache // future blocks are blocks added for later processing 87 | 88 | wg sync.WaitGroup // 89 | quit chan struct{} // shutdown signal, closed in Stop. 90 | running int32 // 0 if chain is running, 1 when stopped 91 | procInterrupt int32 // interrupt signaler for block processing 92 | 93 | engine consensus.Engine 94 | validator Validator // Block and state validator interface 95 | prefetcher Prefetcher 96 | processor Processor // Block transaction processor interface 97 | forker *ForkChoice 98 | vmConfig vm.Config 99 | } 100 | 101 | ``` 102 | ## 向Blockchain中插入Block 103 | `func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals, setHead bool) (int, error)` -------------------------------------------------------------------------------- /CN/06_mining_hash_gpu.md: -------------------------------------------------------------------------------- 1 | # Mining 2 | 3 | ## Block Reward 4 | 5 | ## How to Seal Block 6 | 7 | 其中一个关键的函数是`miner/worker.go`中的`fillTransactions()`函数。这个函数会从transaction pool中的Pending的transactions选取若干的交易并且将他们按照Gas Price和Nonce的顺序进行排序形成新的tx set并传递给`commitTransactions()`函数。在`fillTransactions()`函数会首先处理Local Pool中的交易，然后再处理从网络中接受到的远程交易。 8 | 9 | 目前Block中的Transaction的顺序就是由`fillTransactions()`函数通过调用`types/transaction.go`中的`NewTransactionsByPriceAndNonce()`来决定。 10 | 11 | 也就说如果我们希望修改Block中Transaction的打包顺序和从Transaction Pool选择Transactions的策略的话，我们可以通修改`fillTransactions()`函数。 12 | 13 | `commitTransactions()`函数的主体是一个for循环体。在这个for循环中，函数会从txs中不断拿出头部的tx进行调用`commitTransaction()`函数进行处理。在Transaction那一个Section我们提到的`commitTransaction()`函数会将成功执行的Transaction保存在`env.txs`中。 14 | 15 | -------------------------------------------------------------------------------- /CN/07_p2p_net_node_sync.md: -------------------------------------------------------------------------------- 1 | # How the Geth Nodes communicate with other Ethereum Nodes 2 | -------------------------------------------------------------------------------- /CN/10_state_data_optimization.md: -------------------------------------------------------------------------------- 1 | # State Data Optimization 2 | 3 | 在之前的文章中我们提到了，State Trie/Storage Trie中的数据的写盘是依赖于`trie/database.go`这个文件中提供的函数。这个文件同样起到了Trie结构优化的作用 4 | 5 | 1. Batch Write 6 | 2. Node Pruning 7 | 1. 节点的引用计数 8 | 2. 当节点的引用计数为0时，从数据库中删除掉这个节点。 9 | 10 | ## Batch Write 11 | 12 | ## State Pruning 13 | -------------------------------------------------------------------------------- /CN/11_leveldb_in_practice.md: -------------------------------------------------------------------------------- 1 | # Leveldb in Practice 2 | -------------------------------------------------------------------------------- /CN/12_caching.md: -------------------------------------------------------------------------------- 1 | # Caching in Practice 2 | -------------------------------------------------------------------------------- /CN/13_evm.md: -------------------------------------------------------------------------------- 1 | # EVM in Practice 2 | 3 | ## General 4 | 5 | EVM是Ethereum中最核心的模块，也可以说是Ethereum运行机制中的灵魂模块。 6 | 7 | 如果我们抛开Blockchain的属性，那么EVM的JVM(或者其他的类似的Virtual Machine)。 8 | 9 | EVM，由Stack，Program Counter，Gas available，Memory，Storage，Opcodes组成。 10 | 11 | ## EVM Instructions 12 | 13 | 关于EVM的指令，我们首先关注会与StateDB交互，甚至会引发Disk I/O的指令。这些指令包括`Balance`,`Sload`,`Sstore`, `EXTCODESIZE`,`EXTCODECOPY`,`EXTCODEHASH`,`SELFDESTRUCT`,`LOG0`,`LOG1`,`LOG2`,`LOG3`,`LOG4`,`KECCAK256`。其中`LOG0`,`LOG1`,`LOG2`,`LOG3`,`LOG4`在本质上都调用了同一个底层函数`makeLog`。 14 | 15 | ## EVM Trace 16 | 17 | 我们知道，在以太坊中有两种类型的交易，1. Native的Ether的转账交易 2. 调用合约函数的交易。调用合约的交易，本质上是实行了一段函数代码，由于是图灵完备的，这算代码可以任意的运行。作为用户，我们只需要知道Transaction的最终的运行结果(最终修改的Storage的结果)。但是对于开发人员，我们需要了解交易运行的最终结果，我们还需要了解在Transaction执行过程中的一些中间状态来方便debug和调优。 18 | 19 | 为了满足开发人员的这种需求，go-ethereum提供了EVM Tracing的模块，来trace Transaction执行时EVM中一些值的情况。 20 | 21 | 目前，这种EVM的Trace是以transaction执行时的调用的opcode单位开展的。EVM每执行一个指令，都会将当前Stack，Memory，Storage，Transaction剩余的Gas量，当前执行的指令的信息输出出来。 22 | 23 | ### EVM Logger -------------------------------------------------------------------------------- /CN/14_signer.md: -------------------------------------------------------------------------------- 1 | # Signer: Transaction的合法性证明 -------------------------------------------------------------------------------- /CN/15_rpc.md: -------------------------------------------------------------------------------- 1 | # Ethereum中的API 调用: RPC and IPC 2 | 3 | ## RPC (远程过程调用) 4 | 5 | ### Background 6 | 7 | 一个应用可以通过RPC(Remote Procedure Call)的方式来调用某一个go-ethereum的实例(instance)。 通常，go-ethereum默认的对外暴露的RPC端口地址为8545。 8 | 9 | 在example/deploy/SendTransaction.go中，我们展示了一个通过RPC调用go-ethereum实例来发送Transaction的例子。 10 | 11 | ## IPC (进程间通信) 12 | 13 | ### Background 14 | 15 | IPC(Inter-Process Communication) 用于两个进程间进行通信，和共享数据。与RPC不同的是，IPC主要用在同一台宿主机上的不同的进程间的通信。 16 | 17 | ## Appendix 18 | -------------------------------------------------------------------------------- /CN/20_bft_consensus.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/20_bft_consensus.md -------------------------------------------------------------------------------- /CN/21_rollup.md: -------------------------------------------------------------------------------- 1 | # 扩容！扩容！扩容！从Plasma到Rollup 2 | 3 | ## In General 4 | 5 | ## 一些旧日的想法: Plasma 6 | 7 | ## 弄潮儿Rollup 8 | 9 | 自从[EIP-1559](https://eips.ethereum.org/EIPS/eip-1559)生效后，由于币价的升高，和基础Gas Price的约束，Ethereum Mainnet上的单笔交易费用已经高到了离谱的程度。显然，目前**天价的交易费**以及**有限的Throughput**已经成为了限制Ethereum继续发展的两大难题。幸运的是，**rollup**技术的发展给社区展现了一种似乎可以一招解决两大难题的绝世武学。 10 | 11 | 简单的来说，顾名思义，rollup，就是把一堆的transaction rollup到一个新的transaction。然后，通过某种神奇的技术，使得Ethereum Mainnet只需要验证这个新生成的transaction，就可以保证被Rollup之前的若干的Transaction的确定性，正确性，完整性。举个简单的例子，我们想象一个学校内交学费的场景。过去，每个学生(i.e. Account)都需要通过学校交费系统(i.e. Ethereum Mainnet)单独的将自己的学费转账(Transfer)给学校教务处。假如，现在计算机系有1000个学生，那么系统就要处理1000笔转账的交易(Transaction)。现在，系里新来一位叫做*Rollup*的教授。他在学生中很有号召力。基于他的个人魅力(神奇的魔法)，**私下里**让所有的的学生把钱通过**某种方式**先转给他。当*Prof.Rollup*收集到系里所有学生的学费之后，然后他通过构造一个transaction把所有的学费交给教务处。这个例子大概就是一个Rollup场景的简单抽象。我们把学校的交费系统看作为Ethereum Mainnet 或者说是Layer-1，那么私下里收集学生的学费就是所谓的**Layer-2**进行的了。通常，我们把Layer-1上的交易称为on-chain transaction，把在Layer-2进行上的交易称为off-chain transaction。 12 | 13 | Rollup的好处是显而易见的，假如每次把**N**个transaction rollup成一个，那么对于Mainnet来说，处理一条交易，实际效果等同于处理了之前系统中的**N**个交易。同时，系统实际吞吐量的Upper Bound也实际上上市到了**N*MAX_TPS**的水平。同时，对于用户来说，同样一条交易Mainnet Transaction Fee实际上是被**N**个用户同时负担的。那么理论上，用户需要的实际交易费用也只有之前的**1/N+c**。这里的**c**代表rollup服务提供商收取的交易费，这个值是远远小于layer-1上交易所需要的交易费用的。 14 | 15 | Rollup看上去完美的解决了Ethereum面临的两大难题，通过分层的方式给Ethereum进行了扩容。但是，在实现layer-2 rollup时，还是有很多的细节有待商榷。比如: 16 | 17 | - 怎么保证*Prof.Rollup*一定会把学费交给教务处呢？ (Layer-2 交易的安全性) 18 | - 怎么保证*Prof.Rollup*会把全部的学费都交给教务处呢？(Layer-2 交易的完整性) 19 | - *Prof.Rollup*什么时候才会把学费打给教务处呢？(Layer-2 到Layer-1 跨链交易的时效性问题) 20 | - 等等.. 21 | 22 | 就像华山派的剑宗和气宗分家一样，目前，在如何实现Rollup上，主要分为了两大流派，分别是**Optimism-Rollup**，和**ZK-Rollup**。两种路线各有所长，又各有不足。 23 | 24 | ## 弄潮儿的精英Zk-Rollup 25 | 26 | ZK-Rollup核心是利用了Zero-Knowledge Proof的技术来实现rollup。ZKP里面的细节比较多，在这里我们不展开描述。其中最重要的核心点是ZK-Rollup主要利用了ZK-SNARKs中的可验证性的正确性保障，以及Verification Time相对较快的特性，来保证layer-1上的Miner可以很快的验证大量交易的准确性。 27 | 28 | 但是正因为ZKP的一些特性，使得ZK-Rollup相比于Optimism-Rollup，在开发上的并没有进行的那么顺利。 29 | 30 | 我们知道ZK-SNARKs的计算的基础来自于:*将一个计算电路(Circuit)，转化为R1CS的形式*，继而转化为QAP问题，最终将问题的Witness生成零知识的Proof。如果我们想生成一个Problem/Computation/Function的ZK-SNARKs的Witness/Proof，那么首先我们需要要把这个问题转化为一个Circuit的形式。或者说用Circuit的语言，用R1CS的形式来描述原计算问题。对于简单的计算问题来说，比如加减计算，解方程组，将问题转化为电路的形式并不是特别的困难。但是对于Ethereum上各种支持图灵完备的智能合约的function来说这个问题就变得非常的棘手。主要因为： 31 | 32 | - 转化生成的电路是静态的，虽然电路中有可以复用的部分(Garget)，但是每个Problem/Computation/Function都要构造新的电路。 33 | - 对于通用计算问题，构造出来的电路需要的gate数量可能是惊人的高。这意味着ZK-Rollup可能需要非常长的时间来生成Proof。 34 | - 对于目前EVM架构下的某些计算问题，生成其电路是非常困难的。 35 | 36 | 前两个问题属于General的ZK-SNARKs应用都会遇到的问题，目前已经有一些的研究人员/公司正在尝试解决这个问题。比如AleoHQ的创始人Howard Wu提出的[DIZK](https://www.usenix.org/conference/usenixsecurity18/presentation/wu)通过分布式的Cluster来并发计算Proof，以及Scroll的创始人[Zhang Ye](https://twitter.com/yezhang1998)，提出的[PipeZK](https://www.microsoft.com/en-us/research/publication/pipezk-accelerating-zero-knowledge-proof-with-a-pipelined-architecture/)通过ASIC来加速ZK计算。 37 | 38 | 对于第三个问题，属于Ethereum中的专有的问题，也是目前zk-Rollup实现中最难，最需要攻克的问题。我们知道在Ethereum中，一条调用合约Function的Transaction的执行是基于/通过EVM的来执行的。EVM会将Transaction中的函数调用，基于合约代码，转换成opcodes的形式，保存在Stack中逐条执行。这个过程类似于编译过程中的IR代码生成，或者高级语言到汇编语言的过程。在执行这些opcodes时，本质上是在执行geth中对应的库函数，部分细节可以参考之前的[blog](http://www.hsyodyssey.com/blockchain/2021/07/25/ethereum_txn.html)。那么如果我们想把一个Transaction的合约调用转换成一个电路的话，本质上我们就要基于这个函数调用过程中执行过的opcodes来生成电路。目前的问题是，EVM在设计的时候并没有考虑到将来会被ZK-SNARKs这一问题，所以在它包含的140个opcode中有些是难以构造电路的。 39 | 40 | 结果就是，在目前的ZK-Rollup的解决方案中，大部分**仅支持基础的转账操作**，而**不能支持**通用的图灵完备的计算。也就是说，目前的ZK-Rollup的解决方案都是不完整的，不能完整的发挥Ethereum图灵完备合约的特性，Layer-2又回到了仅支持Token转账的时代。 41 | 42 | 为了解决这个问题，使得Layer-2能支持像现在的Layer-1一样的功能，目前的技术主要在朝向两个方向发展，1. 构建ZK-SNARKs兼容的zkEVM，2.提出新的VM来兼容ZK-SNARKs，并想办法与Ethereum兼容。 43 | -------------------------------------------------------------------------------- /CN/22_ads.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/22_ads.md -------------------------------------------------------------------------------- /CN/23_bloom_filter.md: -------------------------------------------------------------------------------- 1 | # Bloom Filter in Ethereum 2 | 3 | Bloom Filter 是一种可以快速检索的工具。Bloom Filter 本身由是一个长度为m的bit array，k个不相同的hash 函数和源dataset组成。具体的说，Bloom Filter是由k个不同的hash function将源dataset hash到m位的bit array构成。通过Bloom Filter，我们可以快速检测出，一个data是不是在源dataset中（O(k) time）。 4 | 5 | Bloom Filter不保证完全的正确性. In other word, Bloom Filter 会出现false positive的结果。但是，如果一个被检索的data，在bloom filter 中得到了false的反馈，那他一定不在源data之中。 6 | 7 | Bloom Filter是Ethereum的核心功能之一。具体的代码是实现是在“core/types/bloom9.go”文件之中。 8 | 9 | 在文件的起始位置，定义了两个常量BloomByteLength 和 BloomBitLength，大小分别是256和8. 10 | 11 | 在Ethereum中的Bloom Filter是一个长度为256的byte数组组成的。 12 | 13 | Bloom represents a 2048 bit bloom filter 14 | type Bloom [BloomByteLength]byte 15 | 16 | Ethereum 中Bloom Filter使用的SHA Hash Function. 17 | 18 | 基本的思想是，使用三个value的值来判断log是否存在。 19 | 首先对data使用SHA function进行求值。选择hash后的 20 | 这三个value的选择[0,1],[2,3],[4,5]的值对2048取模得到目标位置的下标，并把这几个位置设为1. 21 | 22 | 对待判断的log进行相同的操作。 -------------------------------------------------------------------------------- /CN/24_turing_halting.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/24_turing_halting.md -------------------------------------------------------------------------------- /CN/25_lsm_tree.md: -------------------------------------------------------------------------------- 1 | # Long-Structured Merge-Tree -------------------------------------------------------------------------------- /CN/26_txn_concurrency.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/26_txn_concurrency.md -------------------------------------------------------------------------------- /CN/30_geth_private_network.md: -------------------------------------------------------------------------------- 1 | # 使用geth构建一个私有区块链网络 (Private Ethereum Blockchain) 2 | 3 | ## 创建 Private 网络 4 | 5 | ### 共识算法的选择 6 | 7 | 目前，Geth支持多种共识算法，包括基于PoA(Proof-of-Authority)的Clique共识算法，以及基于PoW(Proof-of-Work)的Ethash共识算法。由于Clique协议在生成区块的时候不需要像PoW协议一样进行Hash计算，因此非常适合用于搭建测试链。比如，Ethereum两个著名的测试网络Rinkeby and Görli都是基于Clique算法搭建的。 8 | 9 | 值得注意的是，在Clique中，Mining Block是没有收益的。所以，如果你想讲Mining Reward作为私有网络中的一项特性的话，请使用Ethash共识算法。 10 | 11 | ### 构建创始区块 12 | 13 | 在运行一个Geth节点之前，我们首先需要创建一个包含创世state信息的genesis.json文件。注意，本例中的Genesis文件没有设置共识算法，所以Geth会默认使用Ethash作为共识算法。 14 | 15 | 在genesis.json文件中，config字段决定了未来整个链的一些基本的设定，通常这些设定在创世区块初始化之后之后是不可修改的。我们来介绍一下其中比较重要的一些设置。 16 | 17 | 首先是chainId字段。我们知道目前在市面上除了Ethereum主网之外，还有很多使用直接使用Geth，或者基于定制化修改的Geth来运行的公有/私有网络。这些客制化区块链网络非常之多，涉及到各个层面，比如Ethereum官方运营的两个测试链网络，币安运行的Binance Smart Chain，以及一些Layer-2的节点，比如Optimism。正如我们之前提到的，考虑到Geth节点之间的通信都是依赖于P2P网络传输，假如两个Geth-base的网络使用了相同的创始数据进行初始化，那么势必会造成混乱，网络数据同步带来混乱。因此，Ethereum的开发人员设计了chainId就是来解决这个问题。chainId用于在P2P的网络世界中来区分这些基于不同的版本/创世节点的网络信息，起到了网络身份证的作用。比如Ethereum主网的chainId是1，Binance Smart Chain主网的chainId是56。更多的关于正在运行中的基于Geth的网络的信息可以参考这个[网站](https://chainlist.org/)。 18 | 19 | alloc字段用于给一些账户初始化一些本网络的Native Token。在创世区块生成之后，网络中的Native Token产生的来源就只有Mining获得的Mining Reward。 20 | 21 | ```json 22 | { 23 | "config": { 24 | "chainId": "Your Private Network id (Int number)", 25 | "homesteadBlock": 0, 26 | "eip150Block": 0, 27 | "eip155Block": 0, 28 | "eip158Block": 0, 29 | "byzantiumBlock": 0, 30 | "constantinopleBlock": 0, 31 | "petersburgBlock": 0, 32 | "istanbulBlock": 0, 33 | "berlinBlock": 0 34 | }, 35 | "alloc": { 36 | "Your Account Address": { 37 | "balance": "10000000000000000000" 38 | } 39 | }, 40 | "coinbase": "0x0000000000000000000000000000000000000000", 41 | "difficulty": "0x200", 42 | "extraData": "", 43 | "gasLimit": "0x2fefd8", 44 | "nonce": "0x00000000000000755", 45 | "mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000", 46 | "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000", 47 | "timestamp": "0x00" 48 | } 49 | ``` 50 | 51 | ### 构建网络 52 | 53 | #### 初始化创世区块 54 | 55 | 在每个新加入网络中节点，首先都需要基于创世区块进行初始化。具体的命令是init 加上 `-datadir` 后面跟上需要存放Chain数据文件路径，最后跟创世区块genesis.json文件所在的路径。 56 | 57 | ```cmd 58 | geth init --datadir genesis.json 59 | ``` 60 | 61 | #### 运行节点 62 | 63 | ```cmd 64 | geth --datadir --networkid --nodiscover --http --rpc --rpcport "8545" --rpcaddr "0.0.0.0" --rpccorsdomain "*" --rpcapi "eth,web3,net,personal,miner" console 2 65 | ``` 66 | 67 | ## Monitoring 68 | 69 | 我们可以使用区块链浏览器来检索链上数据。一般来说，它们通过go-ethereum实例的RPC接口来调用，实例的API，从而获取最新的链上信息。 70 | -------------------------------------------------------------------------------- /CN/31_solidity_in_practice.md: -------------------------------------------------------------------------------- 1 | # Solidity 2 | 3 | ## Instructions 4 | 5 | EVM类似汇编器，负责把合约汇编成更底层的指令(instruction)。每条指令表示了一些基础或者原子行逻辑操作，例如opCreate用于在State Database上创建一个新的Contract，opBalance用于从State Database中获取某个State Object的balance。这些指令的的具体的代码实现位于core/vm/instructions.go 文件中。 6 | 7 | 值得注意的是，这些指令仍然会调用go-ethereum中其他package所提供的API，而不是直接对更底层的数据进行操作。比如，opSstore与opSload指令用于从Storage层存储和读取数据。这两个指令直接调用了StateDB(core/state/statedb.go)与StateObject(core/state/state_object.go)提供的API。关于这些指令的详细介绍可以参考Ethereum Yellow Paper。 8 | 9 | ### opSload 10 | 11 | opSload的代码如下所示。 12 | 13 | ```Golang 14 | func opSload(pc *uint64, interpreter *EVMInterpreter, scope *ScopeContext) ([]byte, error) { 15 | loc := scope.Stack.peek() 16 | hash := common.Hash(loc.Bytes32()) 17 | // 从StateDB中读取到对应的合约中对应的存储Object的值 18 | val := interpreter.evm.StateDB.GetState(scope.Contract.Address(), hash) 19 | loc.SetBytes(val.Bytes()) 20 | return nil, nil 21 | } 22 | ``` 23 | 24 | ### opSstore 25 | 26 | opSstore的代码如下所示。 27 | 28 | ```Golang 29 | func opSstore(pc *uint64, interpreter *EVMInterpreter, scope *ScopeContext) ([]byte, error) { 30 | loc := scope.Stack.pop() 31 | val := scope.Stack.pop() 32 | // 将Stack中的数据写入到StateDB中 33 | interpreter.evm.StateDB.SetState(scope.Contract.Address(), 34 | loc.Bytes32(), val.Bytes32()) 35 | return nil, nil 36 | } 37 | ``` 38 | 39 | 我们注意到，opSstore指令中向合约中的写入逻辑是调用了StateDB中的SetState函数(在core/state/statedb.go中)。SetState函数有三个参数作为input，分别是目标合约的地址，目标storage object的has，以及其更新后的value。其代码如下所示。 40 | 41 | ```Golang 42 | func (s *StateDB) SetState(addr common.Address, key, value common.Hash) { 43 | stateObject := s.GetOrNewStateObject(addr) 44 | if stateObject != nil { 45 | stateObject.SetState(s.db, key, value) 46 | } 47 | } 48 | ``` 49 | 50 | SetState 函数通过调用StateObject的SetState函数来修改Storage的值。 51 | 52 | ```Golang 53 | // SetState updates a value in account storage. 54 | func (s *stateObject) SetState(db Database, key, value common.Hash) { 55 | // If the fake storage is set, put the temporary state update here. 56 | if s.fakeStorage != nil { 57 | s.fakeStorage[key] = value 58 | return 59 | } 60 | // If the new value is the same as old, don't set 61 | prev := s.GetState(db, key) 62 | if prev == value { 63 | return 64 | } 65 | // New value is different, update and journal the change 66 | s.db.journal.append(storageChange{ 67 | account: &s.address, 68 | key: key, 69 | prevalue: prev, 70 | }) 71 | s.setState(key, value) 72 | } 73 | 74 | func (s *stateObject) setState(key, value common.Hash) { 75 | s.dirtyStorage[key] = value 76 | } 77 | ``` 78 | 79 | 这里的dirtStorage起到了一个cache的作用。之后在updated storage root的时候会基于当前dirtyStorage中的信息，在commit函数中统一更新root的值。 80 | -------------------------------------------------------------------------------- /CN/32_oracle.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/32_oracle.md -------------------------------------------------------------------------------- /CN/33_query.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/33_query.md -------------------------------------------------------------------------------- /CN/41_system_tunning.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/41_system_tunning.md -------------------------------------------------------------------------------- /CN/42_developer_view.md: -------------------------------------------------------------------------------- 1 | # 从开发的角度探究go-ethereum的设计思想 2 | 3 | go-ethereum 中大量使用了新的实例来处理业务逻辑。比如在core/state_processor.go中: 4 | 5 | ```go 6 | func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg vm.Config) (types.Receipts, []*types.Log, uint64, error) { 7 | .... 8 | blockContext := NewEVMBlockContext(header, p.bc, nil) 9 | // 新new了一个EVM的实例 10 | vmenv := vm.NewEVM(blockContext, vm.TxContext{}, statedb, p.config, cfg) 11 | .... 12 | } 13 | 14 | ``` 15 | 16 | Go-ethereum很多函数将配置文件/协议作为函数的参数，来动态的修改/升级系统的逻辑。比如: 17 | /core/state_transition.go/refundGas()用于当transaction执行失败时返还给用户的gas，其参数是最高返还的比例(份额)。 18 | ```go 19 | func (st *StateTransition) refundGas(refundQuotient uint64) { 20 | // Apply refund counter, capped to a refund quotient 21 | refund := st.gasUsed() / refundQuotient 22 | if refund > st.state.GetRefund() { 23 | refund = st.state.GetRefund() 24 | } 25 | st.gas += refund 26 | 27 | // Return ETH for remaining gas, exchanged at the original rate. 28 | remaining := new(big.Int).Mul(new(big.Int).SetUint64(st.gas), st.gasPrice) 29 | st.state.AddBalance(st.msg.From(), remaining) 30 | 31 | // Also return remaining gas to the block gas counter so it is 32 | // available for the next transaction. 33 | st.gp.AddGas(st.gas) 34 | } 35 | ``` 36 | 37 | 这个函数的调用在/core/state_transition.go/TransitionDb()调用的时候，输入是根据EIP协议的动态配置params.RefundQuotientEIP3529. 38 | 39 | ```go 40 | func (st *StateTransition) TransitionDb() (*ExecutionResult, error) { 41 | .... 42 | if !st.evm.ChainConfig().IsLondon(st.evm.Context.BlockNumber) { 43 | // Before EIP-3529: refunds were capped to gasUsed / 2 44 | st.refundGas(params.RefundQuotient)} 45 | else { 46 | // After EIP-3529: refunds are capped to gasUsed / 5 47 | st.refundGas(params.RefundQuotientEIP3529) 48 | } 49 | .... 50 | } 51 | ``` -------------------------------------------------------------------------------- /CN/43_metrics.md: -------------------------------------------------------------------------------- 1 | # Metrics in Ethereum and related Blockchain systems 2 | 3 | 在本文中，我们主要总结一下Ethereum中常用的Metrics。 4 | 5 | 6 | ## Metrics in Ethereum 7 | 8 | 描述以太坊中性能的两个重要的metrics是TPS和transaction delay. 9 | 10 | 11 | ## Compare with current relational database 12 | 13 | Blockchain system 并不是传统意义上的Database management system. 14 | 15 | ### Metrics in Blockchain 16 | 17 | - TPS 18 | - Delay time 19 | - Network I/O 20 | - 支持的用户数量：无限的 21 | 22 | ### Metrics in relational database 23 | 24 | - TPS 每秒钟处理的transaction的数量 25 | - QPS 每秒钟查询的transaction的数量 26 | - IOPS 每秒钟IO的读取的的速率 27 | - 支持的用户数量：有限的 28 | -------------------------------------------------------------------------------- /CN/44_golang_ethereum.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/CN/44_golang_ethereum.md -------------------------------------------------------------------------------- /CN/45_eth2.md: -------------------------------------------------------------------------------- 1 | # Looking Forward to the Ethereum2.0 2 | 3 | 今年Ethereum就会切换到2.0版本。在这个版本中PoW的挖矿将成为过去式，Ethereum将切换为PoS的Mining奖励模式。 4 | 5 | Validator的投票权的大小取决于他们质押的Ethereum的数量。 -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Understanding-Ethereum-Go-version 2 | 3 | - Title: Understanding Ethereum: Starting with go-ethereum Source Code｜理解以太坊：Go-Ethereum 源码剖析 4 | - Subject: Ethereum Source Code Analysis | 以太坊源码剖析 5 | - Author: Siyuan Han 6 | - Go-Ethereum Version: v1.10.15 (London Upgrade) 7 | - Updated date: 2022-04 8 | 9 | ## Preface 10 | 11 | ### Background 12 | 13 | 自中本聪发表比特币白皮书至今，已经过了十几年的时光，Blockchain 这一理念，从最开始仅作为支持 Bitcoin 的分布式账本，在这十几年中也在不断演化发展。随着参与人数的增加，不断有新的思想涌入社区。**Blockchain** 逐渐成为了集成了包括*数据库*，*分布式系统*，*密码学*，*点对点网络*，*编译原理*，*静态软件分析*，*众包*，*经济学*，*货币金融学*，*社会学*在内的等多个学科知识的一个全新技术领域。Blockchain 也逐渐从小众的去中心化社区逐渐走向了主流社会的舞台，目前仍是当下**最热度最高**，**技术迭代最快**，**最能引起社会讨论**的技术话题之一。 14 | 15 | 在 Blockchain 固有的 decentralized 的思想下，市面上绝大多数的 Blockchain 系统都已经开源，并会继续以开源的形式持续开发。这就为我们提供了一种的极好的学习 Blockchain 技术的方式：通过结合文档，基于 State-of-the-arts 的 Blockchain Systems 的源代码出发开始学习。 16 | 17 | 一个热门的技术总会有大量不同视角的作者，在不同的时间记录下来的文档资料。作为学习者，不管是探究以加密货币导向（Crypto-based）的 Bitcoin, 还是了解致力于实现通用 Web3.0 框架（General-Purpose）的 Ethereum，社区中有丰厚的文档从 high-level 的角度来讲述它们的基础概念，以及设计的思想。比如，技术社区有非常多的资料来讲述什么是梅克尔树 (Merkle Hash Tree)，什么是梅克尔帕特里夏树 (Merkle Patricia Trie)，什么是有向无环图 (Directed acyclic Graph); BFT (Byzantine Fault Tolerance) 和 PoW (Proof-Of-Work) 共识算法算法的区别；以及介绍 Blockchain 系统为什么可以抵抗双花攻击 (Double-Spending)，或者为什么 Ethereum 会遇到 DAO Attack (Decentralized autonomous organization) 等具体问题。 18 | 19 | 但是，只了解关键组件的实现细节，或者高度抽象的系统工作流，并不代表着可以让读者从真正搞清楚 Blockchain 的**工作原理**，反而很容易让读者在一些关键细节上一头雾水，似懂非懂。比如，当我们谈到 Blockchain 系统中 Transaction 的生命周期时，在文档中经常会读到，“Miner 节点批量地从自己维护的 Transaction pool 中选择一些 Transaction 并打包成一个新的 Block 中”。那么究竟 miner 是怎么从网络中获取到 Transaction？又是继续什么样的策略从 Transaction pool 中选取**多少** Transaction？最终按照又按照什么样的 Order 把 transaction 打包进区块链中的呢？如何打包成功的 Block 是怎么交互给其他节点呢？在我学习的过程中，尝试去搜索了一下，发现鲜有文章从*整体*的系统工作流的角度出发，以**细粒度**的视角对区块链系统中的具体的实现*细节*进行解析。与数据库系统 (Database Management System) 相似，Blockchain 系统同样是一个包含网络层，业务逻辑层，任务解析层，存储层的复杂数据管理系统。对它研究同样需要从系统的实现细节出发，从宏观到微观的了解每个执行逻辑的工作流，才能彻底理解和掌握这门技术的秘密。 20 | 21 | 笔者坚信，随着网络基础架构的不断完善，将带来显著的带宽增加和通信延迟下降。同时随着存储技术和分布式算法的不断发展，未来系统的运行效率将会不断逼近硬件极限。在未来的是五到十年内，云端服务/去中心化系统的效率以及覆盖场景一定还会有很大的提升。未来技术世界一定是两极分化的。一极是以大云计算公司（i.e, Google，MS，Oracle，Snowflake，and Alibaba）为代表的中心化服务商。另一极就是以 Blockchain 技术作为核心的去中心化的世界。在这个世界中，Ethereum 及其生态系统是当之无愧的领头羊。Ethereum 作为通用型 Public Chain 中的翘楚取得了巨大的成功，构建了强大的生态系统。并且吸引到了一大批世界上最优秀的工程师，研究人员的持续的发力，不断的将新思想，新理念，新技术引入到 Ethereum 中，持续的引领 Blockchain 技术发展。同时 Go-Ethereum 作为其优秀的开源实现，已经被广泛的订制，来适应不同的私有/联盟/Layer-2 的场景 (e.g., Quorum, Binance Smart Chain, Optimism)。因此，要想真正掌握好区块链系统的原理，达到可以设计开发区块链系统的水平，研究好 Ethereum 的原理以及其设计思想是非常有必要。 22 | 23 | 本系列文章作为我在博士期间学习/研究的记录，将会从 Ethereum 中具体业务的工作的视角出发，在源码的层面细粒度的解析以太坊系统中各个模块的实现的细节，以及背后的蕴含的技术和设计思想。同时，在阅读源代码中发现的问题也可以会提交 Pr 来贡献社区。本系列基于的代码库是 Go-ethereum version 1.10.*版本。Go-ethereum 是以太坊协议的 Go 语言实现版本，目前由以太坊基金会维护。目前除了 Go-ethereum 之外，Ethereum 还有 C++, Python，Java, Rust 等基于其他语言实现的版本。相比于其他的社区版实现，Go-ethereum 的用户数量最多，开发人员最多，版本更新最频繁，issues 的发现和处理都较快。其他语言的 Ethereum 实现版本因为用户与开发人员的数量相对较少，更新频率相对较低，隐藏问题出现的可能性更高。因此我们选择从 Go-ethereum 代码库作为我们的主要学习资料。 24 | 25 | ### 为什么要阅读区块链系统的源代码？ 26 | 27 | 1. 文档资料相对较少，且**内容浅尝辄止**，**相互独立**。比如，*很多的科普文章都提到，在打包新的 Block 的时候，miner 负责把 a batch of transactions 从 transaction pool 中打包到新的 block 中*。那么我们希望读者思考如下的几个问题： 28 | - Miner 是基于什么样策略从 Transaction Pool 中选择 Transaction 呢？ 29 | - 被选择的 Transactions 又是以怎样的顺序 (Order) 被打包到区块中的呢？ 30 | - 在执行 Transaction 的 EVM 是怎么计算 gas used，从而限定 Block 中 Transaction 的数量的呢？ 31 | - 剩余的 gas 又是怎么返还给 Transaction Proposer 的呢？ 32 | - EVM 是怎么解释 Contract 的 Message Call 并执行的呢？ 33 | - 在执行 Transaction 中是哪个模块，是怎样去修改 Contract 中的持久化变量呢？ 34 | - Smart Contract 中的持久化变量又是以什么样的形式存储在什么地方的呢？ 35 | - 当新的 Block 加入到 Blockchain 中时，World State 又是何时怎样更新的呢？ 36 | - 哪些数据常驻内存，哪些数据又需要保存在 Disk 中呢？ 37 | 38 | 2. 目前的 Blockchain 系统并没有像数据库系统 (DBMS) 那样统一的形成系统性的方法论。在 Ethereum 中每个不同的模块中都集成了大量的细节。从源码的角度出发可以了解到很多容易被忽视的细节。简单的说，一个完整的区块链系统至少包含以下的模块： 39 | - 密码学模块：加解密，签名，安全 hash，Mining 40 | - 网络模块：P2P 节点通信 41 | - 分布式共识模块：PoW, BFT，PoA 42 | - 智能合约解释器模块：Solidity 编译语言，EVM 解释器 43 | - 数据存储模块：数据库，Caching，数据存储，Index，LevelDB 44 | - Log 日志模块 45 | - etc. 46 | 47 | 而在具体实现中，由于设计理念，以及 go 语言的特性（没有继承派生关系），Go-ethereum 中的模块之间相互调用关系相对复杂。因此，只有通过阅读源码的方式才能更好理解不同模块之间的调用关系，以及业务的 workflow 中执行流程/细节。 48 | 49 | ### Blockchain System (BCS) VS Database Management System (DBMS) 50 | 51 | Blockchain 系统在设计层面借鉴了很多数据库系统中的设计逻辑。 52 | 53 | - Blockchain 系统同样也从 Transaction 作为基本的操作载核，包含一个 Parser 模块，Transaction Executor 模块，和一个 Storage 管理模块。 54 | 55 | ## Contents（暂定） 56 | 57 | ### PART ONE - General Source Code Analysis: Basic Workflow and Data Components 58 | 59 | - [00_万物的起点：Geth Start](CN/00_geth.md) 60 | - [01_Account and State](CN/01_account_state.md) 61 | - [02_State Management i: StateDB](CN/02_state_management_statedb.md) 62 | - [03_State Management ii: World State Trie and Storage Trie](CN/03_state_management_stateTrie.md) 63 | - [04_Transaction: 一个 Transaction 的生老病死](CN/04_transaction.md) 64 | - [05_从 Block 到 Blockchain: 链状区块结构的构建](CN/05_block_blockchain.md) 65 | - [06_一个网吧老板是怎么用闲置的电脑进行挖矿的] 66 | - [07_一个新节点是怎么加入网络并同步区块的] 67 | 68 | ### PART TWO - General Source Code Analysis: Lower-level Services 69 | 70 | - [10_状态数据优化：Batch and Pruning] 71 | - [11_Blockchain 的数据是如何持久化的：Leveldb in Practice] 72 | - [12_当 I/O 变成瓶颈：Caching in Practice] 73 | - [13_深入 EVM: 设计与实现] 74 | - [14_Signer: 如何证明 Transaction 的合法性] 75 | - [15_节点的调用 RPC and IPC](CN/15_rpc_ipc.md) 76 | 77 | ### PART THREE - Advanced Topics~P 78 | 79 | - [20_结合 BFT Consensus 解决拜占庭将军问题] 80 | - [21_从 Plasma 到 Rollup](CN/21_rollup.md) 81 | - [22_Authenticated data structures Brief](CN/22_ads.md) 82 | - [23_Bloom Filter](CN/23_bloom_filter.md) 83 | - [24_图灵机和停机问题] 84 | - [25_Log-structured merge-tree in Ethereum](CN/25_lsm_tree.md) 85 | - [26_Concurrency in Ethereum Transaction](CN/26_txn_concurrency.md) 86 | - [27_Zero-knowledge Proof] 87 | 88 | ### PART FOUR - Ethereum in Practice 89 | 90 | - [30_使用 geth 构建一个私有网络](CN/30_geth_private_network.md) 91 | - [31_如何编写 Solidity 语言](CN/31_solidity_in_practice.md) 92 | - [32_使用预言机 (Oracle) 构建随机化的 DApp] 93 | - [33_Query On Ethereum Data] 94 | - [34_layer2 in Practice] 95 | 96 | ### PART FIVE - APPENDIX 97 | 98 | - [40_FQA](#tips) 99 | - [41_Ethereum System Tunning] 100 | - [42_go-ethereum 的开发思想] 101 | - [43_Metrics in Ethereum] 102 | - [44_Golang & Ethereum] 103 | - [45_Looking Forward to the Ethereum2.0] 104 | 105 | ----------------------------------------------------------- 106 | 107 | ## How to measure the level of understanding of a system？ 108 | 109 | 如何衡量对一个系统的理解程度？ 110 | 111 | - Level 4: 掌握（Mastering） 112 | - 在完全理解的基础上，可以设计并编写一个新的系统 113 | - Level 3: 完全理解（Complete Understanding） 114 | - 在理解的基础上，完全掌握系统的各项实现的细节，并能做出优化 115 | - 可以对现有的系统定制化到不同的应用场景 116 | - Level 2: 理解（Understanding） 117 | - 熟练使用系统提供的 API 118 | - 了解系统模块的调用关系 119 | - 能对系统的部分模块进行简单修改/重构 120 | - Level 1: 了解（Brief understanding） 121 | - 了解系统设计的目标，了解系统的应用场景 122 | - 可以使用系统的部分的 API 123 | 124 | 我们希望读者在阅读完本系列之后，对 Ethereum 的理解能够达到 Level 2 - Level 3 的水平。 125 | 126 | ## Some Details 127 | 128 | - 以太坊是基于 State 状态机模型的区块链系统，Miner 在 update new Block 的时候，会直接修改自身的状态（添加区块奖励给自己。所以与 Bitcoin 不同的是，Ethereum 的区块中，并没有类似的 Coinbase 的 transaction。 129 | - 在 core/transaction.go 中，transaction 的的数据结构是有 time.Time 的参数的。但是在下面的 newTransaction 的 function 中只是使用 Local 的 time.now() 对 Transaction.time 进行初始化。 130 | - 在 core/transaction.go 的 transaction 数据结构定义的时候，在 transaction.time 后面的注释写到（Time first seen locally (spam avoidance), Time 只是用于在本地首次看到的时间。 131 | - uncle block 中的 transaction 不会被包括到主链上。 132 | - go-ethereum 有专用函数来控制每次 transaction 执行完，返还给用户的 Gas 的量。有根据 EIP-3529，每次最多返还 50%的 gas. 133 | - 不同的 Contracts 的数据会混合的保存在底层的一个 LevelDB instance 中。 134 | - LevelDB 中保存的是 MPT 的 Node 信息，包括 State Trie 和 Storage Trie。 135 | - 在以太坊更新数据相关的工作流中，通常先调用`Finalise`函数，然后执行`Commit`函数。 136 | 137 | ## 关键函数 138 | 139 | ```go 140 | // 向 leveldb 中更新 Storage 数据 141 | func WritePreimages(db ethdb.KeyValueWriter, preimages map[common.Hash][]byte) 142 | 143 | // 向 Blockchain 中添加新的 Block，会涉及到 StateDB(Memory)/Trie(Memory)/EthDB(Disk) 的更新 144 | func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals, setHead bool) (int, error) 145 | 146 | // insertChain 中调用来执行 Block 中的所有的交易 147 | func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg vm.Config) (types.Receipts, []*types.Log, uint64, error) 148 | 149 | //执行单条 Transaction 的调用 150 | func applyTransaction(msg types.Message, config *params.ChainConfig, bc ChainContext, author *common.Address, gp *GasPool, statedb *state.StateDB, blockNumber *big.Int, blockHash common.Hash, tx *types.Transaction, usedGas *uint64, evm *vm.EVM) (*types.Receipt, error) 151 | 152 | // 状态转移函数 153 | func (st *StateTransition) TransitionDb() (*ExecutionResult, error) 154 | 155 | // 执行合约内 function 156 | func (in *EVMInterpreter) Run(contract *Contract, input []byte, readOnly bool) (ret []byte, err error) 157 | 158 | // opSstore 的调用 159 | func (s *StateDB) SetState(addr common.Address, key, value common.Hash) 160 | // 被修改的 state 的值会首先被放在 StateObject 的 dirtyStorage 中，而不是直接添加到 Trie 或者 Disk Database 中。 161 | func (s *stateObject) setState(key, value common.Hash) 162 | 163 | // 根据当前的 State Trie 的值来重新计算 State Trie 的 Root，并返回改 Root 的值 164 | func (s *StateDB) IntermediateRoot(deleteEmptyObjects bool) common.Hash 165 | 166 | // Finalise 当前内存中的 Cache. 167 | func (s *StateDB) Finalise(deleteEmptyObjects bool) 168 | 169 | // Commit StateDB 中的 Cache 到内存数据库中 170 | func (s *StateDB) Commit(deleteEmptyObjects bool) (common.Hash, error) 171 | 172 | // 将 StateObject 中所有的 dirtyStorage 转存到 PendingStorage 中，并清空 dirtyStorage，并给 prefetcher 赋值 173 | func (s *stateObject) finalise(prefetch bool) 174 | 175 | // 更新 StorageObject 对应的 Trie, from Pending Storage 176 | func (s *stateObject) updateTrie(db Database) Trie 177 | 178 | // 最终获取到新的 StateObject 的 Storage Root 179 | func (t *Trie) hashRoot() (node, node, error) 180 | 181 | // 用于在内存数据库中保存 MPT 节点 182 | func (c *committer) store(n node, db *Database) node 183 | 184 | // 向 rawdb 对应的数据库写数据 (leveldb) 185 | func (db *Database) Commit(node common.Hash, report bool, callback func(common.Hash)) error 186 | 187 | ``` 188 | 189 | ## References 190 | 191 | 本书主要参考了 Go-Ethereum 代码库，Ethereum Yellow Paper，以及 EIP 的具体 Spec。读者可以从下面的 Link 中找到相关的引用资料。 192 | 193 | - [1] Ethereum Yellow Paper [(Paper Link)](https://ethereum.github.io/yellowpaper/paper.pdf) 194 | - [2] Ethereum/Go-Ethereum [(link)](https://github.com/ethereum/go-ethereum) 195 | - [3] Go-ethereum code analysis [(link)](https://github.com/ZtesoftCS/go-ethereum-code-analysis) 196 | - [4] Ethereum Improvement Proposals [(link)](https://github.com/ethereum/EIPs) 197 | - [5] Mastering Bitcoin(Second Edition) 198 | - [6] Mastering Ethereum [(link)](https://github.com/ethereumbook/ethereumbook) 199 | - [7] EIP-2718: Typed Transaction Envelope[(link)](https://eips.ethereum.org/EIPS/eip-2718) 200 | - [8] EIP-2930: Optional access lists [(link)](https://eips.ethereum.org/EIPS/eip-2930) 201 | - [9] EIP-1559: Fee market change for ETH 1.0 chain [(link)](https://eips.ethereum.org/EIPS/eip-1559) 202 | - [10] Ask about Geth: Snapshot acceleration [(link)](https://blog.ethereum.org/2020/07/17/ask-about-geth-snapshot-acceleration/) 203 | 204 | ## Talks 205 | 206 | - Succinct Proofs in Ethereum - Barry Whitehat, Ethereum Foundation [(Youtube)](https://www.youtube.com/watch?v=TtsDNneTDDY) 207 | -------------------------------------------------------------------------------- /example/account/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "encoding/hex" 5 | "fmt" 6 | "math/big" 7 | 8 | "github.com/ethereum/go-ethereum/common" 9 | "github.com/ethereum/go-ethereum/core/rawdb" 10 | "github.com/ethereum/go-ethereum/core/state" 11 | "github.com/ethereum/go-ethereum/core/state/snapshot" 12 | "github.com/ethereum/go-ethereum/crypto" 13 | 14 | solsha3 "github.com/miguelmota/go-solidity-sha3" 15 | ) 16 | 17 | var toHash = common.BytesToHash 18 | 19 | func main() { 20 | var snaps *snapshot.Tree 21 | stateDB, _ := state.New(common.Hash{}, state.NewDatabase(rawdb.NewMemoryDatabase()), snaps) 22 | 23 | account1 := common.HexToAddress("0x1111111111111111111111111111111111111111") 24 | account2 := common.HexToAddress("0x2222222222222222222222222222222222222222") 25 | 26 | stateDB.AddBalance(account1, big.NewInt(2000)) 27 | stateDB.AddBalance(account2, big.NewInt(8888)) 28 | contract := crypto.CreateAddress(account1, stateDB.GetNonce(account1)) 29 | stateDB.CreateAccount(contract) 30 | stateDB.SetCode(contract, []byte("contract code bytes")) 31 | 32 | stateDB.SetNonce(contract, 1) 33 | stateDB.SetState(contract, toHash([]byte("owner")), toHash(account1.Bytes())) 34 | stateDB.SetState(contract, toHash([]byte("name")), toHash([]byte("hsy"))) 35 | 36 | stateDB.SetState(contract, toHash([]byte("online")), toHash([]byte{1})) 37 | stateDB.SetState(contract, toHash([]byte("online")), toHash([]byte{})) 38 | 39 | stateDB.Commit(true) 40 | 41 | // TODO Geth changed this API. 42 | // fmt.Println(string(stateDB.Dump(true, true, true))) 43 | 44 | fmt.Println("------Test Hash-------") 45 | for i := 0; i <= 2; i++ { 46 | hash := solsha3.SoliditySHA3( 47 | solsha3.Uint256(big.NewInt(int64(i))), 48 | ) 49 | fmt.Printf("The hash of slot pos %d: 0x%x\n", i, hash) 50 | } 51 | 52 | // Test the map-type in contract storage 53 | // Map key/ Slot postion 54 | k1 := solsha3.SoliditySHA3([]byte("hsy"), solsha3.Uint256(big.NewInt(int64(1)))) 55 | fmt.Printf("Test the Solidity Map storage Key1: 0x%x\n", k1) 56 | 57 | // Corresponding Object slot index 58 | i1 := solsha3.SoliditySHA3(solsha3.Uint256(hexToBigInt(hex.EncodeToString(k1)))) 59 | fmt.Printf("Test the Solidity Map storage Key1's index: 0x%x\n", i1) 60 | 61 | k2 := solsha3.SoliditySHA3([]byte("lei"), solsha3.Uint256(big.NewInt(int64(1)))) 62 | fmt.Printf("Test the Solidity Map storage Key2: 0x%x\n", k2) 63 | 64 | i2 := solsha3.SoliditySHA3(solsha3.Uint256(hexToBigInt(hex.EncodeToString(k2)))) 65 | fmt.Printf("Test the Solidity Map storage Key2's index: 0x%x\n", i2) 66 | 67 | } 68 | 69 | func hexToBigInt(hex string) *big.Int { 70 | n := new(big.Int) 71 | n, _ = n.SetString(hex[:], 16) 72 | 73 | return n 74 | } 75 | -------------------------------------------------------------------------------- /example/deploy/DeployContract.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "crypto/ecdsa" 6 | "fmt" 7 | "log" 8 | "math/big" 9 | 10 | "github.com/ethereum/go-ethereum/accounts/abi/bind" 11 | "github.com/ethereum/go-ethereum/crypto" 12 | "github.com/ethereum/go-ethereum/ethclient" 13 | "github.com/hsyodyssey/qianmo/ethclient/contracts" 14 | ) 15 | 16 | func DeployContract() { 17 | client, err := ethclient.Dial("The RPC interface address") 18 | if err != nil { 19 | log.Fatal("Unable to connect to network:", err) 20 | } 21 | 22 | privateKey, err := crypto.HexToECDSA("You private Key") 23 | if err != nil { 24 | log.Fatal(err) 25 | } 26 | 27 | publicKey := privateKey.Public() 28 | 29 | publicKeyECDSA, ok := publicKey.(*ecdsa.PublicKey) 30 | 31 | if !ok { 32 | log.Fatal("cannot assert type: publicKey is not of type *ecdsa.PublicKey") 33 | } 34 | 35 | fromAddress := crypto.PubkeyToAddress(*publicKeyECDSA) 36 | nonce, err := client.PendingNonceAt(context.Background(), fromAddress) 37 | if err != nil { 38 | log.Fatal(err) 39 | } 40 | 41 | ChainID, err := client.NetworkID(context.Background()) 42 | if err != nil { 43 | log.Fatal(err) 44 | } 45 | 46 | gasPrice, err := client.SuggestGasPrice(context.Background()) 47 | if err != nil { 48 | log.Fatal(err) 49 | } 50 | 51 | auth, err := bind.NewKeyedTransactorWithChainID(privateKey, ChainID) 52 | if err != nil { 53 | log.Fatal(err) 54 | } 55 | auth.Nonce = big.NewInt(int64(nonce)) 56 | auth.Value = big.NewInt(0) 57 | auth.GasLimit = uint64(400000) 58 | auth.GasPrice = gasPrice 59 | 60 | // Deploy the Contract from the Contract Go file 61 | address, tx, instance, err := contracts.DeployStorage(auth, client) 62 | if err != nil { 63 | log.Fatal(err) 64 | } 65 | 66 | fmt.Println(address.Hex()) 67 | fmt.Println(tx.Hash().Hex()) 68 | 69 | _ = instance 70 | 71 | } 72 | -------------------------------------------------------------------------------- /example/deploy/SendTransaction.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "crypto/ecdsa" 6 | "fmt" 7 | "io/ioutil" 8 | "log" 9 | "math" 10 | "math/big" 11 | 12 | "github.com/ethereum/go-ethereum/common" 13 | "github.com/ethereum/go-ethereum/core/types" 14 | "github.com/ethereum/go-ethereum/crypto" 15 | "github.com/ethereum/go-ethereum/ethclient" 16 | ) 17 | 18 | func SendTx() { 19 | 20 | account := common.HexToAddress("You Account") 21 | 22 | data, _ := ioutil.ReadFile("key") 23 | fmt.Println(data) 24 | fmt.Println(string(data)) 25 | 26 | client, err := ethclient.Dial("The RPC interface address") 27 | if err != nil { 28 | log.Fatal("Unable to connect to network:", err) 29 | } 30 | 31 | // Get the Balance 32 | balance, err := client.BalanceAt(context.Background(), account, nil) 33 | 34 | fbalance := new(big.Float) 35 | fbalance.SetString(balance.String()) 36 | bnbValue := new(big.Float).Quo(fbalance, big.NewFloat(math.Pow10(18))) 37 | 38 | fmt.Println(bnbValue) 39 | 40 | if err != nil { 41 | log.Fatal(err) 42 | } 43 | 44 | privateKey, err := crypto.HexToECDSA("You private Key") 45 | if err != nil { 46 | log.Fatal(err) 47 | } 48 | 49 | publicKey := privateKey.Public() 50 | 51 | publicKeyECDSA, ok := publicKey.(*ecdsa.PublicKey) 52 | 53 | if !ok { 54 | log.Fatal("cannot assert type: publicKey is not of type *ecdsa.PublicKey") 55 | } 56 | 57 | fromAddress := crypto.PubkeyToAddress(*publicKeyECDSA) 58 | 59 | nonce, err := client.PendingNonceAt(context.Background(), fromAddress) 60 | if err != nil { 61 | log.Fatal(err) 62 | } 63 | 64 | // 18 zeros, transfer 1 ETH 65 | value := big.NewInt(1000000000000000000) 66 | 67 | gasLimit := uint64(21000) 68 | 69 | // Get the estimate Gas Price (Based on the average gas price in previous transaction gas price) 70 | gasPrice, err := client.SuggestGasPrice(context.Background()) 71 | if err != nil { 72 | log.Fatal(err) 73 | } 74 | 75 | // Construct the target Address 76 | toAddress := common.HexToAddress("Target Address") 77 | 78 | // Construct the raw Transaction (Unsigned) 79 | tx := types.NewTransaction(nonce, toAddress, value, gasLimit, gasPrice, nil) 80 | 81 | // Get the current ChainID 82 | ChainID, err := client.NetworkID(context.Background()) 83 | if err != nil { 84 | log.Fatal(err) 85 | } 86 | 87 | // Sign the transaction 88 | signedTx, err := types.SignTx(tx, types.NewEIP155Signer(ChainID), privateKey) 89 | 90 | if err != nil { 91 | log.Fatal(err) 92 | } 93 | 94 | // Send Signed Transactions 95 | err = client.SendTransaction(context.Background(), signedTx) 96 | if err != nil { 97 | log.Fatal(err) 98 | } 99 | 100 | } 101 | -------------------------------------------------------------------------------- /example/deploy/contracts/storage.go: -------------------------------------------------------------------------------- 1 | // Code generated - DO NOT EDIT. 2 | // This file is a generated binding and any manual changes will be lost. 3 | 4 | package contracts 5 | 6 | import ( 7 | "math/big" 8 | "strings" 9 | 10 | ethereum "github.com/ethereum/go-ethereum" 11 | "github.com/ethereum/go-ethereum/accounts/abi" 12 | "github.com/ethereum/go-ethereum/accounts/abi/bind" 13 | "github.com/ethereum/go-ethereum/common" 14 | "github.com/ethereum/go-ethereum/core/types" 15 | "github.com/ethereum/go-ethereum/event" 16 | ) 17 | 18 | // Reference imports to suppress errors if they are not otherwise used. 19 | var ( 20 | _ = big.NewInt 21 | _ = strings.NewReader 22 | _ = ethereum.NotFound 23 | _ = bind.Bind 24 | _ = common.Big1 25 | _ = types.BloomLookup 26 | _ = event.NewSubscription 27 | ) 28 | 29 | // StorageABI is the input ABI used to generate the binding from. 30 | const StorageABI = "[{\"inputs\":[],\"name\":\"retrieve\",\"outputs\":[{\"internalType\":\"uint256\",\"name\":\"\",\"type\":\"uint256\"}],\"stateMutability\":\"view\",\"type\":\"function\"},{\"inputs\":[{\"internalType\":\"uint256\",\"name\":\"num\",\"type\":\"uint256\"}],\"name\":\"store\",\"outputs\":[],\"stateMutability\":\"nonpayable\",\"type\":\"function\"}]" 31 | 32 | // StorageFuncSigs maps the 4-byte function signature to its string representation. 33 | var StorageFuncSigs = map[string]string{ 34 | "2e64cec1": "retrieve()", 35 | "6057361d": "store(uint256)", 36 | } 37 | 38 | // StorageBin is the compiled bytecode used for deploying new contracts. 39 | var StorageBin = "0x608060405234801561001057600080fd5b5060ac8061001f6000396000f3fe6080604052348015600f57600080fd5b506004361060325760003560e01c80632e64cec11460375780636057361d14604f575b600080fd5b603d606b565b60408051918252519081900360200190f35b606960048036036020811015606357600080fd5b50356071565b005b60005490565b60005556fea264697066735822122040be9e16a517d5253bf91c0cbe99165f29dcd3ff1171fa3d30e11bb1d4bde5c464736f6c63430007010033" 40 | 41 | // DeployStorage deploys a new Ethereum contract, binding an instance of Storage to it. 42 | func DeployStorage(auth *bind.TransactOpts, backend bind.ContractBackend) (common.Address, *types.Transaction, *Storage, error) { 43 | parsed, err := abi.JSON(strings.NewReader(StorageABI)) 44 | if err != nil { 45 | return common.Address{}, nil, nil, err 46 | } 47 | 48 | address, tx, contract, err := bind.DeployContract(auth, parsed, common.FromHex(StorageBin), backend) 49 | if err != nil { 50 | return common.Address{}, nil, nil, err 51 | } 52 | return address, tx, &Storage{StorageCaller: StorageCaller{contract: contract}, StorageTransactor: StorageTransactor{contract: contract}, StorageFilterer: StorageFilterer{contract: contract}}, nil 53 | } 54 | 55 | // Storage is an auto generated Go binding around an Ethereum contract. 56 | type Storage struct { 57 | StorageCaller // Read-only binding to the contract 58 | StorageTransactor // Write-only binding to the contract 59 | StorageFilterer // Log filterer for contract events 60 | } 61 | 62 | // StorageCaller is an auto generated read-only Go binding around an Ethereum contract. 63 | type StorageCaller struct { 64 | contract *bind.BoundContract // Generic contract wrapper for the low level calls 65 | } 66 | 67 | // StorageTransactor is an auto generated write-only Go binding around an Ethereum contract. 68 | type StorageTransactor struct { 69 | contract *bind.BoundContract // Generic contract wrapper for the low level calls 70 | } 71 | 72 | // StorageFilterer is an auto generated log filtering Go binding around an Ethereum contract events. 73 | type StorageFilterer struct { 74 | contract *bind.BoundContract // Generic contract wrapper for the low level calls 75 | } 76 | 77 | // StorageSession is an auto generated Go binding around an Ethereum contract, 78 | // with pre-set call and transact options. 79 | type StorageSession struct { 80 | Contract *Storage // Generic contract binding to set the session for 81 | CallOpts bind.CallOpts // Call options to use throughout this session 82 | TransactOpts bind.TransactOpts // Transaction auth options to use throughout this session 83 | } 84 | 85 | // StorageCallerSession is an auto generated read-only Go binding around an Ethereum contract, 86 | // with pre-set call options. 87 | type StorageCallerSession struct { 88 | Contract *StorageCaller // Generic contract caller binding to set the session for 89 | CallOpts bind.CallOpts // Call options to use throughout this session 90 | } 91 | 92 | // StorageTransactorSession is an auto generated write-only Go binding around an Ethereum contract, 93 | // with pre-set transact options. 94 | type StorageTransactorSession struct { 95 | Contract *StorageTransactor // Generic contract transactor binding to set the session for 96 | TransactOpts bind.TransactOpts // Transaction auth options to use throughout this session 97 | } 98 | 99 | // StorageRaw is an auto generated low-level Go binding around an Ethereum contract. 100 | type StorageRaw struct { 101 | Contract *Storage // Generic contract binding to access the raw methods on 102 | } 103 | 104 | // StorageCallerRaw is an auto generated low-level read-only Go binding around an Ethereum contract. 105 | type StorageCallerRaw struct { 106 | Contract *StorageCaller // Generic read-only contract binding to access the raw methods on 107 | } 108 | 109 | // StorageTransactorRaw is an auto generated low-level write-only Go binding around an Ethereum contract. 110 | type StorageTransactorRaw struct { 111 | Contract *StorageTransactor // Generic write-only contract binding to access the raw methods on 112 | } 113 | 114 | // NewStorage creates a new instance of Storage, bound to a specific deployed contract. 115 | func NewStorage(address common.Address, backend bind.ContractBackend) (*Storage, error) { 116 | contract, err := bindStorage(address, backend, backend, backend) 117 | if err != nil { 118 | return nil, err 119 | } 120 | return &Storage{StorageCaller: StorageCaller{contract: contract}, StorageTransactor: StorageTransactor{contract: contract}, StorageFilterer: StorageFilterer{contract: contract}}, nil 121 | } 122 | 123 | // NewStorageCaller creates a new read-only instance of Storage, bound to a specific deployed contract. 124 | func NewStorageCaller(address common.Address, caller bind.ContractCaller) (*StorageCaller, error) { 125 | contract, err := bindStorage(address, caller, nil, nil) 126 | if err != nil { 127 | return nil, err 128 | } 129 | return &StorageCaller{contract: contract}, nil 130 | } 131 | 132 | // NewStorageTransactor creates a new write-only instance of Storage, bound to a specific deployed contract. 133 | func NewStorageTransactor(address common.Address, transactor bind.ContractTransactor) (*StorageTransactor, error) { 134 | contract, err := bindStorage(address, nil, transactor, nil) 135 | if err != nil { 136 | return nil, err 137 | } 138 | return &StorageTransactor{contract: contract}, nil 139 | } 140 | 141 | // NewStorageFilterer creates a new log filterer instance of Storage, bound to a specific deployed contract. 142 | func NewStorageFilterer(address common.Address, filterer bind.ContractFilterer) (*StorageFilterer, error) { 143 | contract, err := bindStorage(address, nil, nil, filterer) 144 | if err != nil { 145 | return nil, err 146 | } 147 | return &StorageFilterer{contract: contract}, nil 148 | } 149 | 150 | // bindStorage binds a generic wrapper to an already deployed contract. 151 | func bindStorage(address common.Address, caller bind.ContractCaller, transactor bind.ContractTransactor, filterer bind.ContractFilterer) (*bind.BoundContract, error) { 152 | parsed, err := abi.JSON(strings.NewReader(StorageABI)) 153 | if err != nil { 154 | return nil, err 155 | } 156 | return bind.NewBoundContract(address, parsed, caller, transactor, filterer), nil 157 | } 158 | 159 | // Call invokes the (constant) contract method with params as input values and 160 | // sets the output to result. The result type might be a single field for simple 161 | // returns, a slice of interfaces for anonymous returns and a struct for named 162 | // returns. 163 | func (_Storage *StorageRaw) Call(opts *bind.CallOpts, result *[]interface{}, method string, params ...interface{}) error { 164 | return _Storage.Contract.StorageCaller.contract.Call(opts, result, method, params...) 165 | } 166 | 167 | // Transfer initiates a plain transaction to move funds to the contract, calling 168 | // its default method if one is available. 169 | func (_Storage *StorageRaw) Transfer(opts *bind.TransactOpts) (*types.Transaction, error) { 170 | return _Storage.Contract.StorageTransactor.contract.Transfer(opts) 171 | } 172 | 173 | // Transact invokes the (paid) contract method with params as input values. 174 | func (_Storage *StorageRaw) Transact(opts *bind.TransactOpts, method string, params ...interface{}) (*types.Transaction, error) { 175 | return _Storage.Contract.StorageTransactor.contract.Transact(opts, method, params...) 176 | } 177 | 178 | // Call invokes the (constant) contract method with params as input values and 179 | // sets the output to result. The result type might be a single field for simple 180 | // returns, a slice of interfaces for anonymous returns and a struct for named 181 | // returns. 182 | func (_Storage *StorageCallerRaw) Call(opts *bind.CallOpts, result *[]interface{}, method string, params ...interface{}) error { 183 | return _Storage.Contract.contract.Call(opts, result, method, params...) 184 | } 185 | 186 | // Transfer initiates a plain transaction to move funds to the contract, calling 187 | // its default method if one is available. 188 | func (_Storage *StorageTransactorRaw) Transfer(opts *bind.TransactOpts) (*types.Transaction, error) { 189 | return _Storage.Contract.contract.Transfer(opts) 190 | } 191 | 192 | // Transact invokes the (paid) contract method with params as input values. 193 | func (_Storage *StorageTransactorRaw) Transact(opts *bind.TransactOpts, method string, params ...interface{}) (*types.Transaction, error) { 194 | return _Storage.Contract.contract.Transact(opts, method, params...) 195 | } 196 | 197 | // Retrieve is a free data retrieval call binding the contract method 0x2e64cec1. 198 | // 199 | // Solidity: function retrieve() view returns(uint256) 200 | func (_Storage *StorageCaller) Retrieve(opts *bind.CallOpts) (*big.Int, error) { 201 | var out []interface{} 202 | err := _Storage.contract.Call(opts, &out, "retrieve") 203 | 204 | if err != nil { 205 | return *new(*big.Int), err 206 | } 207 | 208 | out0 := *abi.ConvertType(out[0], new(*big.Int)).(**big.Int) 209 | 210 | return out0, err 211 | 212 | } 213 | 214 | // Retrieve is a free data retrieval call binding the contract method 0x2e64cec1. 215 | // 216 | // Solidity: function retrieve() view returns(uint256) 217 | func (_Storage *StorageSession) Retrieve() (*big.Int, error) { 218 | return _Storage.Contract.Retrieve(&_Storage.CallOpts) 219 | } 220 | 221 | // Retrieve is a free data retrieval call binding the contract method 0x2e64cec1. 222 | // 223 | // Solidity: function retrieve() view returns(uint256) 224 | func (_Storage *StorageCallerSession) Retrieve() (*big.Int, error) { 225 | return _Storage.Contract.Retrieve(&_Storage.CallOpts) 226 | } 227 | 228 | // Store is a paid mutator transaction binding the contract method 0x6057361d. 229 | // 230 | // Solidity: function store(uint256 num) returns() 231 | func (_Storage *StorageTransactor) Store(opts *bind.TransactOpts, num *big.Int) (*types.Transaction, error) { 232 | return _Storage.contract.Transact(opts, "store", num) 233 | } 234 | 235 | // Store is a paid mutator transaction binding the contract method 0x6057361d. 236 | // 237 | // Solidity: function store(uint256 num) returns() 238 | func (_Storage *StorageSession) Store(num *big.Int) (*types.Transaction, error) { 239 | return _Storage.Contract.Store(&_Storage.TransactOpts, num) 240 | } 241 | 242 | // Store is a paid mutator transaction binding the contract method 0x6057361d. 243 | // 244 | // Solidity: function store(uint256 num) returns() 245 | func (_Storage *StorageTransactorSession) Store(num *big.Int) (*types.Transaction, error) { 246 | return _Storage.Contract.Store(&_Storage.TransactOpts, num) 247 | } 248 | -------------------------------------------------------------------------------- /example/private_chain/genesis.json: -------------------------------------------------------------------------------- 1 | { 2 | "config": { 3 | "chainId": 7788, 4 | "homesteadBlock": 0, 5 | "eip150Block": 0, 6 | "eip155Block": 0, 7 | "eip158Block": 0, 8 | "byzantiumBlock": 0, 9 | "constantinopleBlock": 0, 10 | "petersburgBlock": 0, 11 | "istanbulBlock": 0, 12 | "berlinBlock": 0 13 | }, 14 | "alloc": { 15 | "0x8F07CF20cc5A76887Ae07131a633E9DEdCB1edF7": { 16 | "balance": "90000000000000000000000" 17 | }, 18 | "0x570B54D10C343638D4a2630c5498E914F8661571": { 19 | "balance": "80000000000000000000000" 20 | }, 21 | "0x11DFe4Ca0983789716aF72821588Cd2Dc0889562": { 22 | "balance": "10000000000000000000000" 23 | }, 24 | "0xA658c6cB5bd763bd30bEa093522dFb576E348cfb": { 25 | "balance": "10000000000000000000000" 26 | }, 27 | "0x7d5F2f0C9A7f4B9007f273dd743c88FeF10412aA": { 28 | "balance": "10000000000000000000000" 29 | }, 30 | "0x12F051579BfFEbF63de6CDFaFAc69D6242c61573": { 31 | "balance": "10000000000000000000000" 32 | }, 33 | "0x13399Fc5eb5a347F01827E18011b623C2c740198": { 34 | "balance": "10000000000000000000000" 35 | }, 36 | "0xab185D96f112130c3119e3E3D941398cbCAb455a": { 37 | "balance": "10000000000000000000000" 38 | } 39 | }, 40 | "coinbase": "0x0000000000000000000000000000000000000000", 41 | "difficulty": "0x20000", 42 | "extraData": "", 43 | "gasLimit": "0x2fefd8", 44 | "nonce": "0x00000000000000755", 45 | "mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000", 46 | "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000", 47 | "timestamp": "0x00" 48 | } -------------------------------------------------------------------------------- /example/private_chain/init.sh: -------------------------------------------------------------------------------- 1 | # 1. First init new genesis state 2 | geth init --datadir genesis.json 3 | 4 | # 2. Based on Local Chaindata and the Newwork id, start Geth. 5 | geth --datadir --networkid --nodiscover --http --rpc --rpcport "8545" --rpcaddr "0.0.0.0" --rpccorsdomain "*" --rpcapi "eth,web3,net,personal,miner" console 2 6 | 7 | 8 | -------------------------------------------------------------------------------- /example/signature/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "crypto/sha256" 5 | "encoding/hex" 6 | "fmt" 7 | "log" 8 | 9 | "github.com/ethereum/go-ethereum/crypto" 10 | ) 11 | 12 | // 随机产生的一个私钥 13 | var AlicePrivateKey = "289c2857d4598e37fb9647507e47a309d6133539bf21a8b9cb6df88fd5232032" 14 | 15 | func main() { 16 | // 需要加密的数据 17 | msg := sha256.Sum256([]byte("ethereum")) 18 | 19 | // The account's Private Key 20 | privateKey, err := hex.DecodeString(AlicePrivateKey) 21 | 22 | if err != nil { 23 | log.Fatalln("PK Generate Failed", err) 24 | } 25 | 26 | // Generate SK，通过privatekey来生成ECDSA中的对应的私钥SK 27 | ecdsaSK, err := crypto.ToECDSA(privateKey) 28 | if err != nil { 29 | log.Fatalln("SK Generate Failed", err) 30 | } 31 | // 通过ECDSA下的pk来生成Account Address 32 | fmt.Println("Account Address is: ", crypto.PubkeyToAddress(ecdsaSK.PublicKey)) 33 | fmt.Println("Account Private key is: ", hex.EncodeToString(privateKey)) 34 | fmt.Println("-----------------Original-----------------------") 35 | fmt.Println("[Original ECDSA] Account Public key is: ", hex.EncodeToString(crypto.FromECDSAPub(&ecdsaSK.PublicKey))) 36 | fmt.Println("[Original ECDSA] Account Public key is: ", hex.EncodeToString(crypto.CompressPubkey(&ecdsaSK.PublicKey))) 37 | 38 | // Sign the Data 39 | sig, err := crypto.Sign(msg[:], ecdsaSK) 40 | if err != nil { 41 | log.Fatalln("Sign Failed", err) 42 | } 43 | 44 | decodeHex := func(s string) []byte { 45 | b, err := hex.DecodeString(s) 46 | if err != nil { 47 | log.Fatal(err) 48 | } 49 | return b 50 | } 51 | 52 | // 签名后的数据 53 | msgSig := decodeHex(hex.EncodeToString(sig)) 54 | 55 | // Recover the Public Key of the Account from the Sig and the Message 56 | // 通过transaction的原文和签名后的数据来恢复ECDSA下对应的Pk 57 | recoveredPub, err := crypto.Ecrecover(msg[:], msgSig) 58 | if err != nil { 59 | log.Fatal(err) 60 | } 61 | 62 | // Get the Account Public Key 63 | pubKey, _ := crypto.UnmarshalPubkey(recoveredPub) 64 | 65 | // Get the Account Public Key Bytes 66 | recoveredPubBytes := crypto.FromECDSAPub(pubKey) 67 | fmt.Println("------------------Recovered--------------------") 68 | 69 | fmt.Println("[Recovered] Account Public Key: ", hex.EncodeToString(recoveredPubBytes)) 70 | fmt.Println("[Recovered] Account Compressed Public Key: ", hex.EncodeToString(crypto.CompressPubkey(pubKey))) 71 | // fmt.Println("[Recovered] Recover the Public Key: ", recoveredPub) 72 | // fmt.Println("[Recovered] UnmarshalPubkey: ", pubKey) 73 | // fmt.Println("[Recovered] Account Pubkey bytes: ", recoveredPubBytes) 74 | 75 | fmt.Println("----------------------------------------------") 76 | 77 | testPk := decodeHex("037db227d7094ce215c3a0f57e1bcc732551fe351f94249471934567e0f5dc1bf7") 78 | 79 | // Verify by the original public key 80 | longVerify := crypto.VerifySignature(recoveredPub, msg[:], msgSig[:len(msgSig)-1]) 81 | 82 | fmt.Println("[Original Public Key] verify pass?", longVerify) 83 | 84 | // Verify by the compressed public key 85 | shortVerify := crypto.VerifySignature(testPk, msg[:], msgSig[:len(msgSig)-1]) 86 | 87 | fmt.Println("[Compressed Public Key] verify pass?", shortVerify) 88 | 89 | } 90 | -------------------------------------------------------------------------------- /example/workflow/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | ) 7 | 8 | type Node struct { 9 | backend Backend 10 | stop chan struct{} 11 | } 12 | 13 | type Backend struct { 14 | balance int 15 | } 16 | 17 | func (n *Node) Wait() { 18 | <-n.stop 19 | } 20 | 21 | func (b *Backend) Mainloop() { 22 | for ; b.balance != 0; b.balance-- { 23 | time.Sleep(1 * time.Second) 24 | fmt.Println("Current Balance: ", b.balance) 25 | } 26 | } 27 | 28 | func NewNode() *Node { 29 | n := &Node{ 30 | backend: *NewBackend(), 31 | stop: make(chan struct{}), 32 | } 33 | return n 34 | } 35 | 36 | func NewBackend() *Backend { 37 | b := &Backend{ 38 | balance: 100, 39 | } 40 | return b 41 | } 42 | 43 | func main() { 44 | n := NewNode() 45 | go n.backend.Mainloop() 46 | go func() { 47 | // Close the Node after 10 second 48 | time.Sleep(10 * time.Second) 49 | close(n.stop) 50 | }() 51 | n.Wait() 52 | } 53 | -------------------------------------------------------------------------------- /figs/01/account_storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/figs/01/account_storage.png -------------------------------------------------------------------------------- /figs/01/remix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/figs/01/remix.png -------------------------------------------------------------------------------- /figs/02/tx_execu_flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/figs/02/tx_execu_flow.png -------------------------------------------------------------------------------- /figs/04/tx_exec_calls.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helloswx/Understanding-Ethereum-Go-version/4a057eeeac419077a78382543d2a7bb82bee7896/figs/04/tx_exec_calls.png --------------------------------------------------------------------------------