├── LICENSE ├── README.md ├── data.sol ├── patricia.sol └── utils.sol /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 chriseth 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Verifying evm-based sidechain executions 2 | 3 | 4 | Specific smart contract that can be used both off-chain and on-chain for verification. 5 | Exactly the same code will run off- (for actually performing the computations) and on-chain 6 | (for settling disputes). The on-chain code will be much faster because it has two 7 | execution modes (mostly relevant to the underlying data storage): 8 | 9 | 1) direct: 10 | 11 | The smart contract gets the full state (encoded in a certain way or just stored in 12 | storage) plus the transaction as input. It updates the state thereby calculating 13 | 14 | (a) helper values for verification, 15 | (b) Merkle proofs for all reads and writes into the state and 16 | (c) the new root hash 17 | 18 | 2) with helpers / witnesses: 19 | 20 | as in 1, just that the helper values that are generated in 1(a) are also added to the input. 21 | Those are used to speed up the computation like in the following example. 22 | If this is run with ``_helper == 0``, it will produce a value to be found and a value for helper. 23 | If that helper value is used in the next run, it will speed up the search. At the same time, 24 | there is no value for ``_helper`` which can produce a different return value (wrong values 25 | will only cause the transaction to be reverted). 26 | 27 | ```solidity 28 | function findValue(uint[] a, uint x, uint _helper) returns (uint value, uint hepper) { 29 | // a is sorted by increasing value 30 | for (uint i = _helper; i < a.length; i++) { 31 | if (array[i] >= x && (i == 0 || array[i-1] < x)) 32 | return (array[i], i); 33 | } 34 | // it is important to detect an invalid value of _helper 35 | revert(); 36 | } 37 | ``` 38 | 39 | 3) with helpers plus merkle proofs for all state access 40 | 41 | In all three variants, state access has to be performed by special functions: 42 | 43 | * ``readState(MerkleTree _state, bytes _key, bytes _merkleProofInput) returns (bytes _value, bytes _merkleProof)`` 44 | * ``writeState(MerkleTree _state, bytes _key, bytes _value, bytes _merkleProofInput) returns (bytes _merkleProof)`` 45 | 46 | If run in the first mode, the input merkle proofs are empty. Instead they are generated and provided as return value. 47 | ``writeState`` updates the merkle tree in place. 48 | 49 | If run in the second mode, the merkle proof can be provided as input. In this case, (for read) it already contains the 50 | value to be read and is only verified against the state. The function just returns the same proof again. 51 | For ``writeState``, the state is also updated in place, but it is assumed to consist only of the root hash anyway. 52 | 53 | ### More formal writeup (work in progress) 54 | 55 | This technique allows to re-run and and in particular verify transactions such that the second run is much more 56 | efficient than the first run. The use-case is to process transactions in a private chain and verify them on the 57 | public chain only on demand and thus take load off the public chain. Since the second run will not generate 58 | the full state, the operators of the private chain have to ensure data availability and can be punished if 59 | they do not provide required data. 60 | 61 | Let `f` be a fixed function taking three parameters and returning two values: 62 | 63 | - Let `f(s, x) = (s', y, w)` be the result of running `f` in full mode (`0`) on the current state `s` with input `x`, producing the new state `s'`, actual output `y` and witness `w` 64 | 65 | We call `g` a verifier of `f` using the hash function `H` (think of `H` as returning the root of a Merkle tree for structured input), if for all `s` and `x` and `(s', y, w) = f(s, x)` it holds that `g(H(s), x, w) = (H(s'), y)`. 66 | 67 | ### Patricia-Merkle-Tree 68 | 69 | The Patricia-Merkle-Tree encodes an arbitrary partial mapping `m: bits -> bits` as an authenticated data structure with 70 | short proofs. Let `<.> -> bits256` be a hash function. We first transform `m` to `M` by `M() = v` for `m(k) = v`. 71 | 72 | For a prefix `pref` of a key in `M` let `path_M(pref: bits)` be the longest bit string `b` such that `pref . b` 73 | is a unique prefix of keys in `M`. If `pref` is not a prefix of a key in `M`, then `path_M(pref)` is the empty bit string. 74 | 75 | We now define a partial function `node_M` that takes a prefix and returns the value of a node in the Patricia-Merkle-tree. 76 | `` is called the root hash of `M` if `M` is non-empty. If it is empty, then 77 | the root hash is the all-zero bitstring of 256 bits. 78 | 79 | #### Definition 80 | 81 | For an arbitrary prefix of a key in `M`, `node(pref)` is defined as follows: 82 | 83 | node_M(pref) = 84 | - M(pref) if pref is a key in M 85 | - . otherwise 86 | . 87 | length(path_M(pref . '0')): uint8 . 88 | length(path_M(pref . '1')): uint8 . 89 | path_M(pref . '0')) [ padded to multiples of 8 bits ] . 90 | path_M(pref . '1')) [ padded to multiples of 8 bits ] 91 | 92 | #### Definition 93 | 94 | A bit string `p` is called a *branch point* of `M` if it is a longest unique prefix 95 | of a key in `M`, but not equal to a key. 96 | 97 | #### Observation 98 | 99 | For a branch point `p` of `M`, `node_M(p)` only contains recursive mentions 100 | of `node_M` with branch points or keys of `M` as arguments. 101 | 102 | For a key `k` of `M`, `node_M(k)` does not contain recursive mentions 103 | of `node_M`. 104 | 105 | From this it follows that if `p` is a prefix of a key in `M`, then `node_M(p . path_M(p))` does 106 | not contains recursive mentions of `node_M` where the argument is not a prefix 107 | of a key in `M`. 108 | -------------------------------------------------------------------------------- /data.sol: -------------------------------------------------------------------------------- 1 | pragma solidity >=0.5.0 <0.6.0; 2 | 3 | library D { 4 | struct Label { 5 | bytes32 data; 6 | uint length; 7 | } 8 | struct Edge { 9 | bytes32 node; 10 | Label label; 11 | } 12 | struct Node { 13 | Edge[2] children; 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /patricia.sol: -------------------------------------------------------------------------------- 1 | pragma solidity >=0.5.0 <0.6.0; 2 | 3 | import {D} from "./data.sol"; 4 | import {Utils} from "./utils.sol"; 5 | 6 | contract PatriciaTree { 7 | // Mapping of hash of key to value 8 | mapping (bytes32 => bytes) values; 9 | // Particia tree nodes (hash to decoded contents) 10 | mapping (bytes32 => D.Node) nodes; 11 | // The current root hash, keccak256(node(path_M('')), path_M('')) 12 | bytes32 public root; 13 | D.Edge rootEdge; 14 | 15 | // TODO also return the proof 16 | function insert(bytes memory key, bytes memory value) public { 17 | D.Label memory k = D.Label(keccak256(key), 256); 18 | bytes32 valueHash = keccak256(value); 19 | values[valueHash] = value; 20 | // keys.push(key); 21 | D.Edge memory e; 22 | if (rootEdge.node == 0 && rootEdge.label.length == 0) 23 | { 24 | // Empty Trie 25 | e.label = k; 26 | e.node = valueHash; 27 | } 28 | else 29 | { 30 | e = insertAtEdge(rootEdge, k, valueHash); 31 | } 32 | root = edgeHash(e); 33 | rootEdge = e; 34 | } 35 | 36 | function getNode(bytes32 hash) public view returns (uint, bytes32, bytes32, uint, bytes32, bytes32) { 37 | D.Node memory n = nodes[hash]; 38 | return ( 39 | n.children[0].label.length, n.children[0].label.data, n.children[0].node, 40 | n.children[1].label.length, n.children[1].label.data, n.children[1].node 41 | ); 42 | } 43 | 44 | function getRootEdge() public view returns (uint, bytes32, bytes32) { 45 | return (rootEdge.label.length, rootEdge.label.data, rootEdge.node); 46 | } 47 | 48 | // Returns the Merkle-proof for the given key 49 | // Proof format should be: 50 | // - uint branchMask - bitmask with high bits at the positions in the key 51 | // where we have branch nodes (bit in key denotes direction) 52 | // - bytes32[] hashes - hashes of sibling edges 53 | function getProof(bytes memory key) public view returns (uint branchMask, bytes32[] memory _siblings) { 54 | D.Label memory k = D.Label(keccak256(key), 256); 55 | D.Edge memory e = rootEdge; 56 | bytes32[256] memory siblings; 57 | uint length; 58 | uint numSiblings; 59 | while (true) { 60 | (D.Label memory prefix, D.Label memory suffix) = Utils.splitCommonPrefix(k, e.label); 61 | require(prefix.length == e.label.length, "Prefix lenght mismatch label lenght"); 62 | if (suffix.length == 0) { 63 | // Found it 64 | break; 65 | } 66 | length += prefix.length; 67 | branchMask |= uint(1) << (255 - length); 68 | length += 1; 69 | (uint head, D.Label memory tail) = Utils.chopFirstBit(suffix); 70 | siblings[numSiblings++] = edgeHash(nodes[e.node].children[1 - head]); 71 | e = nodes[e.node].children[head]; 72 | k = tail; 73 | } 74 | if (numSiblings > 0) 75 | { 76 | _siblings = new bytes32[](numSiblings); 77 | for (uint i = 0; i < numSiblings; i++) 78 | _siblings[i] = siblings[i]; 79 | } 80 | } 81 | 82 | function verifyProof(bytes32 rootHash, bytes memory key, bytes memory value, uint branchMask, bytes32[] memory siblings) public pure { 83 | D.Label memory k = D.Label(keccak256(key), 256); 84 | D.Edge memory e; 85 | e.node = keccak256(value); 86 | uint b = branchMask; 87 | for (uint i = 0; b != 0; i++) { 88 | uint bitSet = Utils.lowestBitSet(b); 89 | b &= ~(uint(1) << bitSet); 90 | (k, e.label) = Utils.splitAt(k, 255 - bitSet); 91 | uint bit; 92 | (bit, e.label) = Utils.chopFirstBit(e.label); 93 | bytes32[2] memory edgeHashes; 94 | edgeHashes[bit] = edgeHash(e); 95 | edgeHashes[1 - bit] = siblings[siblings.length - i - 1]; 96 | e.node = keccak256(abi.encodePacked(edgeHashes)); 97 | } 98 | e.label = k; 99 | require(rootHash == edgeHash(e), "Bad proof"); 100 | } 101 | 102 | function insertAtNode(bytes32 nodeHash, D.Label memory key, bytes32 value) internal returns (bytes32) { 103 | require(key.length > 1, "Bad key"); 104 | D.Node memory n = nodes[nodeHash]; 105 | (uint256 head, D.Label memory tail) = Utils.chopFirstBit(key); 106 | n.children[head] = insertAtEdge(n.children[head], tail, value); 107 | return replaceNode(nodeHash, n); 108 | } 109 | 110 | function insertAtEdge(D.Edge memory e, D.Label memory key, bytes32 value) internal returns (D.Edge memory) { 111 | require(key.length >= e.label.length, "Key lenght mismatch label lenght"); 112 | (D.Label memory prefix, D.Label memory suffix) = Utils.splitCommonPrefix(key, e.label); 113 | bytes32 newNodeHash; 114 | if (suffix.length == 0) { 115 | // Full match with the key, update operation 116 | newNodeHash = value; 117 | } else if (prefix.length >= e.label.length) { 118 | // Partial match, just follow the path 119 | newNodeHash = insertAtNode(e.node, suffix, value); 120 | } else { 121 | // Mismatch, so let us create a new branch node. 122 | (uint256 head, D.Label memory tail) = Utils.chopFirstBit(suffix); 123 | D.Node memory branchNode; 124 | branchNode.children[head] = D.Edge(value, tail); 125 | branchNode.children[1 - head] = D.Edge(e.node, Utils.removePrefix(e.label, prefix.length + 1)); 126 | newNodeHash = insertNode(branchNode); 127 | } 128 | return D.Edge(newNodeHash, prefix); 129 | } 130 | 131 | function insertNode(D.Node memory n) internal returns (bytes32 newHash) { 132 | bytes32 h = hash(n); 133 | nodes[h].children[0] = n.children[0]; 134 | nodes[h].children[1] = n.children[1]; 135 | return h; 136 | } 137 | 138 | function replaceNode(bytes32 oldHash, D.Node memory n) internal returns (bytes32 newHash) { 139 | delete nodes[oldHash]; 140 | return insertNode(n); 141 | } 142 | 143 | function edgeHash(D.Edge memory e) internal pure returns (bytes32) { 144 | return keccak256(abi.encodePacked(e.node, e.label.length, e.label.data)); 145 | } 146 | 147 | // Returns the hash of the encoding of a node. 148 | function hash(D.Node memory n) internal pure returns (bytes32) { 149 | return keccak256(abi.encodePacked(edgeHash(n.children[0]), edgeHash(n.children[1]))); 150 | } 151 | } 152 | 153 | 154 | contract PatriciaTreeTest is PatriciaTree { 155 | function test() public { 156 | //testInsert(); 157 | testProofs(); 158 | } 159 | 160 | function testInsert() internal { 161 | insert("one", "ONE"); 162 | insert("two", "ONE"); 163 | insert("three", "ONE"); 164 | insert("four", "ONE"); 165 | insert("five", "ONE"); 166 | insert("six", "ONE"); 167 | insert("seven", "ONE"); 168 | // update 169 | insert("one", "TWO"); 170 | } 171 | 172 | function testProofs() internal { 173 | insert("one", "ONE"); 174 | (uint branchMask, bytes32[] memory siblings) = getProof("one"); 175 | verifyProof(root, "one", "ONE", branchMask, siblings); 176 | insert("two", "TWO"); 177 | (branchMask, siblings) = getProof("one"); 178 | verifyProof(root, "one", "ONE", branchMask, siblings); 179 | (branchMask, siblings) = getProof("two"); 180 | verifyProof(root, "two", "TWO", branchMask, siblings); 181 | } 182 | } 183 | -------------------------------------------------------------------------------- /utils.sol: -------------------------------------------------------------------------------- 1 | pragma solidity >=0.5.0 <0.6.0; 2 | 3 | import {D} from "./data.sol"; 4 | 5 | library Utils { 6 | /// Returns a label containing the longest common prefix of `check` and `label` 7 | /// and a label consisting of the remaining part of `label`. 8 | function splitCommonPrefix(D.Label memory label, D.Label memory check) internal pure returns (D.Label memory prefix, D.Label memory labelSuffix) { 9 | return splitAt(label, commonPrefix(check, label)); 10 | } 11 | /// Splits the label at the given position and returns prefix and suffix, 12 | /// i.e. prefix.length == pos and prefix.data . suffix.data == l.data. 13 | function splitAt(D.Label memory l, uint pos) internal pure returns (D.Label memory prefix, D.Label memory suffix) { 14 | require(pos <= l.length && pos <= 256, "Bad pos"); 15 | prefix.length = pos; 16 | if (pos == 0) { 17 | prefix.data = bytes32(0); 18 | } else { 19 | prefix.data = l.data & ~bytes32((uint(1) << (256 - pos)) - 1); 20 | } 21 | suffix.length = l.length - pos; 22 | suffix.data = l.data << pos; 23 | } 24 | /// Returns the length of the longest common prefix of the two labels. 25 | function commonPrefix(D.Label memory a, D.Label memory b) internal pure returns (uint prefix) { 26 | uint length = a.length < b.length ? a.length : b.length; 27 | // TODO: This could actually use a "highestBitSet" helper 28 | uint diff = uint(a.data ^ b.data); 29 | uint mask = 1 << 255; 30 | for (; prefix < length; prefix++) 31 | { 32 | if ((mask & diff) != 0) 33 | break; 34 | diff += diff; 35 | } 36 | } 37 | /// Returns the result of removing a prefix of length `prefix` bits from the 38 | /// given label (i.e. shifting its data to the left). 39 | function removePrefix(D.Label memory l, uint prefix) internal pure returns (D.Label memory r) { 40 | require(prefix <= l.length, "Bad lenght"); 41 | r.length = l.length - prefix; 42 | r.data = l.data << prefix; 43 | } 44 | /// Removes the first bit from a label and returns the bit and a 45 | /// label containing the rest of the label (i.e. shifted to the left). 46 | function chopFirstBit(D.Label memory l) internal pure returns (uint firstBit, D.Label memory tail) { 47 | require(l.length > 0, "Empty element"); 48 | return (uint(l.data >> 255), D.Label(l.data << 1, l.length - 1)); 49 | } 50 | /// Returns the first bit set in the bitfield, where the 0th bit 51 | /// is the least significant. 52 | /// Throws if bitfield is zero. 53 | /// More efficient the smaller the result is. 54 | function lowestBitSet(uint bitfield) internal pure returns (uint bit) { 55 | require(bitfield != 0, "Bad bitfield"); 56 | bytes32 bitfieldBytes = bytes32(bitfield); 57 | // First, find the lowest byte set 58 | uint byteSet = 0; 59 | for (; byteSet < 32; byteSet++) { 60 | if (bitfieldBytes[31 - byteSet] != 0) 61 | break; 62 | } 63 | uint singleByte = uint(uint8(bitfieldBytes[31 - byteSet])); 64 | uint mask = 1; 65 | for (bit = 0; bit < 256; bit ++) { 66 | if ((singleByte & mask) != 0) 67 | return 8 * byteSet + bit; 68 | mask += mask; 69 | } 70 | assert(false); 71 | return 0; 72 | } 73 | /// Returns the value of the `bit`th bit inside `bitfield`, where 74 | /// the least significant is the 0th bit. 75 | function bitSet(uint bitfield, uint bit) internal pure returns (uint) { 76 | return (bitfield & (uint(1) << bit)) != 0 ? 1 : 0; 77 | } 78 | } 79 | 80 | 81 | contract UtilsTest { 82 | function test() public pure { 83 | testLowestBitSet(); 84 | testChopFirstBit(); 85 | testRemovePrefix(); 86 | testCommonPrefix(); 87 | testSplitAt(); 88 | testSplitCommonPrefix(); 89 | } 90 | function testLowestBitSet() internal pure { 91 | require(Utils.lowestBitSet(0x123) == 0, "testLowestBitSet 1"); 92 | require(Utils.lowestBitSet(0x124) == 2, "testLowestBitSet 2"); 93 | require(Utils.lowestBitSet(0x11 << 30) == 30, "testLowestBitSet 3"); 94 | require(Utils.lowestBitSet(1 << 255) == 255, "testLowestBitSet 4"); 95 | } 96 | function testChopFirstBit() internal pure { 97 | D.Label memory l; 98 | l.data = hex"ef1230"; 99 | l.length = 20; 100 | uint bit1; 101 | uint bit2; 102 | uint bit3; 103 | uint bit4; 104 | (bit1, l) = Utils.chopFirstBit(l); 105 | (bit2, l) = Utils.chopFirstBit(l); 106 | (bit3, l) = Utils.chopFirstBit(l); 107 | (bit4, l) = Utils.chopFirstBit(l); 108 | require(bit1 == 1, "testChopFirstBit 1"); 109 | require(bit2 == 1, "testChopFirstBit 2"); 110 | require(bit3 == 1, "testChopFirstBit 3"); 111 | require(bit4 == 0, "testChopFirstBit 4"); 112 | require(l.length == 16, "testChopFirstBit 5"); 113 | require(l.data == hex"F123", "testChopFirstBit 6"); 114 | 115 | l.data = hex"80"; 116 | l.length = 1; 117 | (bit1, l) = Utils.chopFirstBit(l); 118 | require(bit1 == 1, "Fail 7"); 119 | require(l.length == 0, "Fail 8"); 120 | require(l.data == 0, "Fail 9"); 121 | } 122 | function testRemovePrefix() internal pure { 123 | D.Label memory l; 124 | l.data = hex"ef1230"; 125 | l.length = 20; 126 | l = Utils.removePrefix(l, 4); 127 | require(l.length == 16, "testRemovePrefix 1"); 128 | require(l.data == hex"f123", "testRemovePrefix 2"); 129 | l = Utils.removePrefix(l, 15); 130 | require(l.length == 1, "testRemovePrefix 3"); 131 | require(l.data == hex"80", "testRemovePrefix 4"); 132 | l = Utils.removePrefix(l, 1); 133 | require(l.length == 0, "testRemovePrefix 5"); 134 | require(l.data == 0, "testRemovePrefix 6"); 135 | } 136 | function testCommonPrefix() internal pure { 137 | D.Label memory a; 138 | D.Label memory b; 139 | a.data = hex"abcd"; 140 | a.length = 16; 141 | b.data = hex"a000"; 142 | b.length = 16; 143 | require(Utils.commonPrefix(a, b) == 4, "testCommonPrefix 1"); 144 | 145 | b.length = 0; 146 | require(Utils.commonPrefix(a, b) == 0, "testCommonPrefix 2"); 147 | 148 | b.data = hex"bbcd"; 149 | b.length = 16; 150 | require(Utils.commonPrefix(a, b) == 3, "testCommonPrefix 3"); 151 | require(Utils.commonPrefix(b, b) == b.length, "testCommonPrefix 4"); 152 | } 153 | function testSplitAt() internal pure { 154 | D.Label memory a; 155 | a.data = hex"abcd"; 156 | a.length = 16; 157 | (D.Label memory x, D.Label memory y) = Utils.splitAt(a, 0); 158 | require(x.length == 0, "testSplitAt 1"); 159 | require(y.length == a.length, "testSplitAt 2"); 160 | require(y.data == a.data, "testSplitAt 3"); 161 | 162 | (x, y) = Utils.splitAt(a, 4); 163 | require(x.length == 4, "testSplitAt 4"); 164 | require(x.data == hex"a0", "testSplitAt 5"); 165 | require(y.length == 12, "testSplitAt 6"); 166 | require(y.data == hex"bcd0", "testSplitAt 7"); 167 | 168 | (x, y) = Utils.splitAt(a, 16); 169 | require(y.length == 0, "testSplitAt 8"); 170 | require(x.length == a.length, "testSplitAt 9"); 171 | require(x.data == a.data, "testSplitAt 10"); 172 | } 173 | function testSplitCommonPrefix() internal pure { 174 | D.Label memory a; 175 | D.Label memory b; 176 | a.data = hex"abcd"; 177 | a.length = 16; 178 | b.data = hex"a0f570"; 179 | b.length = 20; 180 | (D.Label memory prefix, D.Label memory suffix) = Utils.splitCommonPrefix(b, a); 181 | require(prefix.length == 4, "testSplitCommonPrefix 1"); 182 | require(prefix.data == hex"a0", "testSplitCommonPrefix 2"); 183 | require(suffix.length == 16, "testSplitCommonPrefix 3"); 184 | require(suffix.data == hex"0f57", "testSplitCommonPrefix 4"); 185 | } 186 | } 187 | --------------------------------------------------------------------------------