├── .gitignore ├── Dockerfile ├── LICENSE ├── README.md ├── bin └── sync_wallets ├── client.py ├── docker-compose.yaml ├── requirements.test.txt ├── requirements.txt ├── test_tinychain.py └── tinychain.py /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | .cache 3 | .coverage 4 | .nvimlog 5 | htmlcov 6 | *.dat 7 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.6.2 2 | ADD requirements.txt ./ 3 | RUN pip install -r requirements.txt 4 | ADD tinychain.py ./ 5 | 6 | CMD ["./tinychain.py", "serve"] 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2017 James O'Beirne 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all 11 | copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 19 | SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ⛼ tinychain 2 | 3 | *Putting the rough in "[rough consensus](https://tools.ietf.org/html/rfc7282#section-1)"* 4 | 5 | 6 | Tinychain is a pocket-sized implementation of Bitcoin. Its goal is to 7 | be a compact, understandable, working incarnation of 8 | [the Nakamoto consensus algorithm](https://bitcoin.org/bitcoin.pdf) at the 9 | expense of advanced functionality, speed, and any real usefulness. 10 | 11 | I wrote it primarily to understand Bitcoin better, but hopefully it can serve as 12 | a jumping-off point for programmers who are interested in (but don't have 13 | intimate familiarity with) Bitcoin or cryptocurrency. At the very least, it can 14 | be a piñata for protocol developers who actually know what they're doing. 15 | 16 | ``` 17 | $ cloc --quiet tinychain.py 18 | 19 | ------------------------------------------------------------------------------- 20 | Language files blank comment code 21 | ------------------------------------------------------------------------------- 22 | Python 1 341 174 679 23 | ------------------------------------------------------------------------------- 24 | ``` 25 | 26 | ## Quick start 27 | 28 | - [Install Docker & docker-compose](https://www.docker.com/community-edition#/download) 29 | - Clone this repo: `git clone git@github.com:jamesob/tinychain.git` 30 | - Make sure you're in a Python3.6 environment: `virtualenv --python=python3.6 venv && . venv/bin/activate` 31 | - Grab Python dependencies locally: `pip install -r requirements.txt` 32 | - Run `docker-compose up`. This will spawn two tinychain nodes. 33 | - In another window, run `./bin/sync_wallets`. This brings the wallet data 34 | from the Docker containers onto your host. 35 | ``` 36 | $ ./bin/sync_wallets 37 | 38 | Synced node1's wallet: 39 | [2017-08-05 12:59:34,423][tinychain:1075] INFO your address is 1898KEjkziq9uRCzaVUUoBwzhURt4nrbP8 40 | 0.0 ⛼ 41 | 42 | Synced node2's wallet: 43 | [2017-08-05 12:59:35,876][tinychain:1075] INFO your address is 15YxFVo4EuqvDJH8ey2bY352MVRVpH1yFD 44 | 0.0 ⛼ 45 | ``` 46 | - Try running `./client.py balance -w wallet1.dat`; try it with the other 47 | wallet file. 48 | ``` 49 | $ ./client.py balance -w wallet2.dat 50 | 51 | [2017-08-05 13:00:37,317][tinychain:1075] INFO your address is 15YxFVo4EuqvDJH8ey2bY352MVRVpH1yFD 52 | 0.0 ⛼ 53 | ``` 54 | - Once you see a few blocks go by, try sending some money between the wallets 55 | ``` 56 | $ ./client.py send -w wallet2.dat 1898KEjkziq9uRCzaVUUoBwzhURt4nrbP8 1337 57 | 58 | [2017-08-05 13:08:08,251][tinychain:1077] INFO your address is 1Q2fBbg8XnnPiv1UHe44f2x9vf54YKXh7C 59 | [2017-08-05 13:08:08,361][client:105] INFO built txn Transaction(...) 60 | [2017-08-05 13:08:08,362][client:106] INFO broadcasting txn 2aa89204456207384851a4bbf8bde155eca7fcf30b833495d5b0541f84931919 61 | ``` 62 | - Check on the status of the transaction 63 | ``` 64 | $ ./client.py status e8f63eeeca32f9df28a3a62a366f63e8595cf70efb94710d43626ff4c0918a8a 65 | 66 | [2017-08-05 13:09:21,489][tinychain:1077] INFO your address is 1898KEjkziq9uRCzaVUUoBwzhURt4nrbP8 67 | Mined in 0000000726752f82af3d0f271fd61337035256051a9a1e5881e82d93d8e42d66 at height 5 68 | ``` 69 | 70 | 71 | ## What is Bitcoin? 72 | 73 | In brief terms that map to this code... 74 | 75 | Bitcoin is a way of generating pseudo-anonymous, decentralized trust at the cost 76 | of electricity. The most commonly known (but not sole) application of this is as 77 | a currency or store of value. If that sounds abstruse, general, and mindblowing, 78 | that's because it is. 79 | 80 | In Bitcoin, value is recorded using a `Transaction`, which assigns some 81 | number of coins to an identity (via `TxOut`s) given some cryptographically 82 | unlocked `TxIn`s. TxIns must always refer to previously created but unspent 83 | TxOuts. 84 | 85 | A Transaction is written into history by being included in a `Block`. Each Block 86 | contains a data structure called a [Merkle 87 | Tree](https://en.wikipedia.org/wiki/Merkle_tree) which generates a fingerprint 88 | unique to the set of Transactions being included. The root of that Merkle tree 89 | is included in the block "header" and hashed (`Block.id`) to permanently seal 90 | the existence and inclusion of each Transaction in the block. 91 | 92 | Blocks are linked together in a chain (`active_chain`) by referring to the 93 | previous Block header hash. In order to add a Block to the chain, the contents 94 | of its header must hash to a number under some difficulty target, which is set 95 | based upon how quickly recent blocks have been discovered 96 | (`get_next_work_required()`). This attempts to 97 | normalize the time between block discovery. 98 | 99 | When a block is discovered, it creates a subsidy for the discoverer in the form 100 | of newly minted coins. The discoverer also collects fees from transactions 101 | included in the block, which are the value of inputs minus outputs. The block 102 | reward subsidy decreases logarithmically over time. Eventually the subsidy 103 | goes to zero and miners are incentivized to continue mining purely by a fee 104 | market. 105 | 106 | Nodes in the network are in a never-ending competition to mine and propagate the 107 | next block, and in doing so facilitate the recording of transaction history. 108 | Transactions are submitted to nodes and broadcast across the network, stored 109 | temporarily in `mempool` where they are queued for block inclusion. 110 | 111 | For more comprehensive descriptions of Bitcoin, see 112 | 113 | - [Bitcoin: A Peer-to-Peer Electronic Cash System](https://bitcoin.org/bitcoin.pdf) 114 | by Satoshi Nakamoto 115 | - [Mastering Bitcoin](https://github.com/bitcoinbook/bitcoinbook/) by Andreas 116 | Antonopoulos 117 | - [The Bitcoin Developer Guide](https://bitcoin.org/en/developer-guide) 118 | 119 | 120 | 121 | ## Notable differences from Bitcoin 122 | 123 | - Byte-level representation and endianness are very important when serializing a 124 | data structure to be hashed in Bitcoin and are not reproduced 125 | faithfully here. In fact, serialization of any kind here is very dumbed down 126 | and based entirely on raw strings or JSON. 127 | 128 | - Transaction types are limited to pay-to-public-key-hash (P2PKH), which 129 | facilitate the bare minimum of "sending money." More exotic 130 | [transaction 131 | types](https://bitcoin.org/en/developer-guide#standard-transactions) which 132 | allow m-of-n key signatures and 133 | [Script](https://en.bitcoin.it/wiki/Script)-based unlocking are not 134 | implemented. 135 | 136 | - [Initial Block Download](https://bitcoin.org/en/developer-guide#initial-block-download) 137 | is at best a simplified version of the old "blocks-first" scheme. It eschews 138 | `getdata` and instead returns block payloads directly in `inv`. 139 | 140 | - The longest, valid chain is determined simply by chain length (number of 141 | blocks) vs. [chainwork](https://bitcoin.stackexchange.com/questions/26869/what-is-chainwork). 142 | 143 | - Peer "discovery" is done through environment variable hardcoding. In 144 | bitcoin core, this is done [with DNS seeds](https://en.bitcoin.it/wiki/Transaction_replacement). 145 | 146 | - [Replace by fee](https://en.bitcoin.it/wiki/Transaction_replacement) is absent. 147 | 148 | - Memory usage is egregious. Networking is a hack. 149 | 150 | - Satoshis are instead called Belushis because, well... 151 | 152 | 153 | 154 | 155 | 156 | ## Q&A 157 | 158 | ### How does RPC work? 159 | 160 | We use JSON for over-the-wire serialization. It's slow and unrealistic but 161 | human-readable and easy. We deserialize right into the `.*Msg` classes, 162 | each of which dictates how a particular RPC message is handled via 163 | `.handle()`. 164 | 165 | ### Why doesn't the client track coins we've spent but haven't confirmed yet? 166 | 167 | Yeah I know, the client sucks. I'll take a PR. 168 | 169 | ### How can I add another RPC command to reveal more data from a node? 170 | 171 | Just add a `NamedTuple` subclass with a `handle()` method defined; it registers 172 | automatically. Mimic any existing `*Msg` class. 173 | 174 | 175 | ### Why aren't my changes changing anything? 176 | 177 | Remember to rebuild the Docker container image when you make changes 178 | ``` 179 | docker-compose build && docker-compose up 180 | ``` 181 | 182 | ### How do I run automated tests? 183 | 184 | ``` 185 | pip install -r requirements.test.txt 186 | py.test --cov test_tinychain.py 187 | ``` 188 | 189 | 190 | ### Is this yet another cryptocurrency created solely to Get Rich Quick™? 191 | 192 | A resounding Yes! (if you're dealing in the very illiquid currency of 193 | education) 194 | 195 | Otherwise nah. This thing has 0 real-world value. 196 | 197 | 198 | ### What's with the logo? 199 | 200 | It's a shitty unicode Merkle tree. Give a guy a break here, this is freeware! 201 | 202 | ### Where can I get more of you ranting? 203 | 204 | [@jamesob](https://twitter.com/jamesob) 205 | -------------------------------------------------------------------------------- /bin/sync_wallets: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | docker-compose exec node1 cat wallet.dat > wallet1.dat 4 | echo "Synced node1's wallet:" 5 | ./client.py balance 6 | echo 7 | 8 | docker-compose exec node2 cat wallet.dat > wallet2.dat 9 | echo "Synced node2's wallet:" 10 | ./client.py balance --wallet wallet2.dat 11 | -------------------------------------------------------------------------------- /client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """ 3 | ⛼ tinychain client 4 | 5 | Usage: 6 | client.py balance [options] [--raw] 7 | client.py send [options] 8 | client.py status [options] [--csv] 9 | 10 | Options: 11 | -h --help Show help 12 | -w, --wallet PATH Use a particular wallet file (e.g. `-w ./wallet2.dat`) 13 | -n, --node HOSTNAME The hostname of node to use for RPC (default: localhost) 14 | -p, --port PORT Port node is listening on (default: 9999) 15 | 16 | """ 17 | import logging 18 | import os 19 | import socket 20 | 21 | from docopt import docopt 22 | 23 | import tinychain as t 24 | 25 | 26 | logging.basicConfig( 27 | level=getattr(logging, os.environ.get('TC_LOG_LEVEL', 'INFO')), 28 | format='[%(asctime)s][%(module)s:%(lineno)d] %(levelname)s %(message)s') 29 | logger = logging.getLogger(__name__) 30 | 31 | 32 | def main(args): 33 | args['signing_key'], args['verifying_key'], args['my_addr'] = ( 34 | t.init_wallet(args.get('--wallet'))) 35 | 36 | if args['--port']: 37 | send_msg.port = args['--port'] 38 | if args['--node']: 39 | send_msg.node_hostname = args['--node'] 40 | 41 | if args['balance']: 42 | get_balance(args) 43 | elif args['send']: 44 | send_value(args) 45 | elif args['status']: 46 | txn_status(args) 47 | 48 | 49 | def get_balance(args): 50 | """ 51 | Get the balance of a given address. 52 | """ 53 | val = sum(i.value for i in find_utxos_for_address(args)) 54 | print(val if args['--raw'] else f"{val / t.Params.BELUSHIS_PER_COIN} ⛼ ") 55 | 56 | 57 | def txn_status(args): 58 | """ 59 | Get the status of a transaction. 60 | 61 | Prints [status],[containing block_id],[height mined] 62 | """ 63 | txid = args[''] 64 | as_csv = args['--csv'] 65 | mempool = send_msg(t.GetMempoolMsg()) 66 | 67 | if txid in mempool: 68 | print(f'{txid}:in_mempool,,' if as_csv else 'Found in mempool') 69 | return 70 | 71 | chain = send_msg(t.GetActiveChainMsg()) 72 | 73 | for tx, block, height in t.txn_iterator(chain): 74 | if tx.id == txid: 75 | print( 76 | f'{txid}:mined,{block.id},{height}' if as_csv else 77 | f'Mined in {block.id} at height {height}') 78 | return 79 | 80 | print(f'{txid}:not_found,,' if as_csv else 'Not found') 81 | 82 | 83 | def send_value(args: dict): 84 | """ 85 | Send value to some address. 86 | """ 87 | val, to_addr, sk = int(args['']), args[''], args['signing_key'] 88 | selected = set() 89 | my_coins = list(sorted( 90 | find_utxos_for_address(args), key=lambda i: (i.value, i.height))) 91 | 92 | for coin in my_coins: 93 | selected.add(coin) 94 | if sum(i.value for i in selected) > val: 95 | break 96 | 97 | txout = t.TxOut(value=val, to_address=to_addr) 98 | 99 | txn = t.Transaction( 100 | txins=[make_txin(sk, coin.outpoint, txout) for coin in selected], 101 | txouts=[txout]) 102 | 103 | logger.info(f'built txn {txn}') 104 | logger.info(f'broadcasting txn {txn.id}') 105 | send_msg(txn) 106 | 107 | 108 | def send_msg(data: bytes, node_hostname=None, port=None): 109 | node_hostname = getattr(send_msg, 'node_hostname', 'localhost') 110 | port = getattr(send_msg, 'port', 9999) 111 | 112 | with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: 113 | s.connect((node_hostname, port)) 114 | s.sendall(t.encode_socket_data(data)) 115 | return t.read_all_from_socket(s) 116 | 117 | 118 | def find_utxos_for_address(args: dict): 119 | utxo_set = dict(send_msg(t.GetUTXOsMsg())) 120 | return [u for u in utxo_set.values() if u.to_address == args['my_addr']] 121 | 122 | 123 | def make_txin(signing_key, outpoint: t.OutPoint, txout: t.TxOut) -> t.TxIn: 124 | sequence = 0 125 | pk = signing_key.verifying_key.to_string() 126 | spend_msg = t.build_spend_message(outpoint, pk, sequence, [txout]) 127 | 128 | return t.TxIn( 129 | to_spend=outpoint, unlock_pk=pk, 130 | unlock_sig=signing_key.sign(spend_msg), sequence=sequence) 131 | 132 | 133 | if __name__ == '__main__': 134 | main(docopt(__doc__, version='tinychain client 0.1')) 135 | -------------------------------------------------------------------------------- /docker-compose.yaml: -------------------------------------------------------------------------------- 1 | version: "3" 2 | 3 | services: 4 | 5 | node1: 6 | build: . 7 | image: tinychain 8 | ports: 9 | - "9999:9999" 10 | 11 | node2: 12 | image: tinychain 13 | environment: 14 | TC_PEERS: 'node1' 15 | -------------------------------------------------------------------------------- /requirements.test.txt: -------------------------------------------------------------------------------- 1 | pytest==3.1.1 2 | pytest-cov==2.5.1 3 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | base58==0.2.5 2 | ecdsa==0.13 3 | docopt==0.6.2 4 | -------------------------------------------------------------------------------- /test_tinychain.py: -------------------------------------------------------------------------------- 1 | import time 2 | 3 | import pytest 4 | import ecdsa 5 | 6 | import tinychain as t 7 | from tinychain import Block, TxIn, TxOut, Transaction 8 | from client import make_txin 9 | 10 | 11 | def test_merkle_trees(): 12 | root = t.get_merkle_root('foo', 'bar') 13 | fooh = t.sha256d('foo') 14 | barh = t.sha256d('bar') 15 | 16 | assert root 17 | assert root.val == t.sha256d(fooh + barh) 18 | assert root.children[0].val == fooh 19 | assert root.children[1].val == barh 20 | 21 | root = t.get_merkle_root('foo', 'bar', 'baz') 22 | bazh = t.sha256d('baz') 23 | 24 | assert root 25 | assert len(root.children) == 2 26 | assert root.children[0].val == t.sha256d(fooh + barh) 27 | assert root.children[1].val == t.sha256d(bazh + bazh) 28 | 29 | 30 | def test_serialization(): 31 | op1 = t.OutPoint(txid='c0ffee', txout_idx=0) 32 | op2 = t.OutPoint(txid='c0ffee', txout_idx=1) 33 | txin1 = t.TxIn( 34 | to_spend=op1, unlock_sig=b'oursig', unlock_pk=b'foo', sequence=1) 35 | txin2 = t.TxIn( 36 | to_spend=op2, unlock_sig=b'oursig', unlock_pk=b'foo', sequence=2) 37 | txout = t.TxOut(value=101, to_address='1zxoijw') 38 | txn1 = t.Transaction(txins=[txin1], txouts=[txout], locktime=0) 39 | txn2 = t.Transaction(txins=[txin2], txouts=[txout], locktime=0) 40 | block = t.Block( 41 | 1, 'deadbeef', 'c0ffee', int(time.time()), 100, 100, [txn1, txn2]) 42 | utxo = t.UnspentTxOut( 43 | *txout, txid=txn1.id, txout_idx=0, is_coinbase=False, height=0) 44 | utxo_set = [utxo.outpoint, utxo] 45 | 46 | for obj in ( 47 | op1, op2, txin1, txin2, txout, txn1, txn2, block, utxo, utxo_set): 48 | assert t.deserialize(t.serialize(obj)) == obj 49 | 50 | 51 | def test_build_spend_message(): 52 | txout = t.TxOut(value=101, to_address='1zz8w9') 53 | txin = t.TxIn( 54 | to_spend=t.OutPoint('c0ffee', 0), 55 | unlock_sig=b'oursig', unlock_pk=b'foo', sequence=1) 56 | txn = t.Transaction(txins=[txin], txouts=[txout], locktime=0) 57 | 58 | spend_msg = t.build_spend_message( 59 | txin.to_spend, txin.unlock_pk, txin.sequence, txn.txouts) 60 | 61 | assert spend_msg == ( 62 | b'677c2d8f9843d1cc456e7bfbc507c0f6d07d19c69e6bca0cbaa7bfaea4dd840a') 63 | 64 | # Adding a new output to the txn creates a new spend message. 65 | 66 | txn.txouts.append(t.TxOut(value=1, to_address='1zz')) 67 | assert t.build_spend_message( 68 | txin.to_spend, txin.unlock_pk, txin.sequence, txn.txouts) != spend_msg 69 | 70 | 71 | def test_get_median_time_past(): 72 | t.active_chain = [] 73 | assert t.get_median_time_past(10) == 0 74 | 75 | timestamps = [1, 30, 60, 90, 400] 76 | t.active_chain = [_dummy_block(timestamp=t) for t in timestamps] 77 | 78 | assert t.get_median_time_past(1) == 400 79 | assert t.get_median_time_past(3) == 90 80 | assert t.get_median_time_past(2) == 90 81 | assert t.get_median_time_past(5) == 60 82 | 83 | 84 | def test_dependent_txns_in_single_block(): 85 | t.active_chain = [] 86 | t.mempool = {} 87 | assert t.connect_block(chain1[0]) == t.ACTIVE_CHAIN_IDX 88 | assert t.connect_block(chain1[1]) == t.ACTIVE_CHAIN_IDX 89 | 90 | assert len(t.active_chain) == 2 91 | assert len(t.utxo_set) == 2 92 | 93 | utxo1 = t.utxo_set[list(t.utxo_set.keys())[0]] 94 | txout1 = TxOut(value=901, to_address=utxo1.to_address) 95 | txin1 = make_txin(signing_key, utxo1.outpoint, txout1) 96 | txn1 = t.Transaction(txins=[txin1], txouts=[txout1], locktime=0) 97 | 98 | # Create a transaction that is dependent on the yet-unconfirmed transaction 99 | # above. 100 | txout2 = TxOut(value=9001, to_address=txout1.to_address) 101 | txin2 = make_txin(signing_key, t.OutPoint(txn1.id, 0), txout2) 102 | txn2 = t.Transaction(txins=[txin2], txouts=[txout2], locktime=0) 103 | 104 | # Assert that we don't accept this txn -- too early to spend the coinbase. 105 | 106 | with pytest.raises(t.TxnValidationError) as excinfo: 107 | t.validate_txn(txn2) 108 | assert 'UTXO not ready' in str(excinfo.value) 109 | 110 | t.connect_block(chain1[2]) 111 | 112 | # Now the coinbase has matured to spending. 113 | t.add_txn_to_mempool(txn1) 114 | assert txn1.id in t.mempool 115 | 116 | # In txn2, we're attempting to spend more than is available (9001 vs. 901). 117 | 118 | assert not t.add_txn_to_mempool(txn2) 119 | 120 | with pytest.raises(t.TxnValidationError) as excinfo: 121 | t.validate_txn(txn2) 122 | assert 'Spend value is more than available' in str(excinfo.value) 123 | 124 | # Recreate the transaction with an acceptable value. 125 | txout2 = TxOut(value=901, to_address=txout1.to_address) 126 | txin2 = make_txin(signing_key, t.OutPoint(txn1.id, 0), txout2) 127 | txn2 = t.Transaction(txins=[txin2], txouts=[txout2], locktime=0) 128 | 129 | t.add_txn_to_mempool(txn2) 130 | assert txn2.id in t.mempool 131 | 132 | block = t.assemble_and_solve_block(t.pubkey_to_address( 133 | signing_key.get_verifying_key().to_string())) 134 | 135 | assert t.connect_block(block) == t.ACTIVE_CHAIN_IDX 136 | 137 | assert t.active_chain[-1] == block 138 | assert block.txns[1:] == [txn1, txn2] 139 | assert txn1.id not in t.mempool 140 | assert txn2.id not in t.mempool 141 | assert t.OutPoint(txn1.id, 0) not in t.utxo_set # Spent by txn2. 142 | assert t.OutPoint(txn2.id, 0) in t.utxo_set 143 | 144 | 145 | def test_pubkey_to_address(): 146 | assert t.pubkey_to_address( 147 | b'k\xd4\xd8M3\xc8\xf7h*\xd2\x16O\xe39a\xc9]\x18i\x08\xf1\xac\xb8\x0f' 148 | b'\x9af\xdd\xd1\'\xe2\xc2v\x8eCo\xd3\xc4\xff\x0e\xfc\x9eBzS\\=\x7f' 149 | b'\x7f\x1a}\xeen"\x9f\x9c\x17E\xeaMH\x88\xec\xf5F') == ( 150 | '18kZswtcPRKCcf9GQsJLNFEMUE8V9tCJr') 151 | 152 | 153 | def test_reorg(): 154 | t.active_chain = [] 155 | 156 | for block in chain1: 157 | assert t.connect_block(block) == t.ACTIVE_CHAIN_IDX 158 | 159 | t.side_branches = [] 160 | t.mempool = {} 161 | t.utxo_set = {} 162 | _add_to_utxo_for_chain(t.active_chain) 163 | 164 | def assert_no_change(): 165 | assert t.active_chain == chain1 166 | assert t.mempool == {} 167 | assert [k.txid[:6] for k in t.utxo_set] == [ 168 | '8b7bfc', 'b8a642', '6708b9'] 169 | 170 | assert len(t.utxo_set) == 3 171 | 172 | # No reorg necessary when side branches are empty. 173 | 174 | assert not t.reorg_if_necessary() 175 | 176 | # No reorg necessary when side branch is shorter than the main chain. 177 | 178 | for block in chain2[1:2]: 179 | assert t.connect_block(block) == 1 180 | 181 | assert not t.reorg_if_necessary() 182 | assert t.side_branches == [chain2[1:2]] 183 | assert_no_change() 184 | 185 | # No reorg necessary when side branch is as long as the main chain. 186 | 187 | assert t.connect_block(chain2[2]) == 1 188 | 189 | assert not t.reorg_if_necessary() 190 | assert t.side_branches == [chain2[1:3]] 191 | assert_no_change() 192 | 193 | # No reorg necessary when side branch is a longer but invalid chain. 194 | 195 | # Block doesn't connect to anything because it's invalid. 196 | assert t.connect_block(chain3_faulty[3]) is None 197 | assert not t.reorg_if_necessary() 198 | 199 | # No change in side branches for an invalid block. 200 | assert t.side_branches == [chain2[1:3]] 201 | assert_no_change() 202 | 203 | # Reorg necessary when a side branch is longer than the main chain. 204 | 205 | assert t.connect_block(chain2[3]) == 1 206 | assert t.connect_block(chain2[4]) == 1 207 | 208 | # Chain1 was reorged into side_branches. 209 | assert [len(c) for c in t.side_branches] == [2] 210 | assert [b.id for b in t.side_branches[0]] == [b.id for b in chain1[1:]] 211 | assert t.side_branches == [chain1[1:]] 212 | assert t.mempool == {} 213 | assert [k.txid[:6] for k in t.utxo_set] == [ 214 | '8b7bfc', 'b8a642', '6708b9', '543683', '53f3c1'] 215 | 216 | 217 | def _add_to_utxo_for_chain(chain): 218 | for block in chain: 219 | for tx in block.txns: 220 | for i, txout in enumerate(tx.txouts): 221 | t.add_to_utxo(txout, tx, i, tx.is_coinbase, len(chain)) 222 | 223 | 224 | signing_key = ecdsa.SigningKey.from_string( 225 | b'\xf1\xad2y\xbf\xa2x\xabn\xfbO\x98\xf7\xa7\xb4\xc0\xf4fOzX\xbf\xf6\\\xd2\xcb-\x1d:0 \xa7', 226 | curve=ecdsa.SECP256k1) 227 | 228 | chain1 = [ 229 | # Block id: 000000154275885a72c004d02aaa9524fc0c4896aef0b0f3bcde2de38f9be561 230 | Block(version=0, prev_block_hash=None, merkle_hash='7118894203235a955a908c0abfc6d8fe6edec47b0a04ce1bf7263da3b4366d22', timestamp=1501821412, bits=24, nonce=10126761, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'0', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='143UVyz7ooiAv1pMqbwPPpnH4BV9ifJGFF')], locktime=None)]), 231 | 232 | # Block id: 00000095f785bc8fbd6007b36c2f1c414d66db930e2e7354076c035c8f92700b 233 | Block(version=0, prev_block_hash='000000154275885a72c004d02aaa9524fc0c4896aef0b0f3bcde2de38f9be561', merkle_hash='27661bd9b23552832becf6c18cb6035a3d77b4e66b5520505221a93922eb82f2', timestamp=1501826444, bits=24, nonce=22488415, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'1', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='1Piq91dFUqSb7tdddCWvuGX5UgdzXeoAwA')], locktime=None)]), 234 | 235 | # Block id: 000000f9b679482f24902297fc59c745e759436ac95e93d2c1eff4d5dbd39e33 236 | Block(version=0, prev_block_hash='00000095f785bc8fbd6007b36c2f1c414d66db930e2e7354076c035c8f92700b', merkle_hash='031f45ad7b5ddf198f7dfa88f53c0262fb14c850c5c1faf506258b9dcad32aef', timestamp=1501826556, bits=24, nonce=30715680, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'2', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='1Piq91dFUqSb7tdddCWvuGX5UgdzXeoAwA')], locktime=None)]) 237 | ] 238 | 239 | chain2 = [ 240 | # Block id: 000000154275885a72c004d02aaa9524fc0c4896aef0b0f3bcde2de38f9be561 241 | Block(version=0, prev_block_hash=None, merkle_hash='7118894203235a955a908c0abfc6d8fe6edec47b0a04ce1bf7263da3b4366d22', timestamp=1501821412, bits=24, nonce=10126761, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'0', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='143UVyz7ooiAv1pMqbwPPpnH4BV9ifJGFF')], locktime=None)]), 242 | 243 | # Block id: 000000e4785f0f384d13e24caaddcf6723ee008d6a179428ce9246e1b32e3b2c 244 | Block(version=0, prev_block_hash='000000154275885a72c004d02aaa9524fc0c4896aef0b0f3bcde2de38f9be561', merkle_hash='27661bd9b23552832becf6c18cb6035a3d77b4e66b5520505221a93922eb82f2', timestamp=1501826757, bits=24, nonce=25773772, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'1', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='1Piq91dFUqSb7tdddCWvuGX5UgdzXeoAwA')], locktime=None)]), 245 | 246 | # Block id: 000000a1698495a3b125d9cd08837cdabffa192639588cdda8018ed8f5af3f8c 247 | Block(version=0, prev_block_hash='000000e4785f0f384d13e24caaddcf6723ee008d6a179428ce9246e1b32e3b2c', merkle_hash='031f45ad7b5ddf198f7dfa88f53c0262fb14c850c5c1faf506258b9dcad32aef', timestamp=1501826872, bits=24, nonce=16925076, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'2', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='1Piq91dFUqSb7tdddCWvuGX5UgdzXeoAwA')], locktime=None)]), 248 | 249 | # Up until this point, we're same length as chain1. 250 | 251 | # This block is where chain3_faulty goes bad. 252 | # Block id: 000000ef44dd5a56c89a43b9cff28e51e5fd91624be3a2de722d864ae4f6a853 253 | Block(version=0, prev_block_hash='000000a1698495a3b125d9cd08837cdabffa192639588cdda8018ed8f5af3f8c', merkle_hash='dbf593cf959b3a03ea97bbeb7a44ee3f4841b338d5ceaa5705b637c853c956ef', timestamp=1501826949, bits=24, nonce=12052237, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'3', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='1Piq91dFUqSb7tdddCWvuGX5UgdzXeoAwA')], locktime=None)]), 254 | 255 | # Block id: 256 | Block(version=0, prev_block_hash='000000ef44dd5a56c89a43b9cff28e51e5fd91624be3a2de722d864ae4f6a853', merkle_hash='a3a55fe5e9f9e5e3282333ac4d149fd186f157a3c1d2b2e04af78c20a519f6b9', timestamp=1501827000, bits=24, nonce=752898, txns=[Transaction(txins=[TxIn(to_spend=None, unlock_sig=b'4', unlock_pk=None, sequence=0)], txouts=[TxOut(value=5000000000, to_address='1Piq91dFUqSb7tdddCWvuGX5UgdzXeoAwA')], locktime=None)]) 257 | ] 258 | 259 | # Make this chain invalid. 260 | chain3_faulty = list(chain2) 261 | chain3_faulty[-2] = chain3_faulty[-2]._replace(nonce=1) 262 | 263 | 264 | def _dummy_block(**kwargs): 265 | defaults = dict( 266 | version=1, prev_block_hash='c0ffee', merkle_hash='deadbeef', 267 | timestamp=1, bits=1, nonce=1, txns=[]) 268 | 269 | return t.Block(**{**defaults, **kwargs}) 270 | -------------------------------------------------------------------------------- /tinychain.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """ 3 | ⛼ tinychain 4 | 5 | putting the rough in "rough consensus" 6 | 7 | 8 | Some terminology: 9 | 10 | - Chain: an ordered list of Blocks, each of which refers to the last and 11 | cryptographically preserves a history of Transactions. 12 | 13 | - Transaction (or tx or txn): a list of inputs (i.e. past outputs being spent) 14 | and outputs which declare value assigned to the hash of a public key. 15 | 16 | - PoW (proof of work): the solution to a puzzle which allows the acceptance 17 | of an additional Block onto the chain. 18 | 19 | - Reorg: chain reorganization. When a side branch overtakes the main chain. 20 | 21 | 22 | An incomplete list of unrealistic simplifications: 23 | 24 | - Byte encoding and endianness are very important when serializing a 25 | data structure to be hashed in Bitcoin and are not reproduced 26 | faithfully here. In fact, serialization of any kind here is slipshod and 27 | in many cases relies on implicit expectations about Python JSON 28 | serialization. 29 | 30 | - Transaction types are limited to P2PKH. 31 | 32 | - Initial Block Download eschews `getdata` and instead returns block payloads 33 | directly in `inv`. 34 | 35 | - Peer "discovery" is done through environment variable hardcoding. In 36 | bitcoin core, this is done with DNS seeds. 37 | See https://bitcoin.stackexchange.com/a/3537/56368 38 | 39 | 40 | Resources: 41 | 42 | - https://en.bitcoin.it/wiki/Protocol_rules 43 | - https://en.bitcoin.it/wiki/Protocol_documentation 44 | - https://bitcoin.org/en/developer-guide 45 | - https://github.com/bitcoinbook/bitcoinbook/blob/second_edition/ch06.asciidoc 46 | 47 | 48 | TODO: 49 | 50 | - deal with orphan blocks 51 | - keep the mempool heap sorted by fee 52 | - make use of Transaction.locktime 53 | ? make use of TxIn.sequence; i.e. replace-by-fee 54 | 55 | """ 56 | import binascii 57 | import time 58 | import json 59 | import hashlib 60 | import threading 61 | import logging 62 | import socketserver 63 | import socket 64 | import random 65 | import os 66 | from functools import lru_cache, wraps 67 | from typing import ( 68 | Iterable, NamedTuple, Dict, Mapping, Union, get_type_hints, Tuple, 69 | Callable) 70 | 71 | import ecdsa 72 | from base58 import b58encode_check 73 | 74 | 75 | logging.basicConfig( 76 | level=getattr(logging, os.environ.get('TC_LOG_LEVEL', 'INFO')), 77 | format='[%(asctime)s][%(module)s:%(lineno)d] %(levelname)s %(message)s') 78 | logger = logging.getLogger(__name__) 79 | 80 | 81 | class Params: 82 | # The infamous max block size. 83 | MAX_BLOCK_SERIALIZED_SIZE = 1000000 # bytes = 1MB 84 | 85 | # Coinbase transaction outputs can be spent after this many blocks have 86 | # elapsed since being mined. 87 | # 88 | # This is "100" in bitcoin core. 89 | COINBASE_MATURITY = 2 90 | 91 | # Accept blocks timestamped as being from the future, up to this amount. 92 | MAX_FUTURE_BLOCK_TIME = (60 * 60 * 2) 93 | 94 | # The number of Belushis per coin. #realname COIN 95 | BELUSHIS_PER_COIN = int(100e6) 96 | 97 | TOTAL_COINS = 21_000_000 98 | 99 | # The maximum number of Belushis that will ever be found. 100 | MAX_MONEY = BELUSHIS_PER_COIN * TOTAL_COINS 101 | 102 | # The duration we want to pass between blocks being found, in seconds. 103 | # This is lower than Bitcoin's configuation (10 * 60). 104 | # 105 | # #realname PowTargetSpacing 106 | TIME_BETWEEN_BLOCKS_IN_SECS_TARGET = 1 * 60 107 | 108 | # The number of seconds we want a difficulty period to last. 109 | # 110 | # Note that this differs considerably from the behavior in Bitcoin, which 111 | # is configured to target difficulty periods of (10 * 2016) minutes. 112 | # 113 | # #realname PowTargetTimespan 114 | DIFFICULTY_PERIOD_IN_SECS_TARGET = (60 * 60 * 10) 115 | 116 | # After this number of blocks are found, adjust difficulty. 117 | # 118 | # #realname DifficultyAdjustmentInterval 119 | DIFFICULTY_PERIOD_IN_BLOCKS = ( 120 | DIFFICULTY_PERIOD_IN_SECS_TARGET / TIME_BETWEEN_BLOCKS_IN_SECS_TARGET) 121 | 122 | # The number of right-shifts applied to 2 ** 256 in order to create the 123 | # initial difficulty target necessary for mining a block. 124 | INITIAL_DIFFICULTY_BITS = 24 125 | 126 | # The number of blocks after which the mining subsidy will halve. 127 | # 128 | # #realname SubsidyHalvingInterval 129 | HALVE_SUBSIDY_AFTER_BLOCKS_NUM = 210_000 130 | 131 | 132 | # Used to represent the specific output within a transaction. 133 | OutPoint = NamedTuple('OutPoint', [('txid', str), ('txout_idx', int)]) 134 | 135 | 136 | class TxIn(NamedTuple): 137 | """Inputs to a Transaction.""" 138 | # A reference to the output we're spending. This is None for coinbase 139 | # transactions. 140 | to_spend: Union[OutPoint, None] 141 | 142 | # The (signature, pubkey) pair which unlocks the TxOut for spending. 143 | unlock_sig: bytes 144 | unlock_pk: bytes 145 | 146 | # A sender-defined sequence number which allows us replacement of the txn 147 | # if desired. 148 | sequence: int 149 | 150 | 151 | class TxOut(NamedTuple): 152 | """Outputs from a Transaction.""" 153 | # The number of Belushis this awards. 154 | value: int 155 | 156 | # The public key of the owner of this Txn. 157 | to_address: str 158 | 159 | 160 | class UnspentTxOut(NamedTuple): 161 | value: int 162 | to_address: str 163 | 164 | # The ID of the transaction this output belongs to. 165 | txid: str 166 | txout_idx: int 167 | 168 | # Did this TxOut from from a coinbase transaction? 169 | is_coinbase: bool 170 | 171 | # The blockchain height this TxOut was included in the chain. 172 | height: int 173 | 174 | @property 175 | def outpoint(self): return OutPoint(self.txid, self.txout_idx) 176 | 177 | 178 | class Transaction(NamedTuple): 179 | txins: Iterable[TxIn] 180 | txouts: Iterable[TxOut] 181 | 182 | # The block number or timestamp at which this transaction is unlocked. 183 | # < 500000000: Block number at which this transaction is unlocked. 184 | # >= 500000000: UNIX timestamp at which this transaction is unlocked. 185 | locktime: int = None 186 | 187 | @property 188 | def is_coinbase(self) -> bool: 189 | return len(self.txins) == 1 and self.txins[0].to_spend is None 190 | 191 | @classmethod 192 | def create_coinbase(cls, pay_to_addr, value, height): 193 | return cls( 194 | txins=[TxIn( 195 | to_spend=None, 196 | # Push current block height into unlock_sig so that this 197 | # transaction's ID is unique relative to other coinbase txns. 198 | unlock_sig=str(height).encode(), 199 | unlock_pk=None, 200 | sequence=0)], 201 | txouts=[TxOut( 202 | value=value, 203 | to_address=pay_to_addr)], 204 | ) 205 | 206 | @property 207 | def id(self) -> str: 208 | return sha256d(serialize(self)) 209 | 210 | def validate_basics(self, as_coinbase=False): 211 | if (not self.txouts) or (not self.txins and not as_coinbase): 212 | raise TxnValidationError('Missing txouts or txins') 213 | 214 | if len(serialize(self)) > Params.MAX_BLOCK_SERIALIZED_SIZE: 215 | raise TxnValidationError('Too large') 216 | 217 | if sum(t.value for t in self.txouts) > Params.MAX_MONEY: 218 | raise TxnValidationError('Spend value too high') 219 | 220 | 221 | class Block(NamedTuple): 222 | # A version integer. 223 | version: int 224 | 225 | # A hash of the previous block's header. 226 | prev_block_hash: str 227 | 228 | # A hash of the Merkle tree containing all txns. 229 | merkle_hash: str 230 | 231 | # A UNIX timestamp of when this block was created. 232 | timestamp: int 233 | 234 | # The difficulty target; i.e. the hash of this block header must be under 235 | # (2 ** 256 >> bits) to consider work proved. 236 | bits: int 237 | 238 | # The value that's incremented in an attempt to get the block header to 239 | # hash to a value below `bits`. 240 | nonce: int 241 | 242 | txns: Iterable[Transaction] 243 | 244 | def header(self, nonce=None) -> str: 245 | """ 246 | This is hashed in an attempt to discover a nonce under the difficulty 247 | target. 248 | """ 249 | return ( 250 | f'{self.version}{self.prev_block_hash}{self.merkle_hash}' 251 | f'{self.timestamp}{self.bits}{nonce or self.nonce}') 252 | 253 | @property 254 | def id(self) -> str: return sha256d(self.header()) 255 | 256 | 257 | # Chain 258 | # ---------------------------------------------------------------------------- 259 | 260 | genesis_block = Block( 261 | version=0, prev_block_hash=None, 262 | merkle_hash=( 263 | '7118894203235a955a908c0abfc6d8fe6edec47b0a04ce1bf7263da3b4366d22'), 264 | timestamp=1501821412, bits=24, nonce=10126761, 265 | txns=[Transaction( 266 | txins=[TxIn( 267 | to_spend=None, unlock_sig=b'0', unlock_pk=None, sequence=0)], 268 | txouts=[TxOut( 269 | value=5000000000, 270 | to_address='143UVyz7ooiAv1pMqbwPPpnH4BV9ifJGFF')], locktime=None)]) 271 | 272 | # The highest proof-of-work, valid blockchain. 273 | # 274 | # #realname chainActive 275 | active_chain: Iterable[Block] = [genesis_block] 276 | 277 | # Branches off of the main chain. 278 | side_branches: Iterable[Iterable[Block]] = [] 279 | 280 | # Synchronize access to the active chain and side branches. 281 | chain_lock = threading.RLock() 282 | 283 | 284 | def with_lock(lock): 285 | def dec(func): 286 | @wraps(func) 287 | def wrapper(*args, **kwargs): 288 | with lock: 289 | return func(*args, **kwargs) 290 | return wrapper 291 | return dec 292 | 293 | 294 | orphan_blocks: Iterable[Block] = [] 295 | 296 | # Used to signify the active chain in `locate_block`. 297 | ACTIVE_CHAIN_IDX = 0 298 | 299 | 300 | @with_lock(chain_lock) 301 | def get_current_height(): return len(active_chain) 302 | 303 | 304 | @with_lock(chain_lock) 305 | def txn_iterator(chain): 306 | return ( 307 | (txn, block, height) 308 | for height, block in enumerate(chain) for txn in block.txns) 309 | 310 | 311 | @with_lock(chain_lock) 312 | def locate_block(block_hash: str, chain=None) -> (Block, int, int): 313 | chains = [chain] if chain else [active_chain, *side_branches] 314 | 315 | for chain_idx, chain in enumerate(chains): 316 | for height, block in enumerate(chain): 317 | if block.id == block_hash: 318 | return (block, height, chain_idx) 319 | return (None, None, None) 320 | 321 | 322 | @with_lock(chain_lock) 323 | def connect_block(block: Union[str, Block], 324 | doing_reorg=False, 325 | ) -> Union[None, Block]: 326 | """Accept a block and return the chain index we append it to.""" 327 | # Only exit early on already seen in active_chain when reorging. 328 | search_chain = active_chain if doing_reorg else None 329 | 330 | if locate_block(block.id, chain=search_chain)[0]: 331 | logger.debug(f'ignore block already seen: {block.id}') 332 | return None 333 | 334 | try: 335 | block, chain_idx = validate_block(block) 336 | except BlockValidationError as e: 337 | logger.exception('block %s failed validation', block.id) 338 | if e.to_orphan: 339 | logger.info(f"saw orphan block {block.id}") 340 | orphan_blocks.append(e.to_orphan) 341 | return None 342 | 343 | # If `validate_block()` returned a non-existent chain index, we're 344 | # creating a new side branch. 345 | if chain_idx != ACTIVE_CHAIN_IDX and len(side_branches) < chain_idx: 346 | logger.info( 347 | f'creating a new side branch (idx {chain_idx}) ' 348 | f'for block {block.id}') 349 | side_branches.append([]) 350 | 351 | logger.info(f'connecting block {block.id} to chain {chain_idx}') 352 | chain = (active_chain if chain_idx == ACTIVE_CHAIN_IDX else 353 | side_branches[chain_idx - 1]) 354 | chain.append(block) 355 | 356 | # If we added to the active chain, perform upkeep on utxo_set and mempool. 357 | if chain_idx == ACTIVE_CHAIN_IDX: 358 | for tx in block.txns: 359 | mempool.pop(tx.id, None) 360 | 361 | if not tx.is_coinbase: 362 | for txin in tx.txins: 363 | rm_from_utxo(*txin.to_spend) 364 | for i, txout in enumerate(tx.txouts): 365 | add_to_utxo(txout, tx, i, tx.is_coinbase, len(chain)) 366 | 367 | if (not doing_reorg and reorg_if_necessary()) or \ 368 | chain_idx == ACTIVE_CHAIN_IDX: 369 | mine_interrupt.set() 370 | logger.info( 371 | f'block accepted ' 372 | f'height={len(active_chain) - 1} txns={len(block.txns)}') 373 | 374 | for peer in peer_hostnames: 375 | send_to_peer(block, peer) 376 | 377 | return chain_idx 378 | 379 | 380 | @with_lock(chain_lock) 381 | def disconnect_block(block, chain=None): 382 | chain = chain or active_chain 383 | assert block == chain[-1], "Block being disconnected must be tip." 384 | 385 | for tx in block.txns: 386 | mempool[tx.id] = tx 387 | 388 | # Restore UTXO set to what it was before this block. 389 | for txin in tx.txins: 390 | if txin.to_spend: # Account for degenerate coinbase txins. 391 | add_to_utxo(*find_txout_for_txin(txin, chain)) 392 | for i in range(len(tx.txouts)): 393 | rm_from_utxo(tx.id, i) 394 | 395 | logger.info(f'block {block.id} disconnected') 396 | return chain.pop() 397 | 398 | 399 | def find_txout_for_txin(txin, chain): 400 | txid, txout_idx = txin.to_spend 401 | 402 | for tx, block, height in txn_iterator(chain): 403 | if tx.id == txid: 404 | txout = tx.txouts[txout_idx] 405 | return (txout, tx, txout_idx, tx.is_coinbase, height) 406 | 407 | 408 | @with_lock(chain_lock) 409 | def reorg_if_necessary() -> bool: 410 | reorged = False 411 | frozen_side_branches = list(side_branches) # May change during this call. 412 | 413 | # TODO should probably be using `chainwork` for the basis of 414 | # comparison here. 415 | for branch_idx, chain in enumerate(frozen_side_branches, 1): 416 | fork_block, fork_idx, _ = locate_block( 417 | chain[0].prev_block_hash, active_chain) 418 | active_height = len(active_chain) 419 | branch_height = len(chain) + fork_idx 420 | 421 | if branch_height > active_height: 422 | logger.info( 423 | f'attempting reorg of idx {branch_idx} to active_chain: ' 424 | f'new height of {branch_height} (vs. {active_height})') 425 | reorged |= try_reorg(chain, branch_idx, fork_idx) 426 | 427 | return reorged 428 | 429 | 430 | @with_lock(chain_lock) 431 | def try_reorg(branch, branch_idx, fork_idx) -> bool: 432 | # Use the global keyword so that we can actually swap out the reference 433 | # in case of a reorg. 434 | global active_chain 435 | global side_branches 436 | 437 | fork_block = active_chain[fork_idx] 438 | 439 | def disconnect_to_fork(): 440 | while active_chain[-1].id != fork_block.id: 441 | yield disconnect_block(active_chain[-1]) 442 | 443 | old_active = list(disconnect_to_fork())[::-1] 444 | 445 | assert branch[0].prev_block_hash == active_chain[-1].id 446 | 447 | def rollback_reorg(): 448 | logger.info(f'reorg of idx {branch_idx} to active_chain failed') 449 | list(disconnect_to_fork()) # Force the gneerator to eval. 450 | 451 | for block in old_active: 452 | assert connect_block(block, doing_reorg=True) == ACTIVE_CHAIN_IDX 453 | 454 | for block in branch: 455 | connected_idx = connect_block(block, doing_reorg=True) 456 | if connected_idx != ACTIVE_CHAIN_IDX: 457 | rollback_reorg() 458 | return False 459 | 460 | # Fix up side branches: remove new active, add old active. 461 | side_branches.pop(branch_idx - 1) 462 | side_branches.append(old_active) 463 | 464 | logger.info( 465 | 'chain reorg! New height: %s, tip: %s', 466 | len(active_chain), active_chain[-1].id) 467 | 468 | return True 469 | 470 | 471 | def get_median_time_past(num_last_blocks: int) -> int: 472 | """Grep for: GetMedianTimePast.""" 473 | last_n_blocks = active_chain[::-1][:num_last_blocks] 474 | 475 | if not last_n_blocks: 476 | return 0 477 | 478 | return last_n_blocks[len(last_n_blocks) // 2].timestamp 479 | 480 | 481 | # Chain Persistance 482 | # ---------------------------------------------------------------------------- 483 | 484 | CHAIN_PATH = os.environ.get('TC_CHAIN_PATH', 'chain.dat') 485 | 486 | @with_lock(chain_lock) 487 | def save_to_disk(): 488 | with open(CHAIN_PATH, "wb") as f: 489 | logger.info(f"saving chain with {len(active_chain)} blocks") 490 | f.write(encode_socket_data(list(active_chain))) 491 | 492 | @with_lock(chain_lock) 493 | def load_from_disk(): 494 | if not os.path.isfile(CHAIN_PATH): 495 | return 496 | try: 497 | with open(CHAIN_PATH, "rb") as f: 498 | msg_len = int(binascii.hexlify(f.read(4) or b'\x00'), 16) 499 | new_blocks = deserialize(f.read(msg_len)) 500 | logger.info(f"loading chain from disk with {len(new_blocks)} blocks") 501 | for block in new_blocks: 502 | connect_block(block) 503 | except Exception: 504 | logger.exception('load chain failed, starting from genesis') 505 | 506 | 507 | # UTXO set 508 | # ---------------------------------------------------------------------------- 509 | 510 | utxo_set: Mapping[OutPoint, UnspentTxOut] = {} 511 | 512 | 513 | def add_to_utxo(txout, tx, idx, is_coinbase, height): 514 | utxo = UnspentTxOut( 515 | *txout, 516 | txid=tx.id, txout_idx=idx, is_coinbase=is_coinbase, height=height) 517 | 518 | logger.info(f'adding tx outpoint {utxo.outpoint} to utxo_set') 519 | utxo_set[utxo.outpoint] = utxo 520 | 521 | 522 | def rm_from_utxo(txid, txout_idx): 523 | del utxo_set[OutPoint(txid, txout_idx)] 524 | 525 | 526 | def find_utxo_in_list(txin, txns) -> UnspentTxOut: 527 | txid, txout_idx = txin.to_spend 528 | try: 529 | txout = [t for t in txns if t.id == txid][0].txouts[txout_idx] 530 | except Exception: 531 | return None 532 | 533 | return UnspentTxOut( 534 | *txout, txid=txid, is_coinbase=False, height=-1, txout_idx=txout_idx) 535 | 536 | 537 | # Proof of work 538 | # ---------------------------------------------------------------------------- 539 | 540 | def get_next_work_required(prev_block_hash: str) -> int: 541 | """ 542 | Based on the chain, return the number of difficulty bits the next block 543 | must solve. 544 | """ 545 | if not prev_block_hash: 546 | return Params.INITIAL_DIFFICULTY_BITS 547 | 548 | (prev_block, prev_height, _) = locate_block(prev_block_hash) 549 | 550 | if (prev_height + 1) % Params.DIFFICULTY_PERIOD_IN_BLOCKS != 0: 551 | return prev_block.bits 552 | 553 | with chain_lock: 554 | # #realname CalculateNextWorkRequired 555 | period_start_block = active_chain[max( 556 | prev_height - (Params.DIFFICULTY_PERIOD_IN_BLOCKS - 1), 0)] 557 | 558 | actual_time_taken = prev_block.timestamp - period_start_block.timestamp 559 | 560 | if actual_time_taken < Params.DIFFICULTY_PERIOD_IN_SECS_TARGET: 561 | # Increase the difficulty 562 | return prev_block.bits + 1 563 | elif actual_time_taken > Params.DIFFICULTY_PERIOD_IN_SECS_TARGET: 564 | return prev_block.bits - 1 565 | else: 566 | # Wow, that's unlikely. 567 | return prev_block.bits 568 | 569 | 570 | def assemble_and_solve_block(pay_coinbase_to_addr, txns=None): 571 | """ 572 | Construct a Block by pulling transactions from the mempool, then mine it. 573 | """ 574 | with chain_lock: 575 | prev_block_hash = active_chain[-1].id if active_chain else None 576 | 577 | block = Block( 578 | version=0, 579 | prev_block_hash=prev_block_hash, 580 | merkle_hash='', 581 | timestamp=int(time.time()), 582 | bits=get_next_work_required(prev_block_hash), 583 | nonce=0, 584 | txns=txns or [], 585 | ) 586 | 587 | if not block.txns: 588 | block = select_from_mempool(block) 589 | 590 | fees = calculate_fees(block) 591 | my_address = init_wallet()[2] 592 | coinbase_txn = Transaction.create_coinbase( 593 | my_address, (get_block_subsidy() + fees), len(active_chain)) 594 | block = block._replace(txns=[coinbase_txn, *block.txns]) 595 | block = block._replace(merkle_hash=get_merkle_root_of_txns(block.txns).val) 596 | 597 | if len(serialize(block)) > Params.MAX_BLOCK_SERIALIZED_SIZE: 598 | raise ValueError('txns specified create a block too large') 599 | 600 | return mine(block) 601 | 602 | 603 | def calculate_fees(block) -> int: 604 | """ 605 | Given the txns in a Block, subtract the amount of coin output from the 606 | inputs. This is kept as a reward by the miner. 607 | """ 608 | fee = 0 609 | 610 | def utxo_from_block(txin): 611 | tx = [t.txouts for t in block.txns if t.id == txin.to_spend.txid] 612 | return tx[0][txin.to_spend.txout_idx] if tx else None 613 | 614 | def find_utxo(txin): 615 | return utxo_set.get(txin.to_spend) or utxo_from_block(txin) 616 | 617 | for txn in block.txns: 618 | spent = sum(find_utxo(i).value for i in txn.txins) 619 | sent = sum(o.value for o in txn.txouts) 620 | fee += (spent - sent) 621 | 622 | return fee 623 | 624 | 625 | def get_block_subsidy() -> int: 626 | halvings = len(active_chain) // Params.HALVE_SUBSIDY_AFTER_BLOCKS_NUM 627 | 628 | if halvings >= 64: 629 | return 0 630 | 631 | return 50 * Params.BELUSHIS_PER_COIN // (2 ** halvings) 632 | 633 | 634 | # Signal to communicate to the mining thread that it should stop mining because 635 | # we've updated the chain with a new block. 636 | mine_interrupt = threading.Event() 637 | 638 | 639 | def mine(block): 640 | start = time.time() 641 | nonce = 0 642 | target = (1 << (256 - block.bits)) 643 | mine_interrupt.clear() 644 | 645 | while int(sha256d(block.header(nonce)), 16) >= target: 646 | nonce += 1 647 | 648 | if nonce % 10000 == 0 and mine_interrupt.is_set(): 649 | logger.info('[mining] interrupted') 650 | mine_interrupt.clear() 651 | return None 652 | 653 | block = block._replace(nonce=nonce) 654 | duration = int(time.time() - start) or 0.001 655 | khs = (block.nonce // duration) // 1000 656 | logger.info( 657 | f'[mining] block found! {duration} s - {khs} KH/s - {block.id}') 658 | 659 | return block 660 | 661 | 662 | def mine_forever(): 663 | while True: 664 | my_address = init_wallet()[2] 665 | block = assemble_and_solve_block(my_address) 666 | 667 | if block: 668 | connect_block(block) 669 | save_to_disk() 670 | 671 | 672 | # Validation 673 | # ---------------------------------------------------------------------------- 674 | 675 | 676 | def validate_txn(txn: Transaction, 677 | as_coinbase: bool = False, 678 | siblings_in_block: Iterable[Transaction] = None, 679 | allow_utxo_from_mempool: bool = True, 680 | ) -> Transaction: 681 | """ 682 | Validate a single transaction. Used in various contexts, so the 683 | parameters facilitate different uses. 684 | """ 685 | txn.validate_basics(as_coinbase=as_coinbase) 686 | 687 | available_to_spend = 0 688 | 689 | for i, txin in enumerate(txn.txins): 690 | utxo = utxo_set.get(txin.to_spend) 691 | 692 | if siblings_in_block: 693 | utxo = utxo or find_utxo_in_list(txin, siblings_in_block) 694 | 695 | if allow_utxo_from_mempool: 696 | utxo = utxo or find_utxo_in_mempool(txin) 697 | 698 | if not utxo: 699 | raise TxnValidationError( 700 | f'Could find no UTXO for TxIn[{i}] -- orphaning txn', 701 | to_orphan=txn) 702 | 703 | if utxo.is_coinbase and \ 704 | (get_current_height() - utxo.height) < \ 705 | Params.COINBASE_MATURITY: 706 | raise TxnValidationError(f'Coinbase UTXO not ready for spend') 707 | 708 | try: 709 | validate_signature_for_spend(txin, utxo, txn) 710 | except TxUnlockError: 711 | raise TxnValidationError(f'{txin} is not a valid spend of {utxo}') 712 | 713 | available_to_spend += utxo.value 714 | 715 | if available_to_spend < sum(o.value for o in txn.txouts): 716 | raise TxnValidationError('Spend value is more than available') 717 | 718 | return txn 719 | 720 | 721 | def validate_signature_for_spend(txin, utxo: UnspentTxOut, txn): 722 | pubkey_as_addr = pubkey_to_address(txin.unlock_pk) 723 | verifying_key = ecdsa.VerifyingKey.from_string( 724 | txin.unlock_pk, curve=ecdsa.SECP256k1) 725 | 726 | if pubkey_as_addr != utxo.to_address: 727 | raise TxUnlockError("Pubkey doesn't match") 728 | 729 | try: 730 | spend_msg = build_spend_message( 731 | txin.to_spend, txin.unlock_pk, txin.sequence, txn.txouts) 732 | verifying_key.verify(txin.unlock_sig, spend_msg) 733 | except Exception: 734 | logger.exception('Key verification failed') 735 | raise TxUnlockError("Signature doesn't match") 736 | 737 | return True 738 | 739 | 740 | def build_spend_message(to_spend, pk, sequence, txouts) -> bytes: 741 | """This should be ~roughly~ equivalent to SIGHASH_ALL.""" 742 | return sha256d( 743 | serialize(to_spend) + str(sequence) + 744 | binascii.hexlify(pk).decode() + serialize(txouts)).encode() 745 | 746 | 747 | @with_lock(chain_lock) 748 | def validate_block(block: Block) -> Block: 749 | if not block.txns: 750 | raise BlockValidationError('txns empty') 751 | 752 | if block.timestamp - time.time() > Params.MAX_FUTURE_BLOCK_TIME: 753 | raise BlockValidationError('Block timestamp too far in future') 754 | 755 | if int(block.id, 16) > (1 << (256 - block.bits)): 756 | raise BlockValidationError("Block header doesn't satisfy bits") 757 | 758 | if [i for (i, tx) in enumerate(block.txns) if tx.is_coinbase] != [0]: 759 | raise BlockValidationError('First txn must be coinbase and no more') 760 | 761 | try: 762 | for i, txn in enumerate(block.txns): 763 | txn.validate_basics(as_coinbase=(i == 0)) 764 | except TxnValidationError: 765 | logger.exception(f"Transaction {txn} in {block} failed to validate") 766 | raise BlockValidationError('Invalid txn {txn.id}') 767 | 768 | if get_merkle_root_of_txns(block.txns).val != block.merkle_hash: 769 | raise BlockValidationError('Merkle hash invalid') 770 | 771 | if block.timestamp <= get_median_time_past(11): 772 | raise BlockValidationError('timestamp too old') 773 | 774 | if not block.prev_block_hash and not active_chain: 775 | # This is the genesis block. 776 | prev_block_chain_idx = ACTIVE_CHAIN_IDX 777 | else: 778 | prev_block, prev_block_height, prev_block_chain_idx = locate_block( 779 | block.prev_block_hash) 780 | 781 | if not prev_block: 782 | raise BlockValidationError( 783 | f'prev block {block.prev_block_hash} not found in any chain', 784 | to_orphan=block) 785 | 786 | # No more validation for a block getting attached to a branch. 787 | if prev_block_chain_idx != ACTIVE_CHAIN_IDX: 788 | return block, prev_block_chain_idx 789 | 790 | # Prev. block found in active chain, but isn't tip => new fork. 791 | elif prev_block != active_chain[-1]: 792 | return block, prev_block_chain_idx + 1 # Non-existent 793 | 794 | if get_next_work_required(block.prev_block_hash) != block.bits: 795 | raise BlockValidationError('bits is incorrect') 796 | 797 | for txn in block.txns[1:]: 798 | try: 799 | validate_txn(txn, siblings_in_block=block.txns[1:], 800 | allow_utxo_from_mempool=False) 801 | except TxnValidationError: 802 | msg = f"{txn} failed to validate" 803 | logger.exception(msg) 804 | raise BlockValidationError(msg) 805 | 806 | return block, prev_block_chain_idx 807 | 808 | 809 | # mempool 810 | # ---------------------------------------------------------------------------- 811 | 812 | # Set of yet-unmined transactions. 813 | mempool: Dict[str, Transaction] = {} 814 | 815 | # Set of orphaned (i.e. has inputs referencing yet non-existent UTXOs) 816 | # transactions. 817 | orphan_txns: Iterable[Transaction] = [] 818 | 819 | 820 | def find_utxo_in_mempool(txin) -> UnspentTxOut: 821 | txid, idx = txin.to_spend 822 | 823 | try: 824 | txout = mempool[txid].txouts[idx] 825 | except Exception: 826 | logger.debug("Couldn't find utxo in mempool for %s", txin) 827 | return None 828 | 829 | return UnspentTxOut( 830 | *txout, txid=txid, is_coinbase=False, height=-1, txout_idx=idx) 831 | 832 | 833 | def select_from_mempool(block: Block) -> Block: 834 | """Fill a Block with transactions from the mempool.""" 835 | added_to_block = set() 836 | 837 | def check_block_size(b) -> bool: 838 | return len(serialize(block)) < Params.MAX_BLOCK_SERIALIZED_SIZE 839 | 840 | def try_add_to_block(block, txid) -> Block: 841 | if txid in added_to_block: 842 | return block 843 | 844 | tx = mempool[txid] 845 | 846 | # For any txin that can't be found in the main chain, find its 847 | # transaction in the mempool (if it exists) and add it to the block. 848 | for txin in tx.txins: 849 | if txin.to_spend in utxo_set: 850 | continue 851 | 852 | in_mempool = find_utxo_in_mempool(txin) 853 | 854 | if not in_mempool: 855 | logger.debug(f"Couldn't find UTXO for {txin}") 856 | return None 857 | 858 | block = try_add_to_block(block, in_mempool.txid) 859 | if not block: 860 | logger.debug(f"Couldn't add parent") 861 | return None 862 | 863 | newblock = block._replace(txns=[*block.txns, tx]) 864 | 865 | if check_block_size(newblock): 866 | logger.debug(f'added tx {tx.id} to block') 867 | added_to_block.add(txid) 868 | return newblock 869 | else: 870 | return block 871 | 872 | for txid in mempool: 873 | newblock = try_add_to_block(block, txid) 874 | 875 | if check_block_size(newblock): 876 | block = newblock 877 | else: 878 | break 879 | 880 | return block 881 | 882 | 883 | def add_txn_to_mempool(txn: Transaction): 884 | if txn.id in mempool: 885 | logger.info(f'txn {txn.id} already seen') 886 | return 887 | 888 | try: 889 | txn = validate_txn(txn) 890 | except TxnValidationError as e: 891 | if e.to_orphan: 892 | logger.info(f'txn {e.to_orphan.id} submitted as orphan') 893 | orphan_txns.append(e.to_orphan) 894 | else: 895 | logger.exception(f'txn rejected') 896 | else: 897 | logger.info(f'txn {txn.id} added to mempool') 898 | mempool[txn.id] = txn 899 | 900 | for peer in peer_hostnames: 901 | send_to_peer(txn, peer) 902 | 903 | 904 | # Merkle trees 905 | # ---------------------------------------------------------------------------- 906 | 907 | class MerkleNode(NamedTuple): 908 | val: str 909 | children: Iterable = None 910 | 911 | 912 | def get_merkle_root_of_txns(txns): 913 | return get_merkle_root(*[t.id for t in txns]) 914 | 915 | 916 | @lru_cache(maxsize=1024) 917 | def get_merkle_root(*leaves: Tuple[str]) -> MerkleNode: 918 | """Builds a Merkle tree and returns the root given some leaf values.""" 919 | if len(leaves) % 2 == 1: 920 | leaves = leaves + (leaves[-1],) 921 | 922 | def find_root(nodes): 923 | newlevel = [ 924 | MerkleNode(sha256d(i1.val + i2.val), children=[i1, i2]) 925 | for [i1, i2] in _chunks(nodes, 2) 926 | ] 927 | 928 | return find_root(newlevel) if len(newlevel) > 1 else newlevel[0] 929 | 930 | return find_root([MerkleNode(sha256d(l)) for l in leaves]) 931 | 932 | 933 | # Peer-to-peer 934 | # ---------------------------------------------------------------------------- 935 | 936 | peer_hostnames = {p for p in os.environ.get('TC_PEERS', '').split(',') if p} 937 | 938 | # Signal when the initial block download has completed. 939 | ibd_done = threading.Event() 940 | 941 | 942 | class GetBlocksMsg(NamedTuple): # Request blocks during initial sync 943 | """ 944 | See https://bitcoin.org/en/developer-guide#blocks-first 945 | """ 946 | from_blockid: str 947 | 948 | CHUNK_SIZE = 50 949 | 950 | def handle(self, sock, peer_hostname): 951 | logger.debug(f"[p2p] recv getblocks from {peer_hostname}") 952 | 953 | _, height, _ = locate_block(self.from_blockid, active_chain) 954 | 955 | # If we don't recognize the requested hash as part of the active 956 | # chain, start at the genesis block. 957 | height = height or 1 958 | 959 | with chain_lock: 960 | blocks = active_chain[height:(height + self.CHUNK_SIZE)] 961 | 962 | logger.debug(f"[p2p] sending {len(blocks)} to {peer_hostname}") 963 | send_to_peer(InvMsg(blocks), peer_hostname) 964 | 965 | 966 | class InvMsg(NamedTuple): # Convey blocks to a peer who is doing initial sync 967 | blocks: Iterable[str] 968 | 969 | def handle(self, sock, peer_hostname): 970 | logger.info(f"[p2p] recv inv from {peer_hostname}") 971 | 972 | new_blocks = [b for b in self.blocks if not locate_block(b.id)[0]] 973 | 974 | if not new_blocks: 975 | logger.info('[p2p] initial block download complete') 976 | ibd_done.set() 977 | return 978 | 979 | for block in new_blocks: 980 | connect_block(block) 981 | 982 | new_tip_id = active_chain[-1].id 983 | logger.info(f'[p2p] continuing initial block download at {new_tip_id}') 984 | 985 | with chain_lock: 986 | # "Recursive" call to continue the initial block sync. 987 | send_to_peer(GetBlocksMsg(new_tip_id)) 988 | 989 | 990 | class GetUTXOsMsg(NamedTuple): # List all UTXOs 991 | def handle(self, sock, peer_hostname): 992 | sock.sendall(encode_socket_data(list(utxo_set.items()))) 993 | 994 | 995 | class GetMempoolMsg(NamedTuple): # List the mempool 996 | def handle(self, sock, peer_hostname): 997 | sock.sendall(encode_socket_data(list(mempool.keys()))) 998 | 999 | 1000 | class GetActiveChainMsg(NamedTuple): # Get the active chain in its entirety. 1001 | def handle(self, sock, peer_hostname): 1002 | sock.sendall(encode_socket_data(list(active_chain))) 1003 | 1004 | 1005 | class AddPeerMsg(NamedTuple): 1006 | peer_hostname: str 1007 | 1008 | def handle(self, sock, peer_hostname): 1009 | peer_hostnames.add(self.peer_hostname) 1010 | 1011 | 1012 | def read_all_from_socket(req) -> object: 1013 | data = b'' 1014 | # Our protocol is: first 4 bytes signify msg length. 1015 | msg_len = int(binascii.hexlify(req.recv(4) or b'\x00'), 16) 1016 | 1017 | while msg_len > 0: 1018 | tdat = req.recv(1024) 1019 | data += tdat 1020 | msg_len -= len(tdat) 1021 | 1022 | return deserialize(data.decode()) if data else None 1023 | 1024 | 1025 | def send_to_peer(data, peer=None): 1026 | """Send a message to a (by default) random peer.""" 1027 | global peer_hostnames 1028 | 1029 | peer = peer or random.choice(list(peer_hostnames)) 1030 | tries_left = 3 1031 | 1032 | while tries_left > 0: 1033 | try: 1034 | with socket.create_connection((peer, PORT), timeout=1) as s: 1035 | s.sendall(encode_socket_data(data)) 1036 | except Exception: 1037 | logger.exception(f'failed to send to peer {peer}') 1038 | tries_left -= 1 1039 | time.sleep(2) 1040 | else: 1041 | return 1042 | 1043 | logger.info(f"[p2p] removing dead peer {peer}") 1044 | peer_hostnames = {x for x in peer_hostnames if x != peer} 1045 | 1046 | 1047 | def int_to_8bytes(a: int) -> bytes: return binascii.unhexlify(f"{a:0{8}x}") 1048 | 1049 | 1050 | def encode_socket_data(data: object) -> bytes: 1051 | """Our protocol is: first 4 bytes signify msg length.""" 1052 | to_send = serialize(data).encode() 1053 | return int_to_8bytes(len(to_send)) + to_send 1054 | 1055 | 1056 | class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer): 1057 | pass 1058 | 1059 | 1060 | class TCPHandler(socketserver.BaseRequestHandler): 1061 | 1062 | def handle(self): 1063 | data = read_all_from_socket(self.request) 1064 | peer_hostname = self.request.getpeername()[0] 1065 | peer_hostnames.add(peer_hostname) 1066 | 1067 | if hasattr(data, 'handle') and isinstance(data.handle, Callable): 1068 | logger.info(f'received msg {data} from peer {peer_hostname}') 1069 | data.handle(self.request, peer_hostname) 1070 | elif isinstance(data, Transaction): 1071 | logger.info(f"received txn {data.id} from peer {peer_hostname}") 1072 | add_txn_to_mempool(data) 1073 | elif isinstance(data, Block): 1074 | logger.info(f"received block {data.id} from peer {peer_hostname}") 1075 | connect_block(data) 1076 | 1077 | 1078 | # Wallet 1079 | # ---------------------------------------------------------------------------- 1080 | 1081 | WALLET_PATH = os.environ.get('TC_WALLET_PATH', 'wallet.dat') 1082 | 1083 | 1084 | def pubkey_to_address(pubkey: bytes) -> str: 1085 | if 'ripemd160' not in hashlib.algorithms_available: 1086 | raise RuntimeError('missing ripemd160 hash algorithm') 1087 | 1088 | sha = hashlib.sha256(pubkey).digest() 1089 | ripe = hashlib.new('ripemd160', sha).digest() 1090 | return b58encode_check(b'\x00' + ripe) 1091 | 1092 | 1093 | @lru_cache() 1094 | def init_wallet(path=None): 1095 | path = path or WALLET_PATH 1096 | 1097 | if os.path.exists(path): 1098 | with open(path, 'rb') as f: 1099 | signing_key = ecdsa.SigningKey.from_string( 1100 | f.read(), curve=ecdsa.SECP256k1) 1101 | else: 1102 | logger.info(f"generating new wallet: '{path}'") 1103 | signing_key = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1) 1104 | with open(path, 'wb') as f: 1105 | f.write(signing_key.to_string()) 1106 | 1107 | verifying_key = signing_key.get_verifying_key() 1108 | my_address = pubkey_to_address(verifying_key.to_string()) 1109 | logger.info(f"your address is {my_address}") 1110 | 1111 | return signing_key, verifying_key, my_address 1112 | 1113 | 1114 | # Misc. utilities 1115 | # ---------------------------------------------------------------------------- 1116 | 1117 | class BaseException(Exception): 1118 | def __init__(self, msg): 1119 | self.msg = msg 1120 | 1121 | 1122 | class TxUnlockError(BaseException): 1123 | pass 1124 | 1125 | 1126 | class TxnValidationError(BaseException): 1127 | def __init__(self, *args, to_orphan: Transaction = None, **kwargs): 1128 | super().__init__(*args, **kwargs) 1129 | self.to_orphan = to_orphan 1130 | 1131 | 1132 | class BlockValidationError(BaseException): 1133 | def __init__(self, *args, to_orphan: Block = None, **kwargs): 1134 | super().__init__(*args, **kwargs) 1135 | self.to_orphan = to_orphan 1136 | 1137 | 1138 | def serialize(obj) -> str: 1139 | """NamedTuple-flavored serialization to JSON.""" 1140 | def contents_to_primitive(o): 1141 | if hasattr(o, '_asdict'): 1142 | o = {**o._asdict(), '_type': type(o).__name__} 1143 | elif isinstance(o, (list, tuple)): 1144 | return [contents_to_primitive(i) for i in o] 1145 | elif isinstance(o, bytes): 1146 | return binascii.hexlify(o).decode() 1147 | elif not isinstance(o, (dict, bytes, str, int, type(None))): 1148 | raise ValueError(f"Can't serialize {o}") 1149 | 1150 | if isinstance(o, Mapping): 1151 | for k, v in o.items(): 1152 | o[k] = contents_to_primitive(v) 1153 | 1154 | return o 1155 | 1156 | return json.dumps( 1157 | contents_to_primitive(obj), sort_keys=True, separators=(',', ':')) 1158 | 1159 | 1160 | def deserialize(serialized: str) -> object: 1161 | """NamedTuple-flavored serialization from JSON.""" 1162 | gs = globals() 1163 | 1164 | def contents_to_objs(o): 1165 | if isinstance(o, list): 1166 | return [contents_to_objs(i) for i in o] 1167 | elif not isinstance(o, Mapping): 1168 | return o 1169 | 1170 | _type = gs[o.pop('_type', None)] 1171 | bytes_keys = { 1172 | k for k, v in get_type_hints(_type).items() if v == bytes} 1173 | 1174 | for k, v in o.items(): 1175 | o[k] = contents_to_objs(v) 1176 | 1177 | if k in bytes_keys: 1178 | o[k] = binascii.unhexlify(o[k]) if o[k] else o[k] 1179 | 1180 | return _type(**o) 1181 | 1182 | return contents_to_objs(json.loads(serialized)) 1183 | 1184 | 1185 | def sha256d(s: Union[str, bytes]) -> str: 1186 | """A double SHA-256 hash.""" 1187 | if not isinstance(s, bytes): 1188 | s = s.encode() 1189 | 1190 | return hashlib.sha256(hashlib.sha256(s).digest()).hexdigest() 1191 | 1192 | 1193 | def _chunks(l, n) -> Iterable[Iterable]: 1194 | return (l[i:i + n] for i in range(0, len(l), n)) 1195 | 1196 | 1197 | # Main 1198 | # ---------------------------------------------------------------------------- 1199 | 1200 | PORT = os.environ.get('TC_PORT', 9999) 1201 | 1202 | 1203 | def main(): 1204 | load_from_disk() 1205 | 1206 | workers = [] 1207 | server = ThreadedTCPServer(('0.0.0.0', PORT), TCPHandler) 1208 | 1209 | def start_worker(fnc): 1210 | workers.append(threading.Thread(target=fnc, daemon=True)) 1211 | workers[-1].start() 1212 | 1213 | logger.info(f'[p2p] listening on {PORT}') 1214 | start_worker(server.serve_forever) 1215 | 1216 | if peer_hostnames: 1217 | logger.info( 1218 | f'start initial block download from {len(peer_hostnames)} peers') 1219 | send_to_peer(GetBlocksMsg(active_chain[-1].id)) 1220 | ibd_done.wait(60.) # Wait a maximum of 60 seconds for IBD to complete. 1221 | 1222 | start_worker(mine_forever) 1223 | [w.join() for w in workers] 1224 | 1225 | 1226 | if __name__ == '__main__': 1227 | signing_key, verifying_key, my_address = init_wallet() 1228 | main() 1229 | --------------------------------------------------------------------------------