├── README.md └── blockchain_consensus.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # blockchain_consensus 2 | This is the code for the "Blockchain Consensus Algorithms and Artificial Intelligence" by Siraj Raval on Youtube 3 | 4 | ## Overview 5 | 6 | This is the code for [this](https://youtu.be/5Tr13l0O1Ws) video on Youtube by Siraj Raval. The attached notebook demonstrates the use cases of blockchain and AI combined. I also have code in it that demonstrates the proof of work algorithm from bitcoin. 7 | 8 | ## Dependencies 9 | 10 | none 11 | 12 | ## Usage 13 | 14 | Just run `jupyter notebook` in terminal and the code will pop up in your browser. 15 | 16 | ## Credits 17 | 18 | Credits for this code go to [tenthirtyone](https://github.com/tenthirtyone/Proof_of_Work) i've merely created a wrapper to get people started 19 | -------------------------------------------------------------------------------- /blockchain_consensus.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "collapsed": true 7 | }, 8 | "source": [ 9 | "# An Overview of Blockchain Consensus Protocols \n", 10 | "\n", 11 | "## Demo -- I'll write Python code for a popular blockchain consensus algorithms (Proof of Work) and talk about others like Proof of Stake, Proof of Burn, etc.) \n", 12 | "\n", 13 | "Steps\n", 14 | "\n", 15 | "1. Motivations for Blockchain + AI\n", 16 | "2. Examples of apps that use Blockchain + AI\n", 17 | "3. Programming Proof of Work\n", 18 | "4. Overview of Proof of Stake & Others\n", 19 | "\n", 20 | "### Why Blockchain + AI\n", 21 | "\n", 22 | "![alt text](https://bitcoin.org/img/dev/en-blockchain-overview.svg \"Logo Title Text 1\")\n", 23 | "\n", 24 | "![alt text](https://image.slidesharecdn.com/securingpersonalgenomicdatares-160504111334/95/securing-personal-genomic-data-res-17-638.jpg?cb=1462360504 \"Logo Title Text 1\")\n", 25 | "\n", 26 | "![alt text](https://cdn-images-1.medium.com/max/1600/1*SVMH0_mLM9gH3miMPSsAgw.png \"Logo Title Text 1\")\n", 27 | "\n", 28 | "![alt text](https://image.slidesharecdn.com/lnezeisksjig9jhh3a5a-signature-b4dcd0353009b2265212073ee2dd56e186c0b13a6a9afd1ed93eb0bebb12e902-poli-160810001503/95/deep-learning-more-than-a-fad-2-638.jpg?cb=1470788314 \"Logo Title Text 1\")\n", 29 | "\n", 30 | "![alt text](http://usblogs.pwc.com/emerging-technology/wp-content/uploads/2016/09/yin-yang-blockchain.png \"Logo Title Text 1\")\n", 31 | "\n", 32 | "#### The Yin:\n", 33 | " - AI is about machines doing educated guesswork for us\n", 34 | " - Machines “learn” by processing training datasets \n", 35 | " - Probabilistic methods that head in the direction of most likely reality\n", 36 | " - AI algorithms are central to goals\n", 37 | "\n", 38 | "#### The Yang:\n", 39 | " - Blockchains are more about validation, permanence and greater degrees of certainty and control. .\n", 40 | " - An immutable, verifiable Ledger\n", 41 | " - Intelligence discovers opportunities & blockchains validate them and \n", 42 | " - Validates identities & transactions, automate the validation and generate the audit trail that goes back as far as the chain does.\n", 43 | " - Blockchain algorithms merely a means of permanently validating and recording transaction truth. \n", 44 | " - blockchain data = canonical reference for who did what when, why, & for how much.\n", 45 | " \n", 46 | "AI provides the educated guesswork to make sense of a complicated business environment and suggest a path forward.\n", 47 | "\n", 48 | "A blockchain by contrast would play the role of the truth teller, guarantor and mediator, the one bringing two parties together to forge and document an agreement immutably. Distributes trust across many nodes.\n", 49 | "\n", 50 | "\n", 51 | "### AI + Blockchain use cases\n", 52 | "\n", 53 | "#### OpenMined (Decentralized Marketplace for Data)\n", 54 | "\n", 55 | "![alt text](https://i.imgur.com/Xc6CrOy.jpg \"Logo Title Text 1\")\n", 56 | "\n", 57 | "- Startup can send a model spec to a smart contract\n", 58 | "- The smart contract intializes the model itself. Because the wieghts are randomly intialized by the smart contract and all the information used to intilaize the weights is public, there are no hidden secrets inside the homomorphic encryption that our startup could embed to make it so that the model secretly memorizes the data. \n", 59 | "- The smart contract sends the model spec to the oracle\n", 60 | "- The oracle generates a public private key pair, encrypts the model, then sends it back to the smart contract\n", 61 | "- After training on all the users, the model is sent to the oracle, its decrypted, encrypted with the oracle’s keys, then sent back. \n", 62 | "- Since the oracle acts as a trusted third party, no one can decrypt the model or gradients. \n", 63 | "- A second smart contract that acts as an insurance policy against the oracle sharing secrets. \n", 64 | "\n", 65 | "#### NumerAI (Decentralized AI Hedge Fund)\n", 66 | "\n", 67 | "![alt text](https://qph.ec.quoracdn.net/main-qimg-9cb24ba49c62c1ecda33c99bcee1d057-c \"Logo Title Text 1\")\n", 68 | "\n", 69 | "#### Self Driving Car Network\n", 70 | "\n", 71 | "![alt text](https://cdn-images-1.medium.com/max/1600/0*XdZlw6TNeK8L-1SP.png \"Logo Title Text 1\")\n", 72 | "\n", 73 | "![alt text](https://media.licdn.com/mpr/mpr/shrinknp_800_800/AAEAAQAAAAAAAAXDAAAAJGM3NDlmNGRjLWJiZWUtNDUwYy05ZWYwLWM1MDU2OGNlZTA0OQ.jpg \"Logo Title Text 1\")\n", 74 | "\n", 75 | "### Decentralized / shared control encourages data sharing:\n", 76 | "- Leads to more data, and therefore better models.\n", 77 | "- Leads to qualitatively new data, and therefore qualitatively new models.\n", 78 | "- Allows for shared control of AI training data & models.\n", 79 | "### Immutability / audit trail:\n", 80 | "- Leads to provenance on training/testing data & models, to improve the trustworthiness of the data & models. Data wants reputation too.\n", 81 | "### Native assets / exchanges:\n", 82 | "- Leads to training/testing data & models as intellectual property (IP) assets, which leads to decentralized data & model exchanges. It also gives better control for upstream usage of your data.\n", 83 | "\n", 84 | "#### (The most wild/exciting idea) Artificial Intelligence Distributed Autonomous Organizations\n", 85 | "\n", 86 | "![alt text](https://image.slidesharecdn.com/20170405berlinbdbmeetup-aidaos-170419075411/95/artificial-intelligence-ai-daos-decentralized-autonomous-organizations-bigchaindb-ipdb-meetup-4-april-05-2017-9-638.jpg?cb=1492593203 \"Logo Title Text 1\")\n", 87 | "\n", 88 | "![alt text](https://cdn-images-1.medium.com/max/1600/0*DfsxWVFfaL07ZQnx.GIF \"Logo Title Text 1\")\n", 89 | "\n", 90 | "![alt text](https://image.slidesharecdn.com/20170405berlinbdbmeetup-aidaos-170419075411/95/artificial-intelligence-ai-daos-decentralized-autonomous-organizations-bigchaindb-ipdb-meetup-4-april-05-2017-13-638.jpg?cb=1492593203 \"Logo Title Text 1\")" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "# Popular Consensus Algorithms\n", 98 | "- Proof of work\n", 99 | "- Proof of Stake" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "# Proof of work\n", 107 | "![alt text](https://image.slidesharecdn.com/blockchainbtsym-161201162319/95/blockchain-btsym-16-11-638.jpg?cb=1480610107 \"Logo Title Text 1\")\n", 108 | "\n", 109 | "![alt text](https://image.slidesharecdn.com/blockchainbtsym-161201162319/95/blockchain-btsym-16-12-638.jpg?cb=1480610107 \"Logo Title Text 1\")" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": { 116 | "collapsed": true 117 | }, 118 | "outputs": [], 119 | "source": [ 120 | "#Simple Proof-of-Work example\n", 121 | "\n", 122 | "import sys\n", 123 | "import time\n", 124 | "import hashlib \n", 125 | "from struct import unpack, pack\n", 126 | "\n", 127 | "#The timestamp, message and payload are just the “stuff” that you \n", 128 | "#want to send the network. \n", 129 | "#It could be block data, as with Bitcoin mining.\n", 130 | "\n", 131 | "timestamp = str(time.time())\n", 132 | "message = \"this is a random message\"\n", 133 | "payload = timestamp + message\n", 134 | "\n", 135 | "#The nonce, guess, throttle and target are used to perform the work. \n", 136 | "#The important thing about proof-of-work is that it has to be difficult \n", 137 | "#for the \n", 138 | "#client but easy for the system to check. Only the nonce changes with \n", 139 | "#each guess. \n", 140 | "#The probability of being correct is so low that this system, \n", 141 | "#probabilistically, \n", 142 | "#guarantees many guesses or work, is performed.\n", 143 | "\n", 144 | "nonce = 0\n", 145 | "guess = 999999999999\n", 146 | "throttle = 100000000\n", 147 | "target = 2**64/ throttle\n", 148 | "\n", 149 | "#\n", 150 | "#nonce increments by 1 each time. \n", 151 | "#Effectively counting how many guesses the computer makes. \n", 152 | "#However, that is not a requirement.\n", 153 | "\n", 154 | "#When the system checks the work, \n", 155 | "#it does not have to trace back over each guess. \n", 156 | "#It only checks the nonce submitted with the data.\n", 157 | "\n", 158 | "\n", 159 | "payloadHash = hashlib.sha512(payload).digest()\n", 160 | "\n", 161 | "start = time.time()\n", 162 | "\n", 163 | "#These three lines are our proof-of-work algorithm. \n", 164 | "#It’s a simple loop. This hashes our data with two rounds of SHA256. \n", 165 | "#The first 8 bytes are turned in to a number. \n", 166 | "#Effectively, this is how a computer makes a guess.\n", 167 | "while guess > target:\n", 168 | " nonce+1\n", 169 | " guess, = unpack('>Q', hashlib.sha512(hashlib.sha512(pack('>Q', nonce) +\n", 170 | " payloadHash).digest()).digest())[0:8]\n", 171 | "end = time.time()\n", 172 | "#The nonce is what proves the work was done. \n", 173 | "#In this example, it’s value represents the number of attempts your CPU made before \n", 174 | "#it found a valid guess below the target. Because each guess has the same probability of being \n", 175 | "#under the target the method of generating your nonce does not matter. However, simply incrementing \n", 176 | "#the nonce is cheaper than generating a random number. When work is submitted to the network, \n", 177 | "#the nonce is used to verify the correctness.\n", 178 | "print \"%s:%s:%s:%s:%s:%s:%s\" % (timestamp, message, nonce, guess, payload, target, end-start)" 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "metadata": {}, 184 | "source": [ 185 | "# Proof of Stake\n", 186 | "\n", 187 | "![alt text](http://webcomsystem.net/blog/wp-content/uploads/2017/08/POW-vs-POS.jpg \"Logo Title Text 1\")\n", 188 | "\n", 189 | "- Proof of Stake (PoS) is a category of consensus algorithms for public blockchains that depend on a validator's economic stake in the network. \n", 190 | "- In proof of work (PoW) based public blockchains (e.g. Bitcoin and the current implementation of Ethereum), the algorithm rewards participants who solve cryptographic puzzles in order to validate transactions and create new blocks (i.e. mining). \n", 191 | "- In PoS-based public blockchains (e.g. Ethereum's upcoming Casper implementation), a set of validators take turns proposing and voting on the next block, and the weight of each validator's vote depends on the size of its deposit (i.e. stake). \n", 192 | "- Significant advantages of PoS include security, reduced risk of centralization, and energy efficiency.\n", 193 | "\n", 194 | "In general, a proof of stake algorithm looks as follows. \n", 195 | "\n", 196 | "- The blockchain keeps track of a set of validators\n", 197 | "- Anyone can become a validator by sending a special type of transaction that locks up their coin into a deposit\n", 198 | "- The process of creating and agreeing to new blocks is then done through a consensus algorithm that all current validators can participate in\n", 199 | "- Many ways to assign rewards to validators who participate in the consensus algorithm, two major types: chain-based proof of stake and BFT-style proof of stake.\n", 200 | "\n", 201 | "In chain-based proof of stake\n", 202 | "- the algorithm pseudo-randomly selects a validator during each time slot (eg. every period of 10 seconds might be a time slot)\n", 203 | "- assigns that validator the right to create a single block\n", 204 | "- this block must point to some previous block (normally the block at the end of the previously longest chain), and so over time most blocks converge into a single constantly growing chain.\n", 205 | "\n", 206 | "In BFT-style proof of stake (byzantine fault tolerant)\n", 207 | "- validators are randomly assigned the right to propose blocks\n", 208 | "- agreeing on which block is canonical is done through a multi-round process where every validator sends a \"vote\" for some specific block during each round\n", 209 | "- at the end of the process all (honest and online) validators permanently agree on whether or not any given block is part of the chain. \n", 210 | "- Blocks may still be chained together; the key difference is that consensus on a block can come within one block, and does not depend on the length or size of the chain after it.\n" 211 | ] 212 | }, 213 | { 214 | "cell_type": "markdown", 215 | "metadata": { 216 | "collapsed": true 217 | }, 218 | "source": [ 219 | "More...\n", 220 | "\n", 221 | "### Proof of Activity (hybrid PoW + PoS) - DeCred governenance currency\n", 222 | "\n", 223 | "- Mining kicks off in a traditional proof-of-work fashion (miners race to solve a cryptographic puzzle)\n", 224 | "- Blocks mined do not contain any transactions so winning block will only contain a header and miner's reward address\n", 225 | "- Then, system switches to proof of stake\n", 226 | "- Based on information in the header, a random group of validators is chosen to sign the new block.\n", 227 | "- The more coins in the system a validator owns, the more likely he or she is to be chosen. \n", 228 | "- The template becomes a full-fledged block as soon as all of the validators sign it.\n", 229 | "\n", 230 | "### Proof of Burn\n", 231 | "\n", 232 | "- Instead of pouring money into expensive computer equipment, you 'burn' coins by sending them to an address where they are irretrievable. \n", 233 | "- By committing your coins to never-never land, you earn a lifetime privilege to mine on the system based on a random selection process.\n", 234 | "- Over time, your stake in the system decays, so eventually you will want to burn more coins to increase your odds of being selected in the lottery. (This mimics bitcoin's mining process, where you have to continually invest in more modern computing equipment to maintain hashing power.)\n", 235 | "\n", 236 | "My most desired Proof\n", 237 | "- Proof of unique identity" 238 | ] 239 | }, 240 | { 241 | "cell_type": "code", 242 | "execution_count": null, 243 | "metadata": { 244 | "collapsed": true 245 | }, 246 | "outputs": [], 247 | "source": [] 248 | } 249 | ], 250 | "metadata": { 251 | "kernelspec": { 252 | "display_name": "Python 3", 253 | "language": "python", 254 | "name": "python3" 255 | }, 256 | "language_info": { 257 | "codemirror_mode": { 258 | "name": "ipython", 259 | "version": 3 260 | }, 261 | "file_extension": ".py", 262 | "mimetype": "text/x-python", 263 | "name": "python", 264 | "nbconvert_exporter": "python", 265 | "pygments_lexer": "ipython3", 266 | "version": "3.6.0" 267 | } 268 | }, 269 | "nbformat": 4, 270 | "nbformat_minor": 2 271 | } 272 | --------------------------------------------------------------------------------