├── .gitignore ├── COPYING ├── README.md ├── apibb ├── .gitignore ├── README.md ├── apibb-client.py ├── apibb-server.py └── mkdb.py ├── causeway ├── .gitignore ├── README.md ├── causeway-server.py ├── client.py ├── default_settings.py ├── models.py └── schema.sql ├── dns ├── .gitignore ├── README.md ├── TODO ├── dns-client.py ├── dns-server.py ├── dns.schema ├── example-dns-server.conf ├── httputil.py ├── mkdb.sh └── srvdb.py ├── fortune ├── README.md ├── fortune-client.py └── fortune-server.py ├── kvdb ├── .gitignore ├── README.md ├── kvdb-client.py ├── kvdb-server.py └── mkdb.py ├── kvram ├── README.md ├── kvram-client.py └── kvram-server.py ├── signing ├── README.md ├── mkdb.sh ├── signing-server.py └── signing.schema ├── stegano ├── steg-client.py └── steg-server.py └── turk ├── .gitignore ├── README.md ├── answers.json ├── mkdb.sh ├── srvdb.py ├── turk-client.py ├── turk-server.py ├── turk.schema ├── util.py ├── worktemplate.json └── worktmp.py /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | payment.sqlite3 3 | 4 | -------------------------------------------------------------------------------- /COPYING: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Bloq, Inc. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 21 playground 3 | ============================== 4 | 5 | Experiment with, extend and utilize the playground by installing 6 | [21](https://21.co/) and following the underlying instructions for each project. 7 | 8 | Projects 9 | -------- 10 | 11 | * **apibb**: rendezvous API; advertise your node as providing a service 12 | * **causeway**: key/value storage server geared toward small files with ECSDA signature auth 13 | * **dns**: Dynamic DNS management service 14 | * **fortune**: receive a pithy saying (fortune cookie) 15 | * **kvdb**: single node key/value API, backed by reliable storage 16 | * **kvram**: single node key/value API, backed by memory 17 | 18 | 19 | Incomplete works-in-progress 20 | ---------------------------- 21 | * signing: Bitcoin transaction signing server 22 | * stegano: Steganography server 23 | * **turk**: Mechanical Turk (status: 98% there) 24 | 25 | -------------------------------------------------------------------------------- /apibb/.gitignore: -------------------------------------------------------------------------------- 1 | 2 | apibb.db 3 | 4 | -------------------------------------------------------------------------------- /apibb/README.md: -------------------------------------------------------------------------------- 1 | 2 | apibb - API bulletin board - rendezvous service 3 | =============================================== 4 | 5 | Summary 6 | ------- 7 | There exists a top-level namespace of DNS-like names (valid chars: A-Z,-.), 8 | where each name is a container for many node advertisements. 9 | 10 | NAMES 11 | ----- 12 | * List of names costs 1 satoshi. 13 | * Names cost 10 satoshis/hour to exist, after which they are removed. 14 | * Anyone may pay to extend the lifetime of a name, for up to 30 days. 15 | 16 | ADVERTISEMENTS 17 | -------------- 18 | * List of advertisements contained within one name costs 1 satoshi. 19 | * Advertisements cost 2 satoshis/hour to exist, after which they are removed. 20 | 21 | Other notes 22 | ----------- 23 | * A node may advertise their service URI within a single name for X hours 24 | * All names, and all advertisements expire (if not extended w/ payment) 25 | * Example: Nodes seeking storage services download a list of all nodes 26 | advertising the "storage.v1" service. 27 | 28 | Pricing theory 29 | -------------- 30 | (A) Anybody may pay to create or renew a name for X hours. 31 | 32 | (B) Anybody may pay to advertise a name + URI combination for X hours. 33 | 34 | 35 | Example: 36 | 37 | "escrow.v1": [ 38 | [ "http://192.25.0.1:14001", "public key" ], 39 | [ "http://88.92.0.3:14001", "public key 3" ], 40 | ], 41 | "storage.v1": [ 42 | [ "http://127.0.0.1:10101", "public key" ], 43 | [ "http://127.0.0.2:10101", "public key 2" ], 44 | [ "http://127.0.0.3:10101", "public key 3" ], 45 | ], 46 | 47 | 48 | Future Directions 49 | ----------------- 50 | * Check public key before permitting advertisement to be extended 51 | * If advertisements within a container exceeds 1,000, enable competitive 52 | bidding to remain within the container. 53 | 54 | 55 | 56 | First time setup 57 | ---------------- 58 | $ python3 mkdb.py 59 | 60 | This creates an empty apibb.db file used for backing storage. 61 | 62 | 63 | Running the server 64 | ------------------ 65 | $ python3 apibb-server.py 66 | 67 | 68 | API; 69 | 70 | * name-list = names() 71 | * name.renew(name, delta-hours) 72 | * ad-list = ads(name) 73 | * advertise(name, uri, pubkey, delta-hours) 74 | 75 | 76 | 77 | 1. Get list of names 78 | -------------------- 79 | 80 | HTTP URI: /apibb/1/names 81 | 82 | Params: 83 | 84 | none 85 | 86 | Result: 87 | 88 | JSON list of objects containing: name, creation time, expiration time 89 | 90 | Pricing: 91 | 92 | 1 satoshi 93 | 94 | 95 | 2. Create a name / renew name 96 | ----------------------------- 97 | 98 | HTTP URI: /apibb/1/namerenew 99 | 100 | Params: 101 | 102 | name Name string 103 | hours Number of hours until expiration 104 | (or if renewing, number of hours to add to expiration) 105 | 106 | Result if successful: 107 | 108 | Binary string, "OK" 109 | 110 | Pricing: 111 | 112 | 10 satoshis per hour 113 | 114 | 115 | 3. Show all nodes advertising a service 116 | --------------------------------------- 117 | 118 | HTTP URI: /apibb/1/ads 119 | 120 | Params: 121 | 122 | name Name string 123 | 124 | Result if successful: 125 | 126 | JSON list of objects, each obj describes a single node 127 | advertising the "name" service. 128 | 129 | Pricing: 130 | 131 | 1 satoshi 132 | 133 | 134 | 4. Advertise a service 135 | ---------------------- 136 | 137 | HTTP URI: /apibb/1/advertise 138 | 139 | Params: 140 | 141 | name Name string 142 | uri URI to advertise 143 | pubkey Public key associated with URI 144 | hours Number of hours until expiration 145 | (or if renewing, number of hours to add to expiration) 146 | 147 | Result if successful: 148 | 149 | Binary string, "OK" 150 | 151 | Pricing: 152 | 153 | 2 satoshis per hour 154 | 155 | -------------------------------------------------------------------------------- /apibb/apibb-client.py: -------------------------------------------------------------------------------- 1 | 2 | # 3 | # Command line usage: 4 | # $ python3 apibb-client.py 5 | # $ python3 apibb-client.py namerenew NAME EXPIRE-HOURS 6 | # $ python3 apibb-client.py get.ads NAME 7 | # $ python3 apibb-client.py post.ad NAME URI PUBKEY EXPIRE-HOURS 8 | # 9 | 10 | import json 11 | import os 12 | import sys 13 | import click 14 | 15 | # import from the 21 Developer Library 16 | from two1.commands.config import Config 17 | from two1.lib.wallet import Wallet 18 | from two1.lib.bitrequests import BitTransferRequests 19 | 20 | # set up bitrequest client for BitTransfer requests 21 | wallet = Wallet() 22 | username = Config().username 23 | requests = BitTransferRequests(wallet, username) 24 | 25 | APIBBCLI_VERSION = '0.1' 26 | DEFAULT_ENDPOINT = 'http://localhost:12002/' 27 | 28 | @click.group() 29 | @click.option('--endpoint', '-e', 30 | default=DEFAULT_ENDPOINT, 31 | metavar='STRING', 32 | show_default=True, 33 | help='API endpoint URI') 34 | @click.option('--debug', '-d', 35 | is_flag=True, 36 | help='Turns on debugging messages.') 37 | @click.version_option(APIBBCLI_VERSION) 38 | @click.pass_context 39 | def main(ctx, endpoint, debug): 40 | """ Command-line Interface for the API bulletin board service 41 | """ 42 | 43 | if ctx.obj is None: 44 | ctx.obj = {} 45 | 46 | ctx.obj['endpoint'] = endpoint 47 | 48 | @click.command(name='info') 49 | @click.pass_context 50 | def cmd_info(ctx): 51 | sel_url = ctx.obj['endpoint'] 52 | answer = requests.get(url=sel_url.format()) 53 | print(answer.text) 54 | 55 | @click.command(name='names') 56 | @click.pass_context 57 | def cmd_get_names(ctx): 58 | sel_url = ctx.obj['endpoint'] + 'apibb/1/names' 59 | answer = requests.get(url=sel_url.format()) 60 | print(answer.text) 61 | 62 | @click.command(name='namerenew') 63 | @click.argument('name') 64 | @click.argument('hours') 65 | @click.pass_context 66 | def cmd_name_renew(ctx, name, hours): 67 | sel_url = ctx.obj['endpoint'] + 'apibb/1/namerenew?name={0}&hours={1}' 68 | answer = requests.get(url=sel_url.format(name, hours)) 69 | print(answer.text) 70 | 71 | @click.command(name='ads') 72 | @click.argument('name') 73 | @click.pass_context 74 | def cmd_get_ads(ctx, name): 75 | sel_url = ctx.obj['endpoint'] + 'apibb/1/ads?name={0}' 76 | answer = requests.get(url=sel_url.format(name)) 77 | print(answer.text) 78 | 79 | @click.command(name='post.ad') 80 | @click.argument('name') 81 | @click.argument('uri') 82 | @click.argument('pubkey') 83 | @click.argument('hours') 84 | @click.pass_context 85 | def cmd_advertise(ctx, name, uri, pubkey, hours): 86 | sel_url = ctx.obj['endpoint'] + 'apibb/1/advertise?name={0}&uri={1}&pubkey={2}&hours={3}' 87 | answer = requests.get(url=sel_url.format(name, uri, pubkey, hours)) 88 | print(answer.text) 89 | 90 | main.add_command(cmd_info) 91 | main.add_command(cmd_name_renew) 92 | main.add_command(cmd_get_names) 93 | main.add_command(cmd_get_ads) 94 | main.add_command(cmd_advertise) 95 | 96 | if __name__ == "__main__": 97 | main() 98 | 99 | -------------------------------------------------------------------------------- /apibb/apibb-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | import json 4 | import random 5 | import apsw 6 | import time 7 | 8 | # import flask web microframework 9 | from flask import Flask 10 | from flask import request 11 | 12 | # import from the 21 Developer Library 13 | from two1.lib.wallet import Wallet 14 | from two1.lib.bitserv.flask import Payment 15 | 16 | connection = apsw.Connection("apibb.db") 17 | 18 | name_re = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9-\.]*$") 19 | 20 | app = Flask(__name__) 21 | wallet = Wallet() 22 | payment = Payment(app, wallet) 23 | 24 | def expire_ads(): 25 | cursor = connection.cursor() 26 | cursor.execute("DELETE FROM ads WHERE expires < datetime('now')") 27 | 28 | def expire_names(): 29 | cursor = connection.cursor() 30 | cursor.execute("DELETE FROM names WHERE expires < datetime('now')") 31 | 32 | @app.route('/apibb/1/names') 33 | @payment.required(1) 34 | def get_names(): 35 | cursor = connection.cursor() 36 | rv = [] 37 | for name,created,expires in cursor.execute("SELECT name,created,expires FROM names ORDER BY name"): 38 | obj = { 39 | "name": name, 40 | "created": created, 41 | "expires": expires 42 | } 43 | rv.append(obj) 44 | 45 | return json.dumps(rv) 46 | 47 | def valid_renewal(request): 48 | name = request.args.get('name') 49 | hours = request.args.get('hours') 50 | 51 | if (name_re.match(name) is None or 52 | int(hours) < 1 or 53 | int(hours) > (24 * 30)): 54 | return False 55 | 56 | return True 57 | 58 | def get_renew_price_from_req(request): 59 | if not valid_renewal(request): 60 | return "invalid advertisement" 61 | 62 | hours = int(request.args.get('hours')) 63 | 64 | price = hours * 10 # 10 satoshis per hour 65 | 66 | if price < 10: 67 | price = 10 68 | return price 69 | 70 | @app.route('/apibb/1/namerenew') 71 | @payment.required(get_renew_price_from_req) 72 | def name_renew(): 73 | if not valid_renewal(request): 74 | return "invalid renewal" 75 | 76 | expire_names() 77 | 78 | name = request.args.get('name') 79 | hours = int(request.args.get('hours')) 80 | 81 | cursor = connection.cursor() 82 | expires = 0 83 | for v in cursor.execute("SELECT expires FROM names WHERE name = ?", (name,)): 84 | expires = v[0] 85 | 86 | print("EXPIRES " + str(expires)) 87 | 88 | if expires == 0: 89 | cursor.execute("INSERT INTO names VALUES(?, datetime('now'), datetime('now', '+" + str(hours) + " hours'))", (name,)) 90 | else: 91 | cursor.execute("UPDATE names SET expires = datetime(?, '+" + str(hours) + " hours') WHERE name = ?", (expires, name)) 92 | 93 | return "OK" 94 | 95 | def valid_advertisement(cursor, request): 96 | name = request.args.get('name') 97 | uri = request.args.get('uri') 98 | pubkey = request.args.get('pubkey') 99 | hours = request.args.get('hours') 100 | 101 | if (name_re.match(name) is None or 102 | len(uri) < 1 or 103 | len(uri) > 512 or 104 | len(pubkey) < 32 or 105 | len(pubkey) > 512 or 106 | int(hours) < 1 or 107 | int(hours) > (24 * 30)): 108 | return False 109 | 110 | expires = None 111 | for v in cursor.execute("SELECT strftime('%s', expires) FROM names WHERE name = ? AND expires > datetime('now')", (name,)): 112 | expires = v 113 | 114 | if expires is None: 115 | return False 116 | 117 | # curtime = int(time.time()) 118 | # curtime_deltap = curtime + (int(hours) * 60 * 60) 119 | # if curtime_deltap > expires: 120 | # return False 121 | 122 | return True 123 | 124 | def get_advertise_price_from_req(request): 125 | cursor = connection.cursor() 126 | if not valid_advertisement(cursor, request): 127 | return "invalid advertisement" 128 | 129 | hours = int(request.args.get('hours')) 130 | 131 | price = hours * 2 # 2 satoshis per hour 132 | 133 | if price < 2: 134 | price = 2 135 | return price 136 | 137 | @app.route('/apibb/1/advertise') 138 | @payment.required(get_advertise_price_from_req) 139 | def advertise(): 140 | cursor = connection.cursor() 141 | if not valid_advertisement(cursor, request): 142 | return "invalid advertisement" 143 | 144 | name = request.args.get('name') 145 | uri = request.args.get('uri') 146 | pubkey = request.args.get('pubkey') 147 | hours = request.args.get('hours') 148 | 149 | cursor.execute("INSERT INTO ads VALUES(?, ?, ?, datetime('now'), datetime('now','+" + str(hours) + " hours'))", (name, uri, pubkey)) 150 | 151 | return "OK" 152 | 153 | @app.route('/apibb/1/ads') 154 | @payment.required(1) 155 | def get_advertisements(): 156 | name = request.args.get('name') 157 | 158 | rv = [] 159 | cursor = connection.cursor() 160 | for uri,pk,created,expires in cursor.execute("SELECT uri,pubkey,created,expires FROM ads WHERE name = ? AND expires > datetime('now')", (name,)): 161 | obj = { 162 | "uri": uri, 163 | "pubkey": pk, 164 | "created": created, 165 | "expires": expires 166 | } 167 | rv.append(obj) 168 | 169 | return json.dumps(rv) 170 | 171 | @app.route('/') 172 | def get_info(): 173 | # API endpoint metadata - export list of services 174 | info_obj = {[ 175 | "name": "apibb/1", # service 'apibb', version '1' 176 | "pricing-type": "per-rpc", # indicates layout of 'pricing' 177 | "pricing": [ 178 | { 179 | "rpc": "names", 180 | "per-req": 1, # 1 satoshi per request 181 | }, 182 | { 183 | "rpc": "namerenew", 184 | "per-hour": 10, # 10 satoshis per hour 185 | "minimum": 10, # 10 satoshi minimum 186 | }, 187 | { 188 | "rpc": "ads", 189 | "per-req": 1, # 1 satoshi per request 190 | }, 191 | { 192 | "rpc": "advertise", 193 | "per-hour": 2, # 2 satoshis per hour 194 | "minimum": 2, # 2 satoshi minimum 195 | }, 196 | ] 197 | ]} 198 | 199 | body = json.dumps(info_obj, indent=2) 200 | return (body, 200, { 201 | 'Content-length': len(body), 202 | 'Content-type': 'application/json', 203 | }) 204 | 205 | if __name__ == '__main__': 206 | app.run(host='0.0.0.0', port=12002) 207 | 208 | -------------------------------------------------------------------------------- /apibb/mkdb.py: -------------------------------------------------------------------------------- 1 | 2 | import apsw 3 | 4 | 5 | connection = apsw.Connection("apibb.db") 6 | cursor = connection.cursor() 7 | cursor.execute("CREATE TABLE names(name TEXT PRIMARY KEY, created INTEGER, expires INTEGER)") 8 | cursor.execute("CREATE TABLE ads(name TEXT, uri TEXT, pubkey TEXT, created INTEGER, expires INTEGER)") 9 | 10 | -------------------------------------------------------------------------------- /causeway/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache* 2 | *.sqlite3 3 | settings.py 4 | -------------------------------------------------------------------------------- /causeway/README.md: -------------------------------------------------------------------------------- 1 | # causeway 2 | 3 | A storage service geared toward small files with ECDSA signature auth that works 4 | with 21 5 | 6 | This project is for a server that will store and return data for a certain 7 | amount of time and accept updates if they are signed by a user's payment 8 | address. 9 | 10 | ## REST API 11 | 12 | * All requests via HTTP GET except where noted. 13 | * Data returned as JSON, formatted with indent=4 for now. 14 | 15 | ### /get 16 | Parameters 17 | key - used to retrieve value 18 | 19 | Returns 20 | key - the key that was requested 21 | value - the value stored for the key 22 | 23 | Note: Charges bandwidth against sale record associated with key/value. 24 | 25 | ### /price 26 | Parameters 27 | None 28 | 29 | Returns 30 | price - satoshis for 1 MB storage + 50 MB transfer 31 | 32 | ### /buy 33 | Parameters 34 | contact - email address to notify on expiration 35 | address - owner of new hosting bucket 36 | 37 | Returns 38 | result - success or error 39 | buckets - listing free space, reamining bandwidth, and expiration 40 | 41 | 42 | ### /put (POST) 43 | Parameters 44 | key - string 45 | value - string 46 | address - account to charge for this data 47 | nonce - latest unused 32-byte string retrieved via /nonce 48 | signature - signature over concat(key + value + address + nonce) by 49 | private key for address 50 | 51 | Returns 52 | status - "success" or "error: " + error reason 53 | 54 | ### /delete (POST) 55 | Parameters 56 | key - string 57 | address - account that owns this key 58 | nonce = latest unused 32-byte string retrieved via /nonce 59 | signature - signature over concat(key + address + nonce) by 60 | private key for address 61 | 62 | ### /nonce 63 | Parameters 64 | address - manually entered account requesting a nonce, users will need to 65 | pay to register in order to be eligible for nonces 66 | 67 | Returns 68 | nonce - random 32-byte string 69 | 70 | Note: nonce will later be stored until used or next nonce generated for address 71 | 72 | ### /help 73 | Parameters 74 | None 75 | 76 | Returns 77 | List of available endpoints 78 | 79 | ### /status 80 | Parameters 81 | None 82 | 83 | Returns 84 | uptime - time in seconds that the service has been running 85 | stored - bytes stored 86 | free - bytes free 87 | price - satoshis for 1 MB storage + 50 MB transfer 88 | 89 | 90 | ## Installation 91 | 92 | ### raspbian 93 | 94 | First choose where you will host your database, this database will host 95 | operational as well as customer-uploaded data. 96 | 97 | sudo apt-get install python3-flask-sqlalchemy sqlite3 98 | sqlite3 /path/to/db/causeway.db < schema.sql 99 | 100 | Then you'll need to copy default\_settings.py to settings.py and change DATABASE 101 | to the full path where you created the database. 102 | 103 | *** 104 | ** Roadmap ** 105 | 106 | * Hosting is purchased in buckets which expire after one month. 107 | * A bucket holds 1 MB of data and comes with 50 MB of transfer. 108 | * If a bucket expires, key/values are redistributed to an owner's newer buckets if possible, 109 | otherwise, the owner is notified via email that expiration is affecting hosting. 110 | * Data is kept if bandwidth is exceeded just no longer served until more is purchased. 111 | 112 | ### /address 113 | Parameters 114 | contact - email or Bitmessage address to contact on expiration 115 | address - account this will be used to fund 116 | signature - signature for concatenation of contact and address by 117 | private key for address 118 | 119 | Returns 120 | address - a dummy string, later an address suitable for funding an account 121 | 122 | ### /balance 123 | Parameters 124 | address - account on which to report balance 125 | nonce - latest unused 32-byte string retrieved via /nonce 126 | signature - signature over concat(address and last nonce received via /nonce call) 127 | 128 | Returns 129 | balance - satoshis worth of value left on account 130 | 131 | -------------------------------------------------------------------------------- /causeway/causeway-server.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | ''' 3 | Causeway Server - key/value storage server geared toward small files with ECSDA signature auth 4 | 5 | Usage: 6 | python3 server.py 7 | ''' 8 | import os, json, random, time, string 9 | from settings import DATABASE, PRICE, DATA_DIR, SERVER_PORT 10 | 11 | from flask import Flask 12 | from flask import request 13 | from flask import abort, url_for 14 | from flask.ext.sqlalchemy import SQLAlchemy 15 | 16 | from two1.lib.wallet import Wallet 17 | from two1.lib.bitserv.flask import Payment 18 | 19 | from models import * 20 | 21 | app = Flask(__name__) 22 | app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + DATABASE 23 | db = SQLAlchemy(app) 24 | wallet = Wallet() 25 | payment = Payment(app, wallet) 26 | 27 | # start time 28 | start_time = time.time() 29 | stored = 0 30 | 31 | @app.route('/') 32 | @app.route('/help') 33 | def home(): 34 | '''Return service, pricing and endpoint information''' 35 | home_obj = [{"name": "causeway/1", # service 'causeway', version '1' 36 | "pricing-type": "per-mb", # pricing is listed per 1000000 bytes 37 | "pricing" : [{"rpc": "buy", 38 | "per-req": 0, 39 | "per-unit": PRICE, 40 | "description": "1 MB hosting, 50 MB bandwidth, 1 year expiration" 41 | }, 42 | {"rpc": "get", 43 | "per-req": 0, 44 | "per-mb": 0 45 | }, 46 | {"rpc": "put", 47 | "per-req": 0, 48 | "per-mb": 0 49 | }, 50 | 51 | # default 52 | {"rpc": True, # True indicates default 53 | "per-req": 0, 54 | "per-mb": 0 55 | }], 56 | "description": "This Causeway server provides microhosting services. Download the "\ 57 | "client and server at https://github.com/jgarzik/playground21/archive/master.zip" 58 | } 59 | ] 60 | 61 | body = json.dumps(home_obj, indent=2) 62 | 63 | return (body, 200, {'Content-length': len(body), 64 | 'Content-type': 'application/json', 65 | } 66 | ) 67 | 68 | @app.route('/status') 69 | def status(): 70 | '''Return general info about server instance. ''' 71 | uptime = str(int(time.time() - start_time)) 72 | st = os.statvfs(DATA_DIR) 73 | free = st.f_bavail * st.f_frsize 74 | body = json.dumps({'uptime': uptime, 75 | 'stored': str(stored), 76 | 'free': str(free), 77 | 'price': str(PRICE) 78 | }, indent=2 79 | ) 80 | return (body, 200, {'Content-length': len(body), 81 | 'Content-type': 'application/json', 82 | } 83 | ) 84 | 85 | @app.route('/price') 86 | def price(): 87 | '''Return price for 1MB storage with bundled 50MB transfer.''' 88 | body = json.dumps({'price': PRICE}) 89 | return (body, 200, {'Content-length': len(body), 90 | 'Content-type': 'application/json', 91 | } 92 | ) 93 | 94 | @app.route('/buy') 95 | @payment.required(PRICE) 96 | def buy_hosting(): 97 | '''Registers one hosting bucket to account on paid request.''' 98 | # extract account address from client request 99 | owner = request.args.get('address') 100 | contact = request.args.get('contact') 101 | 102 | # check if user exists 103 | o = db.session.query(Owner).get(owner) 104 | if o is None: 105 | # create them 106 | o = Owner(owner) 107 | db.session.add(o) 108 | db.session.commit() 109 | 110 | # owner should now exist, create sale record for address 111 | s = Sale(owner, contact, 1, 30, PRICE) 112 | db.session.add(s) 113 | db.session.commit() 114 | 115 | body = json.dumps({'result': 'success', 116 | 'buckets': s.get_buckets()}, indent=2) 117 | return (body, 200, {'Content-length': len(body), 118 | 'Content-type': 'application/json', 119 | } 120 | ) 121 | 122 | @app.route('/put', methods=['POST']) 123 | def put(): 124 | '''Store a key-value pair.''' 125 | # get size of file sent 126 | # Validate JSON body w/ API params 127 | try: 128 | body = request.data.decode('utf-8') 129 | in_obj = json.loads(body) 130 | except: 131 | return ("JSON Decode failed", 400, {'Content-Type':'text/plain'}) 132 | 133 | k = in_obj['key'] 134 | v = in_obj['value'] 135 | o = in_obj['address'] 136 | n = in_obj['nonce'] 137 | s = in_obj['signature'] 138 | 139 | # check signature 140 | owner = Owner.query.filter_by(address=o).first() 141 | if owner.nonce not in n or wallet.verify_bitcoin_message(k + v + o + n, s, o): 142 | body = json.dumps({'error': 'Incorrect signature.'}) 143 | code = 401 144 | else: 145 | size = len(k) + len(v) 146 | 147 | # check if owner has enough free storage 148 | # get free space from each of owner's buckets 149 | result = db.engine.execute('select * from sale where julianday("now") - \ 150 | julianday(sale.created) < sale.term order by sale.created desc') 151 | # choose newest bucket that has enough space 152 | sale_id = None 153 | for row in result: 154 | if (row[7] + size) < (1024 * 1024): 155 | sale_id = row[0] 156 | 157 | if sale_id is None: # we couldn't find enough free space 158 | body = json.dumps({'error': 'Insufficient storage space.'}) 159 | code = 403 160 | else: 161 | # check if key already exists and is owned by the same owner 162 | kv = db.session.query(Kv).filter_by(key=k).filter_by(owner=o).first() 163 | 164 | if kv is None: 165 | kv = Kv(k, v, o, sale_id) 166 | db.session.add(kv) 167 | db.session.commit() 168 | else: 169 | kv.value = v 170 | db.session.commit() 171 | 172 | s = db.session.query(Sale).get(sale_id) 173 | s.bytes_used = s.bytes_used + size 174 | db.session.commit() 175 | body = json.dumps({'result': 'success'}) 176 | code = 201 177 | 178 | return (body, code, {'Content-length': len(body), 179 | 'Content-type': 'application/json', 180 | } 181 | ) 182 | 183 | @app.route('/delete', methods=['POST']) 184 | def delete(): 185 | '''Delete a key-value pair.''' 186 | # Validate JSON body w/ API params 187 | try: 188 | body = request.data.decode('utf-8') 189 | in_obj = json.loads(body) 190 | except: 191 | return ("JSON Decode failed", 400, {'Content-Type':'text/plain'}) 192 | 193 | k = in_obj['key'] 194 | o = in_obj['address'] 195 | n = in_obj['nonce'] 196 | s = in_obj['signature'] 197 | 198 | # check signature 199 | owner = Owner.query.filter_by(address=o).first() 200 | if owner.nonce not in n or wallet.verify_bitcoin_message(k + o + n, s, o): 201 | body = json.dumps({'error': 'Incorrect signature.'}) 202 | code = 401 203 | else: 204 | # check if key already exists and is owned by the same owner 205 | kv = db.session.query(Kv).filter_by(key=k).filter_by(owner=o).first() 206 | if kv is None: 207 | body = json.dumps({'error': 'Key not found or not owned by caller.'}) 208 | code = 404 209 | else: 210 | # free up storage quota and remove kv 211 | size = len(kv.value) 212 | sale_id = kv.sale 213 | s = db.session.query(Sale).get(sale_id) 214 | s.bytes_used = s.bytes_used - size 215 | db.session.delete(kv) 216 | db.session.commit() 217 | body = json.dumps({'result': 'success'}) 218 | code = 200 219 | 220 | return (body, code, {'Content-length': len(body), 221 | 'Content-type': 'application/json', 222 | } 223 | ) 224 | 225 | @app.route('/get') 226 | def get(): 227 | '''Get a key-value pair.''' 228 | 229 | key = request.args.get('key') 230 | 231 | kv = Kv.query.filter_by(key=key).first() 232 | 233 | if kv is None: 234 | body = json.dumps({'error': 'Key not found.'}) 235 | code = 404 236 | else: 237 | body = json.dumps({'key': key, 'value': kv.value}) 238 | code = 200 239 | 240 | # calculate size and check against quota on kv's sale record 241 | return (body, code, {'Content-length': len(body), 242 | 'Content-type': 'application/json', 243 | } 244 | ) 245 | 246 | @app.route('/nonce') 247 | def nonce(): 248 | '''Return 32-byte nonce for generating non-reusable signatures..''' 249 | # check if user exists 250 | o = db.session.query(Owner).get(request.args.get('address')) 251 | if o is None: 252 | return abort(500) 253 | 254 | # if nonce is set for user return it, else make a new one 255 | if o.nonce and len(o.nonce) == 32: 256 | body = json.dumps({'nonce': o.nonce}) 257 | # if not, create one and store it 258 | else: 259 | print("storing") 260 | n = ''.join(random.SystemRandom().choice(string.hexdigits) for _ in range(32)) 261 | o.nonce = n.lower() 262 | db.session.commit() 263 | body = json.dumps({'nonce': o.nonce}) 264 | 265 | return (body, 200, {'Content-length': len(body), 266 | 'Content-type': 'application/json', 267 | } 268 | ) 269 | 270 | @app.route('/address') 271 | def get_deposit_address(): 272 | '''Return new or unused deposit address for on-chain funding.''' 273 | # check if user exists 274 | o = db.session.query(Owner).get(request.args.get('address')) 275 | if o is None: 276 | return abort(500) 277 | 278 | address = request.args.get('address') 279 | message = request.args.get('contact') + "," + address 280 | signature = request.args.get('signature') 281 | 282 | print(len(signature)) 283 | if len(signature) == 88 and wallet.verify_bitcoin_message(message, signature, address): 284 | body = json.dumps({'address': 'hereyago'}) 285 | else: 286 | body = json.dumps({'error': 'Invalid signature'}) 287 | 288 | return (body, 200, {'Content-length': len(body), 289 | 'Content-type': 'application/json', 290 | } 291 | ) 292 | 293 | def has_no_empty_params(rule): 294 | '''Testing rules to identify routes.''' 295 | defaults = rule.defaults if rule.defaults is not None else () 296 | arguments = rule.arguments if rule.arguments is not None else () 297 | return len(defaults) >= len(arguments) 298 | 299 | @app.route('/info') 300 | def info(): 301 | '''Returns list of defined routes.''' 302 | links = [] 303 | for rule in app.url_map.iter_rules(): 304 | # Filter out rules we can't navigate to in a browser 305 | # and rules that require parameters 306 | if "GET" in rule.methods and has_no_empty_params(rule): 307 | url = url_for(rule.endpoint, **(rule.defaults or {})) 308 | links.append(url) 309 | 310 | return json.dumps(links, indent=2) 311 | 312 | if __name__ == '__main__': 313 | app.debug = True 314 | app.run(host='0.0.0.0', port=SERVER_PORT) 315 | -------------------------------------------------------------------------------- /causeway/client.py: -------------------------------------------------------------------------------- 1 | #.!/usr/bin/env python3 2 | 3 | import sys, json, os, argparse 4 | 5 | # import from the 21 Developer Library 6 | from two1.commands.config import Config 7 | from two1.lib.wallet import Wallet 8 | from two1.lib.bitrequests import BitTransferRequests 9 | 10 | # set up bitrequest client for BitTransfer requests 11 | wallet = Wallet() 12 | username = Config().username 13 | requests = BitTransferRequests(wallet, username) 14 | 15 | # server address 16 | def buy(args): 17 | primary_address = wallet.get_payout_address() 18 | sel_url = "{0}buy?address={1}&contact={2}" 19 | answer = requests.get(url=sel_url.format(args.url, primary_address, args.contact)) 20 | if answer.status_code != 200: 21 | print("Could not make offchain payment. Please check that you have sufficient balance.") 22 | else: 23 | print(answer.text) 24 | 25 | def put(args): 26 | primary_address = wallet.get_payout_address() 27 | message = args.key + args.value + primary_address + args.nonce 28 | signature = wallet.sign_message(message) 29 | 30 | data = {"key": args.key, 31 | "value": args.value, 32 | "nonce": args.nonce, 33 | "signature": signature, 34 | "address": primary_address} 35 | 36 | sel_url = "{0}put" 37 | body = json.dumps(data) 38 | headers = {'Content-Type': 'application/json'} 39 | answer = requests.post(url=sel_url.format(args.url), headers=headers, data=body) 40 | print(answer.text) 41 | 42 | def delete(args): 43 | primary_address = wallet.get_payout_address() 44 | message = args.key + primary_address + args.nonce 45 | signature = wallet.sign_message(message) 46 | 47 | data = {"key": args.key, 48 | "nonce": args.nonce, 49 | "signature": signature, 50 | "address": primary_address} 51 | sel_url = "{0}delete" 52 | body = json.dumps(data) 53 | headers = {'Content-Type': 'application/json'} 54 | answer = requests.post(url=sel_url.format(args.url), headers=headers, data=body) 55 | print(answer.text) 56 | 57 | def get(args): 58 | sel_url = "{0}get?key={1}" 59 | answer = requests.get(url=sel_url.format(args.url, args.key)) 60 | print(answer.text) 61 | 62 | def buy_file(server_url = 'http://localhost:5000/'): 63 | 64 | # get the file listing from the server 65 | response = requests.get(url=server_url+'files') 66 | file_list = json.loads(response.text) 67 | 68 | # print the file list to the console 69 | for file in range(len(file_list)): 70 | print("{}. {}\t{}".format(file+1, file_list[str(file+1)][0], file_list[str(file+1)][1])) 71 | 72 | try: 73 | # prompt the user to input the index number of the file to be purchased 74 | sel = input("Please enter the index of the file that you would like to purchase:") 75 | 76 | # check if the input index is valid key in file_list dict 77 | if sel in file_list: 78 | print('You selected {} in our database'.format(file_list[sel][0])) 79 | 80 | #create a 402 request with the server payout address 81 | sel_url = server_url+'buy?selection={0}&payout_address={1}' 82 | answer = requests.get(url=sel_url.format(int(sel), wallet.get_payout_address()), stream=True) 83 | if answer.status_code != 200: 84 | print("Could not make an offchain payment. Please check that you have sufficient balance.") 85 | else: 86 | # open a file with the same name as the file being purchased and stream the data into it. 87 | filename = file_list[str(sel)][0] 88 | with open(filename,'wb') as fd: 89 | for chunk in answer.iter_content(4096): 90 | fd.write(chunk) 91 | fd.close() 92 | print('Congratulations, you just purchased a file for bitcoin!') 93 | else: 94 | print("That is an invalid selection.") 95 | 96 | except ValueError: 97 | print("That is an invalid input. Only numerical inputs are accepted.") 98 | 99 | def nonce(args): 100 | primary_address = wallet.get_payout_address() 101 | sel_url = args.url + 'nonce?address={0}' 102 | answer = requests.get(url=sel_url.format(primary_address)) 103 | print(answer.text) 104 | 105 | def address(args): 106 | primary_address = wallet.get_payout_address() 107 | sel_url = args.url + 'address?contact={0}&address={1}&signature={2}' 108 | answer = requests.get(url=sel_url.format(args.contact, primary_address, args.signature)) 109 | print(answer.text) 110 | 111 | def help(args): 112 | print("Please run with --help") 113 | 114 | if __name__ == '__main__': 115 | parser = argparse.ArgumentParser(description="Interact with Causeway server") 116 | #parser.set_defaults(func=help) 117 | subparsers = parser.add_subparsers(help="Commands") 118 | 119 | parser_buy = subparsers.add_parser('buy', help="Purchase hosting bucket") 120 | parser_buy.add_argument('url', help='Url of the Causeway server with trailing slash.') 121 | #parser_buy.add_argument('address', help='Address used as username for the service.') 122 | parser_buy.add_argument('contact', help='Email address to contact on expiration.') 123 | parser_buy.set_defaults(func=buy) 124 | 125 | parser_put = subparsers.add_parser('put', help="Set or update a value for a key") 126 | parser_put.add_argument('url', help='Url of the Causeway server with trailing slash.') 127 | #parser_put.add_argument('address', help='Address used as username for the service.') 128 | parser_put.add_argument('key', help='Data storage key') 129 | parser_put.add_argument('value', help='Data stored by key') 130 | parser_put.add_argument('nonce', help='Nonce for signature uniqueness.') 131 | parser_put.set_defaults(func=put) 132 | 133 | parser_delete = subparsers.add_parser('delete', help="Delete a key/value pair.") 134 | parser_delete.add_argument('url', help='Url of the Causeway server with trailing slash.') 135 | #parser_delete.add_argument('address', help='Address used as username for the service.') 136 | parser_delete.add_argument('key', help='Data storage key') 137 | parser_delete.add_argument('nonce', help='Nonce for signature uniqueness.') 138 | parser_delete.set_defaults(func=delete) 139 | 140 | parser_get = subparsers.add_parser('get', help="Download the value stored with a key") 141 | parser_get.add_argument('url', help='Url of the Causeway server with trailing slash.') 142 | parser_get.add_argument('key', help='Key to retrieve') 143 | parser_get.set_defaults(func=get) 144 | 145 | parser_nonce = subparsers.add_parser('nonce', help="Get nonce for the address") 146 | parser_nonce.add_argument('url', help='Url of the Causeway server with trailing slash.') 147 | #parser_nonce.add_argument('address', help='Address used as username for the service.') 148 | parser_nonce.set_defaults(func=nonce) 149 | 150 | parser_address = subparsers.add_parser('address', help="Get a deposit address") 151 | parser_address.add_argument('url', help='Url of the Causeway server with trailing slash.') 152 | parser_address.add_argument('contact', help='Email address to contact on expiration.') 153 | parser_address.add_argument('address', help='Address used as username for the service.') 154 | parser_address.add_argument('signature', help='Signature of "contact,address" using address\' privkey') 155 | parser_address.set_defaults(func=address) 156 | 157 | args = parser.parse_args() 158 | args.func(args) 159 | -------------------------------------------------------------------------------- /causeway/default_settings.py: -------------------------------------------------------------------------------- 1 | '''Configuration file - copy to settings.py and fill in your own settings.''' 2 | 3 | SERVER_PORT = 5000 4 | 5 | DATABASE = '/home/twenty/var/sqlite/causeway.db' 6 | 7 | DATA_DIR = '/home/twenty/var/storage' 8 | 9 | # Price in satoshis for 1MB storage and 50MB transfer 10 | PRICE = 1000 11 | -------------------------------------------------------------------------------- /causeway/models.py: -------------------------------------------------------------------------------- 1 | 2 | from settings import * 3 | 4 | from flask import Flask 5 | from flask.ext.sqlalchemy import SQLAlchemy 6 | app = Flask(__name__) 7 | app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + DATABASE 8 | db = SQLAlchemy(app) 9 | 10 | class Owner(db.Model): 11 | __tablename__ = 'owner' 12 | 13 | address = db.Column(db.String(64), primary_key=True) 14 | nonce = db.Column(db.String(32), unique=True) 15 | balance = db.Column(db.Integer) 16 | bad_attempts = db.Column(db.Integer) 17 | 18 | def __init__(self, address, nonce=None, balance=0, bad_attempts=0): 19 | self.address = address 20 | self.nonce = nonce 21 | self.balance = balance 22 | self.bad_attempts = bad_attempts 23 | 24 | def __repr__(self): 25 | return '' % self.address 26 | 27 | class Kv(db.Model): 28 | __tablename__ = 'kv' 29 | 30 | key = db.Column(db.String(64), primary_key=True) 31 | value = db.Column(db.String(8192)) 32 | owner = db.Column(db.String(64)) 33 | sale = db.Column(db.Integer) #aka bucket 34 | 35 | def __init__(self, key, value, owner, sale): 36 | self.key = key 37 | self.value = value 38 | self.owner = owner 39 | self.sale = sale 40 | 41 | def __repr__(self): 42 | return "" % self.key 43 | 44 | class Sale(db.Model): 45 | __tablename__ = 'sale' 46 | 47 | id = db.Column(db.Integer, primary_key=True) 48 | owner = db.Column(db.String(64)) # owner address 49 | contact = db.Column(db.String(255)) # owner's contact email address 50 | created = db.Column(db.DateTime, default=db.func.current_timestamp()) 51 | term = db.Column(db.Integer) # term in days 52 | amount = db.Column(db.Integer) # units purchased 53 | price = db.Column(db.Integer) # satoshis paid per unit 54 | bytes_used = db.Column(db.Integer) 55 | 56 | #s = Sale(owner, contact, 1, 30, PRICE) 57 | def __init__(self, owner, contact, amount, term, price, id=None): 58 | self.owner = owner 59 | self.contact = contact 60 | self.amount = amount 61 | self.term = term 62 | self.price = price 63 | self.bytes_used = 0 64 | self.id = id 65 | 66 | def get_buckets(self): 67 | sales = Sale.query.filter_by(owner=self.owner).all() 68 | result = [] 69 | for s in sales: 70 | result.append({"created":str(s.created), "bytes_free": str(1024*1024 - s.bytes_used)}) 71 | return result 72 | 73 | def __repr__(self): 74 | return '' % self.id 75 | -------------------------------------------------------------------------------- /causeway/schema.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE owner ( 2 | address varchar(64) primary key, 3 | nonce varchar(32), 4 | balance integer, 5 | bad_attempts integer 6 | ); 7 | 8 | CREATE TABLE kv ( 9 | key varchar(64) primary key, 10 | value blob, 11 | owner varchar(64), 12 | sale integer, /* which sale/bucket is this stored under */ 13 | foreign key(owner) references owner(address) 14 | ); 15 | 16 | CREATE TABLE wallet ( 17 | address varchar(64) primary key, 18 | contact varchar(256), 19 | owner varchar(64) 20 | ); 21 | 22 | CREATE TABLE sale ( 23 | id integer primary key, 24 | owner varchar(64), 25 | created text, 26 | amount integer, 27 | term integer, 28 | contact varchar(255), 29 | price integer, 30 | bytes_used integer, 31 | foreign key(owner) references owner(address) 32 | ); 33 | 34 | CREATE TABLE log ( 35 | created text, 36 | ip varchar(45), /* max length of ipv6 address */ 37 | action varchar(10), 38 | bytes integer, 39 | owner varchar(64), 40 | message text, 41 | foreign key(owner) references owner(address) 42 | ); 43 | -------------------------------------------------------------------------------- /dns/.gitignore: -------------------------------------------------------------------------------- 1 | 2 | __pycache__/ 3 | dns.db 4 | dns-server.conf 5 | 6 | nsupdate.log 7 | 8 | *.key 9 | *.private 10 | 11 | -------------------------------------------------------------------------------- /dns/README.md: -------------------------------------------------------------------------------- 1 | 2 | dns - Dynamic DNS management service 3 | ==================================== 4 | 5 | Summary: Dynamic DNS domain management system 6 | 7 | Once connected to your BIND9 DNS servers, this server enables you to 8 | monetize your domain by selling 9 | 10 | YOUR-NAME-HERE.example.com 11 | 12 | Customers buy "my-silly-name.example.com" from this service, 13 | and control all DNS address records under that DNS domain. 14 | 15 | Demonstrates: 16 | 17 | * Pricing in USD dollars (converted to satoshis) 18 | 19 | * Flask error handling, file upload. 20 | 21 | * Public key authentication - Remote client provides public key from a 21 22 | wallet. Server verifies signature before updating records. 23 | 24 | * Permissioned SQL database access. 25 | 26 | * Providing request/response via JSON documents. 27 | 28 | Status: Tested and feature complete client + server. 29 | 30 | First time setup 31 | ---------------- 32 | 33 | $ ./mkdb.sh 34 | 35 | Running the server 36 | ------------------ 37 | 38 | $ python3 dns-server.py 39 | 40 | 41 | 42 | API 43 | === 44 | 45 | 0. Show endpoint API metadata 46 | ----------------------------- 47 | HTTP URI: GET / 48 | 49 | Params: 50 | 51 | None 52 | 53 | Result: 54 | 55 | application/json document with standard API endpoint descriptors. 56 | 57 | Example: 58 | 59 | [ { 60 | "name": "dns/1", 61 | "website": "https://github.com/jgarzik/playground21/tree/master/dns", 62 | "pricing": [ 63 | { 64 | "rpc": "domains", 65 | "per-req": 0 66 | }, 67 | { 68 | "rpc": "host.register", 69 | "per-day": 56 70 | }, 71 | { 72 | "rpc": "simpleRegister", 73 | "per-day": 56 74 | }, 75 | { 76 | "rpc": "records.update", 77 | "per-req": 564 78 | }, 79 | { 80 | "rpc": "host.delete", 81 | "per-req": 0 82 | } 83 | ], 84 | "pricing-type": "per-rpc" 85 | } ] 86 | 87 | 88 | 1. List DNS domains 89 | ------------------- 90 | Show DNS domains, e.g. "example.com", available for use at this service. 91 | 92 | HTTP URI: GET /dns/1/domains 93 | 94 | Params: 95 | 96 | None 97 | 98 | Result: 99 | 100 | application/json document with the following data: 101 | List of domains (string) 102 | (or an HTTP 4xx, 5xx error) 103 | 104 | Pricing: 105 | 106 | Free 107 | 108 | Example: 109 | 110 | [ 111 | "example.com", 112 | "bcapi.xyz" 113 | ] 114 | 115 | 116 | 2. Register host name 117 | --------------------- 118 | HTTP URI: POST /dns/1/host.register 119 | 120 | Params: 121 | 122 | In HTTP body, a application/json document containing the following keys: 123 | 124 | name: name to register. Must be valid DNS name. 125 | domain: domain name under which name will be registered 126 | pkh: (optional) public key hash for permissioned updates 127 | days: (optional) number of days to keep name registered (1-365) 128 | hosts: (optional) list of objects whose keys are: 129 | ttl: DNS TTL, in seconds 130 | rec_type: DNS record type ('A' and 'AAAA' supported) 131 | address: IPv4 or IPv6 address 132 | 133 | Result: 134 | 135 | application/json document with the following data: true 136 | (or an HTTP 4xx, 5xx error) 137 | 138 | Pricing: 139 | 140 | US$0.0002/day 141 | 142 | Example: 143 | 144 | { 145 | "name": "test3", 146 | "domain": "example.com", 147 | "pkh": "1M3iEX7daqd9psQC8PsxN7ZE3GjoAe6k7d", 148 | "days": 1, 149 | "hosts": [ 150 | {"ttl": 30, "address": "127.0.0.1", "rec_type": "A"} 151 | ] 152 | } 153 | 154 | 155 | 3. Update host records 156 | ---------------------- 157 | Replace _all_ DNS records associated a host, with the specified list. An 158 | empty list deletes all records. 159 | 160 | HTTP URI: POST /dns/1/records.update 161 | 162 | Params: 163 | 164 | In HTTP body, a application/json document containing the following keys: 165 | 166 | name: name to register. Must be valid DNS name. 167 | domain: domain name under which name will be registered 168 | hosts: (optional) list of objects whose keys are: 169 | ttl: DNS TTL, in seconds 170 | rec_type: DNS record type ('A' and 'AAAA' supported) 171 | address: IPv4 or IPv6 address 172 | 173 | Header X-Bitcoin-Sig contains signature of encoded json document, signed with key used in host.register. 174 | 175 | Result: 176 | 177 | application/json document with the following data: true 178 | (or an HTTP 4xx, 5xx error) 179 | 180 | Pricing: 181 | 182 | US$0.002 183 | 184 | Example: 185 | 186 | { 187 | "name": "test" 188 | "domain": "example.com", 189 | "hosts": [ 190 | { 191 | "address": "127.0.0.1", 192 | "ttl": 30, 193 | "rec_type": "A" 194 | }, 195 | { 196 | "address": "::1", 197 | "ttl": 60, 198 | "rec_type": "AAAA" 199 | } 200 | ], 201 | } 202 | 203 | 204 | 205 | 4. Delete host 206 | -------------- 207 | Remove _all_ DNS records associated a host, as well as the host itself. 208 | 209 | HTTP URI: POST /dns/1/host.delete 210 | 211 | Params: 212 | 213 | In HTTP body, a application/json document containing the following keys: 214 | 215 | name: name to register. Must be valid DNS name. 216 | domain: domain name under which name will be registered 217 | pkh: public key hash for permissioned updates 218 | 219 | Header X-Bitcoin-Sig contains signature of encoded json document, signed with key used in host.register. 220 | 221 | Result: 222 | 223 | application/json document with the following data: true 224 | (or an HTTP 4xx, 5xx error) 225 | 226 | Pricing: 227 | 228 | Free 229 | 230 | Example: 231 | 232 | { 233 | "domain": "example.com", 234 | "name": "test3", 235 | "pkh": "1M3iEX7daqd9psQC8PsxN7ZE3GjoAe6k7d" 236 | } 237 | 238 | 239 | 5. Register host name (simplified interface) 240 | -------------------------------------------- 241 | HTTP URI: GET /dns/1/simpleRegister 242 | 243 | Params: 244 | 245 | HTTP query string parameters: 246 | 247 | name: name to register. Must be valid DNS name. 248 | domain: domain name under which name will be registered 249 | days: Number of days to register 250 | ip: IPv4 or IPv6 address (e.g. 127.0.0.1) 251 | 252 | Result: 253 | 254 | application/json document with the following data: true 255 | (or an HTTP 4xx, 5xx error) 256 | 257 | Pricing: 258 | 259 | US$0.0002/day 260 | 261 | Example: 262 | 263 | Register **test2.example.com** for **4** days at address **127.0.0.1**. 264 | 265 | GET /dns/1/simpleRegister?name=test2&domain=example.com&days=4&ip=127.0.0.1 266 | 267 | -------------------------------------------------------------------------------- /dns/TODO: -------------------------------------------------------------------------------- 1 | 2 | - add host.renew 3 | - if no pkh, anyone can renew 4 | 5 | - expire records from DNS, when lifetime ends 6 | 7 | -------------------------------------------------------------------------------- /dns/dns-client.py: -------------------------------------------------------------------------------- 1 | 2 | # 3 | # Command line usage: 4 | # $ python3 dns-client.py --help 5 | # 6 | 7 | import json 8 | import os 9 | import sys 10 | import click 11 | import pprint 12 | 13 | # import from the 21 Developer Library 14 | from two1.commands.config import Config 15 | from two1.lib.wallet import Wallet 16 | from two1.lib.bitrequests import BitTransferRequests 17 | 18 | pp = pprint.PrettyPrinter(indent=2) 19 | 20 | # set up bitrequest client for BitTransfer requests 21 | wallet = Wallet() 22 | username = Config().username 23 | requests = BitTransferRequests(wallet, username) 24 | 25 | DNSCLI_VERSION = '0.1' 26 | DEFAULT_ENDPOINT = 'http://localhost:12005/' 27 | 28 | @click.group() 29 | @click.option('--endpoint', '-e', 30 | default=DEFAULT_ENDPOINT, 31 | metavar='STRING', 32 | show_default=True, 33 | help='API endpoint URI') 34 | @click.option('--debug', '-d', 35 | is_flag=True, 36 | help='Turns on debugging messages.') 37 | @click.version_option(DNSCLI_VERSION) 38 | @click.pass_context 39 | def main(ctx, endpoint, debug): 40 | """ Command-line Interface for the DDNS API service 41 | """ 42 | 43 | if ctx.obj is None: 44 | ctx.obj = {} 45 | 46 | ctx.obj['endpoint'] = endpoint 47 | 48 | @click.command(name='info') 49 | @click.pass_context 50 | def cmd_info(ctx): 51 | sel_url = ctx.obj['endpoint'] 52 | answer = requests.get(url=sel_url.format()) 53 | print(answer.text) 54 | 55 | @click.command(name='domains') 56 | @click.pass_context 57 | def cmd_domains(ctx): 58 | sel_url = ctx.obj['endpoint'] + 'dns/1/domains' 59 | answer = requests.get(url=sel_url.format()) 60 | print(answer.text) 61 | 62 | @click.command(name='register') 63 | @click.argument('name') 64 | @click.argument('domain') 65 | @click.argument('days') 66 | @click.argument('recordlist', nargs=-1) 67 | @click.pass_context 68 | def cmd_register(ctx, name, domain, days, recordlist): 69 | 70 | pubkey = wallet.get_message_signing_public_key() 71 | addr = pubkey.address() 72 | print("Registering with key %s" % (addr,)) 73 | 74 | records = [] 75 | for arg in recordlist: 76 | words = arg.split(',') 77 | host_obj = { 78 | 'ttl': int(words[0]), 79 | 'rec_type': words[1], 80 | 'address': words[2], 81 | } 82 | records.append(host_obj) 83 | 84 | req_obj = { 85 | 'name': name, 86 | 'domain': domain, 87 | 'days': int(days), 88 | 'pkh': addr, 89 | 'hosts': records, 90 | } 91 | 92 | sel_url = ctx.obj['endpoint'] + 'dns/1/host.register' 93 | body = json.dumps(req_obj) 94 | headers = {'Content-Type': 'application/json'} 95 | answer = requests.post(url=sel_url.format(), headers=headers, data=body) 96 | print(answer.text) 97 | 98 | @click.command(name='simpleregister') 99 | @click.argument('name') 100 | @click.argument('domain') 101 | @click.argument('days') 102 | @click.argument('ipaddress') 103 | @click.pass_context 104 | def cmd_simpleRegister(ctx, name, domain, days, ipaddress): 105 | sel_url = ctx.obj['endpoint'] + 'dns/1/simpleRegister?name={0}&domain={1}&days={2}&ip={3}' 106 | answer = requests.get(url=sel_url.format(name, domain, days, ipaddress)) 107 | print(answer.text) 108 | 109 | @click.command(name='update') 110 | @click.argument('name') 111 | @click.argument('domain') 112 | @click.argument('pkh') 113 | @click.argument('records', nargs=-1) 114 | @click.pass_context 115 | def cmd_update(ctx, name, domain, pkh, records): 116 | req_obj = { 117 | 'name': name, 118 | 'domain': domain, 119 | 'hosts': [], 120 | } 121 | for record in records: 122 | words = record.split(',') 123 | host_obj = { 124 | 'ttl': int(words[0]), 125 | 'rec_type': words[1], 126 | 'address': words[2], 127 | } 128 | req_obj['hosts'].append(host_obj) 129 | 130 | 131 | body = json.dumps(req_obj) 132 | sig_str = wallet.sign_bitcoin_message(body, pkh) 133 | if not wallet.verify_bitcoin_message(body, sig_str, pkh): 134 | print("Cannot self-verify message") 135 | sys.exit(1) 136 | 137 | sel_url = ctx.obj['endpoint'] + 'dns/1/records.update' 138 | headers = { 139 | 'Content-Type': 'application/json', 140 | 'X-Bitcoin-Sig': sig_str, 141 | } 142 | answer = requests.post(url=sel_url.format(), headers=headers, data=body) 143 | print(answer.text) 144 | 145 | @click.command(name='delete') 146 | @click.argument('name') 147 | @click.argument('domain') 148 | @click.argument('pkh') 149 | @click.pass_context 150 | def cmd_delete(ctx, name, domain, pkh): 151 | req_obj = { 152 | 'name': name, 153 | 'domain': domain, 154 | 'pkh': pkh 155 | } 156 | 157 | body = json.dumps(req_obj) 158 | sig_str = wallet.sign_bitcoin_message(body, pkh) 159 | if not wallet.verify_bitcoin_message(body, sig_str, pkh): 160 | print("Cannot self-verify message") 161 | sys.exit(1) 162 | 163 | sel_url = ctx.obj['endpoint'] + 'dns/1/host.delete' 164 | headers = { 165 | 'Content-Type': 'application/json', 166 | 'X-Bitcoin-Sig': sig_str, 167 | } 168 | answer = requests.post(url=sel_url.format(), headers=headers, data=body) 169 | print(answer.text) 170 | 171 | main.add_command(cmd_info) 172 | main.add_command(cmd_domains) 173 | main.add_command(cmd_register) 174 | main.add_command(cmd_simpleRegister) 175 | main.add_command(cmd_update) 176 | main.add_command(cmd_delete) 177 | 178 | if __name__ == "__main__": 179 | main() 180 | 181 | -------------------------------------------------------------------------------- /dns/dns-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import binascii 4 | import srvdb 5 | import re 6 | import base58 7 | import ipaddress 8 | import subprocess 9 | import time 10 | import pprint 11 | from httputil import httpjson, http400, http404, http500 12 | 13 | # import flask web microframework 14 | from flask import Flask 15 | from flask import request 16 | from flask import abort 17 | 18 | # import from the 21 Developer Library 19 | from two1.lib.bitcoin.txn import Transaction 20 | from two1.lib.bitcoin.crypto import PublicKey 21 | from two1.lib.wallet import Wallet, exceptions 22 | from two1.lib.bitserv.flask import Payment 23 | 24 | server_config = json.load(open("dns-server.conf")) 25 | 26 | USCENT=2824 27 | DNS_SERVER1=server_config["DNS_SERVER1"] 28 | NSUPDATE_KEYFILE=server_config["NSUPDATE_KEYFILE"] 29 | NSUPDATE_LOG=server_config["NSUPDATE_LOG"] 30 | nsupdate_logging=server_config["NSUPDATE_LOGGING"] 31 | 32 | pp = pprint.PrettyPrinter(indent=2) 33 | 34 | db = srvdb.SrvDb(server_config["DB_PATHNAME"]) 35 | 36 | app = Flask(__name__) 37 | wallet = Wallet() 38 | payment = Payment(app, wallet) 39 | 40 | name_re = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9-]*$") 41 | name_ns_re = re.compile(r"^ns[0-9]+") 42 | 43 | def valid_name(name): 44 | if not name or len(name) < 1 or len(name) > 64: 45 | return False 46 | if not name_re.match(name): 47 | return False 48 | return True 49 | 50 | def reserved_name(name): 51 | if name_ns_re.match(name): 52 | return True 53 | return False 54 | 55 | def nsupdate_cmd(name, domain, host_records): 56 | pathname = "%s.%s." % (name, domain) 57 | 58 | cmd = "server %s\n" % (DNS_SERVER1,) 59 | cmd += "zone %s.\n" % (domain,) 60 | cmd += "update delete %s\n" % (pathname,) 61 | 62 | for rec in host_records: 63 | cmd += "update add %s %d %s %s\n" % (pathname, rec[4], rec[2], rec[3]) 64 | 65 | cmd += "show\n" 66 | cmd += "send\n" 67 | 68 | return cmd.encode('utf-8') 69 | 70 | def nsupdate_exec(name, domain, host_records): 71 | nsupdate_input = nsupdate_cmd(name, domain, host_records) 72 | args = [ 73 | "/usr/bin/nsupdate", 74 | "-k", NSUPDATE_KEYFILE, 75 | "-v", 76 | ] 77 | proc = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 78 | try: 79 | outs, errs = proc.communicate(input=nsupdate_input, timeout=10) 80 | except subprocess.TimeoutExpired: 81 | proc.kill() 82 | outs, errs = proc.communicate(input=nsupdate_input) 83 | 84 | if nsupdate_logging: 85 | with open(NSUPDATE_LOG, 'a') as f: 86 | f.write("timestamp %lu\n" % (int(time.time()),)) 87 | f.write(outs.decode('utf-8') + "\n") 88 | f.write(errs.decode('utf-8') + "\n") 89 | f.write("---------------------------------------------\n") 90 | 91 | if proc.returncode is None or proc.returncode != 0: 92 | return False 93 | return True 94 | 95 | @app.route('/dns/1/domains') 96 | def get_domains(): 97 | try: 98 | domains = db.domains() 99 | except: 100 | abort(500) 101 | 102 | return httpjson(domains) 103 | 104 | def parse_hosts(name, domain, in_obj): 105 | host_records = [] 106 | try: 107 | if (not 'hosts' in in_obj): 108 | return host_records 109 | 110 | hosts = in_obj['hosts'] 111 | for host in hosts: 112 | rec_type = host['rec_type'] 113 | ttl = int(host['ttl']) 114 | 115 | if ttl < 30 or ttl > (24 * 60 * 60 * 7): 116 | return "Invalid TTL" 117 | 118 | if rec_type == 'A': 119 | address = ipaddress.IPv4Address(host['address']) 120 | elif rec_type == 'AAAA': 121 | address = ipaddress.IPv6Address(host['address']) 122 | else: 123 | return "Invalid rec type" 124 | 125 | host_rec = (name, domain, rec_type, str(address), ttl) 126 | host_records.append(host_rec) 127 | 128 | except: 129 | return "JSON validation exception" 130 | 131 | return host_records 132 | 133 | def store_host(name, domain, days, pkh, host_records): 134 | # Add to database. Rely on db to filter out dups. 135 | try: 136 | db.add_host(name, domain, days, pkh) 137 | if len(host_records) > 0: 138 | if not nsupdate_exec(name, domain, host_records): 139 | http500("nsupdate failure") 140 | db.update_records(name, domain, host_records) 141 | except: 142 | return http400("Host addition rejected") 143 | 144 | return httpjson(True) 145 | 146 | def get_price_register_days(days): 147 | if days < 1 or days > 365: 148 | return 0 149 | 150 | price = int(USCENT / 50) * days 151 | 152 | return price 153 | 154 | def get_price_register(request): 155 | try: 156 | body = request.data.decode('utf-8') 157 | in_obj = json.loads(body) 158 | days = int(in_obj['days']) 159 | except: 160 | return 0 161 | 162 | return get_price_register_days(days) 163 | 164 | @app.route('/dns/1/host.register', methods=['POST']) 165 | @payment.required(get_price_register) 166 | def cmd_host_register(): 167 | 168 | # Validate JSON body w/ API params 169 | try: 170 | body = request.data.decode('utf-8') 171 | in_obj = json.loads(body) 172 | except: 173 | return http400("JSON Decode failed") 174 | 175 | try: 176 | if (not 'name' in in_obj or 177 | not 'domain' in in_obj): 178 | return http400("Missing name/domain") 179 | 180 | name = in_obj['name'] 181 | domain = in_obj['domain'] 182 | pkh = None 183 | days = 1 184 | if 'pkh' in in_obj: 185 | pkh = in_obj['pkh'] 186 | if 'days' in in_obj: 187 | days = int(in_obj['days']) 188 | 189 | if not valid_name(name) or days < 1 or days > 365: 190 | return http400("Invalid name/days") 191 | if not db.valid_domain(domain): 192 | return http404("Domain not found") 193 | if pkh: 194 | base58.b58decode_check(pkh) 195 | if (len(pkh) < 20) or (len(pkh) > 40): 196 | return http400("Invalid pkh") 197 | except: 198 | return http400("JSON validation exception") 199 | 200 | # Check against reserved host name list 201 | if reserved_name(name): 202 | return http400("Reserved name. Name not available for registration.") 203 | 204 | # Validate and collect host records for updating 205 | host_records = parse_hosts(name, domain, in_obj) 206 | if isinstance(host_records, str): 207 | return http400(host_records) 208 | 209 | return store_host(name, domain, days, pkh, host_records) 210 | 211 | def get_price_register_simple(request): 212 | try: 213 | name = request.args.get('name') 214 | domain = request.args.get('domain') 215 | days = int(request.args.get('days')) 216 | ip = request.args.get('ip') 217 | address = ipaddress.ip_address(ip) 218 | if (not valid_name(name) or days < 1 or days > 365 or 219 | not db.valid_domain(domain)): 220 | return 0 221 | except: 222 | return 0 223 | 224 | return get_price_register_days(days) 225 | 226 | @app.route('/dns/1/simpleRegister') 227 | @payment.required(get_price_register_simple) 228 | def cmd_host_simpleRegister(): 229 | try: 230 | name = request.args.get('name') 231 | domain = request.args.get('domain') 232 | days = int(request.args.get('days')) 233 | ip = request.args.get('ip') 234 | 235 | if not valid_name(name) or days < 1 or days > 365: 236 | return http400("Invalid name/days") 237 | if not db.valid_domain(domain): 238 | return http404("Domain not found") 239 | except: 240 | return http400("Invalid name / domain / days supplied") 241 | 242 | try: 243 | address = ipaddress.ip_address(ip) 244 | except: 245 | return http400("Invalid IP address supplied") 246 | 247 | if isinstance(address, ipaddress.IPv4Address): 248 | rec_type = 'A' 249 | elif isinstance(address, ipaddress.IPv6Address): 250 | rec_type = 'AAAA' 251 | else: 252 | return http500("bonkers") 253 | 254 | # Check against reserved host name list 255 | if reserved_name(name): 256 | return http400("Reserved name. Name not available for registration.") 257 | 258 | # Validate and collect host records 259 | host_records = [] 260 | host_rec = (name, domain, rec_type, str(address), 1000) 261 | host_records.append(host_rec) 262 | 263 | return store_host(name, domain, days, None, host_records) 264 | 265 | @app.route('/dns/1/records.update', methods=['POST']) 266 | @payment.required(int(USCENT / 5)) 267 | def cmd_host_update(): 268 | 269 | # Validate JSON body w/ API params 270 | try: 271 | body = request.data.decode('utf-8') 272 | in_obj = json.loads(body) 273 | except: 274 | return http400("JSON Decode failed") 275 | 276 | # Validate JSON object basics 277 | try: 278 | if (not 'name' in in_obj or 279 | not 'domain' in in_obj or 280 | not 'hosts' in in_obj): 281 | return http400("Missing name/hosts") 282 | 283 | name = in_obj['name'] 284 | domain = in_obj['domain'] 285 | if not valid_name(name): 286 | return http400("Invalid name") 287 | if not db.valid_domain(domain): 288 | return http404("Domain not found") 289 | except: 290 | return http400("JSON validation exception") 291 | 292 | # Validate and collect host records for updating 293 | host_records = parse_hosts(name, domain, in_obj) 294 | if isinstance(host_records, str): 295 | return http400(host_records) 296 | 297 | # Verify host exists, and is not expired 298 | try: 299 | hostinfo = db.get_host(name, domain) 300 | if hostinfo is None: 301 | return http404("Unknown name") 302 | except: 303 | return http500("DB Exception") 304 | 305 | # Check permission to update 306 | pkh = hostinfo['pkh'] 307 | if pkh is None: 308 | abort(403) 309 | sig_str = request.headers.get('X-Bitcoin-Sig') 310 | try: 311 | if not sig_str or not wallet.verify_bitcoin_message(body, sig_str, pkh): 312 | abort(403) 313 | except: 314 | abort(403) 315 | 316 | # Add to database. Rely on db to filter out dups. 317 | try: 318 | if not nsupdate_exec(name, domain, host_records): 319 | http500("nsupdate failure") 320 | db.update_records(name, domain, host_records) 321 | except: 322 | return http400("DB Exception") 323 | 324 | return httpjson(True) 325 | 326 | @app.route('/dns/1/host.delete', methods=['POST']) 327 | def cmd_host_delete(): 328 | 329 | # Validate JSON body w/ API params 330 | try: 331 | body = request.data.decode('utf-8') 332 | in_obj = json.loads(body) 333 | except: 334 | return http400("JSON Decode failed") 335 | 336 | # Validate JSON object basics 337 | try: 338 | if (not 'name' in in_obj or 339 | not 'domain' in in_obj or 340 | not 'pkh' in in_obj): 341 | return http400("Missing name/pkh") 342 | 343 | name = in_obj['name'] 344 | domain = in_obj['domain'] 345 | pkh = in_obj['pkh'] 346 | if (not valid_name(name) or (len(pkh) < 10)): 347 | return http400("Invalid name") 348 | if not db.valid_domain(domain): 349 | return http404("Domain not found") 350 | except: 351 | return http400("JSON validation exception") 352 | 353 | # Verify host exists, and is not expired 354 | try: 355 | hostinfo = db.get_host(name, domain) 356 | if hostinfo is None: 357 | return http404("Unknown name") 358 | except: 359 | return http500("DB Exception - get host") 360 | 361 | # Check permission to update 362 | if (hostinfo['pkh'] is None) or (pkh != hostinfo['pkh']): 363 | abort(403) 364 | sig_str = request.headers.get('X-Bitcoin-Sig') 365 | try: 366 | if not sig_str or not wallet.verify_bitcoin_message(body, sig_str, pkh): 367 | abort(403) 368 | except: 369 | abort(403) 370 | 371 | # Remove from database. Rely on db to filter out dups. 372 | try: 373 | if not nsupdate_exec(name, domain, []): 374 | http500("nsupdate failure") 375 | db.delete_host(name, domain) 376 | except: 377 | return http400("DB Exception - delete host") 378 | 379 | return httpjson(True) 380 | 381 | @app.route('/') 382 | def get_info(): 383 | # API endpoint metadata - export list of services 384 | info_obj = [{ 385 | "name": "dns/1", 386 | "website": "https://github.com/jgarzik/playground21/tree/master/dns", 387 | "pricing-type": "per-rpc", 388 | "pricing": [ 389 | { 390 | "rpc": "domains", 391 | "per-req": 0, 392 | }, 393 | { 394 | "rpc": "host.register", 395 | "per-day": int(USCENT / 50), 396 | }, 397 | { 398 | "rpc": "simpleRegister", 399 | "per-day": int(USCENT / 50), 400 | }, 401 | { 402 | "rpc": "records.update", 403 | "per-req": int(USCENT / 5), 404 | }, 405 | { 406 | "rpc": "host.delete", 407 | "per-req": 0, 408 | }, 409 | ] 410 | }] 411 | return httpjson(info_obj) 412 | 413 | if __name__ == '__main__': 414 | app.run(host='0.0.0.0', port=12005) 415 | 416 | -------------------------------------------------------------------------------- /dns/dns.schema: -------------------------------------------------------------------------------- 1 | CREATE TABLE domains ( 2 | name TEXT PRIMARY KEY 3 | ); 4 | 5 | CREATE TABLE hosts ( 6 | name TEXT NOT NULL, 7 | domain TEXT NOT NULL, 8 | time_create INTEGER NOT NULL, 9 | time_expire INTEGER NOT NULL, 10 | owner_pkh TEXT, 11 | FOREIGN KEY(domain) REFERENCES domains(name) 12 | ); 13 | 14 | CREATE UNIQUE INDEX hosts_idx ON hosts (name, domain); 15 | 16 | CREATE TABLE records ( 17 | name TEXT NOT NULL, 18 | domain TEXT NOT NULL, 19 | rec_type TEXT NOT NULL, 20 | value TEXT NOT NULL, 21 | ttl INTEGER NOT NULL, 22 | FOREIGN KEY(name) REFERENCES hosts(name), 23 | FOREIGN KEY(domain) REFERENCES domains(name) 24 | ); 25 | 26 | CREATE INDEX record_idx ON records (name, domain); 27 | 28 | -------------------------------------------------------------------------------- /dns/example-dns-server.conf: -------------------------------------------------------------------------------- 1 | { 2 | "DNS_SERVER1": "127.0.0.1", 3 | "NSUPDATE_KEYFILE": "Kexample.com.+111+22334.key", 4 | "NSUPDATE_LOG": "nsupdate.log", 5 | "NSUPDATE_LOGGING": true, 6 | "DB_PATHNAME": "dns.db" 7 | } 8 | -------------------------------------------------------------------------------- /dns/httputil.py: -------------------------------------------------------------------------------- 1 | 2 | import json 3 | 4 | def httpjson(val): 5 | body = json.dumps(val, indent=2) 6 | return (body, 200, { 7 | 'Content-length': len(body), 8 | 'Content-type': 'application/json', 9 | }) 10 | 11 | def http400(msg): 12 | if not msg: 13 | msg = "Invalid request" 14 | return (msg, 400, {'Content-Type':'text/plain'}) 15 | 16 | def http404(msg): 17 | if not msg: 18 | msg = "Not found" 19 | return (msg, 404, {'Content-Type':'text/plain'}) 20 | 21 | def http500(msg): 22 | if not msg: 23 | msg = "Internal server error" 24 | return (msg, 500, {'Content-Type':'text/plain'}) 25 | 26 | -------------------------------------------------------------------------------- /dns/mkdb.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | FN=dns.db 4 | 5 | if [ -f $FN ] 6 | then 7 | echo Database $FN already exists. Will not overwrite 8 | exit 1 9 | fi 10 | sqlite3 $FN < dns.schema 11 | 12 | -------------------------------------------------------------------------------- /dns/srvdb.py: -------------------------------------------------------------------------------- 1 | 2 | import apsw 3 | import time 4 | 5 | class SrvDb(object): 6 | def __init__(self, filename): 7 | self.connection = apsw.Connection(filename) 8 | 9 | def valid_domain(self, domain): 10 | cursor = self.connection.cursor() 11 | 12 | row = cursor.execute("SELECT COUNT(*) FROM domains WHERE name = ?", (domain,)).fetchone() 13 | if not row or (int(row[0] < 1)): 14 | return False 15 | return True 16 | 17 | def domains(self): 18 | cursor = self.connection.cursor() 19 | 20 | # retrieve sorted domain list 21 | rows = [] 22 | for row in cursor.execute("SELECT name FROM domains ORDER BY name"): 23 | rows.append(row[0]) 24 | return rows 25 | 26 | def add_host(self, name, domain, days, pkh): 27 | cursor = self.connection.cursor() 28 | 29 | # Create, expiration times 30 | tm_creat = int(time.time()) 31 | tm_expire = tm_creat + (days * 24 * 60 * 60) 32 | 33 | # Add hash metadata to db 34 | cursor.execute("INSERT INTO hosts VALUES(?, ?, ?, ?, ?)", (name, domain, tm_creat, tm_expire, pkh)) 35 | 36 | return True 37 | 38 | def get_host(self, name, domain): 39 | cursor = self.connection.cursor() 40 | 41 | curtime = int(time.time()) 42 | row = cursor.execute("SELECT * FROM hosts WHERE name = ? AND domain = ? AND time_expire > ?", (name, domain, curtime)).fetchone() 43 | if not row: 44 | return None 45 | obj = { 46 | 'name': row[0], 47 | 'domain': row[1], 48 | 'create': int(row[2]), 49 | 'expire': int(row[3]), 50 | 'pkh': row[4], 51 | } 52 | return obj 53 | 54 | def update_records(self, name, domain, host_records): 55 | cursor = self.connection.cursor() 56 | 57 | cursor.execute("DELETE FROM records WHERE name = ? AND domain = ?", (name, domain)) 58 | 59 | for host_rec in host_records: 60 | cursor.execute("INSERT INTO records VALUES(?, ?, ?, ?, ?)", host_rec) 61 | 62 | def delete_host(self, name, domain): 63 | cursor = self.connection.cursor() 64 | 65 | cursor.execute("DELETE FROM records WHERE name = ? AND domain = ?", (name, domain)) 66 | cursor.execute("DELETE FROM hosts WHERE name = ? AND domain = ?", (name, domain)) 67 | 68 | -------------------------------------------------------------------------------- /fortune/README.md: -------------------------------------------------------------------------------- 1 | 2 | fortune - Fortune cookie API 3 | ============================ 4 | 5 | Summary: Receive a pithy saying for 10 satoshis. 6 | 7 | First time setup 8 | ---------------- 9 | 10 | $ sudo apt-get install fortune-mod 11 | 12 | 13 | Running the server 14 | ------------------ 15 | 16 | $ python3 fortune-server.py 17 | 18 | 19 | 20 | API; 21 | 22 | 1. Get fortune 23 | -------------- 24 | 25 | HTTP URI: /fortune 26 | 27 | Params: None 28 | 29 | Result: 30 | 31 | Plain text fortune 32 | 33 | Pricing: 34 | 35 | 10 satoshis 36 | 37 | 38 | -------------------------------------------------------------------------------- /fortune/fortune-client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | 3 | # 4 | # Command line usage: 5 | # $ python3 fortune-client.py # Get pithy saying 6 | # $ python3 fortune-client.py info # Get server metadata 7 | # 8 | 9 | import json 10 | import os 11 | import sys 12 | import click 13 | 14 | # import from the 21 Developer Library 15 | from two1.commands.config import Config 16 | from two1.lib.wallet import Wallet 17 | from two1.lib.bitrequests import BitTransferRequests 18 | 19 | # set up bitrequest client for BitTransfer requests 20 | wallet = Wallet() 21 | username = Config().username 22 | requests = BitTransferRequests(wallet, username) 23 | 24 | FORTUNECLI_VERSION = '1.0' 25 | DEFAULT_ENDPOINT = 'http://localhost:12012/' 26 | 27 | @click.group(invoke_without_command=True) 28 | @click.option('--endpoint', '-e', 29 | default=DEFAULT_ENDPOINT, 30 | metavar='STRING', 31 | show_default=True, 32 | help='API endpoint URI') 33 | @click.option('--debug', '-d', 34 | is_flag=True, 35 | help='Turns on debugging messages.') 36 | @click.version_option(FORTUNECLI_VERSION) 37 | @click.pass_context 38 | def main(ctx, endpoint, debug): 39 | """ Command-line Interface for the fortune cookie service 40 | """ 41 | 42 | if ctx.obj is None: 43 | ctx.obj = {} 44 | 45 | ctx.obj['endpoint'] = endpoint 46 | 47 | if ctx.invoked_subcommand is None: 48 | cmd_fortune(ctx) 49 | 50 | 51 | def cmd_fortune(ctx): 52 | sel_url = ctx.obj['endpoint'] + 'fortune' 53 | answer = requests.get(url=sel_url.format()) 54 | print(answer.text) 55 | 56 | 57 | @click.command(name='info') 58 | @click.pass_context 59 | def cmd_info(ctx): 60 | sel_url = ctx.obj['endpoint'] 61 | answer = requests.get(url=sel_url.format()) 62 | print(answer.text) 63 | 64 | main.add_command(cmd_info) 65 | 66 | if __name__ == "__main__": 67 | main() 68 | 69 | -------------------------------------------------------------------------------- /fortune/fortune-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import random 4 | import subprocess 5 | 6 | # import flask web microframework 7 | from flask import Flask 8 | from flask import request 9 | from flask import abort 10 | 11 | # import from the 21 Developer Library 12 | from two1.lib.wallet import Wallet 13 | from two1.lib.bitserv.flask import Payment 14 | 15 | app = Flask(__name__) 16 | wallet = Wallet() 17 | payment = Payment(app, wallet) 18 | 19 | def get_fortune_text(): 20 | proc = subprocess.Popen(["/usr/games/fortune"], stdout=subprocess.PIPE, stderr=subprocess.PIPE) 21 | try: 22 | outs, errs = proc.communicate(timeout=10) 23 | except TimeoutExpired: 24 | proc.kill() 25 | outs, errs = proc.communicate() 26 | 27 | if errs: 28 | return None 29 | 30 | return outs 31 | 32 | # endpoint to get a value from the server 33 | @app.route('/fortune') 34 | @payment.required(10) 35 | def get_fortune(): 36 | fortune = get_fortune_text() 37 | if fortune is None: 38 | abort(500) 39 | 40 | return fortune 41 | 42 | @app.route('/') 43 | def get_info(): 44 | info_obj = { 45 | "name": "fortune", 46 | "version": 100, 47 | "pricing": { 48 | "/fortune" : { 49 | "minimum" : 10 50 | }, 51 | } 52 | 53 | } 54 | body = json.dumps(info_obj, indent=2) 55 | return (body, 200, { 56 | 'Content-length': len(body), 57 | 'Content-type': 'application/json', 58 | }) 59 | 60 | if __name__ == '__main__': 61 | app.run(host='0.0.0.0', port=12012) 62 | 63 | -------------------------------------------------------------------------------- /kvdb/.gitignore: -------------------------------------------------------------------------------- 1 | 2 | keyvalue.db 3 | 4 | -------------------------------------------------------------------------------- /kvdb/README.md: -------------------------------------------------------------------------------- 1 | 2 | kvdb - Simple key/value API 3 | =========================== 4 | 5 | Summary: key/value database. Simple, reliable storage API. 6 | 7 | Pricing theory: Each request charges per byte, with a minimum price floor. 8 | 9 | TODO/Caveats: 10 | - Entries are never deleted. Potential for running out of storage space (while 11 | being paid to do so). 12 | 13 | 14 | First time setup 15 | ---------------- 16 | $ python3 mkdb.py 17 | 18 | This creates an empty keyvalue.db file used for backing storage. 19 | 20 | 21 | Running the server 22 | ------------------ 23 | $ python3 kvdb-server.py 24 | 25 | 26 | API; 27 | 28 | 1. Get value 29 | ------------ 30 | 31 | HTTP URI: /get 32 | 33 | Params: 34 | key Binary string, 1-512 bytes 35 | 36 | Result if key found: 37 | Binary string, 0-1000000 bytes 38 | Result if key not found: 39 | Binary string, 0 bytes 40 | 41 | Pricing: 42 | Byte length of value, in satoshis. Minimum 1. 43 | 44 | 45 | 2. Store key+value 46 | ------------------ 47 | 48 | HTTP URI: /put 49 | 50 | Params: 51 | key Binary string, 1-512 bytes 52 | value Binary string, 0-1000000 bytes 53 | 54 | Result if successfully stored: 55 | Binary string, "stored" 56 | Otherwise: 57 | Binary string, containing an error message 58 | 59 | Pricing: 60 | Byte length of key + 61 | Byte length of value, in satoshis. Minimum 2. 62 | 63 | 64 | -------------------------------------------------------------------------------- /kvdb/kvdb-client.py: -------------------------------------------------------------------------------- 1 | 2 | # 3 | # Command line usage: 4 | # $ python3 kvdb-client.py # Get service info (JSON) 5 | # $ python3 kvdb-client.py KEY # Get value from server, given KEY 6 | # $ python3 kvdb-client.py KEY VALUE # Store KEY and VALUE on server 7 | # 8 | 9 | import json 10 | import os 11 | import sys 12 | 13 | # import from the 21 Developer Library 14 | from two1.commands.config import Config 15 | from two1.lib.wallet import Wallet 16 | from two1.lib.bitrequests import BitTransferRequests 17 | 18 | # set up bitrequest client for BitTransfer requests 19 | wallet = Wallet() 20 | username = Config().username 21 | requests = BitTransferRequests(wallet, username) 22 | 23 | # server address 24 | server_url = 'http://localhost:12003/' 25 | 26 | def cmd_get(key): 27 | sel_url = server_url + 'get?key={0}' 28 | answer = requests.get(url=sel_url.format(key)) 29 | print(answer.text) 30 | 31 | def cmd_put(key, value): 32 | sel_url = server_url + 'put?key={0}&value={1}' 33 | answer = requests.get(url=sel_url.format(key, value)) 34 | print(answer.text) 35 | 36 | def cmd_info(): 37 | sel_url = server_url 38 | answer = requests.get(url=sel_url.format()) 39 | print(answer.text) 40 | 41 | if __name__ == '__main__': 42 | if len(sys.argv) == 2: 43 | cmd_get(sys.argv[1]) 44 | elif len(sys.argv) == 3: 45 | cmd_put(sys.argv[1], sys.argv[2]) 46 | else: 47 | cmd_info() 48 | 49 | -------------------------------------------------------------------------------- /kvdb/kvdb-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import random 4 | import apsw 5 | 6 | # import flask web microframework 7 | from flask import Flask 8 | from flask import request 9 | 10 | # import from the 21 Developer Library 11 | from two1.lib.wallet import Wallet 12 | from two1.lib.bitserv.flask import Payment 13 | 14 | connection = apsw.Connection("keyvalue.db") 15 | 16 | app = Flask(__name__) 17 | wallet = Wallet() 18 | payment = Payment(app, wallet) 19 | 20 | def sqldb_query(key): 21 | cursor = connection.cursor() 22 | for v in cursor.execute("SELECT v FROM tab WHERE k = ?", (key,)): 23 | return v 24 | return '' 25 | 26 | def sqldb_store(key, value): 27 | cursor = connection.cursor() 28 | cursor.execute("REPLACE INTO tab(k, v) VALUES(?, ?)", (key, value)) 29 | 30 | def get_get_price_from_request(request): 31 | key = request.args.get('key') 32 | price = len(sqldb_query(key)) 33 | if price < 1: 34 | price = 1 35 | return price 36 | 37 | # endpoint to get a value from the server 38 | @app.route('/get') 39 | @payment.required(get_get_price_from_request) 40 | def load_value(): 41 | key = request.args.get('key') 42 | cursor = connection.cursor() 43 | return sqldb_query(key) 44 | 45 | def get_put_price_from_request(request): 46 | key = request.args.get('key') 47 | value = request.args.get('value') 48 | total = len(key) + len(value) 49 | if total < 2: 50 | total = 2 51 | return total 52 | 53 | @app.route('/put') 54 | @payment.required(get_put_price_from_request) 55 | def store_value(): 56 | key = request.args.get('key') 57 | value = request.args.get('value') 58 | 59 | if len(key) < 1 or len(key) > 512: 60 | return "invalid key size" 61 | if len(value) > 1000000: 62 | return "value too large" 63 | 64 | sqldb_store(key, value) 65 | 66 | return "stored" 67 | 68 | @app.route('/') 69 | def get_info(): 70 | info_obj = { 71 | "name": "kvdb", 72 | "version": 100, 73 | "pricing": { 74 | "/get" : { 75 | "minimum" : 1 76 | }, 77 | "/put" : { 78 | "minimum" : 2 79 | } 80 | } 81 | 82 | } 83 | body = json.dumps(info_obj, indent=2) 84 | return (body, 200, { 85 | 'Content-length': len(body), 86 | 'Content-type': 'application/json', 87 | }) 88 | 89 | if __name__ == '__main__': 90 | app.run(host='0.0.0.0', port=12003) 91 | 92 | -------------------------------------------------------------------------------- /kvdb/mkdb.py: -------------------------------------------------------------------------------- 1 | 2 | import apsw 3 | 4 | 5 | connection = apsw.Connection("keyvalue.db") 6 | cursor = connection.cursor() 7 | cursor.execute("CREATE TABLE tab(k BLOB PRIMARY KEY, v BLOB)") 8 | 9 | -------------------------------------------------------------------------------- /kvram/README.md: -------------------------------------------------------------------------------- 1 | 2 | kvram - Simple key/value API 3 | ============================ 4 | 5 | Summary: In-memory key/value database. Simple, unreliable RAM storage API. 6 | 7 | Pricing theory: Each request charges per byte, with a minimum price floor. 8 | 9 | Future work / caveats: 10 | 11 | * Entries are never stored. Each server restart empties database. 12 | * Entries are never deleted. 13 | * Potential for running out of memory (while being paid to do so). 14 | 15 | 16 | First time setup 17 | ---------------- 18 | None 19 | 20 | 21 | Running the server 22 | ------------------ 23 | $ python3 kvram-server.py 24 | 25 | The server starts with an empty in-memory database. 26 | 27 | 28 | API; 29 | 30 | 1. Get value 31 | ------------ 32 | 33 | HTTP URI: /get 34 | 35 | Params: 36 | 37 | key Binary string, 1-512 bytes 38 | 39 | Result if key found: 40 | 41 | Binary string, 0-1000000 bytes 42 | 43 | Result if key not found: 44 | 45 | Binary string, 0 bytes 46 | 47 | Pricing: 48 | 49 | Byte length of value, in satoshis. Minimum 1. 50 | 51 | 52 | 2. Store key+value 53 | ------------------ 54 | 55 | HTTP URI: /put 56 | 57 | Params: 58 | 59 | key Binary string, 1-512 bytes 60 | value Binary string, 0-1000000 bytes 61 | 62 | Result if successfully stored: 63 | 64 | Binary string, "stored" 65 | 66 | Otherwise: 67 | 68 | Binary string, containing an error message 69 | 70 | Pricing: 71 | 72 | Byte length of key + 73 | Byte length of value, in satoshis. Minimum 2. 74 | 75 | 76 | -------------------------------------------------------------------------------- /kvram/kvram-client.py: -------------------------------------------------------------------------------- 1 | 2 | # 3 | # Command line usage: 4 | # $ python3 kvram-client.py # Get service info (JSON) 5 | # $ python3 kvram-client.py KEY # Get value from server, given KEY 6 | # $ python3 kvram-client.py KEY VALUE # Store KEY and VALUE on server 7 | # 8 | 9 | import json 10 | import os 11 | import sys 12 | 13 | # import from the 21 Developer Library 14 | from two1.commands.config import Config 15 | from two1.lib.wallet import Wallet 16 | from two1.lib.bitrequests import BitTransferRequests 17 | 18 | # set up bitrequest client for BitTransfer requests 19 | wallet = Wallet() 20 | username = Config().username 21 | requests = BitTransferRequests(wallet, username) 22 | 23 | # server address 24 | server_url = 'http://localhost:12001/' 25 | 26 | def cmd_get(key): 27 | sel_url = server_url + 'get?key={0}' 28 | answer = requests.get(url=sel_url.format(key)) 29 | print(answer.text) 30 | 31 | def cmd_put(key, value): 32 | sel_url = server_url + 'put?key={0}&value={1}' 33 | answer = requests.get(url=sel_url.format(key, value)) 34 | print(answer.text) 35 | 36 | def cmd_info(): 37 | sel_url = server_url 38 | answer = requests.get(url=sel_url.format()) 39 | print(answer.text) 40 | 41 | if __name__ == '__main__': 42 | if len(sys.argv) == 2: 43 | cmd_get(sys.argv[1]) 44 | elif len(sys.argv) == 3: 45 | cmd_put(sys.argv[1], sys.argv[2]) 46 | else: 47 | cmd_info() 48 | 49 | -------------------------------------------------------------------------------- /kvram/kvram-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import random 4 | 5 | # import flask web microframework 6 | from flask import Flask 7 | from flask import request 8 | 9 | # import from the 21 Developer Library 10 | from two1.lib.wallet import Wallet 11 | from two1.lib.bitserv.flask import Payment 12 | 13 | app = Flask(__name__) 14 | wallet = Wallet() 15 | payment = Payment(app, wallet) 16 | 17 | db = {} 18 | 19 | def get_get_price_from_request(request): 20 | key = request.args.get('key') 21 | price = 0 22 | if key in db: 23 | price = len(db[key]) 24 | if price < 1: 25 | price = 1 26 | return price 27 | 28 | # endpoint to get a value from the server 29 | @app.route('/get') 30 | @payment.required(get_get_price_from_request) 31 | def load_value(): 32 | key = request.args.get('key') 33 | if key in db: 34 | return db[key] 35 | else: 36 | return '' 37 | 38 | def get_put_price_from_request(request): 39 | key = request.args.get('key') 40 | value = request.args.get('value') 41 | total = len(key) + len(value) 42 | if total < 2: 43 | total = 2 44 | return total 45 | 46 | @app.route('/put') 47 | @payment.required(get_put_price_from_request) 48 | def store_value(): 49 | key = request.args.get('key') 50 | value = request.args.get('value') 51 | 52 | if len(key) < 1 or len(key) > 512: 53 | return "invalid key size" 54 | if len(value) > 1000000: 55 | return "value too large" 56 | 57 | db[key] = value 58 | 59 | return "stored" 60 | 61 | @app.route('/') 62 | def get_info(): 63 | info_obj = { 64 | "name": "kvram", 65 | "version": 100, 66 | "pricing": { 67 | "/get" : { 68 | "minimum" : 1 69 | }, 70 | "/put" : { 71 | "minimum" : 2 72 | } 73 | } 74 | 75 | } 76 | body = json.dumps(info_obj, indent=2) 77 | return (body, 200, { 78 | 'Content-length': len(body), 79 | 'Content-type': 'application/json', 80 | }) 81 | 82 | if __name__ == '__main__': 83 | app.run(host='0.0.0.0', port=12001) 84 | 85 | -------------------------------------------------------------------------------- /signing/README.md: -------------------------------------------------------------------------------- 1 | 2 | signing - Bitcoin transaction signing server 3 | ============================================ 4 | 5 | WARNING: Work-in-progress. Status: untested + feature complete. 6 | 7 | Summary: Create account at signing server. Sign a bitcoin transaction. 8 | 9 | Demonstrates: 10 | 11 | * Participating in a P2SH, P2PKH, and/or multisig contract, by generating and providing a public key upon request. 12 | 13 | * Signing a bitcoin transaction 14 | 15 | * Broadcasting a bitcoin transaction to the bitcoin network 16 | 17 | * Public key authentication (only signs transaction for contract based on public key supplied at first contact) 18 | 19 | 20 | Running the server 21 | ------------------ 22 | 23 | $ python3 signing-server.py 24 | 25 | 26 | 27 | API; 28 | 29 | 1. New contract 30 | --------------- 31 | 32 | HTTP URI: GET /new 33 | 34 | Params: 35 | 36 | owner: Hex-encoded ECDSA public key 37 | 38 | Result: 39 | 40 | application/json document with the following keys: 41 | id: contract id (number) 42 | contract_key: public key associated with this contract, which the signing server will use for signing future bitcoin transactions 43 | 44 | 45 | Pricing: 46 | 47 | 1000 satoshis 48 | 49 | 50 | 51 | 2. Sign contract 52 | ---------------- 53 | HTTP URI: PUT /sign/[contract id] 54 | 55 | Params: 56 | 57 | In HTTP body, a application/json document containing the following keys: 58 | 59 | msg: signed message, wrapping a hex-encoded bitcoin transaction 60 | sig: base64-encoded signature 61 | input_index: index inside BTC tx to sign 62 | hash_type: hash type of signature to apply 63 | script: hex-encoded scriptPubKey / redeem script 64 | broadcast: if true, broadcast signed TX to network 65 | 66 | Result: 67 | 68 | text/plain document containing signed, hex-encoded transaction 69 | 70 | Pricing: 71 | 72 | 1000 satoshis 73 | 74 | 75 | 76 | -------------------------------------------------------------------------------- /signing/mkdb.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | FN=signing.sqlite3 4 | 5 | if [ -f $FN ] 6 | then 7 | echo Database $FN already exists. Will not overwrite 8 | exit 1 9 | fi 10 | sqlite3 $FN < signing.schema 11 | 12 | -------------------------------------------------------------------------------- /signing/signing-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import apsw 3 | import json 4 | import binascii 5 | 6 | # import flask web microframework 7 | from flask import Flask 8 | from flask import request 9 | from flask import abort 10 | 11 | # import from the 21 Developer Library 12 | from two1.lib.bitcoin.txn import Transaction 13 | from two1.lib.bitcoin.crypto import PublicKey 14 | from two1.lib.wallet import Wallet, exceptions 15 | from two1.lib.bitserv.flask import Payment 16 | 17 | connection = apsw.Connection("signing.db") 18 | 19 | SRV_ACCT = 'signing' 20 | 21 | app = Flask(__name__) 22 | wallet = Wallet() 23 | payment = Payment(app, wallet) 24 | 25 | # Create wallet account for these contracts 26 | try: 27 | wallet.create_account(SRV_ACCT) 28 | except exceptions.AccountCreationError: 29 | pass 30 | 31 | def srvdb_last_idx(cursor): 32 | row = cursor.execute("SELECT MAX(hd_index) FROM metadata").fetchone() 33 | if row is None: 34 | return 0 35 | return int(row[0]) 36 | 37 | @app.route('/new') 38 | @payment.required(1000) 39 | def cmd_new(): 40 | # Get hex-encoded input (owner) public key 41 | try: 42 | owner_pubkey = PublicKey.from_bytes(request.args.get('owner')) 43 | except: 44 | abort(400) 45 | 46 | # Generate next available HD wallet index 47 | try: 48 | cursor = connection.cursor() 49 | next_idx = srvdb_last_idx(cursor) + 1 50 | except: 51 | abort(500) 52 | 53 | # Derive HD public key 54 | acct = wallet._check_and_get_accounts([SRV_ACCT]) 55 | hd_pubkey = acct.get_public_key(False, next_idx) 56 | pubkey = binascii.hexlify(hd_public.compressed_bytes()) 57 | address = hd_pubkey.address() 58 | owner_b64 = owner_pubkey.to_base64() 59 | 60 | # Add public key to db 61 | try: 62 | cursor.execute("INSERT INTO metadata VALUES(?, ?, ?, ?)", (address, pubkey, next_idx, owner_b64)) 63 | except: 64 | abort(500) 65 | 66 | # Return contract id, public key 67 | obj = { 68 | 'id': next_idx, 69 | 'contract_key': pubkey, 70 | } 71 | 72 | body = json.dumps(ret_obj, indent=2) 73 | return (body, 200, { 74 | 'Content-length': len(body), 75 | 'Content-type': 'application/json', 76 | }) 77 | 78 | 79 | @app.route('/sign/', methods=['PUT']) 80 | @payment.required(1000) 81 | def cmd_sign(id): 82 | body = request.data 83 | body_len = len(body) 84 | 85 | # get associated metadata 86 | try: 87 | cursor = connection.cursor() 88 | row = cursor.execute("SELECT * FROM metadata WHERE hd_index = ?", (id,)).fetchone() 89 | if row is None: 90 | abort(404) 91 | 92 | hd_index = int(row[0]) 93 | owner_key = PublicKey.from_bytes(row[3]) 94 | except: 95 | abort(500) 96 | 97 | # check content-length 98 | clen_str = request.headers.get('content-length') 99 | if clen_str is None: 100 | abort(400) 101 | clen = int(clen_str) 102 | if clen != body_len: 103 | abort(400) 104 | 105 | # check content-type 106 | ctype = request.headers.get('content-type') 107 | if ctype is None or ctype != 'application/json': 108 | abort(400) 109 | 110 | # parse JSON body 111 | try: 112 | in_obj = json.loads(body) 113 | if (not 'msg' in in_obj or not 'sig' in in_obj or 114 | not 'hash_type' in in_obj or 115 | not 'input_idx' in in_obj or not 'script' in in_obj): 116 | abort(400) 117 | hash_type = int(in_obj['hash_type']) 118 | input_idx = int(in_obj['input_idx']) 119 | script = Script.from_bytes(binascii.unhexlify(in_obj['script'])) 120 | tx_hex = binascii.unhexlify(in_obj['msg']) 121 | 122 | broadcast = False 123 | if 'broadcast' in in_obj and in_obj['broadcast'] == True: 124 | broadcast = True 125 | except: 126 | abort(400) 127 | 128 | # validate base64-encoded signature on hex-encoded transaction 129 | try: 130 | rc = PublicKey.verify_bitcoin(tx_hex, in_obj['sig'], owner_key.address()) 131 | if not rc: 132 | abort(400) 133 | 134 | tx = Transaction.from_hex(tx_hex) 135 | except: 136 | abort(400) 137 | 138 | # get HD wallet account, privkey for this contract 139 | acct = wallet._check_and_get_accounts([SRV_ACCT]) 140 | hd_privkey = acct.get_private_key(False, hd_index) 141 | 142 | # try to sign the input 143 | try: 144 | tx.sign_input(input_idx, hash_type, hd_privkey, script) 145 | except: 146 | abort(400) 147 | 148 | # broadcast transaction to network 149 | if broadcast: 150 | wallet.broadcast(tx) 151 | 152 | # return updated transaction 153 | output_data = tx.to_hex() 154 | 155 | return (output_data, 200, { 156 | 'Content-length': len(output_data), 157 | 'Content-type': 'text/plain', 158 | }) 159 | 160 | 161 | @app.route('/') 162 | def get_info(): 163 | info_obj = { 164 | "name": "signing", 165 | "version": 100, 166 | "pricing": { 167 | "/new" : { 168 | "minimum" : 1000 169 | }, 170 | "/sign" : { 171 | "minimum" : 1000 172 | }, 173 | } 174 | 175 | } 176 | body = json.dumps(info_obj, indent=2) 177 | return (body, 200, { 178 | 'Content-length': len(body), 179 | 'Content-type': 'application/json', 180 | }) 181 | 182 | if __name__ == '__main__': 183 | app.run(host='0.0.0.0', port=12004) 184 | 185 | -------------------------------------------------------------------------------- /signing/signing.schema: -------------------------------------------------------------------------------- 1 | CREATE TABLE metadata ( 2 | hd_index INTEGER PRIMARY KEY, 3 | address TEXT NOT NULL, 4 | pubkey TEXT NOT NULL, 5 | owner_pubkey TEXT NOT NULL 6 | ); 7 | -------------------------------------------------------------------------------- /stegano/steg-client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | 3 | # 4 | # Command line usage: 5 | # $ python3 steg-client.py --help 6 | # 7 | 8 | import json 9 | import os 10 | import sys 11 | import click 12 | 13 | # import from the 21 Developer Library 14 | from two1.commands.config import Config 15 | from two1.lib.wallet import Wallet 16 | from two1.lib.bitrequests import BitTransferRequests 17 | 18 | # set up bitrequest client for BitTransfer requests 19 | wallet = Wallet() 20 | username = Config().username 21 | requests = BitTransferRequests(wallet, username) 22 | 23 | STEGCLI_VERSION = '1.0' 24 | DEFAULT_ENDPOINT = 'http://localhost:12018/' 25 | 26 | @click.group(invoke_without_command=True) 27 | @click.option('--endpoint', '-e', 28 | default=DEFAULT_ENDPOINT, 29 | metavar='STRING', 30 | show_default=True, 31 | help='API endpoint URI') 32 | @click.option('--debug', '-d', 33 | is_flag=True, 34 | help='Turns on debugging messages.') 35 | @click.version_option(STEGCLI_VERSION) 36 | @click.pass_context 37 | def main(ctx, endpoint, debug): 38 | """ Command-line Interface for the steganography service 39 | """ 40 | 41 | if ctx.obj is None: 42 | ctx.obj = {} 43 | 44 | ctx.obj['endpoint'] = endpoint 45 | 46 | 47 | 48 | @click.command(name='info') 49 | @click.pass_context 50 | def cmd_info(ctx): 51 | sel_url = ctx.obj['endpoint'] 52 | answer = requests.get(url=sel_url.format()) 53 | print(answer.text) 54 | 55 | 56 | @click.command(name='encode') 57 | @click.argument('message', type=click.File('rb')) 58 | @click.argument('file', type=click.File('rb')) 59 | @click.pass_context 60 | def cmd_encode(ctx, message, file): 61 | sel_url = ctx.obj['endpoint'] + 'encode' 62 | files = { 'message': message, 'file': file } 63 | answer = requests.post(url=sel_url.format(), files=files) 64 | print(answer.text) 65 | 66 | 67 | @click.command(name='decode') 68 | @click.argument('file', type=click.File('rb')) 69 | @click.pass_context 70 | def cmd_decode(ctx, file): 71 | sel_url = ctx.obj['endpoint'] + 'decode' 72 | files = { 'file': file } 73 | answer = requests.post(url=sel_url.format(), files=files) 74 | print(answer.text) 75 | 76 | 77 | main.add_command(cmd_info) 78 | main.add_command(cmd_encode) 79 | main.add_command(cmd_decode) 80 | 81 | if __name__ == "__main__": 82 | main() 83 | 84 | -------------------------------------------------------------------------------- /stegano/steg-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import random 4 | import subprocess 5 | import tempfile 6 | 7 | # import flask web microframework 8 | from flask import Flask 9 | from flask import request 10 | from flask import abort 11 | from werkzeug import secure_filename 12 | 13 | # import from the 21 Developer Library 14 | from two1.lib.wallet import Wallet 15 | from two1.lib.bitserv.flask import Payment 16 | 17 | app = Flask(__name__) 18 | wallet = Wallet() 19 | payment = Payment(app, wallet) 20 | 21 | @app.route('/encode', methods=['POST']) 22 | @payment.required(10) 23 | def encode(): 24 | clen_str = request.headers.get('content-length') 25 | if (not clen_str or int(clen_str) > 16000000): 26 | abort(400) 27 | 28 | file = request.files['file'] 29 | msg_file = request.files['message'] 30 | if not file or not msg_file: 31 | abort(400) 32 | 33 | filename = secure_filename(file.filename) 34 | msg_filename = secure_filename(msg_file.filename) 35 | file.save(filename) 36 | msg_file.save(msg_filename) 37 | 38 | ctype = file.content_type 39 | if not ctype: 40 | ctype = 'application/octet-stream' 41 | 42 | tmp_out = tempfile.NamedTemporaryFile(suffix=".txt") 43 | 44 | proc = subprocess.Popen(["/usr/bin/outguess", "-d", msg_filename, filename, tmp_out.name], stdout=subprocess.PIPE, stderr=subprocess.PIPE) 45 | try: 46 | outs, errs = proc.communicate(timeout=10) 47 | except TimeoutExpired: 48 | proc.kill() 49 | outs, errs = proc.communicate() 50 | 51 | os.remove(filename) 52 | os.remove(msg_filename) 53 | 54 | if errs: 55 | body = outs.decode('utf-8') + "\n" + errs.decode('utf-8') 56 | return (body, 200, { 57 | 'Content-length': len(body), 58 | 'Content-type': 'text/plain', 59 | }) 60 | # abort(500) 61 | 62 | body = tmp_out.read() 63 | return (body, 200, { 64 | 'Content-length': len(body), 65 | 'Content-type': ctype, 66 | }) 67 | 68 | @app.route('/decode', methods=['POST']) 69 | @payment.required(10) 70 | def decode(): 71 | ctype = request.headers.get('content-type') 72 | if not ctype: 73 | ctype = 'application/octet-stream' 74 | 75 | clen_str = request.headers.get('content-length') 76 | if (not clen_str or int(clen_str) > 16000000): 77 | abort(400) 78 | 79 | file = request.files['file'] 80 | if not file: 81 | abort(400) 82 | 83 | in_filename, in_ext = os.path.splitext(file.filename) 84 | if not in_ext: 85 | in_ext = ".jpg" 86 | 87 | tmp_in = tempfile.NamedTemporaryFile(suffix=in_ext) 88 | tmp_in.write(file.read()) 89 | 90 | tmp_out = tempfile.NamedTemporaryFile(suffix=".txt") 91 | 92 | proc = subprocess.Popen(["/usr/bin/outguess", "-r", tmp_in.name, tmp_out.name], stdout=subprocess.PIPE, stderr=subprocess.PIPE) 93 | try: 94 | outs, errs = proc.communicate(timeout=10) 95 | except TimeoutExpired: 96 | proc.kill() 97 | outs, errs = proc.communicate() 98 | 99 | if errs: 100 | body = outs.decode('utf-8') + "\n" + errs.decode('utf-8') 101 | return (body, 200, { 102 | 'Content-length': len(body), 103 | 'Content-type': 'text/plain', 104 | }) 105 | # abort(500) 106 | 107 | body = tmp_out.read() 108 | return (body, 200, { 109 | 'Content-length': len(body), 110 | 'Content-type': ctype, 111 | }) 112 | 113 | 114 | @app.route('/') 115 | def get_info(): 116 | info_obj = { 117 | "name": "stegano", 118 | "version": 100, 119 | "pricing": { 120 | "/encode" : { 121 | "minimum" : 10 122 | }, 123 | "/decode" : { 124 | "minimum" : 10 125 | }, 126 | } 127 | 128 | } 129 | body = json.dumps(info_obj, indent=2) 130 | return (body, 200, { 131 | 'Content-length': len(body), 132 | 'Content-type': 'application/json', 133 | }) 134 | 135 | if __name__ == '__main__': 136 | app.run(host='0.0.0.0', port=12018, debug=True) 137 | 138 | -------------------------------------------------------------------------------- /turk/.gitignore: -------------------------------------------------------------------------------- 1 | 2 | __pycache__/ 3 | turk.db 4 | 5 | -------------------------------------------------------------------------------- /turk/README.md: -------------------------------------------------------------------------------- 1 | 2 | turk - Mechanical Turk service 3 | ============================== 4 | 5 | WARNING: Untested code ahead. Needs lots of testing. 6 | 7 | Summary: API for automating and comparing work by human(?) workers. 8 | 9 | How it works: 10 | 11 | 1. Supervisor submits an image, and a list of questions about an image. A 12 | minimum number of workers, and a bitcoin reward, is specified. 13 | 14 | 2. Workers download the image, answer the question(s), submit results. 15 | 16 | 3. API collects work. When X workers have submitted answers, they are 17 | compared for matches. The most matches - most accurate - workers receive 18 | the reward. 19 | 20 | 21 | Status: Final compare-work-and-perform-payouts step is UNTESTED. All else works. 22 | 23 | 24 | First time setup 25 | ---------------- 26 | 27 | $ ./mkdb.sh 28 | 29 | Running the server 30 | ------------------ 31 | 32 | $ python3 turk-server.py 33 | 34 | 35 | 36 | API 37 | === 38 | 39 | 1. Get task to work on 40 | ---------------------- 41 | 42 | HTTP URI: GET /task/ 43 | 44 | Params: 45 | 46 | 47 | Result: 48 | 49 | 50 | Pricing: 51 | 52 | 53 | 54 | 55 | 2. Submit work to supervisor 56 | ---------------------------- 57 | HTTP URI: POST /task 58 | 59 | Params: 60 | 61 | In HTTP body, a application/json document containing the following keys: 62 | 63 | 64 | Result: 65 | 66 | 67 | Pricing: 68 | 69 | 70 | 71 | 72 | 3. Get list of tasks 73 | ---------------------- 74 | 75 | HTTP URI: GET /tasks.list 76 | 77 | Params: 78 | 79 | None 80 | 81 | Result: 82 | 83 | application/json document with the following data: 84 | (or an HTTP 4xx, 5xx error) 85 | 86 | Pricing: 87 | 88 | 89 | 90 | 91 | 4. Supervisor creates new task to receive work 92 | ---------------------------------------------- 93 | 94 | HTTP URI: POST /task.new 95 | 96 | Params: 97 | 98 | In HTTP body, a application/json document containing the following keys: 99 | 100 | 101 | Result: 102 | 103 | application/json document with the following data: true 104 | (or an HTTP 4xx, 5xx error) 105 | 106 | Pricing: 107 | 108 | 109 | 5. Register as new worker to receive tasks 110 | ------------------------------------------ 111 | 112 | HTTP URI: POST /worker.new 113 | 114 | Params: 115 | 116 | In HTTP body, a application/json document containing the following keys: 117 | 118 | 119 | Result: 120 | 121 | application/json document with the following data: true 122 | (or an HTTP 4xx, 5xx error) 123 | 124 | Pricing: 125 | 126 | 127 | 128 | -------------------------------------------------------------------------------- /turk/answers.json: -------------------------------------------------------------------------------- 1 | { 2 | "work_type": "image-question", 3 | "answers": [ 4 | "shorter" 5 | ] 6 | } 7 | -------------------------------------------------------------------------------- /turk/mkdb.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | FN=turk.db 4 | 5 | if [ -f $FN ] 6 | then 7 | echo Database $FN already exists. Will not overwrite 8 | exit 1 9 | fi 10 | sqlite3 $FN < turk.schema 11 | 12 | -------------------------------------------------------------------------------- /turk/srvdb.py: -------------------------------------------------------------------------------- 1 | 2 | import apsw 3 | import time 4 | import binascii 5 | import json 6 | 7 | class SrvDb(object): 8 | def __init__(self, filename): 9 | self.connection = apsw.Connection(filename) 10 | 11 | def worker_add(self, pkh, payout_addr): 12 | cursor = self.connection.cursor() 13 | 14 | cursor.execute("INSERT INTO workers VALUES(?, ?, 0, 0, 0)", (pkh, payout_addr)) 15 | 16 | def worker_inc_req(self, pkh): 17 | cursor = self.connection.cursor() 18 | 19 | cursor.execute("UPDATE workers SET tasks_req = tasks_req + 1 WHERE auth_pkh = ?", (pkh,)) 20 | 21 | def worker_inc_done(self, pkh): 22 | cursor = self.connection.cursor() 23 | 24 | cursor.execute("UPDATE workers SET tasks_done = tasks_done + 1 WHERE auth_pkh = ?", (pkh,)) 25 | 26 | def worker_get(self, pkh): 27 | cursor = self.connection.cursor() 28 | 29 | row = cursor.execute("SELECT * FROM workers WHERE auth_pkh = ?", (pkh,)).fetchone() 30 | if row is None: 31 | return None 32 | obj = { 33 | 'pkh': pkh, 34 | 'payout_addr': row[1], 35 | 'tasks_req': int(row[2]), 36 | 'tasks_done': int(row[3]), 37 | 'tasks_accepted': int(row[4]), 38 | } 39 | return obj 40 | 41 | def task_add(self, id, summary, pkh, image, image_ctype, template, min_workers, reward): 42 | cursor = self.connection.cursor() 43 | 44 | tstamp = int(time.time()) 45 | cursor.execute("INSERT INTO tasks VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?)", (id, summary, pkh, image, image_ctype, template, min_workers, reward, tstamp)) 46 | 47 | def task_close(self, id): 48 | cursor = self.connection.cursor() 49 | 50 | cursor.execute("UPDATE tasks SET time_closed=datetime('now') WHERE id = ?", (id,)) 51 | 52 | def tasks(self): 53 | cursor = self.connection.cursor() 54 | 55 | tasks = [] 56 | for row in cursor.execute("SELECT id,summary,min_workers,reward,time_create FROM tasks ORDER BY time_create DESC"): 57 | obj = { 58 | 'id': row[0], 59 | 'summary': row[1], 60 | 'min_workers': int(row[2]), 61 | 'reward': int(row[3]), 62 | 'time_create': int(row[4]), 63 | } 64 | tasks.append(obj) 65 | 66 | return tasks 67 | 68 | def task_get(self, id): 69 | cursor = self.connection.cursor() 70 | 71 | row = cursor.execute("SELECT * FROM tasks WHERE id = ?", (id,)).fetchone() 72 | if row is None: 73 | return None 74 | 75 | obj = { 76 | 'summary': row[1], 77 | 'pkh': row[2], 78 | 'image': binascii.hexlify(row[3]).decode('utf-8'), 79 | 'image_ctype': row[4], 80 | 'template': json.loads(row[5]), 81 | 'min_workers': int(row[6]), 82 | 'reward': int(row[7]), 83 | 'time_create': int(row[8]), 84 | } 85 | return obj 86 | 87 | def answer_add(self, id, pkh, answers): 88 | cursor = self.connection.cursor() 89 | 90 | tstamp = int(time.time()) 91 | cursor.execute("INSERT INTO answers VALUES(?, ?, ?, ?)", (id, pkh, answers, tstamp)) 92 | 93 | def answers_get(self, id): 94 | cursor = self.connection.cursor() 95 | 96 | answers = [] 97 | for row in cursor.execute("SELECT * FROM answers WHERE id = ?", (id,)): 98 | obj = { 99 | 'worker': row[1], 100 | 'answers': json.loads(row[2]), 101 | 'time_submit': int(row[3]), 102 | } 103 | answers.append(obj) 104 | 105 | return answers 106 | -------------------------------------------------------------------------------- /turk/turk-client.py: -------------------------------------------------------------------------------- 1 | 2 | # 3 | # Command line usage: 4 | # $ python3 turk-client.py --help 5 | # 6 | 7 | import json 8 | import os 9 | import sys 10 | import click 11 | import binascii 12 | import time 13 | import pprint 14 | 15 | import worktmp 16 | import util 17 | 18 | # import from the 21 Developer Library 19 | from two1.commands.config import Config 20 | from two1.lib.wallet import Wallet 21 | from two1.lib.bitrequests import BitTransferRequests 22 | 23 | pp = pprint.PrettyPrinter(indent=2) 24 | 25 | # set up bitrequest client for BitTransfer requests 26 | wallet = Wallet() 27 | username = Config().username 28 | requests = BitTransferRequests(wallet, username) 29 | 30 | TURKCLI_VERSION = '0.1' 31 | DEFAULT_ENDPOINT = 'http://localhost:12007/' 32 | 33 | @click.group() 34 | @click.option('--endpoint', '-e', 35 | default=DEFAULT_ENDPOINT, 36 | metavar='STRING', 37 | show_default=True, 38 | help='API endpoint URI') 39 | @click.option('--debug', '-d', 40 | is_flag=True, 41 | help='Turns on debugging messages.') 42 | @click.version_option(TURKCLI_VERSION) 43 | @click.pass_context 44 | def main(ctx, endpoint, debug): 45 | """ Command-line Interface for the Mechanical turk API service 46 | """ 47 | 48 | if ctx.obj is None: 49 | ctx.obj = {} 50 | 51 | ctx.obj['endpoint'] = endpoint 52 | 53 | @click.command(name='info') 54 | @click.pass_context 55 | def cmd_info(ctx): 56 | sel_url = ctx.obj['endpoint'] 57 | answer = requests.get(url=sel_url.format()) 58 | print(answer.text) 59 | 60 | @click.command(name='submit.task') 61 | @click.argument('id') 62 | @click.argument('pkh') 63 | @click.argument('answersfile', type=click.File('r')) 64 | @click.pass_context 65 | def cmd_task_submit(ctx, id, pkh, answersfile): 66 | try: 67 | answers_obj = json.load(answersfile) 68 | except: 69 | print("Unable to decode JSON work answers") 70 | sys.exit(1) 71 | 72 | tstamp = int(time.time()) 73 | req_obj = { 74 | 'pkh': pkh, 75 | 'id': id, 76 | 'tstamp': tstamp, 77 | 'answers': answers_obj, 78 | } 79 | 80 | body = json.dumps(req_obj) 81 | 82 | sig_str = wallet.sign_bitcoin_message(body, pkh) 83 | if not wallet.verify_bitcoin_message(body, sig_str, pkh): 84 | print("Error: cannot self-verify signed message") 85 | sys.exit(1) 86 | 87 | sel_url = ctx.obj['endpoint'] + 'task' 88 | headers = { 89 | 'Content-Type': 'application/json', 90 | 'X-Bitcoin-Sig': sig_str, 91 | } 92 | answer = requests.post(url=sel_url.format(), headers=headers, data=body) 93 | print(answer.text) 94 | 95 | @click.command(name='get.task') 96 | @click.argument('id') 97 | @click.argument('pkh') 98 | @click.pass_context 99 | def cmd_task_get(ctx, id, pkh): 100 | # Build, hash and sign pseudo-header 101 | tstamp = int(time.time()) 102 | msg = util.hash_task_phdr(id, pkh, tstamp) 103 | sig_str = wallet.sign_bitcoin_message(msg, pkh) 104 | if not wallet.verify_bitcoin_message(msg, sig_str, pkh): 105 | print("Error: cannot self-verify signed message") 106 | sys.exit(1) 107 | 108 | # Send request to endpoint 109 | sel_url = ctx.obj['endpoint'] + 'task/' + id 110 | headers = { 111 | 'X-Bitcoin-PKH': pkh, 112 | 'X-Bitcoin-Sig': sig_str, 113 | 'X-Timestamp': str(tstamp), 114 | } 115 | answer = requests.get(url=sel_url.format(), headers=headers) 116 | print(answer.text) 117 | 118 | @click.command(name='tasklist') 119 | @click.pass_context 120 | def cmd_task_list(ctx): 121 | sel_url = ctx.obj['endpoint'] + 'tasks.list' 122 | answer = requests.get(url=sel_url.format()) 123 | print(answer.text) 124 | 125 | @click.command(name='new.task') 126 | @click.argument('summary') 127 | @click.argument('imagefile', type=click.File('rb')) 128 | @click.argument('content_type') 129 | @click.argument('templatefile', type=click.File('r')) 130 | @click.argument('min_workers') 131 | @click.argument('reward') 132 | @click.pass_context 133 | def cmd_task_new(ctx, summary, imagefile, content_type, templatefile, min_workers, reward): 134 | try: 135 | template_obj = json.load(templatefile) 136 | except: 137 | print("Unable to decode JSON work template") 138 | sys.exit(1) 139 | 140 | wt = worktmp.WorkTemplate() 141 | wt.set(template_obj) 142 | if not wt.valid(): 143 | print("JSON work template not valid") 144 | sys.exit(1) 145 | 146 | auth_pubkey = wallet.get_message_signing_public_key() 147 | auth_pkh = auth_pubkey.address() 148 | 149 | print("Registering as supervisor pubkey " + auth_pkh) 150 | 151 | req_obj = { 152 | 'pkh': auth_pkh, 153 | 'summary': summary, 154 | 'image': binascii.hexlify(imagefile.read()).decode('utf-8'), 155 | 'image_ctype': content_type, 156 | 'template': template_obj, 157 | 'min_workers': int(min_workers), 158 | 'reward': int(reward), 159 | } 160 | 161 | body = json.dumps(req_obj) 162 | 163 | sel_url = ctx.obj['endpoint'] + 'task.new' 164 | headers = { 'Content-Type': 'application/json', } 165 | answer = requests.post(url=sel_url.format(), headers=headers, data=body) 166 | print(answer.text) 167 | 168 | @click.command(name='register') 169 | @click.pass_context 170 | def cmd_register(ctx): 171 | auth_pubkey = wallet.get_message_signing_public_key() 172 | auth_pkh = auth_pubkey.address() 173 | 174 | print("Registering as worker pubkey " + auth_pkh) 175 | 176 | req_obj = { 177 | 'pkh': auth_pkh, 178 | 'payout_addr': wallet.get_payout_address(), 179 | } 180 | 181 | body = json.dumps(req_obj) 182 | 183 | sel_url = ctx.obj['endpoint'] + 'worker.new' 184 | headers = { 'Content-Type': 'application/json', } 185 | answer = requests.post(url=sel_url.format(), headers=headers, data=body) 186 | print(answer.text) 187 | 188 | main.add_command(cmd_info) 189 | main.add_command(cmd_task_new) 190 | main.add_command(cmd_task_get) 191 | main.add_command(cmd_task_submit) 192 | main.add_command(cmd_task_list) 193 | main.add_command(cmd_register) 194 | 195 | if __name__ == "__main__": 196 | main() 197 | 198 | -------------------------------------------------------------------------------- /turk/turk-server.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import binascii 4 | import hashlib 5 | import srvdb 6 | import re 7 | import base58 8 | import ipaddress 9 | import pprint 10 | import time 11 | 12 | import worktmp 13 | import util 14 | 15 | # import flask web microframework 16 | from flask import Flask 17 | from flask import request 18 | from flask import abort 19 | 20 | # import from the 21 Developer Library 21 | from two1.lib.bitcoin.txn import Transaction 22 | from two1.lib.bitcoin.crypto import PublicKey 23 | from two1.lib.wallet import Wallet, exceptions 24 | from two1.lib.bitserv.flask import Payment 25 | 26 | USCENT=2801 27 | 28 | pp = pprint.PrettyPrinter(indent=2) 29 | 30 | db = srvdb.SrvDb('turk.db') 31 | 32 | app = Flask(__name__) 33 | wallet = Wallet() 34 | payment = Payment(app, wallet) 35 | 36 | def check_timestamp(tstamp): 37 | curtime = int(time.time()) 38 | min_time = min(tstamp, curtime) 39 | max_time = max(tstamp, curtime) 40 | time_diff = max_time - min_time 41 | if (time_diff > 15): 42 | return False 43 | return True 44 | 45 | @app.route('/task/') 46 | @payment.required(int(USCENT / 100)) 47 | def get_task(id): 48 | try: 49 | pkh_str = request.headers.get('X-Bitcoin-PKH') 50 | sig_str = request.headers.get('X-Bitcoin-Sig') 51 | 52 | tstamp = int(request.headers.get('X-Timestamp')) 53 | if not check_timestamp(tstamp): 54 | return ("Clock drift", 403, {'Content-Type':'text/plain'}) 55 | 56 | msg = util.hash_task_phdr(id, pkh_str, tstamp) 57 | if not wallet.verify_bitcoin_message(msg, sig_str, pkh_str): 58 | return ("Permission denied", 403, {'Content-Type':'text/plain'}) 59 | except: 60 | return ("Permission denied", 403, {'Content-Type':'text/plain'}) 61 | 62 | try: 63 | worker = db.worker_get(pkh_str) 64 | if worker is None: 65 | return ("Permission denied", 403, {'Content-Type':'text/plain'}) 66 | 67 | task = db.task_get(id) 68 | if task is None: 69 | abort(404) 70 | 71 | db.worker_inc_req(pkh_str) 72 | except: 73 | abort(500) 74 | 75 | body = json.dumps(task, indent=2) 76 | return (body, 200, { 77 | 'Content-length': len(body), 78 | 'Content-type': 'application/json', 79 | }) 80 | 81 | def process_work(id, task): 82 | answers = db.answers_get(id) 83 | if len(answers) < task.min_workers: 84 | return 85 | 86 | # TODO compare answer data more comprehensively 87 | scores = [] 88 | for ai in range(len(answers)): 89 | answer_obj = answers[ai] 90 | score = 0 91 | for i in range(len(answer)): 92 | if i == ai: 93 | continue 94 | compare_answer_obj = answers[i] 95 | match = True 96 | for j in range(len(answer_obj['answers'])): 97 | if answer_obj['answers'][j] != answer_obj['answers'][j]: 98 | match = False 99 | if match: 100 | score = score + 1 101 | scores.append(score) 102 | 103 | # close task 104 | db.task_close(id) 105 | 106 | # issue worker payouts 107 | worker_reward = int(task.reward / len(answers)) 108 | for score in scores: 109 | if score == 0: 110 | continue 111 | worker = db.worker_get(answer['worker']) 112 | if worker: 113 | wallet.sendto(worker['payout_addr'], worker_reward) 114 | 115 | @app.route('/task', methods=['POST']) 116 | @payment.required(USCENT * 1) 117 | def cmd_task_submit(): 118 | 119 | # Validate JSON body w/ API params 120 | try: 121 | body = request.data.decode('utf-8') 122 | in_obj = json.loads(body) 123 | except: 124 | return ("JSON Decode failed", 400, {'Content-Type':'text/plain'}) 125 | 126 | # Validate JSON object basics 127 | try: 128 | if (not 'pkh' in in_obj or 129 | not 'id' in in_obj or 130 | not 'tstamp' in in_obj or 131 | not 'answers' in in_obj): 132 | return ("Missing params", 400, {'Content-Type':'text/plain'}) 133 | 134 | sig_str = request.headers.get('X-Bitcoin-Sig') 135 | 136 | pkh = in_obj['pkh'] 137 | id = in_obj['id'] 138 | tstamp = int(in_obj['tstamp']) 139 | answers = in_obj['answers'] 140 | 141 | base58.b58decode_check(pkh) 142 | if not check_timestamp(tstamp): 143 | return ("Clock drift", 403, {'Content-Type':'text/plain'}) 144 | except: 145 | return ("JSON validation exception", 400, {'Content-Type':'text/plain'}) 146 | 147 | # Validate signature 148 | try: 149 | if not sig_str or not wallet.verify_bitcoin_message(body, sig_str, pkh): 150 | return ("Permission denied", 403, {'Content-Type':'text/plain'}) 151 | except: 152 | return ("Permission denied", 403, {'Content-Type':'text/plain'}) 153 | 154 | # Validate known worker and task 155 | try: 156 | worker = db.worker_get(pkh) 157 | if worker is None: 158 | return ("Permission denied", 403, {'Content-Type':'text/plain'}) 159 | 160 | task = db.task_get(id) 161 | if task is None: 162 | abort(404) 163 | except: 164 | abort(500) 165 | 166 | # Self-check work template 167 | wt = worktmp.WorkTemplate() 168 | wt.set(task['template']) 169 | if not wt.valid(): 170 | return ("JSON template self-validation failed", 500, {'Content-Type':'text/plain'}) 171 | 172 | # Validate answers match work template 173 | wt.set_answers(answers) 174 | if not wt.answers_valid(): 175 | return ("JSON answers validation failed", 400, {'Content-Type':'text/plain'}) 176 | 177 | # Store answer in db 178 | try: 179 | answers_json = json.dumps(answers) 180 | db.answer_add(id, pkh, answers_json) 181 | db.worker_inc_done(pkh) 182 | except: 183 | return ("Initial answer storage failed", 400, {'Content-Type':'text/plain'}) 184 | 185 | # If we have enough answers, compare work and payout 186 | process_work(id, task) 187 | 188 | body = json.dumps(True, indent=2) 189 | return (body, 200, { 190 | 'Content-length': len(body), 191 | 'Content-type': 'application/json', 192 | }) 193 | 194 | @app.route('/tasks.list') 195 | @payment.required(10) 196 | def get_tasks(): 197 | try: 198 | tasks = db.tasks() 199 | except: 200 | abort(500) 201 | 202 | body = json.dumps(tasks, indent=2) 203 | return (body, 200, { 204 | 'Content-length': len(body), 205 | 'Content-type': 'application/json', 206 | }) 207 | 208 | @app.route('/task.new', methods=['POST']) 209 | @payment.required(USCENT * 10) 210 | def cmd_task_new(): 211 | 212 | # Validate JSON body w/ API params 213 | try: 214 | body = request.data.decode('utf-8') 215 | in_obj = json.loads(body) 216 | except: 217 | return ("JSON Decode failed", 400, {'Content-Type':'text/plain'}) 218 | 219 | # Validate JSON object basics 220 | try: 221 | if (not 'pkh' in in_obj or 222 | not 'summary' in in_obj or 223 | not 'image' in in_obj or 224 | not 'image_ctype' in in_obj or 225 | not 'template' in in_obj or 226 | not 'min_workers' in in_obj or 227 | not 'reward' in in_obj): 228 | return ("Missing params", 400, {'Content-Type':'text/plain'}) 229 | 230 | pkh = in_obj['pkh'] 231 | summary = in_obj['summary'] 232 | image = binascii.unhexlify(in_obj['image']) 233 | image_ctype = in_obj['image_ctype'] 234 | template = in_obj['template'] 235 | min_workers = int(in_obj['min_workers']) 236 | reward = int(in_obj['reward']) 237 | 238 | base58.b58decode_check(pkh) 239 | except: 240 | return ("JSON validation exception", 400, {'Content-Type':'text/plain'}) 241 | 242 | # Check work template 243 | wt = worktmp.WorkTemplate() 244 | wt.set(template) 245 | if not wt.valid(): 246 | return ("JSON template validation failed", 400, {'Content-Type':'text/plain'}) 247 | 248 | # Generate unique id 249 | time_str = str(int(time.time())) 250 | md = hashlib.sha256() 251 | md.update(time_str.encode('utf-8')) 252 | md.update(body.encode('utf-8')) 253 | id = md.hexdigest() 254 | 255 | # Add worker to database. Rely on db to filter out dups. 256 | try: 257 | template_json = json.dumps(template) 258 | db.task_add(id, summary, pkh, image, image_ctype, template_json, min_workers, reward) 259 | except: 260 | return ("DB Exception - add task", 400, {'Content-Type':'text/plain'}) 261 | 262 | return (id, 200, { 263 | 'Content-length': len(body), 264 | 'Content-type': 'text/plain', 265 | }) 266 | 267 | @app.route('/worker.new', methods=['POST']) 268 | @payment.required(USCENT * 10) 269 | def cmd_worker_new(): 270 | 271 | # Validate JSON body w/ API params 272 | try: 273 | body = request.data.decode('utf-8') 274 | in_obj = json.loads(body) 275 | except: 276 | return ("JSON Decode failed", 400, {'Content-Type':'text/plain'}) 277 | 278 | # Validate JSON object basics 279 | try: 280 | if (not 'payout_addr' in in_obj or 281 | not 'pkh' in in_obj): 282 | return ("Missing name/pkh", 400, {'Content-Type':'text/plain'}) 283 | 284 | pkh = in_obj['pkh'] 285 | payout_addr = in_obj['payout_addr'] 286 | 287 | base58.b58decode_check(pkh) 288 | base58.b58decode_check(payout_addr) 289 | except: 290 | return ("JSON validation exception", 400, {'Content-Type':'text/plain'}) 291 | 292 | # Add worker to database. Rely on db to filter out dups. 293 | try: 294 | db.worker_add(pkh, payout_addr) 295 | except: 296 | return ("DB Exception - add worker", 400, {'Content-Type':'text/plain'}) 297 | 298 | body = json.dumps(True, indent=2) 299 | return (body, 200, { 300 | 'Content-length': len(body), 301 | 'Content-type': 'application/json', 302 | }) 303 | 304 | @app.route('/') 305 | def get_info(): 306 | info_obj = { 307 | "name": "turk", 308 | "version": 100, 309 | "pricing": { 310 | "/worker.new" : { 311 | "minimum" : (USCENT * 10) 312 | }, 313 | "/task.new" : { 314 | "minimum" : (USCENT * 10) 315 | }, 316 | "/task/" : { 317 | "minimum" : int(USCENT / 100) 318 | }, 319 | "/tasks.list" : { 320 | "minimum" : 10 321 | }, 322 | } 323 | 324 | } 325 | body = json.dumps(info_obj, indent=2) 326 | return (body, 200, { 327 | 'Content-length': len(body), 328 | 'Content-type': 'application/json', 329 | }) 330 | 331 | if __name__ == '__main__': 332 | app.run(host='0.0.0.0', port=12007, debug=True) 333 | 334 | -------------------------------------------------------------------------------- /turk/turk.schema: -------------------------------------------------------------------------------- 1 | CREATE TABLE workers ( 2 | auth_pkh TEXT PRIMARY KEY, 3 | payout_addr TEXT NOT NULL, 4 | tasks_req INTEGER NOT NULL, 5 | tasks_done INTEGER NOT NULL, 6 | tasks_accepted INTEGER NOT NULL 7 | ); 8 | 9 | CREATE TABLE tasks ( 10 | id TEXT PRIMARY KEY, 11 | summary TEXT NOT NULL, 12 | auth_pkh TEXT NOT NULL, 13 | image BLOB NOT NULL, 14 | image_ctype TEXT NOT NULL, 15 | template_json TEXT NOT NULL, 16 | min_workers INTEGER NOT NULL, 17 | reward INTEGER NOT NULL, 18 | time_create INTEGER NOT NULL, 19 | time_closed INTEGER 20 | ); 21 | 22 | CREATE TABLE answers ( 23 | id TEXT NOT NULL, 24 | worker_pkh TEXT NOT NULL, 25 | answers_json TEXT NOT NULL, 26 | time_submit INTEGER NOT NULL 27 | ); 28 | 29 | CREATE INDEX answers_idx ON answers (id); 30 | 31 | -------------------------------------------------------------------------------- /turk/util.py: -------------------------------------------------------------------------------- 1 | 2 | import hashlib 3 | 4 | def hash_task_phdr(id, pkh, tstamp): 5 | md = hashlib.sha256() 6 | md.update(id.encode('utf-8')) 7 | md.update(pkh.encode('utf-8')) 8 | md.update(str(tstamp).encode('utf-8')) 9 | return md.hexdigest() 10 | 11 | -------------------------------------------------------------------------------- /turk/worktemplate.json: -------------------------------------------------------------------------------- 1 | { 2 | "work_type": "image-question", 3 | "questions": [ 4 | "What is the name of the street?" 5 | ], 6 | "notes": "excluding suffixes such as St, Dr", 7 | "keywords": ["image","english"] 8 | } 9 | -------------------------------------------------------------------------------- /turk/worktmp.py: -------------------------------------------------------------------------------- 1 | 2 | import json 3 | 4 | class WorkTemplate(object): 5 | def __init__(self, str_in=None): 6 | self.obj = None 7 | if not str_in is None: 8 | self.obj = json.loads(str_in) 9 | 10 | def set(self, obj_in): 11 | self.obj = obj_in 12 | 13 | def set_answers(self, obj_in): 14 | self.answers = obj_in 15 | 16 | def load(self, str_in): 17 | self.obj = json.loads(str_in) 18 | 19 | def valid_questionlist(self): 20 | if (not 'questions' in self.obj or 21 | not isinstance(self.obj['questions'], list)): 22 | return False 23 | 24 | for q in self.obj['questions']: 25 | if not isinstance(q, str): 26 | return False 27 | 28 | if ('notes' in self.obj and 29 | not isinstance(self.obj['notes'], str)): 30 | return False 31 | 32 | return True 33 | 34 | def valid_answerlist(self): 35 | if (not 'answers' in self.answers or 36 | not isinstance(self.answers['answers'], list)): 37 | return False 38 | 39 | if len(self.obj['questions']) != len(self.answers['answers']): 40 | return False 41 | 42 | for a in self.answers['answers']: 43 | if not isinstance(a, str): 44 | return False 45 | 46 | return True 47 | 48 | def valid(self): 49 | if (self.obj is None or 50 | not 'work_type' in self.obj or 51 | not isinstance(self.obj['work_type'], str) or 52 | not 'keywords' in self.obj or 53 | not isinstance(self.obj['keywords'], list)): 54 | return False 55 | 56 | for kw in self.obj['keywords']: 57 | if not isinstance(kw, str): 58 | return False 59 | 60 | wt = self.obj['work_type'] 61 | if wt == 'image-question': 62 | return self.valid_questionlist() 63 | 64 | return False 65 | 66 | def answers_valid(self): 67 | if (self.answers is None or 68 | not 'work_type' in self.answers or 69 | not isinstance(self.answers['work_type'], str)): 70 | return False 71 | 72 | wt = self.answers['work_type'] 73 | if wt == 'image-question': 74 | return self.valid_answerlist() 75 | 76 | return False 77 | 78 | --------------------------------------------------------------------------------