├── .gitignore ├── README.md ├── rpcauth.py └── terraform ├── .terraform.lock.hcl ├── bitcoind-deployment.tf ├── default.tf ├── kubernetes.tf ├── project.tf └── variables.tf /.gitignore: -------------------------------------------------------------------------------- 1 | .terraform 2 | terraform.tfstate 3 | terraform.tfstate.backup 4 | terraform/input.tfvars 5 | *.out -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Bitcoin Full Node on GCP Kubernetes Cluster 2 | 3 | There are [many reasons](https://blog.keys.casa/why-run-a-node/) to 4 | run a Bitcoin full node. Running it in the cloud has a few extra 5 | benefits: 6 | 7 | - A fully indexed Bitcoin full node can take up to 300Gi disk space 8 | and non-trivial amount of CPU and memory, which is quite a bit for an 9 | everyday laptop which is also used for other purposes. 10 | - A cloud based Bitcoin full node can be accessed from anywhere 11 | through multiple devices, using standard tooling such as 12 | [gcloud](https://cloud.google.com/sdk/gcloud/) and 13 | [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/). 14 | - If you want to develop or test a Bitcoin application, it is easy to point 15 | local development environment to the Bitcoin full node through [port 16 | forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/) 17 | and then smoothly deploy it to the cloud later. 18 | 19 | Code contained this repository helps to run a Bitcoin full node in 20 | [Google Cloud Platform](https://cloud.google.com/) using [Kubernetes 21 | Engine](https://cloud.google.com/kubernetes-engine/). It can be used 22 | as a starting point to build and deploy Bitcoin powered applications. 23 | 24 | ## Prerequisit 25 | 26 | * [Terraform](https://www.terraform.io/) (>0.13) 27 | * [Google Cloud SDK](https://cloud.google.com/sdk/) 28 | 29 | ## Setup 30 | 31 | 1. If you have not setup GCP for your google account already, you can 32 | try to set it up [here](https://cloud.google.com/gcp/). For new 33 | users, Google offers 300 USD credits. To start the trial, Google 34 | requires registration of payment method. According to the [terms 35 | and conditions](https://cloud.google.com/terms/free-trial/) for GCP 36 | free trial, mining cryptocurrency is not allowed, but running a 37 | full node is *not* mining. 38 | 2. After GCP is setup, you can find your billing account id 39 | [here](https://console.cloud.google.com/billing) 40 | 3. Bitcoind RPC requires basic authentication. Use 41 | [rpcauth.py](rpcauth.py) to generate the `rpcauth` value, as shown below: 42 | 43 | ``` 44 | $ ./rpcauth.py username password 45 | String to be appended to bitcoin.conf: 46 | rpcauth=username:6436a49a71eabc219d79c38980be0a54$7bc37525a942213ee5ac1ae8cf7f802616b96bf2dd01a1e0121629be86c5d425 47 | ``` 48 | 49 | 4. Go into terraform directory, run 50 | 51 | ``` 52 | terraform init 53 | terraform plan -var 'project_id=YOUR_PROJECT_ID' \ 54 | -var 'project_billing_account=YOUR_BILLING_ACCOUNT_ID_FROM_STEP_2' \ 55 | -var 'bitcoin_rpcauth=YOUR_RPCAUTH_CREATED_FROM_STEP_3' \ 56 | -out plan.out 57 | 58 | ## Confirm the output and then run 59 | terraform apply "plan.out" 60 | ``` 61 | 62 | By default, machine type `e2-standard-2` is used, you can change it to 63 | something else using `kubernetes_node_pool_machine_type` 64 | variable. Similiarly, `project_name`, `region`, `zone` and 65 | `bitcoin_version` can be configured as well through terraform 66 | variables. 67 | 68 | If everything is executed successfully, a Bitcoin node should be 69 | running in the `bitcoin` namespace in the kubernetes cluster. 70 | 71 | ## Usage 72 | 73 | If you want to access the Bitcoin full node locally, you need to have simply run the 74 | following command: 75 | 76 | ``` 77 | kubectl port-forward svc/bitcoind 8332 78 | ``` 79 | Then you can interact with the bitcoind as if it is running locally on 80 | your machine 81 | 82 | ``` 83 | $ bitcoin-cli getrawtransaction e13d407be6085f1d0a0513f47577340f41f45d0b7250f3c554e8e05590113d7a true 84 | "txid": "e13d407be6085f1d0a0513f47577340f41f45d0b7250f3c554e8e05590113d7a", 85 | "hash": "7871c106d47a4d45784d427d990adc7e221aa6fade5d3c0bf39314812cea9ee6", 86 | "version": 1, 87 | "size": 371, 88 | "vsize": 206, 89 | "weight": 821, 90 | "locktime": 0, 91 | "vin": [ 92 | { 93 | 94 | .... 95 | ``` 96 | 97 | ## Cost 98 | 99 | ## Issues 100 | 101 | If you see something like this after you run `terraform apply` 102 | 103 | ``` 104 | Error: googleapi: Error 403: Kubernetes Engine API is not enabled for this project. Please ensure it is enabled in Google Cloud Console and try again: visit https://console.cloud.google.com/apis/api/container.googleapis.com/overview?projec 105 | t=YOUR_PROJECT_ID to do so., forbidden 106 | ``` 107 | 108 | You maybe need to run the following command to enable it. 109 | 110 | ``` 111 | gcloud services enable container.googleapis.com --project=YOUR_PROJECT_ID 112 | ``` -------------------------------------------------------------------------------- /rpcauth.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # Copyright (c) 2015-2018 The Bitcoin Core developers 3 | # Distributed under the MIT software license, see the accompanying 4 | # file COPYING or http://www.opensource.org/licenses/mit-license.php. 5 | 6 | from argparse import ArgumentParser 7 | from base64 import urlsafe_b64encode 8 | from binascii import hexlify 9 | from getpass import getpass 10 | from os import urandom 11 | 12 | import hmac 13 | 14 | def generate_salt(size): 15 | """Create size byte hex salt""" 16 | return hexlify(urandom(size)).decode() 17 | 18 | def generate_password(): 19 | """Create 32 byte b64 password""" 20 | return urlsafe_b64encode(urandom(32)).decode('utf-8') 21 | 22 | def password_to_hmac(salt, password): 23 | m = hmac.new(bytearray(salt, 'utf-8'), bytearray(password, 'utf-8'), 'SHA256') 24 | return m.hexdigest() 25 | 26 | def main(): 27 | parser = ArgumentParser(description='Create login credentials for a JSON-RPC user') 28 | parser.add_argument('username', help='the username for authentication') 29 | parser.add_argument('password', help='leave empty to generate a random password or specify "-" to prompt for password', nargs='?') 30 | args = parser.parse_args() 31 | 32 | if not args.password: 33 | args.password = generate_password() 34 | elif args.password == '-': 35 | args.password = getpass() 36 | 37 | # Create 16 byte hex salt 38 | salt = generate_salt(16) 39 | password_hmac = password_to_hmac(salt, args.password) 40 | 41 | print('String to be appended to bitcoin.conf:') 42 | print('rpcauth={0}:{1}${2}'.format(args.username, salt, password_hmac)) 43 | print('Your password:\n{0}'.format(args.password)) 44 | 45 | if __name__ == '__main__': 46 | main() 47 | -------------------------------------------------------------------------------- /terraform/.terraform.lock.hcl: -------------------------------------------------------------------------------- 1 | # This file is maintained automatically by "terraform init". 2 | # Manual edits may be lost in future updates. 3 | 4 | provider "registry.terraform.io/hashicorp/google" { 5 | version = "3.85.0" 6 | hashes = [ 7 | "h1:NFLAKeIbO6e3PaqsLIPqf55m7Gta0e8FRbyyeoHLL5A=", 8 | "zh:08d7856f7a57c2a6f4d49340fa4a7ba63d6d63a96669172d580eb4339ba2231c", 9 | "zh:0f70abdbbdea8d4617df6582ecb9626ba5b31aa1401326fd564844d62b2b11b6", 10 | "zh:2259e6518fbb0092d8f387749ea9add91362bb0f45c1d21829ff46f77b0f88e7", 11 | "zh:29cb85bacec2fc4a1bd9fae4cc065e16c9f1a10c1ab76e0131fa55dc9a369d01", 12 | "zh:4271ccfe6b4f45a1eea2586ecaf8e3fa6d93d187795a96642b7b979fe578f7cc", 13 | "zh:6cc85edb48b0d04599d4ba02b109ad9d3348342bc77ec09323c734a2c5e53bc9", 14 | "zh:995cd51b1ceb4f1e968f7a94c90258b5501325f1d7d6bbbc21d39acd04b55d51", 15 | "zh:a039acd66af0c3fd5b14bc9a153ba1db3ae24e6e766d2e3a435c2101e1b09e38", 16 | "zh:a6b2a9f70375fb93c3274ece7d3a90a6e54c9e24e123c6f219a3e17cb282fc57", 17 | "zh:bc2cc035f469b7e9efe907e009b9a8c28f1e4f23673b0039a50983e829fc49cf", 18 | "zh:eb50a38f75e57cfede9e99e03c8b9aba03b3c79d22d9650e868022fd1c130fdd", 19 | ] 20 | } 21 | 22 | provider "registry.terraform.io/hashicorp/google-beta" { 23 | version = "3.85.0" 24 | hashes = [ 25 | "h1:qU6zAqpXYhUQFVu+82j2D0idJ3H6i4g4oRbOx5NjXeE=", 26 | "zh:1435c576a6b5f7d301f318b1d18a879ccf02723c8ab37a8b7ca62593e284ff11", 27 | "zh:458b92562d5d9649653ec0a74c091215bf07ba32823e436e4e642349a3cd3b2b", 28 | "zh:4a75e9c8742dacc167f5da8a59aafe10c2b136efdbafea6ae941f273499e1655", 29 | "zh:62d11b3421b9cb7d44849f2e92717cdf0175bb9342e6a2653a40199228625c95", 30 | "zh:711abcb17367d0defa46d35fc0c2d36ec000cffe0e3b53b1b9315595584c2e2a", 31 | "zh:a44be49d11b0037c49959dac2fd44d417bc6066b312cbdc2a64cdc6a222bcb04", 32 | "zh:b9b0988e162239d0b180259b9bfd4097c7a72d17d9a118df7e3df262ebc7e93b", 33 | "zh:bb4ea355bad5662112775bca2e7b82d6c0831a7df677499e3a71512525450f5b", 34 | "zh:c3bf57b09225d1c86d7b2005dbfbdad97a6824f5a05eba22675b3eb14f21dd72", 35 | "zh:d0fee444fb02034e7c59d49254959fa354727a8bfd43355e3feffc04d762ac7f", 36 | "zh:ecb5ab606dbc2504f90600790ca84098c11ddab2521113214595d26ceac2823b", 37 | ] 38 | } 39 | 40 | provider "registry.terraform.io/hashicorp/kubernetes" { 41 | version = "2.5.0" 42 | hashes = [ 43 | "h1:gWcWwaZRr45P/3uejuRWOPfAOIX0tuyt2oYe+c7yJgY=", 44 | "zh:00aa1fe7c5a1872d2861a0c45dab7a927e445ed5ea49aed497d61cd739926b5d", 45 | "zh:12bd420f2535f1b02f03147764d91846dceda30e7c08a9f637ff1f26829f21c0", 46 | "zh:1e9c3385bb3921adc259e830e2f1c68458395f8bd6528a3850b76d9b949dd708", 47 | "zh:2c8f51546ae3f44044373cabfd8216b37a22e88a2acde85d35cbd6d0e918f519", 48 | "zh:3fad611703d62a07b0c8c39b23c8aeba31538248a941be71ce7556450253064e", 49 | "zh:796065da7245d7aa062426ceafd7cfd839a755f9ce0be487ea18649830042da3", 50 | "zh:84a20140556ddf9cec3a8287c902424fb76c2a8a11aa7d6ca1a1b28710695932", 51 | "zh:8ba5feb317f009516bf9b35a55ef60775d622c605821726b773dfe50b3674fce", 52 | "zh:997b3a4fd3e0794de1f6a92d751391c787114235849bfad7f909147b6c4515ea", 53 | "zh:b7cfe2ef65f9b64fdcc045c2e351c6ab31baf74690d3e11a03e349248d0e92f2", 54 | "zh:e2fe05d8637b0836ac70965032d9fe61d7a5cc6006d958c58ef033104060928e", 55 | ] 56 | } 57 | -------------------------------------------------------------------------------- /terraform/bitcoind-deployment.tf: -------------------------------------------------------------------------------- 1 | resource "kubernetes_namespace" "bitcoin" { 2 | metadata { 3 | name = "bitcoin" 4 | 5 | labels = { 6 | name = "bitcoin" 7 | } 8 | } 9 | 10 | depends_on = [ 11 | google_container_cluster.cluster 12 | ] 13 | } 14 | 15 | resource "kubernetes_deployment" "bitcoind" { 16 | metadata { 17 | name = "bitcoind" 18 | namespace = kubernetes_namespace.bitcoin.metadata.0.name 19 | labels = { 20 | app = "bitcoind" 21 | } 22 | } 23 | 24 | spec { 25 | replicas = 1 26 | 27 | selector { 28 | match_labels = { 29 | app = "bitcoind" 30 | } 31 | } 32 | 33 | template { 34 | metadata { 35 | name = "bitcoind" 36 | labels = { 37 | app = "bitcoind" 38 | } 39 | } 40 | 41 | spec { 42 | container { 43 | image = "nicolasdorier/docker-bitcoin:${var.bitcoin_version}" 44 | name = "bitcoind" 45 | image_pull_policy = "Always" 46 | 47 | env { 48 | name = "BITCOIN_EXTRA_ARGS" 49 | value = "rpcauth=${var.bitcoin_rpcauth}\ntxindex=1" 50 | } 51 | 52 | termination_message_path = "/dev/termination-log" 53 | 54 | volume_mount { 55 | mount_path = "/data" 56 | name = "bitcoin-blockchain-data" 57 | } 58 | } 59 | 60 | dns_policy = "ClusterFirst" 61 | restart_policy = "Always" 62 | termination_grace_period_seconds = 30 63 | 64 | volume { 65 | name = "bitcoin-blockchain-data" 66 | persistent_volume_claim { 67 | claim_name = "bitcoin-blockchain-pvc-claim" 68 | } 69 | } 70 | } 71 | } 72 | } 73 | } 74 | 75 | resource "kubernetes_service" "bitcoind" { 76 | metadata { 77 | name = "bitcoind" 78 | namespace = kubernetes_namespace.bitcoin.metadata.0.name 79 | } 80 | 81 | spec { 82 | selector = { 83 | app = "bitcoind" 84 | } 85 | 86 | port { 87 | port = 8332 88 | target_port = 8332 89 | } 90 | } 91 | } 92 | 93 | resource "kubernetes_persistent_volume_claim" "bitcoin-blockchain-pvc-claim" { 94 | metadata { 95 | name = "bitcoin-blockchain-pvc-claim" 96 | namespace = kubernetes_namespace.bitcoin.metadata.0.name 97 | } 98 | 99 | spec { 100 | access_modes = ["ReadWriteOnce"] 101 | 102 | resources { 103 | requests = { 104 | storage = "450Gi" 105 | } 106 | } 107 | 108 | storage_class_name = "standard" 109 | } 110 | } -------------------------------------------------------------------------------- /terraform/default.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_version = ">=0.13.0" 3 | } 4 | -------------------------------------------------------------------------------- /terraform/kubernetes.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | node_pool_ingress_tag = "cluster-ingress" 3 | } 4 | 5 | provider "kubernetes" { 6 | config_path = "~/.kube/config" 7 | } 8 | 9 | resource "google_container_cluster" "cluster" { 10 | provider = google-beta 11 | project = var.project_id 12 | name = "bitcoin-full-node" 13 | min_master_version = "latest" 14 | 15 | location = var.zone 16 | 17 | remove_default_node_pool = true 18 | initial_node_count = 1 19 | 20 | master_auth { 21 | # Disables basic auth 22 | username = "" 23 | password = "" 24 | 25 | client_certificate_config { 26 | issue_client_certificate = false 27 | } 28 | } 29 | 30 | enable_legacy_abac = false 31 | 32 | workload_identity_config { 33 | identity_namespace = "${var.project_id}.svc.id.goog" 34 | } 35 | 36 | depends_on = [ 37 | google_project_service.project_service_compute, 38 | google_project_service.project_service_iam, 39 | google_project_service.project_service_logging, 40 | google_project_service.project_service_monitoring, 41 | google_project_service.project_service_stackdriver 42 | ] 43 | } 44 | 45 | resource "google_container_node_pool" "bitcoin_nodes_01" { 46 | project = var.project_id 47 | name = "bitcoin-node-pool-01" 48 | cluster = google_container_cluster.cluster.name 49 | location = var.zone 50 | 51 | initial_node_count = 1 52 | 53 | autoscaling { 54 | min_node_count = 1 55 | max_node_count = 2 56 | } 57 | 58 | node_config { 59 | preemptible = true 60 | machine_type = var.kubernetes_node_pool_machine_type 61 | 62 | metadata = { 63 | disable-legacy-endpoints = "true" 64 | } 65 | 66 | labels = { 67 | preemptible-node = true 68 | } 69 | 70 | disk_size_gb = 25 71 | 72 | tags = [local.node_pool_ingress_tag] 73 | 74 | oauth_scopes = [ 75 | "https://www.googleapis.com/auth/devstorage.read_only", 76 | "https://www.googleapis.com/auth/logging.write", 77 | "https://www.googleapis.com/auth/monitoring", 78 | ] 79 | } 80 | 81 | management { 82 | auto_repair = true 83 | } 84 | 85 | depends_on = [ 86 | google_container_cluster.cluster 87 | ] 88 | } 89 | 90 | -------------------------------------------------------------------------------- /terraform/project.tf: -------------------------------------------------------------------------------- 1 | resource "google_project" "project" { 2 | name = var.project_name 3 | project_id = var.project_id 4 | billing_account = var.project_billing_account 5 | skip_delete = true 6 | } 7 | 8 | # Enable Google services on the project 9 | resource "google_project_service" "project_service_compute" { 10 | project = var.project_id 11 | service = "compute.googleapis.com" 12 | disable_dependent_services = true 13 | depends_on = [ 14 | google_project.project 15 | ] 16 | } 17 | 18 | resource "google_project_service" "project_service_iam" { 19 | project = var.project_id 20 | service = "iam.googleapis.com" 21 | disable_dependent_services = true 22 | depends_on = [ 23 | google_project.project 24 | ] 25 | } 26 | 27 | resource "google_project_service" "project_service_logging" { 28 | project = var.project_id 29 | service = "logging.googleapis.com" 30 | disable_dependent_services = true 31 | depends_on = [ 32 | google_project.project 33 | ] 34 | } 35 | 36 | resource "google_project_service" "project_service_monitoring" { 37 | project = var.project_id 38 | service = "monitoring.googleapis.com" 39 | disable_dependent_services = true 40 | depends_on = [ 41 | google_project.project 42 | ] 43 | } 44 | 45 | resource "google_project_service" "project_service_stackdriver" { 46 | project = var.project_id 47 | service = "stackdriver.googleapis.com" 48 | disable_dependent_services = true 49 | depends_on = [ 50 | google_project.project 51 | ] 52 | } 53 | 54 | resource "google_compute_project_default_network_tier" "default" { 55 | project = var.project_id 56 | network_tier = "STANDARD" 57 | } -------------------------------------------------------------------------------- /terraform/variables.tf: -------------------------------------------------------------------------------- 1 | variable "project_name" { 2 | type = string 3 | description = "Name of the Google project." 4 | default = "Bitcoin Full Node" 5 | } 6 | 7 | variable "project_id" { 8 | type = string 9 | description = "Id of the Google project for running bitcoin full node." 10 | } 11 | 12 | variable "region" { 13 | type = string 14 | description = "Google project region." 15 | default = "europe-west1" 16 | } 17 | 18 | variable "zone" { 19 | type = string 20 | description = "Google project zone." 21 | default = "europe-west1-d" 22 | } 23 | 24 | variable "kubernetes_node_pool_machine_type" { 25 | type = string 26 | description = "Machine type for Kubernetes node pool." 27 | default = "e2-standard-2" 28 | } 29 | 30 | variable "project_billing_account" { 31 | type = string 32 | description = "Billing account for the project." 33 | } 34 | 35 | variable "bitcoin_version" { 36 | type = string 37 | description = "Bitcoin core version. See https://hub.docker.com/r/nicolasdorier/docker-bitcoin for more details" 38 | default = "0.18.1" 39 | } 40 | 41 | variable "bitcoin_rpcauth" { 42 | type = string 43 | description = "Basic authentication of the bitcoin rpc, corresponding to the rpcauth option of bitcoind. Run `rpcpython.py username password` to generate the string here." 44 | } 45 | --------------------------------------------------------------------------------