├── .env.sample ├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md └── workflows │ └── add_issue_to_project.yml ├── .gitignore ├── DVNode.png ├── README.md ├── config └── teku_config.yaml ├── docker-compose.yml ├── grafana ├── dashboards.yml ├── dashboards │ └── single_node_dashboard.json ├── datasource.yml └── grafana.ini ├── jwt └── jwt.hex └── prometheus └── prometheus.yml /.env.sample: -------------------------------------------------------------------------------- 1 | # Replace the ENR variables below with participant ENRs to create cluster-definition.json for create dkg command. 2 | # NOTE: You cannot declare and use variables (like $ENR1) as the current file is an environment file not a script. 3 | CHARON_OPERATOR_ENRS=ENR1,ENR2,ENR3,ENR4 4 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: "\U0001F41EBug report" 3 | about: Report a bug or problem with running charon-distributed-validator-node 4 | labels: Bug 5 | --- 6 | 14 | 15 | # 🐞 Bug Report 16 | 17 | ### Description 18 | 19 | A clear and concise description of the problem... 20 | 21 | ### Has this worked before in a previous version? 22 | 23 | 24 | Yes, the previous version in which this bug was not present was: .... 25 | 26 | ## 🔬 Minimal Reproduction 27 | 28 | 31 | 32 | ## 🔥 Error 33 | 34 |

35 | 
36 | 
37 | 
38 | 
39 | 40 | 41 | ## 🌍 Your Environment 42 | 43 | **Operating System:** 44 | 45 |
46 |   
47 | 
48 |   
49 | 
50 | 51 | **What version of Charon are you running? (Which release)** 52 | 53 |
54 |   
55 | 
56 |   
57 | 
58 | 59 | **Anything else relevant (validator index / public key)?** 60 | 61 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: "\U0001F680Feature request" 3 | about: Suggest a feature for charon-distributed-validator-node 4 | labels: Enhancement 5 | --- 6 | 14 | 15 | # 🚀 Feature Request 16 | 17 | ## Problem to be solved 18 | 19 | A clear and concise description of the problem or missing capability... 20 | 21 | ## Proposed Solution 22 | 23 | If you have a solution in mind, please describe it. 24 | 25 | ## Describe alternatives you've considered 26 | 27 | Have you considered any alternative solutions or workarounds? 28 | 29 | ## Out of Scope 30 | 31 | What do you consider to be beyond the scope of this ticket? -------------------------------------------------------------------------------- /.github/workflows/add_issue_to_project.yml: -------------------------------------------------------------------------------- 1 | # This is a basic workflow adding every issue in this repo to the new Project Management Board 2 | name: Add Issue To Project 3 | 4 | # Controls when the workflow will run - new issues 5 | on: 6 | issues: 7 | types: 8 | - opened 9 | 10 | # This workflow contains a single job called "build" 11 | jobs: 12 | build: 13 | runs-on: ubuntu-latest 14 | # Steps 15 | steps: 16 | - name: Add Issue To Project 17 | uses: actions/add-to-project@v0.3.0 18 | with: 19 | # URL of the project to add issues to 20 | project-url: https://github.com/orgs/ObolNetwork/projects/7 21 | # A GitHub personal access token with write access to the project, need org admin's token with repo and project permissions, need to store the token outside the script if public 22 | github-token: ${{ secrets.GH_ORG_ADMIN_SECRET }} 23 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .env 2 | charon-enr-private-key 3 | validator_keys/ 4 | keystore-* 5 | deposit-data.json 6 | cluster-definition.json 7 | cluster-lock.json 8 | .DS_Store 9 | data/ -------------------------------------------------------------------------------- /DVNode.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NodesBlocks/charon-distributed-validator-node/002502aeeb152a6d5f923fd082a7a4faec1ffc67/DVNode.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Obol Logo](https://obol.tech/obolnetwork.png) 2 | 3 |

Charon Distributed Validator Node

4 | 5 | This repo contains the [docker-compose](https://docs.docker.com/compose/) files needed to run one node in a [charon](https://github.com/ObolNetwork/charon) [Distributed Validator Cluster](https://docs.obol.tech/docs/int/key-concepts#distributed-validator-cluster). 6 | 7 | A distributed validator node is a machine running: 8 | 9 | - An Ethereum Execution client 10 | - An Ethereum Consensus client 11 | - An Ethereum Distributed Validator client 12 | - An Ethereum Validator client 13 | 14 | ![Distributed Validator Node](DVNode.png) 15 | 16 | # Quickstart 17 | 18 | The following instructions aim to assist a group of users coordinating together to create a distributed validator cluster between them. Only one person needs to do [step 2](#step-2-leader-creates-the-dkg-configuration-file-and-distributes-it-to-everyone-else) and [step 5](#step-5-activate-the-deposit-data) in the quickstart process. 19 | 20 | ## Pre-requisites 21 | 22 | Ensure you have [docker](https://docs.docker.com/engine/install/) and [git](https://git-scm.com/downloads) installed. Also, make sure `docker` is running before executing the commands below. 23 | 24 | ## Step 1. Creating and backing up a private key for charon 25 | 26 | The first step of running a cluster is preparing for a distributed key generation ceremony. To do this everyone must create an [ENR](https://docs.obol.tech/docs/int/faq#what-is-an-enr) for their charon client. This ENR is a public/private key pair, and allows the other charon clients in the DKG to identify and connect to your node. 27 | 28 | ```sh 29 | # Clone this repo 30 | git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git 31 | 32 | # Change directory 33 | cd charon-distributed-validator-node 34 | 35 | # Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory 36 | docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.9.0 create enr 37 | ``` 38 | 39 | You should expect to see a console output like 40 | 41 | Created ENR private key: .charon/charon-enr-private-key 42 | enr:-JG4QGQpV4qYe32QFUAbY1UyGNtNcrVMip83cvJRhw1brMslPeyELIz3q6dsZ7GblVaCjL_8FKQhF6Syg-O_kIWztimGAYHY5EvPgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQKzMe_GFPpSqtnYl-mJr8uZAUtmkqccsAx7ojGmFy-FY4N0Y3CCDhqDdWRwgg4u 43 | 44 | ## ⚠️ Attention 45 | Please make sure to create a backup of the private key at `.charon/charon-enr-private-key`. Be careful not to commit it to git! **If you lose this file you won't be able to take part in the DKG ceremony.** 46 | 47 | If you are taking part in an organised Obol testnet, submit the created ENR public address (the console output starting with `enr:-...` not the contents of the private key file) to the appropriate typeform. 48 | 49 | ## Step 2. Leader creates the DKG configuration file and distributes it to everyone else 50 | 51 | One person, in the cluster or otherwise, will prepare the configuration file for the distributed key generation ceremony using the `charon create dkg` command. For the official Obol testnets, this step will be completed by an Obol core team member or the cluster captain and the definition file will be distributed to the cluster members for DKG completion. 52 | 53 | In future, step 1 and step 2 of this guide will use the [Obol Distributed Validator Launchpad](https://docs.obol.tech/docs/dvk/distributed_validator_launchpad) to facilitate and verify these files are created in an authenticated manner. 54 | 55 | ``` 56 | # Prepare an environment variable file 57 | cp .env.sample .env 58 | 59 | # Set the ENRs of all the operators participating in the DKG ceremony in the .env file variable CHARON_OPERATOR_ENRS 60 | 61 | # Set FEE_RECIPIENT_ADDRESS and WITHDRAWAL_ADDRESS to ETH1 addresses of your choice. 62 | # NAME can be any random string like "Obol Team" 63 | docker run --rm -v "$(pwd):/opt/charon" --env-file .env obolnetwork/charon:v0.9.0 create dkg --name=$NAME --fee-recipient-address=$FEE_RECIPIENT_ADDRESS --withdrawal-address=$WITHDRAWAL_ADDRESS 64 | 65 | # The above command prepares a DKG configuration file. 66 | ``` 67 | 68 | This command should output a file at `.charon/cluster-definition.json`. This file needs to be shared with the other operators in a cluster. 69 | 70 | ## Step 3. Run the DKG 71 | 72 | After receiving the `cluster-definition.json` file created by the leader, cluster members should ideally save it in the `.charon/` folder that was created during step 1, alternatively the `--definition-file` flag can override the default expected location for this file. 73 | 74 | Every cluster member then participates in the DKG ceremony. For Charon v1, this needs to happen synchronously between participants at an agreed time. 75 | 76 | ``` 77 | # Participate in DKG ceremony, this will create .charon/cluster-lock.json, .charon/deposit-data.json and .charon/validator_keys 78 | docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.9.0 dkg --p2p-bootnode-relay 79 | ``` 80 | 81 | Assuming the DKG is successful, a number of artefacts will be created in the `.charon` folder. These include: 82 | 83 | - A `deposit-data.json` file. This contains the information needed to activate the validator on the Ethereum network. 84 | - A `cluster-lock.json` file. This contains the information needed by charon to operate the distributed validator cluster with its peers. 85 | - A `validator_keys/` folder. This folder contains the private key shares and passwords for the created distributed validators. 86 | 87 | At this point you should make a backup of the `.charon/validator_keys` folder as replacing lost private keys is not straightforward at this point in charon's development. The `cluster-lock` and `deposit-data` files are identical for each operator and can be copied if lost. 88 | 89 | If taking part in the official Athena testnet, one cluster member will have to submit the `cluster-lock` and `deposit-data` files to the Obol Team, setting the stage for activation. 90 | 91 | ## Step 4. Start the Distributed Validator Cluster 92 | 93 | With the DKG ceremony over, the last phase before activation is to prepare your node for validating over the long term. This repo is configured to sync an execution layer client (`geth`) and a consensus layer client (`lighthouse`). 94 | 95 | Before completing these instructions, you should assign a static local IP address to your device (extending the DHCP reservation indefinitely or removing the device from the DCHP pool entirely if you prefer), and port forward the TCP protocol on the public port `:3610` on your router to your device's local IP address on the same port. This step is different for every person's home internet, and can be complicated by the presence of dynamic public IP addresses. We are currently working on making this as easy as possible, but for the time being, a distributed validator cluster isn't going to work very resiliently if all charon nodes cannot talk directly to one another and instead need to have an intermediary node forwarding traffic to them. 96 | 97 | **Caution**: If you manually update `docker-compose` to mount `lighthouse` from your locally synced `~/.lighthouse`, the whole chain database may get deleted. It'd be best not to manually update as `lighthouse` checkpoint-syncs so the syncing doesn't take much time. 98 | 99 | **NOTE**: If you have a `geth` node already synced, you can simply copy over the directory. For ex: `cp -r ~/.ethereum/goerli data/geth`. This makes everything faster since you start from a synced geth node. 100 | 101 | ``` 102 | # Delete lighthouse data if it exists 103 | rm -r ./data/lighthouse 104 | 105 | # Spin up a Distributed Validator Node with a Validator Client 106 | docker-compose up 107 | 108 | # Open Grafana dashboard 109 | open http://localhost:3000/d/singlenode/ 110 | ``` 111 | 112 | You should use the grafana dashboard to infer whether your cluster is healthy. In particular you should check: 113 | 114 | - That your charon client can connect to the configured beacon client. 115 | - That your charon client can connect to all peers 116 | 117 | You might notice that there are logs indicating that a validator cannot be found and that APIs are returning 404. This is to be expected at this point, as the validator public keys listed in the lock file have not been deposited and acknowledged on the consensus layer yet (usually ~16 hours after the deposit is made). 118 | 119 | To turn off your node after checking the health of the cluster you can run: 120 | 121 | ``` 122 | # Shut down the currently running distributed validator node 123 | docker-compose down 124 | ``` 125 | 126 | ## Step 5. Activate the deposit data 127 | 128 | If you and your team have gotten to this phase of the quickstart, and you have successfully created a distributed validator together, and you have connected all of your charon clients together such that the monitoring indicates that they are all healthy and ready to operate, one person may process to activate this deposit data with the existing [staking launchpad](https://prater.launchpad.ethereum.org/). 129 | 130 | This process can take a minimum of 16 hours, with the maximum time to activation being dictated by the length of the activation queue, which can be weeks. You can leave your distributed validator cluster offline until closer to the activation period if you would prefer. You can also use this time to improve and harden your monitoring and alerting for the cluster. 131 | 132 | If you have gotten this far through the process, and whether you succeed or fail at running the distributed validator successfully on the testnet, we would like to hear your feedback on the process and where you encountered difficulties. Please open issues in either this repo if the problem is deployment related, or the [charon](https://github.com/ObolNetwork/charon) repo if the issue is directly related to the client. 133 | 134 | ## Step 6. Add Central Monitoring Token 135 | 136 | You will be provided with a Central Monitoring Token used to push distributed validator metrics to our central prometheus service to monitor, analyze and improve your cluster's performance. The token needs to be added in prometheus/prometheus.yml replacing `$PROM_REMOTE_WRITE_TOKEN`. The token will look like: 137 | `eyJtZXNzYWdlIjoiSldUIFJ1bGVzISIsImlhdCI6MTQ1OTQ0ODExOSwiZXhwIjoxNDU5NDU0NTE5fQ`. Final prometheus/prometheus.yml would look something like: 138 | ``` 139 | global: 140 | scrape_interval: 12s # Set the scrape interval to every 12 seconds. Default is every 1 minute. 141 | evaluation_interval: 12s # Evaluate rules every 12 seconds. The default is every 1 minute. 142 | 143 | remote_write: 144 | - url: https://prometheus-prod-10-prod-us-central-0.grafana.net/api/prom/push 145 | authorization: 146 | credentials: 436764:eyJtZXNzYWdlIjoiSldUIFJ1bGVzISIsImlhdCI6MTQ1OTQ0ODExOSwiZXhwIjoxNDU5NDU0NTE5fQ 147 | name: obol-prom 148 | 149 | scrape_configs: 150 | - job_name: 'charon' 151 | static_configs: 152 | - targets: ['charon:3620'] 153 | - job_name: 'teku' 154 | static_configs: 155 | - targets: ['teku:8008'] 156 | - job_name: 'node-exporter' 157 | static_configs: 158 | - targets: ['node-exporter:9100'] 159 | ``` 160 | 161 | Thanks for trying our quickstart guide! 162 | 163 | # Project Status 164 | 165 | It is still early days for the Obol Network and everything is under active development. 166 | It is NOT ready for mainnet. 167 | Keep checking in for updates, [here](https://github.com/ObolNetwork/charon/#supported-consensus-layer-clients) is the latest on charon's supported clients and duties. 168 | 169 | # FAQs: 170 | 1. How do I get my ENR if I want to generate it again? 171 | - `cd` to the directory where your private keys are located (ex: `cd /path/to/charon/enr/private/key`) 172 | - Run `docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.9.0 enr`. This prints the ENR on your screen. 173 | - **Please note that this ENR is not the same as the one generated when you created it for the first time**. This is because the process of generating ENRs includes the current timestamp. 174 | 175 | 2. What do I do if lose my `charon-enr-private-key`? 176 | - For now, ENR rotation/replacement is not supported, it will be supported in a future release. 177 | - Therefore, it's advised to always keep a backup of your `private-key` in a secure location (ex: cloud storage, USB Flash drive etc.) 178 | 179 | 3. I have run the command in `Step 1` but I can't find the keys anywhere. 180 | - The `charon-enr-private-key` is generated inside a hidden folder `.charon`. 181 | - To view it, run `ls -al` in your terminal. 182 | - You can then copy the key to your `~/Downloads` folder for easy access by running `cp .charon/charon-enr-private-key ~/Downloads`. This step maybe a bit different for windows. 183 | - Else, if you are on `macOS`, press `Cmd + Shift + . ` to view the `.charon` folder in the `finder` application. 184 | 185 | 4. Why does Teku throw a keystore file error? 186 | - Teku sometimes logs an error which looks like: 187 | `Keystore file /opt/charon/validator_keys/keystore-0.json.lock already in use.` 188 | - This can be solved by deleting the file(s) ending with `.lock` in the folder `.charon/validator_keys`. 189 | - It is caused by an unsafe shut down of Teku (usually by double pressing Ctrl+C to shutdown containers faster). 190 | 191 | 5. How to fix the grafana dashboard? 192 | - Sometimes, grafana dashboard doesn't load any data first time around 193 | - You can solve this by following the steps below: 194 | - Click the Wheel Icon > Datasources 195 | - Click prometheus 196 | - Change the "Access" field from `Server (default)` to `Browser`. Press "Save & Test". It should fail. 197 | - Change the "Access" field back to `Server (default)` and press "Save & Test". You should be presented with a green success icon saying "Data source is working" and you can return to the dashboard page. 198 | 199 | 6. How to fix `permission denied` errors? 200 | - Permission denied errors can come up in a variety of manners, particularly on Linux and WSL for Windows systems. 201 | - In the interest of security, the charon docker image runs as a non-root user, and this user often does not have the permissions to write in the directory you have checked out the code to. 202 | - This can be generally be fixed with some of the following: 203 | - Running docker commands with `sudo`, if you haven't [setup docker to be run as a non-root](https://docs.docker.com/engine/install/linux-postinstall/) user. 204 | - Changing the permissions of the `.charon` folder with the commands: 205 | - `mkdir .charon` (if it doesn't already exist) 206 | - `sudo chmod -R 666 .charon` 207 | 208 | 7. I see a lot of errors after running `docker-compose up`. 209 | - It's because both `geth` and `lighthouse` start syncing and so there's connectivity issues among the containers. 210 | - Simply let the containers run for a while. You won't observe frequent errors when geth finishes syncing. 211 | -------------------------------------------------------------------------------- /config/teku_config.yaml: -------------------------------------------------------------------------------- 1 | beacon-node-api-endpoint: "http://charon:3600" 2 | metrics-enabled: true 3 | metrics-host-allowlist: "*" 4 | metrics-interface: "0.0.0.0" 5 | metrics-port: "8008" 6 | network: "goerli" 7 | validator-keys: "/opt/charon/validator_keys:/opt/charon/validator_keys" 8 | validators-graffiti: "Obol Distributed Validator" 9 | validators-proposer-default-fee-recipient: "0x0000000000000000000000000000000000000000" 10 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3.8" 2 | 3 | services: 4 | # _ _ 5 | # __ _ ___| |_| |__ 6 | # / _` |/ _ \ __| '_ \ 7 | # | (_| | __/ |_| | | | 8 | # \__, |\___|\__|_| |_| 9 | # |___/ 10 | 11 | geth: 12 | image: ethereum/client-go:v1.10.21 13 | ports: 14 | - 30303:30303/tcp 15 | - 30303:30303/udp 16 | - 8545:8545 17 | - 8551:8551/tcp 18 | - 8551:8551/udp 19 | command: | 20 | --goerli 21 | --http 22 | --http.addr=0.0.0.0 23 | --http.port=8545 24 | --http.vhosts="*" 25 | --http.api="db,eth,net,engine,rpc,web3" 26 | --authrpc.jwtsecret="/root/jwt/jwt.hex" 27 | --authrpc.addr=0.0.0.0 28 | --authrpc.port=8551 29 | --authrpc.vhosts="*" 30 | networks: [dvnode] 31 | volumes: 32 | - ./data/geth:/root/.ethereum 33 | - ./jwt:/root/jwt 34 | restart: on-failure 35 | 36 | # _ _ _ _ _ 37 | # | (_) __ _| |__ | |_| |__ ___ _ _ ___ ___ 38 | # | | |/ _` | '_ \| __| '_ \ / _ \| | | / __|/ _ \ 39 | # | | | (_| | | | | |_| | | | (_) | |_| \__ \ __/ 40 | # |_|_|\__, |_| |_|\__|_| |_|\___/ \__,_|___/\___| 41 | # |___/ 42 | 43 | lighthouse: 44 | image: sigp/lighthouse:v2.5.1 45 | depends_on: 46 | - geth 47 | ports: 48 | - 9000:9000/tcp 49 | - 9000:9000/udp 50 | - 5052:5052 51 | command: | 52 | lighthouse bn 53 | --network=goerli 54 | --purge-db 55 | --checkpoint-sync-url=https://goerli.checkpoint-sync.ethdevops.io 56 | --execution-endpoint=http://geth:8551 57 | --execution-jwt=/opt/jwt/jwt.hex 58 | --http 59 | --http-address=0.0.0.0 60 | --http-port=5052 61 | networks: [dvnode] 62 | volumes: 63 | - ./data/lighthouse:/opt/app/beacon 64 | - ./jwt:/opt/jwt 65 | restart: on-failure 66 | 67 | # _ 68 | # ___| |__ __ _ _ __ ___ _ __ 69 | # / __| '_ \ / _` | '__/ _ \| '_ \ 70 | # | (__| | | | (_| | | | (_) | | | | 71 | # \___|_| |_|\__,_|_| \___/|_| |_| 72 | 73 | charon: 74 | # Pegged charon version (update this for each release). 75 | # image: obolnetwork/charon:${CHARON_VERSION} 76 | image: ghcr.io/obolnetwork/charon:4453fee 77 | depends_on: 78 | - lighthouse 79 | environment: 80 | CHARON_BEACON_NODE_ENDPOINTS: http://lighthouse:5052 81 | CHARON_JAEGER_ADDRESS: jaeger:6831 82 | CHARON_VALIDATOR_API_ADDRESS: 0.0.0.0:3600 83 | CHARON_P2P_TCP_ADDRESS: 0.0.0.0:3610 84 | CHARON_MONITORING_ADDRESS: 0.0.0.0:3620 85 | CHARON_P2P_UDP_ADDRESS: 0.0.0.0:3630 86 | CHARON_P2P_BOOTNODE_RELAY: "true" 87 | CHARON_LOG_LEVEL: info 88 | CHARON_JAEGER_SERVICE: charon 89 | CHARON_P2P_EXTERNAL_HOSTNAME: charon 90 | networks: [dvnode] 91 | volumes: 92 | - .charon:/opt/charon/.charon 93 | restart: on-failure 94 | 95 | # _ _ 96 | # | |_ ___| | ___ _ 97 | # | __/ _ \ |/ / | | | 98 | # | || __/ <| |_| | 99 | # \__\___|_|\_\\__,_| 100 | 101 | teku: 102 | image: consensys/teku:22.8.0 103 | depends_on: 104 | - charon 105 | - lighthouse 106 | ports: 107 | - 8008:8008 108 | command: | 109 | validator-client 110 | --config-file "/opt/charon/teku/teku_config.yaml" 111 | networks: [dvnode] 112 | volumes: 113 | - ".charon/validator_keys:/opt/charon/validator_keys" 114 | - "./config:/opt/charon/teku" 115 | restart: on-failure 116 | 117 | # _ _ _ 118 | # _ __ ___ ___ _ __ (_) |_ ___ _ __(_)_ __ __ _ 119 | # | '_ ` _ \ / _ \| '_ \| | __/ _ \| '__| | '_ \ / _` | 120 | # | | | | | | (_) | | | | | || (_) | | | | | | | (_| | 121 | # |_| |_| |_|\___/|_| |_|_|\__\___/|_| |_|_| |_|\__, | 122 | # |___/ 123 | 124 | prometheus: 125 | image: prom/prometheus:latest 126 | ports: 127 | - "9090:9090" 128 | networks: [dvnode] 129 | volumes: 130 | - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml 131 | 132 | grafana: 133 | image: grafana/grafana:latest 134 | depends_on: [prometheus] 135 | ports: 136 | - "3000:3000" 137 | networks: [dvnode] 138 | volumes: 139 | - ./grafana/datasource.yml:/etc/grafana/provisioning/datasources/datasource.yml 140 | - ./grafana/dashboards.yml:/etc/grafana/provisioning/dashboards/datasource.yml 141 | - ./grafana/grafana.ini:/etc/grafana/grafana.ini:ro 142 | - ./grafana/dashboards:/etc/dashboards 143 | 144 | node-exporter: 145 | image: prom/node-exporter:latest 146 | ports: 147 | - "9100:9100" 148 | networks: [dvnode] 149 | 150 | jaeger: 151 | image: jaegertracing/all-in-one:latest 152 | ports: 153 | - "16686:16686" 154 | networks: [dvnode] 155 | 156 | networks: 157 | dvnode: 158 | -------------------------------------------------------------------------------- /grafana/dashboards.yml: -------------------------------------------------------------------------------- 1 | apiVersion: 1 2 | 3 | providers: 4 | - name: dashboards 5 | type: file 6 | updateIntervalSeconds: 30 7 | options: 8 | path: /etc/dashboards 9 | -------------------------------------------------------------------------------- /grafana/dashboards/single_node_dashboard.json: -------------------------------------------------------------------------------- 1 | { 2 | "annotations": { 3 | "list": [ 4 | { 5 | "$$hashKey": "object:139", 6 | "builtIn": 1, 7 | "datasource": { 8 | "type": "datasource", 9 | "uid": "grafana" 10 | }, 11 | "enable": true, 12 | "hide": true, 13 | "iconColor": "rgba(0, 211, 255, 1)", 14 | "name": "Annotations & Alerts", 15 | "target": { 16 | "limit": 100, 17 | "matchAny": false, 18 | "tags": [], 19 | "type": "dashboard" 20 | }, 21 | "type": "dashboard" 22 | } 23 | ] 24 | }, 25 | "editable": true, 26 | "fiscalYearStartMonth": 0, 27 | "graphTooltip": 0, 28 | "links": [], 29 | "liveNow": false, 30 | "panels": [ 31 | { 32 | "datasource": { 33 | "type": "prometheus", 34 | "uid": "PBFA97CFB590B2093" 35 | }, 36 | "fieldConfig": { 37 | "defaults": { 38 | "color": { 39 | "mode": "thresholds" 40 | }, 41 | "mappings": [], 42 | "thresholds": { 43 | "mode": "absolute", 44 | "steps": [ 45 | { 46 | "color": "green", 47 | "value": null 48 | }, 49 | { 50 | "color": "red", 51 | "value": 80 52 | } 53 | ] 54 | } 55 | }, 56 | "overrides": [] 57 | }, 58 | "gridPos": { 59 | "h": 1, 60 | "w": 24, 61 | "x": 0, 62 | "y": 0 63 | }, 64 | "id": 38, 65 | "options": { 66 | "colorMode": "none", 67 | "graphMode": "none", 68 | "justifyMode": "auto", 69 | "orientation": "auto", 70 | "reduceOptions": { 71 | "calcs": [ 72 | "lastNotNull" 73 | ], 74 | "fields": "", 75 | "values": false 76 | }, 77 | "textMode": "none" 78 | }, 79 | "pluginVersion": "9.0.2", 80 | "targets": [ 81 | { 82 | "datasource": { 83 | "type": "prometheus", 84 | "uid": "PBFA97CFB590B2093" 85 | }, 86 | "expr": "1", 87 | "refId": "A" 88 | } 89 | ], 90 | "title": "CHARON", 91 | "type": "stat" 92 | }, 93 | { 94 | "datasource": { 95 | "type": "prometheus", 96 | "uid": "PBFA97CFB590B2093" 97 | }, 98 | "fieldConfig": { 99 | "defaults": { 100 | "color": { 101 | "mode": "thresholds" 102 | }, 103 | "custom": { 104 | "align": "auto", 105 | "displayMode": "auto", 106 | "inspect": false 107 | }, 108 | "mappings": [], 109 | "thresholds": { 110 | "mode": "absolute", 111 | "steps": [ 112 | { 113 | "color": "green", 114 | "value": null 115 | }, 116 | { 117 | "color": "red", 118 | "value": 80 119 | } 120 | ] 121 | } 122 | }, 123 | "overrides": [] 124 | }, 125 | "gridPos": { 126 | "h": 4, 127 | "w": 6, 128 | "x": 0, 129 | "y": 1 130 | }, 131 | "id": 32, 132 | "options": { 133 | "footer": { 134 | "fields": "", 135 | "reducer": [ 136 | "sum" 137 | ], 138 | "show": false 139 | }, 140 | "showHeader": false 141 | }, 142 | "pluginVersion": "9.0.2", 143 | "targets": [ 144 | { 145 | "datasource": { 146 | "type": "prometheus", 147 | "uid": "PBFA97CFB590B2093" 148 | }, 149 | "expr": "(\n sum(app_git_commit) by (git_hash)\n)\n + on(job) group_left(version)\n(\n 0 * sum(app_version) by (version)\n)\n + on(job) group_left(lock_hash)\n(\n 0 * sum(app_lock_hash) by (lock_hash)\n)", 150 | "refId": "A" 151 | } 152 | ], 153 | "title": "Version Info", 154 | "transformations": [ 155 | { 156 | "id": "labelsToFields", 157 | "options": { 158 | "mode": "rows" 159 | } 160 | } 161 | ], 162 | "type": "table" 163 | }, 164 | { 165 | "datasource": { 166 | "type": "prometheus", 167 | "uid": "PBFA97CFB590B2093" 168 | }, 169 | "fieldConfig": { 170 | "defaults": { 171 | "color": { 172 | "mode": "thresholds" 173 | }, 174 | "mappings": [ 175 | { 176 | "options": { 177 | "0": { 178 | "color": "red", 179 | "index": 0, 180 | "text": "Inactive" 181 | }, 182 | "1": { 183 | "color": "green", 184 | "index": 1, 185 | "text": "Active" 186 | } 187 | }, 188 | "type": "value" 189 | } 190 | ], 191 | "thresholds": { 192 | "mode": "absolute", 193 | "steps": [ 194 | { 195 | "color": "green", 196 | "value": null 197 | }, 198 | { 199 | "color": "red", 200 | "value": 80 201 | } 202 | ] 203 | } 204 | }, 205 | "overrides": [] 206 | }, 207 | "gridPos": { 208 | "h": 4, 209 | "w": 4, 210 | "x": 6, 211 | "y": 1 212 | }, 213 | "id": 42, 214 | "options": { 215 | "colorMode": "value", 216 | "graphMode": "none", 217 | "justifyMode": "auto", 218 | "orientation": "auto", 219 | "reduceOptions": { 220 | "calcs": [ 221 | "lastNotNull" 222 | ], 223 | "fields": "", 224 | "values": false 225 | }, 226 | "textMode": "auto" 227 | }, 228 | "pluginVersion": "9.0.2", 229 | "targets": [ 230 | { 231 | "datasource": { 232 | "type": "prometheus", 233 | "uid": "PBFA97CFB590B2093" 234 | }, 235 | "expr": "app_monitoring_readyz", 236 | "refId": "A" 237 | } 238 | ], 239 | "title": "Charon status", 240 | "type": "stat" 241 | }, 242 | { 243 | "datasource": { 244 | "type": "prometheus", 245 | "uid": "PBFA97CFB590B2093" 246 | }, 247 | "fieldConfig": { 248 | "defaults": { 249 | "color": { 250 | "mode": "continuous-YlBl" 251 | }, 252 | "decimals": 1, 253 | "mappings": [], 254 | "thresholds": { 255 | "mode": "absolute", 256 | "steps": [ 257 | { 258 | "color": "green", 259 | "value": null 260 | } 261 | ] 262 | }, 263 | "unit": "dthms" 264 | }, 265 | "overrides": [] 266 | }, 267 | "gridPos": { 268 | "h": 4, 269 | "w": 4, 270 | "x": 10, 271 | "y": 1 272 | }, 273 | "id": 6, 274 | "options": { 275 | "colorMode": "background", 276 | "graphMode": "none", 277 | "justifyMode": "center", 278 | "orientation": "auto", 279 | "reduceOptions": { 280 | "calcs": [ 281 | "lastNotNull" 282 | ], 283 | "fields": "", 284 | "values": false 285 | }, 286 | "textMode": "auto" 287 | }, 288 | "pluginVersion": "9.0.2", 289 | "targets": [ 290 | { 291 | "datasource": { 292 | "type": "prometheus", 293 | "uid": "PBFA97CFB590B2093" 294 | }, 295 | "editorMode": "code", 296 | "exemplar": false, 297 | "expr": "(time() - app_start_time_secs)", 298 | "instant": false, 299 | "interval": "", 300 | "legendFormat": "{{job}}", 301 | "range": true, 302 | "refId": "A" 303 | } 304 | ], 305 | "title": "Running Since", 306 | "transformations": [], 307 | "type": "stat" 308 | }, 309 | { 310 | "datasource": { 311 | "type": "prometheus", 312 | "uid": "PBFA97CFB590B2093" 313 | }, 314 | "fieldConfig": { 315 | "defaults": { 316 | "color": { 317 | "mode": "continuous-BlPu" 318 | }, 319 | "mappings": [ 320 | { 321 | "id": 0, 322 | "op": "=", 323 | "text": "N/A", 324 | "type": 1, 325 | "value": "null" 326 | } 327 | ], 328 | "thresholds": { 329 | "mode": "absolute", 330 | "steps": [ 331 | { 332 | "color": "green", 333 | "value": null 334 | } 335 | ] 336 | }, 337 | "unit": "none" 338 | }, 339 | "overrides": [] 340 | }, 341 | "gridPos": { 342 | "h": 4, 343 | "w": 5, 344 | "x": 14, 345 | "y": 1 346 | }, 347 | "id": 4, 348 | "links": [], 349 | "options": { 350 | "colorMode": "background", 351 | "graphMode": "none", 352 | "justifyMode": "auto", 353 | "orientation": "horizontal", 354 | "reduceOptions": { 355 | "calcs": [ 356 | "mean" 357 | ], 358 | "fields": "", 359 | "values": false 360 | }, 361 | "textMode": "auto" 362 | }, 363 | "pluginVersion": "9.0.2", 364 | "targets": [ 365 | { 366 | "expr": "core_scheduler_current_slot", 367 | "interval": "", 368 | "legendFormat": "{{job}}", 369 | "refId": "A" 370 | } 371 | ], 372 | "title": "Latest Slot", 373 | "type": "stat" 374 | }, 375 | { 376 | "datasource": { 377 | "type": "prometheus", 378 | "uid": "PBFA97CFB590B2093" 379 | }, 380 | "fieldConfig": { 381 | "defaults": { 382 | "color": { 383 | "mode": "thresholds" 384 | }, 385 | "mappings": [], 386 | "thresholds": { 387 | "mode": "absolute", 388 | "steps": [ 389 | { 390 | "color": "semi-dark-red", 391 | "value": null 392 | }, 393 | { 394 | "color": "semi-dark-orange", 395 | "value": 1 396 | }, 397 | { 398 | "color": "semi-dark-yellow", 399 | "value": 2 400 | }, 401 | { 402 | "color": "semi-dark-green", 403 | "value": 3 404 | } 405 | ] 406 | } 407 | }, 408 | "overrides": [] 409 | }, 410 | "gridPos": { 411 | "h": 4, 412 | "w": 5, 413 | "x": 19, 414 | "y": 1 415 | }, 416 | "id": 34, 417 | "options": { 418 | "colorMode": "background", 419 | "graphMode": "none", 420 | "justifyMode": "auto", 421 | "orientation": "auto", 422 | "reduceOptions": { 423 | "calcs": [ 424 | "lastNotNull" 425 | ], 426 | "fields": "", 427 | "values": false 428 | }, 429 | "textMode": "auto" 430 | }, 431 | "pluginVersion": "9.0.2", 432 | "targets": [ 433 | { 434 | "datasource": { 435 | "type": "prometheus", 436 | "uid": "PBFA97CFB590B2093" 437 | }, 438 | "expr": "sum(p2p_ping_success)", 439 | "refId": "A" 440 | } 441 | ], 442 | "title": "Number of connected peers", 443 | "type": "stat" 444 | }, 445 | { 446 | "datasource": { 447 | "type": "prometheus", 448 | "uid": "PBFA97CFB590B2093" 449 | }, 450 | "description": "", 451 | "fieldConfig": { 452 | "defaults": { 453 | "color": { 454 | "mode": "palette-classic" 455 | }, 456 | "custom": { 457 | "axisLabel": "", 458 | "axisPlacement": "auto", 459 | "barAlignment": 0, 460 | "drawStyle": "line", 461 | "fillOpacity": 0, 462 | "gradientMode": "none", 463 | "hideFrom": { 464 | "legend": false, 465 | "tooltip": false, 466 | "viz": false 467 | }, 468 | "lineInterpolation": "linear", 469 | "lineWidth": 2, 470 | "pointSize": 5, 471 | "scaleDistribution": { 472 | "type": "linear" 473 | }, 474 | "showPoints": "auto", 475 | "spanNulls": false, 476 | "stacking": { 477 | "group": "A", 478 | "mode": "none" 479 | }, 480 | "thresholdsStyle": { 481 | "mode": "off" 482 | } 483 | }, 484 | "links": [], 485 | "mappings": [], 486 | "thresholds": { 487 | "mode": "absolute", 488 | "steps": [ 489 | { 490 | "color": "green", 491 | "value": null 492 | }, 493 | { 494 | "color": "red", 495 | "value": 80 496 | } 497 | ] 498 | }, 499 | "unit": "s" 500 | }, 501 | "overrides": [] 502 | }, 503 | "gridPos": { 504 | "h": 9, 505 | "w": 8, 506 | "x": 0, 507 | "y": 5 508 | }, 509 | "id": 2, 510 | "options": { 511 | "legend": { 512 | "calcs": [], 513 | "displayMode": "list", 514 | "placement": "bottom" 515 | }, 516 | "tooltip": { 517 | "mode": "single", 518 | "sort": "none" 519 | } 520 | }, 521 | "pluginVersion": "9.0.4", 522 | "targets": [ 523 | { 524 | "datasource": { 525 | "type": "prometheus", 526 | "uid": "PBFA97CFB590B2093" 527 | }, 528 | "expr": "histogram_quantile(0.90, sum(rate(app_eth2_latency_seconds_bucket[30s])) by (le,endpoint)) ", 529 | "instant": false, 530 | "interval": "", 531 | "legendFormat": "{{endpoint}}", 532 | "refId": "A" 533 | } 534 | ], 535 | "title": "Beacon Node Latency (90%)", 536 | "type": "timeseries" 537 | }, 538 | { 539 | "datasource": { 540 | "type": "prometheus", 541 | "uid": "PBFA97CFB590B2093" 542 | }, 543 | "description": "Host machine's memory usage", 544 | "fieldConfig": { 545 | "defaults": { 546 | "color": { 547 | "mode": "palette-classic" 548 | }, 549 | "custom": { 550 | "axisLabel": "GB", 551 | "axisPlacement": "auto", 552 | "barAlignment": 0, 553 | "drawStyle": "line", 554 | "fillOpacity": 40, 555 | "gradientMode": "opacity", 556 | "hideFrom": { 557 | "legend": false, 558 | "tooltip": false, 559 | "viz": false 560 | }, 561 | "lineInterpolation": "linear", 562 | "lineWidth": 1, 563 | "pointSize": 5, 564 | "scaleDistribution": { 565 | "type": "linear" 566 | }, 567 | "showPoints": "never", 568 | "spanNulls": true, 569 | "stacking": { 570 | "group": "A", 571 | "mode": "normal" 572 | }, 573 | "thresholdsStyle": { 574 | "mode": "off" 575 | } 576 | }, 577 | "mappings": [], 578 | "max": 16000000000, 579 | "min": 0, 580 | "thresholds": { 581 | "mode": "absolute", 582 | "steps": [ 583 | { 584 | "color": "green", 585 | "value": null 586 | }, 587 | { 588 | "color": "red", 589 | "value": 80 590 | } 591 | ] 592 | }, 593 | "unit": "decbytes" 594 | }, 595 | "overrides": [] 596 | }, 597 | "gridPos": { 598 | "h": 9, 599 | "w": 9, 600 | "x": 8, 601 | "y": 5 602 | }, 603 | "id": 40, 604 | "options": { 605 | "legend": { 606 | "calcs": [], 607 | "displayMode": "list", 608 | "placement": "bottom" 609 | }, 610 | "tooltip": { 611 | "mode": "single", 612 | "sort": "none" 613 | } 614 | }, 615 | "targets": [ 616 | { 617 | "datasource": { 618 | "type": "prometheus", 619 | "uid": "PBFA97CFB590B2093" 620 | }, 621 | "editorMode": "code", 622 | "expr": "node_memory_MemTotal_bytes - node_memory_MemFree_bytes - node_memory_Buffers_bytes - node_memory_Cached_bytes - node_memory_SwapCached_bytes - node_memory_Slab_bytes - node_memory_PageTables_bytes - node_memory_VmallocUsed_bytes", 623 | "legendFormat": "apps", 624 | "range": true, 625 | "refId": "A" 626 | }, 627 | { 628 | "datasource": { 629 | "type": "prometheus", 630 | "uid": "PBFA97CFB590B2093" 631 | }, 632 | "editorMode": "code", 633 | "expr": "node_memory_Buffers_bytes", 634 | "hide": false, 635 | "legendFormat": "buffers", 636 | "range": true, 637 | "refId": "B" 638 | }, 639 | { 640 | "datasource": { 641 | "type": "prometheus", 642 | "uid": "PBFA97CFB590B2093" 643 | }, 644 | "editorMode": "code", 645 | "expr": "node_memory_Cached_bytes", 646 | "hide": false, 647 | "legendFormat": "cached", 648 | "range": true, 649 | "refId": "C" 650 | }, 651 | { 652 | "datasource": { 653 | "type": "prometheus", 654 | "uid": "PBFA97CFB590B2093" 655 | }, 656 | "editorMode": "code", 657 | "expr": "node_memory_MemFree_bytes", 658 | "hide": false, 659 | "legendFormat": "free", 660 | "range": true, 661 | "refId": "D" 662 | }, 663 | { 664 | "datasource": { 665 | "type": "prometheus", 666 | "uid": "PBFA97CFB590B2093" 667 | }, 668 | "editorMode": "code", 669 | "expr": "node_memory_Slab_bytes", 670 | "hide": false, 671 | "legendFormat": "slab", 672 | "range": true, 673 | "refId": "E" 674 | } 675 | ], 676 | "title": "Memory Usage", 677 | "type": "timeseries" 678 | }, 679 | { 680 | "datasource": { 681 | "type": "prometheus", 682 | "uid": "PBFA97CFB590B2093" 683 | }, 684 | "fieldConfig": { 685 | "defaults": { 686 | "color": { 687 | "mode": "palette-classic" 688 | }, 689 | "custom": { 690 | "axisLabel": "", 691 | "axisPlacement": "auto", 692 | "barAlignment": 0, 693 | "drawStyle": "line", 694 | "fillOpacity": 0, 695 | "gradientMode": "none", 696 | "hideFrom": { 697 | "legend": false, 698 | "tooltip": false, 699 | "viz": false 700 | }, 701 | "lineInterpolation": "linear", 702 | "lineWidth": 2, 703 | "pointSize": 5, 704 | "scaleDistribution": { 705 | "type": "linear" 706 | }, 707 | "showPoints": "auto", 708 | "spanNulls": false, 709 | "stacking": { 710 | "group": "A", 711 | "mode": "none" 712 | }, 713 | "thresholdsStyle": { 714 | "mode": "off" 715 | } 716 | }, 717 | "links": [], 718 | "mappings": [], 719 | "thresholds": { 720 | "mode": "absolute", 721 | "steps": [ 722 | { 723 | "color": "green", 724 | "value": null 725 | }, 726 | { 727 | "color": "red", 728 | "value": 80 729 | } 730 | ] 731 | }, 732 | "unit": "s" 733 | }, 734 | "overrides": [] 735 | }, 736 | "gridPos": { 737 | "h": 9, 738 | "w": 7, 739 | "x": 17, 740 | "y": 5 741 | }, 742 | "id": 13, 743 | "maxDataPoints": 1296, 744 | "options": { 745 | "legend": { 746 | "calcs": [], 747 | "displayMode": "list", 748 | "placement": "bottom" 749 | }, 750 | "tooltip": { 751 | "mode": "single", 752 | "sort": "none" 753 | } 754 | }, 755 | "pluginVersion": "9.0.4", 756 | "targets": [ 757 | { 758 | "datasource": { 759 | "type": "prometheus", 760 | "uid": "PBFA97CFB590B2093" 761 | }, 762 | "editorMode": "code", 763 | "exemplar": true, 764 | "expr": "histogram_quantile(0.90, sum(rate(p2p_ping_latency_secs_bucket[30s])) by (le,peer)) ", 765 | "interval": "", 766 | "legendFormat": "{{peer}}", 767 | "range": true, 768 | "refId": "A" 769 | } 770 | ], 771 | "title": "Peer ping latency (90%)", 772 | "type": "timeseries" 773 | }, 774 | { 775 | "datasource": { 776 | "type": "prometheus", 777 | "uid": "PBFA97CFB590B2093" 778 | }, 779 | "fieldConfig": { 780 | "defaults": { 781 | "color": { 782 | "mode": "thresholds" 783 | }, 784 | "mappings": [], 785 | "thresholds": { 786 | "mode": "absolute", 787 | "steps": [ 788 | { 789 | "color": "green", 790 | "value": null 791 | }, 792 | { 793 | "color": "red", 794 | "value": 80 795 | } 796 | ] 797 | } 798 | }, 799 | "overrides": [] 800 | }, 801 | "gridPos": { 802 | "h": 1, 803 | "w": 24, 804 | "x": 0, 805 | "y": 14 806 | }, 807 | "id": 36, 808 | "options": { 809 | "colorMode": "none", 810 | "graphMode": "none", 811 | "justifyMode": "auto", 812 | "orientation": "auto", 813 | "reduceOptions": { 814 | "calcs": [ 815 | "lastNotNull" 816 | ], 817 | "fields": "", 818 | "values": false 819 | }, 820 | "textMode": "none" 821 | }, 822 | "pluginVersion": "9.0.2", 823 | "targets": [ 824 | { 825 | "datasource": { 826 | "type": "prometheus", 827 | "uid": "PBFA97CFB590B2093" 828 | }, 829 | "editorMode": "code", 830 | "expr": "1", 831 | "legendFormat": "__auto", 832 | "range": true, 833 | "refId": "A" 834 | } 835 | ], 836 | "title": "VALIDATOR", 837 | "type": "stat" 838 | }, 839 | { 840 | "datasource": { 841 | "type": "prometheus", 842 | "uid": "PBFA97CFB590B2093" 843 | }, 844 | "fieldConfig": { 845 | "defaults": { 846 | "color": { 847 | "mode": "thresholds" 848 | }, 849 | "mappings": [ 850 | { 851 | "options": { 852 | "match": "null", 853 | "result": { 854 | "text": "N/A" 855 | } 856 | }, 857 | "type": "special" 858 | } 859 | ], 860 | "thresholds": { 861 | "mode": "absolute", 862 | "steps": [ 863 | { 864 | "color": "#F2495C", 865 | "value": null 866 | }, 867 | { 868 | "color": "rgba(237, 129, 40, 0.89)", 869 | "value": 0 870 | }, 871 | { 872 | "color": "#299c46", 873 | "value": 1 874 | } 875 | ] 876 | }, 877 | "unit": "none" 878 | }, 879 | "overrides": [] 880 | }, 881 | "gridPos": { 882 | "h": 4, 883 | "w": 4, 884 | "x": 0, 885 | "y": 15 886 | }, 887 | "id": 25, 888 | "links": [], 889 | "maxDataPoints": 100, 890 | "options": { 891 | "colorMode": "none", 892 | "graphMode": "none", 893 | "justifyMode": "auto", 894 | "orientation": "horizontal", 895 | "reduceOptions": { 896 | "calcs": [ 897 | "lastNotNull" 898 | ], 899 | "fields": "", 900 | "values": false 901 | }, 902 | "textMode": "auto" 903 | }, 904 | "pluginVersion": "9.0.2", 905 | "targets": [ 906 | { 907 | "datasource": { 908 | "type": "prometheus", 909 | "uid": "PBFA97CFB590B2093" 910 | }, 911 | "expr": "validator_local_validator_count", 912 | "interval": "", 913 | "legendFormat": "", 914 | "refId": "A" 915 | } 916 | ], 917 | "title": "Number of validators", 918 | "type": "stat" 919 | }, 920 | { 921 | "datasource": { 922 | "type": "prometheus", 923 | "uid": "PBFA97CFB590B2093" 924 | }, 925 | "fieldConfig": { 926 | "defaults": { 927 | "color": { 928 | "mode": "continuous-blues" 929 | }, 930 | "mappings": [ 931 | { 932 | "options": { 933 | "match": "null", 934 | "result": { 935 | "text": "N/A" 936 | } 937 | }, 938 | "type": "special" 939 | } 940 | ], 941 | "thresholds": { 942 | "mode": "absolute", 943 | "steps": [ 944 | { 945 | "color": "green", 946 | "value": null 947 | }, 948 | { 949 | "color": "red", 950 | "value": 80 951 | } 952 | ] 953 | }, 954 | "unit": "none" 955 | }, 956 | "overrides": [] 957 | }, 958 | "gridPos": { 959 | "h": 4, 960 | "w": 4, 961 | "x": 4, 962 | "y": 15 963 | }, 964 | "id": 23, 965 | "links": [], 966 | "maxDataPoints": 100, 967 | "options": { 968 | "colorMode": "background", 969 | "graphMode": "none", 970 | "justifyMode": "auto", 971 | "orientation": "horizontal", 972 | "reduceOptions": { 973 | "calcs": [ 974 | "mean" 975 | ], 976 | "fields": "", 977 | "values": false 978 | }, 979 | "textMode": "auto" 980 | }, 981 | "pluginVersion": "9.0.2", 982 | "targets": [ 983 | { 984 | "expr": "validator_current_epoch", 985 | "interval": "", 986 | "legendFormat": "", 987 | "refId": "A" 988 | } 989 | ], 990 | "title": "Current Epoch", 991 | "type": "stat" 992 | }, 993 | { 994 | "datasource": { 995 | "type": "prometheus", 996 | "uid": "PBFA97CFB590B2093" 997 | }, 998 | "fieldConfig": { 999 | "defaults": { 1000 | "color": { 1001 | "fixedColor": "yellow", 1002 | "mode": "thresholds" 1003 | }, 1004 | "mappings": [], 1005 | "thresholds": { 1006 | "mode": "absolute", 1007 | "steps": [ 1008 | { 1009 | "color": "semi-dark-blue", 1010 | "value": null 1011 | }, 1012 | { 1013 | "color": "semi-dark-green", 1014 | "value": 0 1015 | } 1016 | ] 1017 | } 1018 | }, 1019 | "overrides": [] 1020 | }, 1021 | "gridPos": { 1022 | "h": 4, 1023 | "w": 4, 1024 | "x": 8, 1025 | "y": 15 1026 | }, 1027 | "id": 29, 1028 | "options": { 1029 | "colorMode": "background", 1030 | "graphMode": "none", 1031 | "justifyMode": "auto", 1032 | "orientation": "auto", 1033 | "reduceOptions": { 1034 | "calcs": [ 1035 | "last" 1036 | ], 1037 | "fields": "", 1038 | "values": false 1039 | }, 1040 | "textMode": "auto" 1041 | }, 1042 | "pluginVersion": "9.0.2", 1043 | "targets": [ 1044 | { 1045 | "expr": "validator_beacon_node_published_attestation_total", 1046 | "interval": "", 1047 | "legendFormat": "", 1048 | "refId": "A" 1049 | } 1050 | ], 1051 | "title": "Attestations Published", 1052 | "type": "stat" 1053 | }, 1054 | { 1055 | "datasource": { 1056 | "type": "prometheus", 1057 | "uid": "PBFA97CFB590B2093" 1058 | }, 1059 | "fieldConfig": { 1060 | "defaults": { 1061 | "color": { 1062 | "fixedColor": "dark-yellow", 1063 | "mode": "fixed" 1064 | }, 1065 | "mappings": [], 1066 | "thresholds": { 1067 | "mode": "absolute", 1068 | "steps": [ 1069 | { 1070 | "color": "red", 1071 | "value": null 1072 | }, 1073 | { 1074 | "color": "green", 1075 | "value": "" 1076 | } 1077 | ] 1078 | } 1079 | }, 1080 | "overrides": [] 1081 | }, 1082 | "gridPos": { 1083 | "h": 4, 1084 | "w": 4, 1085 | "x": 12, 1086 | "y": 15 1087 | }, 1088 | "id": 30, 1089 | "options": { 1090 | "colorMode": "background", 1091 | "graphMode": "none", 1092 | "justifyMode": "auto", 1093 | "orientation": "auto", 1094 | "reduceOptions": { 1095 | "calcs": [ 1096 | "last" 1097 | ], 1098 | "fields": "", 1099 | "values": false 1100 | }, 1101 | "textMode": "auto" 1102 | }, 1103 | "pluginVersion": "9.0.2", 1104 | "targets": [ 1105 | { 1106 | "expr": "validator_beacon_node_published_block_total", 1107 | "interval": "", 1108 | "legendFormat": "", 1109 | "refId": "A" 1110 | } 1111 | ], 1112 | "title": "Blocks Proposed", 1113 | "type": "stat" 1114 | }, 1115 | { 1116 | "datasource": { 1117 | "type": "prometheus", 1118 | "uid": "PBFA97CFB590B2093" 1119 | }, 1120 | "description": "Number of Validators with the given status.", 1121 | "fieldConfig": { 1122 | "defaults": { 1123 | "color": { 1124 | "mode": "fixed" 1125 | }, 1126 | "custom": { 1127 | "align": "center", 1128 | "displayMode": "auto", 1129 | "filterable": false, 1130 | "inspect": false 1131 | }, 1132 | "mappings": [], 1133 | "thresholds": { 1134 | "mode": "absolute", 1135 | "steps": [ 1136 | { 1137 | "color": "green", 1138 | "value": null 1139 | }, 1140 | { 1141 | "color": "red", 1142 | "value": 80 1143 | } 1144 | ] 1145 | }, 1146 | "unit": "none" 1147 | }, 1148 | "overrides": [] 1149 | }, 1150 | "gridPos": { 1151 | "h": 4, 1152 | "w": 8, 1153 | "x": 16, 1154 | "y": 15 1155 | }, 1156 | "id": 21, 1157 | "options": { 1158 | "footer": { 1159 | "fields": "", 1160 | "reducer": [ 1161 | "sum" 1162 | ], 1163 | "show": true 1164 | }, 1165 | "showHeader": false, 1166 | "sortBy": [] 1167 | }, 1168 | "pluginVersion": "9.0.2", 1169 | "targets": [ 1170 | { 1171 | "datasource": { 1172 | "type": "prometheus", 1173 | "uid": "PBFA97CFB590B2093" 1174 | }, 1175 | "expr": "validator_local_validator_counts > 0", 1176 | "interval": "", 1177 | "legendFormat": "{{status}}", 1178 | "refId": "A" 1179 | } 1180 | ], 1181 | "title": "Validator Statuses", 1182 | "transformations": [ 1183 | { 1184 | "id": "reduce", 1185 | "options": { 1186 | "labelsToFields": false, 1187 | "reducers": [ 1188 | "lastNotNull" 1189 | ] 1190 | } 1191 | } 1192 | ], 1193 | "type": "table" 1194 | }, 1195 | { 1196 | "datasource": { 1197 | "type": "prometheus", 1198 | "uid": "PBFA97CFB590B2093" 1199 | }, 1200 | "fieldConfig": { 1201 | "defaults": { 1202 | "color": { 1203 | "mode": "palette-classic" 1204 | }, 1205 | "custom": { 1206 | "axisLabel": "", 1207 | "axisPlacement": "auto", 1208 | "barAlignment": 0, 1209 | "drawStyle": "line", 1210 | "fillOpacity": 0, 1211 | "gradientMode": "none", 1212 | "hideFrom": { 1213 | "legend": false, 1214 | "tooltip": false, 1215 | "viz": false 1216 | }, 1217 | "lineInterpolation": "linear", 1218 | "lineWidth": 2, 1219 | "pointSize": 5, 1220 | "scaleDistribution": { 1221 | "type": "linear" 1222 | }, 1223 | "showPoints": "auto", 1224 | "spanNulls": false, 1225 | "stacking": { 1226 | "group": "A", 1227 | "mode": "none" 1228 | }, 1229 | "thresholdsStyle": { 1230 | "mode": "off" 1231 | } 1232 | }, 1233 | "mappings": [], 1234 | "thresholds": { 1235 | "mode": "absolute", 1236 | "steps": [ 1237 | { 1238 | "color": "green", 1239 | "value": null 1240 | }, 1241 | { 1242 | "color": "red", 1243 | "value": 80 1244 | } 1245 | ] 1246 | } 1247 | }, 1248 | "overrides": [] 1249 | }, 1250 | "gridPos": { 1251 | "h": 8, 1252 | "w": 8, 1253 | "x": 0, 1254 | "y": 19 1255 | }, 1256 | "id": 27, 1257 | "options": { 1258 | "legend": { 1259 | "calcs": [], 1260 | "displayMode": "list", 1261 | "placement": "bottom" 1262 | }, 1263 | "tooltip": { 1264 | "mode": "single", 1265 | "sort": "none" 1266 | } 1267 | }, 1268 | "pluginVersion": "8.5.5", 1269 | "targets": [ 1270 | { 1271 | "datasource": { 1272 | "type": "prometheus", 1273 | "uid": "PBFA97CFB590B2093" 1274 | }, 1275 | "editorMode": "code", 1276 | "exemplar": false, 1277 | "expr": "validator_duties_performed{result=\"failed\"}", 1278 | "format": "time_series", 1279 | "instant": false, 1280 | "interval": "", 1281 | "legendFormat": "{{type}}", 1282 | "range": true, 1283 | "refId": "A" 1284 | } 1285 | ], 1286 | "title": "Duties Failed", 1287 | "transformations": [], 1288 | "type": "timeseries" 1289 | } 1290 | ], 1291 | "refresh": "10s", 1292 | "schemaVersion": 36, 1293 | "style": "dark", 1294 | "tags": [], 1295 | "templating": { 1296 | "list": [] 1297 | }, 1298 | "time": { 1299 | "from": "now-5m", 1300 | "to": "now" 1301 | }, 1302 | "timepicker": { 1303 | "refresh_intervals": [ 1304 | "5s", 1305 | "10s", 1306 | "30s", 1307 | "1m", 1308 | "5m", 1309 | "15m", 1310 | "30m", 1311 | "1h", 1312 | "2h", 1313 | "1d" 1314 | ] 1315 | }, 1316 | "timezone": "", 1317 | "title": "Single Charon Node Dashboard", 1318 | "uid": "singlenode", 1319 | "version": 1, 1320 | "weekStart": "" 1321 | } -------------------------------------------------------------------------------- /grafana/datasource.yml: -------------------------------------------------------------------------------- 1 | # config file version 2 | apiVersion: 1 3 | 4 | # list of datasources that should be deleted from the database 5 | deleteDatasources: 6 | - name: Prometheus 7 | orgId: 1 8 | 9 | # list of datasources to insert/update depending 10 | # whats available in the database 11 | datasources: 12 | # name of the datasource. Required 13 | - name: Prometheus 14 | # datasource type. Required 15 | type: prometheus 16 | # org id. will default to orgId 1 if not specified 17 | orgId: 1 18 | # url 19 | url: http://prometheus:9090 20 | # database password, if used 21 | password: 22 | # database user, if used 23 | user: 24 | # database name, if used 25 | database: 26 | # enable/disable basic auth 27 | basicAuth: false 28 | # enable/disable with credentials headers 29 | withCredentials: 30 | # mark as default datasource. Max one per org 31 | isDefault: true 32 | # fields that will be converted to json and stored in json_data 33 | jsonData: 34 | graphiteVersion: "1.1" 35 | tlsAuth: false 36 | tlsAuthWithCACert: false 37 | # json object of data that will be encrypted. 38 | secureJsonData: 39 | tlsCACert: "..." 40 | tlsClientCert: "..." 41 | tlsClientKey: "..." 42 | version: 1 43 | # allow users to edit datasources from the UI. 44 | editable: true -------------------------------------------------------------------------------- /grafana/grafana.ini: -------------------------------------------------------------------------------- 1 | [auth.anonymous] 2 | enabled = true 3 | org_role = Admin -------------------------------------------------------------------------------- /jwt/jwt.hex: -------------------------------------------------------------------------------- 1 | 7074a5bf6bd6dae368fa598249d57edfcbccc67a1205b2c8d5d2fe7b800663aa -------------------------------------------------------------------------------- /prometheus/prometheus.yml: -------------------------------------------------------------------------------- 1 | global: 2 | scrape_interval: 12s # Set the scrape interval to every 12 seconds. Default is every 1 minute. 3 | evaluation_interval: 12s # Evaluate rules every 12 seconds. The default is every 1 minute. 4 | 5 | remote_write: 6 | - url: https://prometheus-prod-10-prod-us-central-0.grafana.net/api/prom/push 7 | authorization: 8 | credentials: 436764:$PROM_REMOTE_WRITE_TOKEN 9 | name: obol-prom 10 | 11 | scrape_configs: 12 | - job_name: 'charon' 13 | static_configs: 14 | - targets: ['charon:3620'] 15 | - job_name: 'teku' 16 | static_configs: 17 | - targets: ['teku:8008'] 18 | - job_name: 'node-exporter' 19 | static_configs: 20 | - targets: ['node-exporter:9100'] --------------------------------------------------------------------------------