├── LICENSE ├── README-en.md ├── README.md ├── audit-en.md ├── audit.md └── images ├── arch-en.png ├── arch-p1-en.png ├── arch-p1.png ├── arch-p2-en.png ├── arch-p2.png ├── arch-p3-en.png ├── arch-p3.png ├── arch-p4-en.png ├── arch-p4.png ├── arch.png └── eosasia-arch.png /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README-en.md: -------------------------------------------------------------------------------- 1 | # EOS BP Nodes Security Checklist 2 | 3 | > By SlowMist Security Team & Joinsec Team 4 | > 5 | > Thanks IMEOS.ONE, EOS Asia, EOS Store for help 6 | 7 | #### [中文版](./README.md) 8 | 9 | ### [The Block Producer Security Audit](./audit-en.md) 10 | 11 | ## Architecture Core Goals 12 | 13 | 1. Protect the normal communication and operation of BPs 14 | 1. Enhance the overall anti-attack capability of the main network 15 | 1. Protect node security 16 | 17 | ## Challenges 18 | 19 | 1. DDoS during the boot phase of main network launch 20 | 1. RPC abuse 21 | 1. Communication failure 22 | 23 | ## Architecture Core Design 24 | 25 | 1. BP server isolation 26 | 1. Multiple jumping nodes 27 | 1. Multi-link high availability 28 | 29 | ## Core Defense 30 | 31 | 1. RPC should be turned off by default. If it needs to be turned on, security measures such as port obfuscation, advanced DDoS protection must be employed. 32 | 1. BP Communication Multilink Design 33 | - The BP server should not be exposed on the public network. It communicates via jump servers，so there should be a large number of jump servers. 34 | - When there is a major failure of jump server/relay network, BP servers synchronize through a private network (1. Private Secret Node 2. Private VPN Link). 35 | 1. Protect BP Servers from being discovered and located in global scale network scans. Use most common ports such as 80, 443 or 22 as P2P/RPC ports instead of 9876/8888, which can effectively increase the attacker's cost to identify BPs. **Note: **setting the ports to totally random ports might also help. 36 | 37 | ## Recommended Overall Architecture 38 | 39 | ![Architecture](./images/arch-en.png) 40 | 41 | Architecture Overview: 42 | 43 | In order to deal with possible DDoS attacks, multiple links should be prepared for every node cluster. When under attack, a node cluster can communicate through a backup link at any time to ensure that the main network continues to produce. 44 | 45 | First of all, each node cluster should have at least 2 public full nodes and 2 private full nodes. The IPs of the public full nodes can be publicized to allow communications between Dapps and the main network. The IPs of the other two private full nodes are only available to other selected BPs and not exposed to the public, so as to avoid DDoS attacks. 46 | 47 | If the IPs of the private full nodes are somehow compromised and known to the attacker (for example, through scanning all IPv4 addresses of the entire network) and a DDoS attack is performed, the nodes will not be able to communicate with each other. In this case, a VPN can be established using encrypted tunnels to ensure uninterrupted normal communication among the 21 BPs. 48 | 49 | ### Architectural Design Description 50 | 51 | #### 1. Public Nodes (publicly available in the community node list) 52 | 53 | ![Architecture](./images/arch-p1-en.png) 54 | 55 | In the absence of an attack, the peripheral nodes communicate through published public nodes. 56 | 57 | #### 2. Private Nodes (privately available to other trusted BPs) 58 | 59 | ![Architecture](./images/arch-p2-en.png) 60 | 61 | When public nodes listed in the community node list are under attack and become unavailable, communications can be performed through private nodes. (Private nodes can be discovered by global scale network scans. So they are not completely safe) 62 | 63 | #### 3. Secret Node on VPN (the last secret communication channel among trusted nodes) 64 | 65 | ![Architecture](./images/arch-p3-en.png) 66 | 67 | When all nodes on the internet (public/private) are compromised, attacked and unavailable, the secret nodes will still be able to communicate through an isolated VPN among trusted BPs to ensure basic block production. 68 | 69 | #### 4. RPC API Node 70 | 71 | ![Architecture](./images/arch-p4-en.png) 72 | 73 | The full node serving RPC queries should be completely isolated from BP and well defended, to ensure that attacks on it will not affect BP. 74 | 75 | ## Security Reinforcement Plan 76 | 77 | ### 1. RPC Security 78 | 79 | #### 1.1 Block RPC 80 | 81 | When not mandatory, external access to RPC should be blocked. The `config.ini` configuration content is as follows: 82 | 83 | - Set server address to empty string `http-server-address =` 84 | - Comment out `https-server-address` 85 | 86 | #### 1.2 Turn on SSL 87 | 88 | When external access to RPC is mandatory, it is recommended to disable HTTP and serve on HTTPS only. The `config.ini` configuration content is as follows: 89 | 90 | - Comment out `http-server-address`, or set it to `127.0.0.1:8888` 91 | - Set `https-server-address` to `0.0.0.0:443` 92 | - Set `https-certificate-chain-file` and `https-private-key-file` to the paths of certificate chain file path and private key file respectively. Note that both files must be of PEM format. 93 | - Configure the certificate chain file and private key file permissions to 600 94 | 95 | #### 1.3 Disable `wallet_plugin` and `wallet_api_plugin` 96 | 97 | When allowing external access to RPC, `wallet_plugin` and `wallet_api_plugin` **MUST NOT** be loaded. If `wallet_plugin` and `wallet_api_plugin` are loaded, the attacker will be able to obtain private keys of unlocked accounts by calling `/v1/wallet/list_keys`. In addition, the attacker will also be able to block all normal unlock account requests by repeatedly calling `/v1/wallet/lock_all`. 98 | 99 | #### 1.4 Disable `producer_api_plugin` 100 | 101 | When allowing external access to RPC, `producer_api_plugin` **MUST NOT** be loaded. If `producer_api_plugin` is loaded, the attacker will be able to stop the BP from block production by calling `/v1/producer/pause`. 102 | 103 | ### 2. Configuration Security 104 | 105 | #### 2.1 Generate Active Multi-Signature Keys 106 | 107 | It is a great risk to put the public and private keys of the BP account in plain text in `config.ini`. It is recommended to generate an active multi-signature for this account to increase the difficulty of asset withdrawal. Examples are as follows: 108 | 109 | ``` 110 | Grant shrimp2 and shrimp3 the permissions of shrimp1 111 | 112 | cleos set account permission shrimp1 active '{"threshold":2,"keys":[{"key":"EOS6tjMy84SYqQEUcUXQeMLmeBo99aakJCbieu2TSMk2Agn6nTwmX","weight":2}],"accounts":[{"permission":{"actor":"shrimp2","permission":"active"},"weight":1},{"permission":{"actor":"shrimp3","permission":"active"},"weight":1}],"waits":[]}' owner 113 | ``` 114 | 115 | #### 2.2 Turn On Logging 116 | 117 | Configure `logconf` parameters in the configuration file to allow necessary RPC requests logging. 118 | 119 | #### 2.3 Optimize Docker Defaults 120 | 121 | The configuration in the official repository `https://github.com/EOSIO/eos/blob/master/Docker/config.ini` is too generic. ~~Plug-ins such as `wallet_api_plugin` are enabled by default, which is a great risk. ~~ (repo updated) It is recommended to modify the configuration before running docker build. We strongly recommend using LXD instead of Docker. 122 | 123 | #### 2.4 Optimize max-clients 124 | 125 | ~~Configure max-clients=0 in the configuration file to increase the number of concurrent connections on the P2P port to unlimited. (process is still single-threaded) At the same time, optimize system parameters (ulimit) and kernel parameters to enhance protection against malicious connection attacks.~~ 126 | 127 | The [commit](https://github.com/EOSIO/eos/commit/d7dff4f1df4a3ab462ef4a60a24ca2be1449df2d) fixed ["malicious P2P connections from same IP"](https://github.com/EOSIO/eos/issues/3497) issue and added a new default parameter `max_nodes_per_host = 1`. So `max-clients` no longer has to be 0 and can be set accordingly. 128 | 129 | #### 2.5 Non-root launch nodeos 130 | 131 | After the compilation is complete, create a normal user account and use this account to launch nodeos, avoid using root in order to reduce risks. 132 | 133 | #### 2.6 Listen random port 134 | 135 | - `p2p-listen-endpoint = ip:` 136 | - `http-server-address = ip:` 137 | 138 | Every time nodeos is started, it will randomly listen to a port. If it is externally serviced, it is recommended to use the configuration method in [Host Security](#4-host-security). 139 | 140 | ### 3. Network Security 141 | 142 | #### 3.1 Network Architecture 143 | 144 | In response to network blockage of main network in possible DDoS attacks, it is recommended to have a backup network configured in advance, for example, a private VPN network. For reference the architecture diagram of [EOS Asia](https://www.eosasia.one/) is as follows: 145 | 146 | ![EOS Asia Architecture](./images/eosasia-arch.png) 147 | 148 | #### 3.2 Cloud Service Providers 149 | 150 | Tested by the Slow Mist security team, Google Cloud and AWS have better performance against DDoS attacks. And after an attack, the mentioned service providers do not limit network access of affected servers, which allows quick service recovery. So they are recommended. (please be careful when choosing a cloud service provider. Many cloud service providers will directly shut down the server in the event of a DDoS attack without users’ consent) 151 | 152 | #### 3.3 DDoS Defense 153 | 154 | To cope with possible DDoS attacks, it is recommended that block producers be configured with DDoS protection services such as Cloudflare and AWS Shield in advanced. 155 | 156 | ### 4. Host Security 157 | 158 | * Protect BP Servers from being discovered and located in global scale network scans. Use most common ports such as 80, 443 or 22 as P2P/RPC ports instead of 9876/8888, which can effectively increase the attacker's cost to identify BPs. **Note:** setting the ports to totally random ports might also help. 159 | * Close irrelevant service ports and customize strict security rules on AWS or Google Cloud. 160 | * Change the default SSH port 22. Configure SSH to allow only private key authentication. Restrict SSH access to only authorized IP addresses of operations team. 161 | * In the case of ample budget, it is recommended to deploy excellent HIDS software to prevent the server from being hacked. 162 | 163 | ### 5. Threat Intelligence 164 | 165 | * It is strongly recommended to do collection, storage and analysis work of the related important logs, which include: the complete communication log between RPC and P2P port, the host system log, node related programs run log, etc. Storage and analysis can be done by building applications with open source programs like ELK(ElasticSearch, Logstash, Kibana), or by buying service from great business platforms. 166 | * If a cloud service from mature provider is used, the console has a number of threat intelligence related modules to refer to, and find exceptions. 167 | * When a major vulnerability or related attack information occurs at the node, the emergency plan shall be launched immediately, including disaster preparedness strategy and upgrade strategy. 168 | * Community intelligence should be shared. 169 | 170 | ## Contributors 171 | 172 | * HelloEOS 173 | * EOS Asia 174 | * EOSBIXIN 175 | * EOS Pacific 176 | * UnlimitedEOS 177 | * EOS Cannon 178 | * EOSpace 179 | * Blockgenic 180 | * EOSeco 181 | * EOSLaoMao 182 | * OneChain 183 | 184 | Thanks to these bps for participating in the node security test, they have accumulated valuable data for community safety. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # EOS bp nodes security checklist（EOS超级节点安全执行指南） 2 | 3 | > by 慢雾安全团队 & Joinsec Team 4 | > 5 | > 感谢 IMEOS.ONE, EOS Asia, EOS Store 倾力相助 6 | 7 | #### [English](./README-en.md) 8 | 9 | ### [超级节点安全审计方案](audit.md) 10 | 11 | ## 目录 12 | 13 | * [架构核心目标](#架构核心目标) 14 | * [面临的主要问题](#面临的主要问题) 15 | * [架构核心设计](#架构核心设计) 16 | * [核心防御](#核心防御) 17 | * [推荐总架构](#推荐总架构) 18 | * [架构各部分设计说明](#架构各部分设计说明) 19 | * [1. 公开节点（对外公开在社区节点列表里）](#1-公开节点对外公开在社区节点列表里) 20 | * [2. 私密节点（只对其他可信BP节点私密分享的通信节点）](#2-私密节点只对其他可信bp节点私密分享的通信节点) 21 | * [3. VPN 加密节点（各可信节点间最后的秘密的通信信道）](#3-vpn-加密节点各可信节点间最后的秘密的通信信道) 22 | * [4. RPC API 节点](#4-rpc-api-节点) 23 | * [安全加固方案](#安全加固方案) 24 | * [1. RPC 安全](#1-rpc-安全) 25 | * [1.1 屏蔽 RPC](#11-屏蔽-rpc) 26 | * [1.2 开启 SSL](#12-开启-ssl) 27 | * [1.3 禁用 wallet_plugin 和 wallet_api_plugin](#13-禁用-wallet_plugin-和-wallet_api_plugin) 28 | * [1.4 禁用producer_api_plugin](#14-禁用producer_api_plugin) 29 | * [2. 配置安全](#2-配置安全) 30 | * [2.1 生成 Active 多签密钥](#21-生成-active-多签密钥) 31 | * [2.2 开启日志记录](#22-开启日志记录) 32 | * [2.3 Docker 默认参数优化](#23-docker-默认参数优化) 33 | * [2.4 max-clients参数优化](#24-max-clients参数优化) 34 | * [2.5 非 root 启动 nodeos](#25-非-root-启动-nodeos) 35 | * [3. 网络安全](#3-网络安全) 36 | * [3.1 网络架构](#31-网络架构) 37 | * [3.2 云服务商](#32-云服务商) 38 | * [3.3 DDoS 防御](#33-ddos-防御) 39 | * [4. 主机安全](#4-主机安全) 40 | * [5. 威胁情报](#5-威胁情报) 41 | * [致谢](#致谢) 42 | 43 | ## 架构核心目标 44 | 45 | 1. 保护出块服务器正常通信与运行 46 | 2. 增强初始主网整体抗攻击能力 47 | 3. 保护节点安全 48 | 49 | ## 面临的主要问题 50 | 51 | 1. 对初始状态主网进行 DDoS 52 | 2. RPC 功能滥用 53 | 3. 通信故障 54 | 55 | ## 架构核心设计 56 | 57 | 1. BP 服务器隔离 58 | 2. 多跳转节点（小节点流量转发，大节点高防护） 59 | 3. 多链路高可用 60 | 61 | ## 核心防御 62 | 63 | 1. 默认关闭 RPC。必须打开时，混淆端口，并架设高防等保护 64 | 2. BP 通信多链路设计 65 | - BP 服务器不在公网上暴露，通过跳板服务器（跳板服务器数量要大）进行通信； 66 | - 在外网公布的跳板服务器大面积瘫痪时，通过私有网络（1.私有秘密节点 2.私有 VPN 链路）来同步区块 67 | 3. 防止全网扫描定位高防后的服务器，修改同步端口 9876（同理 RPC 的 8888）至全网最大存活数量的端口 80、443 或 22，这样可以有效抬高攻击者定位成本。 68 | 69 | ## 推荐总架构 70 | 71 | ![Architecture](./images/arch.png) 72 | 73 | 架构说明： 74 | 75 | 为了应对可能的 DDoS 攻击，节点应准备多条链路，在攻击到来后，可以随时通过备用链路进行通信，确保主网顺利启动，并持续出块。 76 | 77 | 首先，每个节点应至少准备 2 个公开 full node、2 个私密 full node，公开 full node 的 IP 可以对外公开，以供 Dapp 和主网正常通信。另 2 个私密 full node 的 IP 只告知被选中的其他超级节点，不对外公开，以避免同时遭遇 DDoS 攻击。 78 | 79 | 假如私密 full node 的 IP 也被攻击者获知（例如扫描全网所有 IPv4 地址）并遭遇 DDoS 攻击导致节点间无法正常通信，此时可通过 VPN 加密链路连接到虚拟内网，确保 21 个超级节点之间不间断的正常通信。 80 | 81 | ### 架构各部分设计说明 82 | 83 | #### 1. 公开节点（对外公开在社区节点列表里） 84 | 85 | ![Architecture](./images/arch-p1.png) 86 | 87 | 在没有攻击情况下，外围节点通过对外公布的公开节点进行通信。 88 | 89 | #### 2. 私密节点（只对其他可信BP节点私密分享的通信节点） 90 | 91 | ![Architecture](./images/arch-p2.png) 92 | 93 | 当攻击者通过公开的节点列表攻击公开节点造成公开节点不可用时，则可通过私密节点进行通信。（私密节点可被全网扫描发现，所以并不是完全安全） 94 | 95 | #### 3. VPN 加密节点（各可信节点间最后的秘密的通信信道） 96 | 97 | ![Architecture](./images/arch-p3.png) 98 | 99 | 当公网节点都被发现，并且攻击者进行攻击导致对公网 full node 服务器全部阻塞，最后则由私有 VPN 网络在隔离的虚拟内网内进行通信，保证最基础的出块正常。 100 | 101 | #### 4. RPC API 节点 102 | 103 | ![Architecture](./images/arch-p4.png) 104 | 105 | 查询用 RPC 所在 full node 与 BP 完全隔离并架设防御，保证外网对 RPC 的攻击不能影响到 BP。 106 | 107 | 108 | ## 安全加固方案 109 | 110 | ### 1. RPC 安全 111 | 112 | #### 1.1 屏蔽 RPC 113 | 114 | 如无必要，建议禁止 RPC 对外访问，`config.ini`配置内容如下： 115 | 116 | - 配置为空值`http-server-address =` 117 | - 注释`https-server-address` 118 | 119 | #### 1.2 开启 SSL 120 | 121 | 如果确实需要对外提供 RPC 服务，建议禁用 HTTP 协议，使用 HTTPS，`config.ini`配置内容如下： 122 | 123 | - 注释`http-server-address`，或者配置为`127.0.0.1:8888` 124 | - 配置`https-server-address`为`0.0.0.0:443` 125 | - 配置`https-certificate-chain-file` 和 `https-private-key-file` 为证书链文件路径和私钥文件路径，注意两个文件格式必须为 PEM 126 | - 配置证书链文件和私钥文件权限为 600 127 | 128 | #### 1.3 禁用 `wallet_plugin` 和 `wallet_api_plugin` 129 | 130 | 在对外提供 RPC 服务的场景下，**一定不要加载** `wallet_plugin` 和 `wallet_api_plugin`。如果加载了`wallet_plugin` 和 `wallet_api_plugin`，攻击者就可以通过 RPC API `/v1/wallet/list_keys` 获取已解锁账户的私钥。此外，攻击者还可以恶意循环调用`/v1/wallet/lock_all`使节点上的账户无法解锁。 131 | 132 | #### 1.4 禁用`producer_api_plugin` 133 | 134 | 在对外提供 RPC 服务的场景下，**一定不要加载** `producer_api_plugin `。如果加载了`producer_api_plugin `，攻击者就可以通过 RPC API `/v1/producer/pause` 远程控制节点停止生产。 135 | 136 | ### 2. 配置安全 137 | 138 | #### 2.1 生成 Active 多签密钥 139 | 140 | 由于超级节点账户的公私钥明文配置在`config.ini`中，存在较大的风险，建议对这个账户生成 Active 多签，提高资产转出门槛。举例如下： 141 | 142 | ``` 143 | 授予 shrimp2 和 shrimp3 拥有 shrimp1 的权限 144 | 145 | cleos set account permission shrimp1 active '{"threshold":2,"keys":[{"key":"EOS6tjMy84SYqQEUcUXQeMLmeBo99aakJCbieu2TSMk2Agn6nTwmX","weight":2}],"accounts":[{"permission":{"actor":"shrimp2","permission":"active"},"weight":1},{"permission":{"actor":"shrimp3","permission":"active"},"weight":1}],"waits":[]}' owner 146 | ``` 147 | 148 | #### 2.2 开启日志记录 149 | 150 | 在配置文件中配置`logconf`参数，记录必要的 RPC 请求日志。 151 | 152 | #### 2.3 Docker 默认参数优化 153 | 154 | 官方仓库`https://github.com/EOSIO/eos/blob/master/Docker/config.ini`中的配置过于宽泛，~~加载了`wallet_api_plugin`等插件，存在较大风险~~（官方已优化），建议在`docker build`之前修改配置。 155 | 156 | #### 2.4 `max-clients`参数优化 157 | 158 | ~~在配置文件中配置`max-clients = 0` 提升 P2P 端口并发连接数为无限制，同时优化`ulimit`系统参数和内核参数，增强恶意连接攻击承受能力。~~ 159 | 160 | 官方在 [这个提交](https://github.com/EOSIO/eos/commit/d7dff4f1df4a3ab462ef4a60a24ca2be1449df2d)中修复了[P2P单节点恶意连接的问题](https://github.com/EOSIO/eos/issues/3497)，并新增了默认配置`max_nodes_per_host = 1`。所以`max-clients`不需要设置为0，可以根据节点性能酌情配置。 161 | 162 | #### 2.5 非 root 启动 nodeos 163 | 164 | 建议编译完成后，创建普通用户账号，并使用该账号启动 nodeos，避免使用 root，降低风险。 165 | 166 | #### 2.6 监听随机端口 167 | 168 | - `p2p-listen-endpoint = ip:` 169 | - `http-server-address = ip:` 170 | 171 | 每次启动会随机监听一个端口，如果是对外服务的，建议采用 [主机安全](#4-主机安全) 中的配置方法 172 | 173 | ### 3. 网络安全 174 | 175 | #### 3.1 网络架构 176 | 177 | 为应对可能的 DDoS 攻击导致节点主网络阻塞的问题，建议提前配置备份网络，例如私密 VPN 网络。具体可参考 [EOS Asia](https://www.eosasia.one/) 的架构图： 178 | 179 | ![EOS Asia Architecture](./images/eosasia-arch.png) 180 | 181 | #### 3.2 云服务商 182 | 183 | 经慢雾安全团队测试，Google Cloud、AWS 及 UCloud 等具有更好的抗 DDoS 攻击的性能，并且在 DDoS 攻击过后服务商不会临时封锁服务器，可以极为快速的恢复网络访问，推荐超级节点使用。（请谨慎选择云服务商，许多云服务商在遭遇 DDoS 等攻击时会直接关闭服务器） 184 | 185 | #### 3.3 DDoS 防御 186 | 187 | 为应对可能发生的 DDoS 攻击，建议超级节点提前配置 Cloudflare、AWS Shield 等 DDoS 高防服务。 188 | 189 | ### 4. 主机安全 190 | 191 | - 防止全网扫描定位高防后的服务器，修改同步端口 9876 （同理 RPC 的 8888）至全网最大存活数量的端口 80、443 或 22，这样可以有效抬高攻击者定位成本。 192 | - 关闭不相关的其他服务端口，并在 AWS 或 Google Cloud 上定制严格的安全规则。 193 | - 更改 SSH 默认的 22 端口，配置 SSH 只允许用 key （并对 key 加密）登录，禁止密码登录，并限制访问 SSH 端口的 IP 只能为我方运维 IP。 194 | - 在预算充足的情况下，推荐部署优秀的 HIDS（或者强烈建议参考开源的 OSSEC 相关做法），及时应对服务器被入侵。 195 | 196 | ### 5. 威胁情报 197 | 198 | - 强烈建议做好相关重要日志的采集、储存与分析工作，这些日志包括：RPC 与 P2P 端口的完整通信日志、主机的系统日志、节点相关程序的运行日志等。储存与分析工作可以选择自建类似 ELK(ElasticSearch, Logstash, Kibana) 这样的开源方案，也可以购买优秀的商业平台。 199 | - 如果使用了成熟的云服务商，他们的控制台有不少威胁情报相关模块可重点参考，以及时发现异常。 200 | - 当节点出现重大漏洞或相关攻击情报，第一时间启动应急预案，包括灾备策略与升级策略。 201 | - 社区情报互通有无。 202 | 203 | ## 致谢 204 | 205 | 在此非常感谢 206 | 207 | * HelloEOS 208 | * EOS Asia 209 | * EOSBIXIN 210 | * EOS Pacific 211 | * UnlimitedEOS 212 | * EOS Cannon 213 | * EOSpace 214 | * Blockgenic 215 | * EOSeco 216 | * EOSLaoMao 217 | * OneChain 218 | 219 | 等社区节点参与到节点安全测试中，为社区安全积累了宝贵的数据。 220 | -------------------------------------------------------------------------------- /audit-en.md: -------------------------------------------------------------------------------- 1 | # The Block Producer Security Audit 2 | 3 | > by SlowMist Security Team & Joinsec Team 4 | > 5 | > Thanks IMEOS.ONE for translation 6 | 7 | #### [中文版](./audit.md) 8 | 9 | ## Contents 10 | 11 | * [Audit Core Goals](#audit-core-goals) 12 | * [Audit Core Direction](#audit-core-direction) 13 | * [Audit Coverage](#audit-coverage) 14 | * [1. Block Producer Self Audit](#1-block-producer-self-audit) 15 | * [1.1 Architecture Audit](#11-architecture-audit) 16 | * [1.2 RPC Security Audit](#12-rpc-security-audit) 17 | * [1.3 Configuration Security Audit](#13-configuration-security-audit) 18 | * [2. Team Security Audit](#2-team-security-audit) 19 | * [2.1  Infrastructure Audit](#21-infrastructure-audit) 20 | * [2.2 BP vulnerability Audit](#22-bp-vulnerability-audit) 21 | * [2.3 Performance Audit under DDoS](#23-performance-audit-under-ddos) 22 | * [Contributors](#contributors) 23 | 24 | ## Audit Core Goals 25 | 26 | Auditing the security status of BP exposed on the public network to find the common security issues 27 | Auditing the BP architecture performance under DDoS 28 | Aimed on EOS, perform a custom payloads attack test to detecting the robustness of overll frame. 29 | 30 | ## Audit Core Direction 31 | 32 | Finding vulnerabilities that could cause the entire BP nodes to stop blocking 33 | The vulnerabilities that Single-point blocking attacks caused by architectural defects could lead to BP node failures 34 | The vulnerabilities that service misconfiguration causes server to be remotely attacked and controlled 35 | The vulnerabilities that BP's sensitive information (especially the server SSH connection private key leaks on GitHub) 36 | 37 | ## Audit Coverage 38 | 39 | In order to achieve the most efficient and the most comprehensive effect, block producer security audit give priority to with self-audit (many sensitive servers and operations should not be directly exposed to third parties and need to rely on BP to self-audit) and security team provides professional guidance and collaboration. 40 | 41 | ### 1. Block Producer Self Audit 42 | 43 | Refer: [EOS BP Nodes Security Checklist](./README-en.md) 44 | 45 | #### 1.1 Architecture Audit 46 | 47 | 1. Whether the BP's server is fully isolated from the external network to ensuring that if a malicious attack from the external network does not directly affect the BP server out of the block 48 | 1. Whether the BP's have multi-link design to prevents BP can not to synchronize with other BPs cause by single points of failure (or DDoS to single point) 49 | 1. Whether the BP has the necessary security hardening (such as whether to properly deploy high-prevention DDoS attacks in the periphery of some core communication nodes and proper deployment of HIDS) 50 | 51 | #### 1.2 RPC Security Audit 52 | 53 | 1. Whether to restrict the RPC services for non-essential BP 54 | 1. If open RPC services, whether to disable unnecessary wallet_plugin、wallet_api_plugin and producer_api_plugin 55 | 1. Is SSL enabled on RPC 56 | 57 | #### 1.3 Configuration Security Audit 58 | 59 | 1. Confirm whether configured of active multi-signature keys is correct 60 | 1. Whether to turn on logging, whether more security logging plug-ins, etc. under certain conditions 61 | 1. Whether the parameter configuration of max-clients is reasonable, whether it is easiest to attacked by P2P connection full number of connections and could not be synchronized. 62 | 1. Whether to start the BP program with non-root authority 63 | 1. Whether to change the default port of the SSH service, whether the server SSH is configured with a white list, and setting to allow only login with key(encrypted key) and prohibit login with password 64 | 65 | 66 | ### 2. Team Security Audit 67 | 68 | #### 2.1 Infrastructure audit 69 | 70 | 1. whether the server provider is a quality supplier with security. Since building infrastructure security requires huge amounts of money and efforts, if choose some small services, it is highly possible that the Internet service providers will cutdown the connection when server surfering hacks. Therefore, it is a very cost-effective and safe choice to stand on the shoulders of giants. After testing, UCloud, AWS, Google Cloud, etc. have very good anti-attack and post-attack resilience. 71 | 1. Performing security audit of the real open port service of the BPC's public network IP will prevent the operator from exposing vulnerable points due to incorrect configuration of service and security rules. 72 | 73 | #### 2.2 BP vulnerability audit 74 | 75 | 1. Audit the ability of nodes to counter full-network scanning and the ability to hide real public network IP. For example,  whether the bp uses the default port configuration (e.g., use the default port 8888, 9876, or other ports like this kind, etc.) led to the vulnerability of exposing to entire network scanning, and easy to be attacked (currently we already know some test nodes have encountered RPC scanning and malicious call). 76 | 1. Audit whether the sensitive information of node is leaked on the public network, such as on GitHub. 77 | 1. Whether the RPC port is capable of making malicious calls 78 | 1. If the BP node deploys other programs other than the main EOSIO program, vulnerability attack and defense audit is conducted for the third party programs. 79 | 1. Whether the node has settle down a customized emergency response plan. 80 | 81 | #### 2.3 Performance Audit under DDoS 82 | 83 | 1. Performing field testing against the Layer4 level of p2p port, to test the ability of anti UDP Flood, anti TCP Flood, etc(including all kinds of mainstream reflex attack). Test the stability of nodes with real attack traffic. 84 | 1. Performing field testing against the Layer7 level of RPC port to test the ability of anti-CC attracking. Detect the stability of the BP using a large number of attack nodes with high concurrent requests that consume server performance. 85 | 86 | ## Contributors 87 | 88 | Special thanks to: 89 | 90 | * HelloEOS 91 | * EOS Asia 92 | * EOSBIXIN 93 | * EOS Pacific 94 | * UnlimitedEOS 95 | * EOS Cannon 96 | * EOSpace 97 | * Blockgenic 98 | * EOSeco 99 | * EOSLaoMao 100 | * OneChain 101 | * EOS Store 102 | * EOS Beijing 103 | * MotionEOS 104 | * EOSvillage 105 | * EOS AntPool 106 | * EOS Gravity 107 | 108 | Thanks to these bps for participating in the node security test, they have accumulated valuable data for community safety. 109 | -------------------------------------------------------------------------------- /audit.md: -------------------------------------------------------------------------------- 1 | # 超级节点安全审计方案 2 | 3 | > by 慢雾安全团队 & Joinsec Team 4 | 5 | #### [English](./audit-en.md) 6 | 7 | ## 目录 8 | 9 | * [审计核心目标](#审计核心目标) 10 | * [审计的核心方向](#审计的核心方向) 11 | * [审计内容](#审计内容) 12 | * [1. 节点自我审计](#1-节点自我审计) 13 | * [1.1 架构审计](#11-架构审计) 14 | * [1.2 RPC 安全审计](#12-rpc-安全审计) 15 | * [1.3 安全配置审计](#13-安全配置审计) 16 | * [2. 安全团队审计](#2-安全团队审计) 17 | * [2.1 基础设施审计](#21-基础设施审计) 18 | * [2.2 节点脆弱性审计](#22-节点脆弱性审计) 19 | * [2.3 抗 DDoS 能力审计](#23-抗-ddos-能力审计) 20 | * [致谢](#致谢) 21 | 22 | ## 审计核心目标 23 | 24 | 1. 审计超级节点在公网暴露服务的安全状况，查找传统安全中的常见安全问题 25 | 1. 审计超级节点架构的抗 DDoS 能力 26 | 1. 针对 EOS 特性，进行定制化载荷攻击测试，检测整体框架壮硕性 27 | 28 | ## 审计的核心方向 29 | 30 | 1. 查找可造成整个节点停止出块的漏洞问题 31 | 1. 架构缺陷造成的单点阻塞攻击就能导致节点瘫痪的问题 32 | 1. 服务错误配置导致服务器可被远程攻击及控制的问题 33 | 1. 节点敏感信息泄露（特别是服务器 SSH 连接私钥在 GitHub 上泄漏）等问题 34 | 35 | ## 审计内容 36 | 37 | 审计以节点自我审计为主（许多敏感的服务器与操作不应该直接暴露给第三方，需要依赖节点自我审计），安全团队提供职业的指导与协作配合，这样达到最高效及最全面的效果。 38 | 39 | ### 1. 节点自我审计 40 | 41 | 参考：[EOS超级节点安全执行指南](./README.md) 42 | 43 | #### 1.1 架构审计 44 | 45 | 1. BP 服务器是否达到与外网的充分隔离，保证若有外网恶意攻击不会直接影响BP服务器出块 46 | 1. BP 是否多链路设计，防止出现单点故障（或针对单点的 DDoS ）导致 BP 无法与其他节点同步 47 | 1. 节点是否有必要的安全加固（如是否在一些核心通信节点外围正确的部署高防抵御 DDoS 攻击，以及适当的部署 HIDS） 48 | 49 | #### 1.2 RPC 安全审计 50 | 51 | 1. 是否有对非必要的节点 RPC 服务进行限制 52 | 1. 若开启 RPC 服务，是否有禁用不必要的 `wallet_plugin`、`wallet_api_plugin`及`producer_api_plugin` 53 | 1. RPC 是否启用 SSL 54 | 55 | #### 1.3 安全配置审计 56 | 57 | 1. Active 多签密钥是否配置正确 58 | 1. 是否开启日志记录，有条件下是否有开启更多安全日志记录插件等 59 | 1. max-clients 参数配置是否合理，是否易遭受 P2P 连接打满连接数，导致无法同步 60 | 1. 是否使用非 root 权限启动节点程序 61 | 1. 是否更改 SSH 服务默认端口，服务器 SSH 是否配置白名单，并且设置只允许 key（并对 key 加密）登录，禁止密码登录 62 | 63 | ### 2. 安全团队审计 64 | 65 | #### 2.1 基础设施审计 66 | 67 | 1. 服务器提供商是否是优质的安全供应商，构建基础设施安全需要耗费巨量经费与精力，若选择一些小型服务商及可能出现服务器被攻击就被服务商主动断网的情况。所以站在巨人的肩膀上是个非常高性价比与安全的选择，经过测试目前 UCloud、AWS、Google Cloud 等拥有非常好的抗攻击与攻击后自我恢复能力。 68 | 1. 节点公网 IP 真实开放端口服务审计，防止运维人员未正确配置服务与安全规则导致脆弱点暴露。 69 | 70 | #### 2.2 节点脆弱性审计 71 | 72 | 1. 审计节点对抗全网扫描，隐藏真实公网IP的能力。比如 BP 节点是否使用默认端口配置（如使用默认端口 8888、9876 这类特征端口等）导致容易被全网扫描定位，及攻击利用（目前我们已经知道有部分测试节点遭遇了 RPC 扫描与恶意调用）。 73 | 1. 审计节点敏感信息是否在公网上泄漏，如在 GitHub 上暴露等。 74 | 1. 审计 RPC 端口是否可进行恶意调用。 75 | 1. 若节点部署除 EOS 主程序外的其他程序，则针对第三方程序进行针对性脆弱性攻防审计。 76 | 1. 审计节点是否有定制合适的应急响应方案。 77 | 78 | #### 2.3 抗 DDoS 能力审计 79 | 80 | 1. 针对 P2P 端口的 Layer4 层的抗 UDP Flood 、TCP Flood 等（含各种主流反射型攻击）进行实战型测试。利用真实的攻击流量，来检验节点的的稳定性。 81 | 1. 针对 RPC 端口的 Layer7 层的抗 CC 攻击进行实战型测试。利用大量攻击节点高并发请求消耗服务器性能来检测节点稳定性。 82 | 83 | ## 致谢 84 | 85 | 在此非常感谢 86 | 87 | * HelloEOS 88 | * EOS Asia 89 | * EOS Store 90 | * EOSBIXIN 91 | * EOS Pacific 92 | * UnlimitedEOS 93 | * EOS Cannon 94 | * EOSpace 95 | * Blockgenic 96 | * EOSeco 97 | * EOSLaoMao 98 | * OneChain 99 | * EOS Store 100 | * EOS Beijing 101 | * MotionEOS 102 | * EOSvillage 103 | * EOS AntPool 104 | * EOS Gravity 105 | 106 | 等社区节点参与到节点安全测试中，为社区安全积累了宝贵的数据。 -------------------------------------------------------------------------------- /images/arch-en.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-en.png -------------------------------------------------------------------------------- /images/arch-p1-en.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p1-en.png -------------------------------------------------------------------------------- /images/arch-p1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p1.png -------------------------------------------------------------------------------- /images/arch-p2-en.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p2-en.png -------------------------------------------------------------------------------- /images/arch-p2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p2.png -------------------------------------------------------------------------------- /images/arch-p3-en.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p3-en.png -------------------------------------------------------------------------------- /images/arch-p3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p3.png -------------------------------------------------------------------------------- /images/arch-p4-en.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p4-en.png -------------------------------------------------------------------------------- /images/arch-p4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch-p4.png -------------------------------------------------------------------------------- /images/arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/arch.png -------------------------------------------------------------------------------- /images/eosasia-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/slowmist/eos-bp-nodes-security-checklist/9e26e1c63c885994efd2b90d1db395ee577796e1/images/eosasia-arch.png --------------------------------------------------------------------------------