├── .github └── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── SECURITY.md ├── assets └── logo.svg ├── chain_endpoints.py ├── check.sh ├── config.sh ├── disk_check.sh ├── endpoint_storage.py └── ram_check.sh /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: Bug report 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | --- 11 | name: 🐛 Bug Report 12 | about: Report an issue to help improve the monitoring tools 13 | title: '[BUG] ' 14 | labels: bug 15 | assignees: '' 16 | --- 17 | 18 | ## 🔍 Bug Description 19 | 20 | 21 | ## 📝 Steps To Reproduce 22 | 1. Configure script with '...' 23 | 2. Run command '....' 24 | 3. See error 25 | 26 | ## ✔️ Expected Behavior 27 | 28 | 29 | ## ❌ Current Behavior 30 | 31 | 32 | ## 🖥️ Environment 33 | - OS: [e.g., Ubuntu 22.04] 34 | - Chain: [e.g., cosmoshub-4] 35 | - Node Version: [e.g., 0.34.24] 36 | 37 | ## 📋 Logs 38 | ```bash 39 | Paste any relevant logs here 40 | ``` 41 | 42 | ## 💡 Possible Solution 43 | 44 | 45 | ## 📸 Screenshots 46 | 47 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: Feature request 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | --- 11 | name: 💡 Feature Request 12 | about: Suggest an idea for this monitoring tool 13 | title: '[FEATURE] ' 14 | labels: enhancement 15 | assignees: '' 16 | --- 17 | 18 | ## 🎯 Feature Description 19 | 20 | 21 | ## 🔍 Problem Statement 22 | 23 | 24 | ## 💭 Proposed Solution 25 | 26 | 27 | ## 🔄 Alternatives Considered 28 | 29 | 30 | ## ⚙️ Implementation Details 31 | 32 | 33 | ## 📋 Additional Context 34 | 35 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Cosmos Guardian 2 | 3 | The best way to contribute to our project is to delegate your tokens to our validator. This helps us continue developing AI-powered tools that make blockchain and Cosmos ecosystem more accessible to everyone, fight scams, and create helpful repositories like this one. 4 | 5 | ## Why Delegate to Us? 6 | - 🤖 We're building AI tools to help everyone interact with Cosmos ecosystem 7 | - 🛡️ Active participation in fighting scams and protecting the community 8 | - 🔧 Creating open-source tools like this monitoring solution 9 | - 💡 Providing support and guidance at any technical level 10 | - 🌟 Making blockchain technology more accessible through AI innovation 11 | 12 | ## Our Validator 13 | ### Quasar 🤖🧬🧠 validator 14 | 15 |
16 | 17 | [![REStake](https://img.shields.io/badge/Delegate%20with-REStake-blue?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAADKSURBVHgBjZLNDYJAEIVnNvwJB0ta4UBIC7QFE46GsAXpwLQAJZCQEFrAErQESjAe1BW/dmfCRjeyh/cl+97MvN0BGOMrDMOJEGLQNA2hg9rSnNLGGPmrIndKeZ7PRcmrCCHger1yxq/QEWgNsL6z0+mE0+kU0SqCIEAURYjjGHVdQ2sNzjmWyyU451gsFqiqCkVRwHt/b4Q2R3RGtB0/9r/e4GXbNnzf39ESUUqhaRp0XQelFJRSTxGtEaWU6PseSZJgvV7/RLQm4Z1PxGv/7+oGxgkb6s/pnJ4AAAAASUVORK5CYII=)](https://restake.app/cosmoshub/cosmosvaloper12syxdn3qs7fxua3khvewsvdvrx6xw8cjlsknnm/stake) 18 | [![Mintscan](https://img.shields.io/badge/View%20on-Mintscan-green?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAEMSURBVHgBjVKhUsNAEH27t4UOFBkyTFyxZYisSJ0D/AA/QGdwnaEOfqAOB/wAfICDOsrUtUNnU1k6TDrtss/cgQTS3L3s7d7u2+PuCS4IguBR3IVaEASPYRiqpmkgxrDv+wlZuVwuV+v1+l0paskmSZIZY+xYVdWU2Ww2ryQaCoIgBqAQ0dgYc1DjOEY5yo3jGMaYA1mWZbF4wl7FP0aWZbDWot/vg/8Dz/Po7BzHGdi2DSKCUgrT6fQbkbbWfpVluR8MBs+DweDEGEOapu8iYk9cRDZVVe3CMHwjhXieNxaRNxH5FpE1gAP1/J/InZ2IwDmXaK2PQohP51xKjRHRiDEWaa3nnPMPrTWUUhM65wJ/R8nHzVHs1wAAAABJRU5ErkJggg==)](https://www.mintscan.io/cosmos/validators/cosmosvaloper12syxdn3qs7fxua3khvewsvdvrx6xw8cjlsknnm) 19 | [![Ping.pub](https://img.shields.io/badge/View%20on-Ping.pub-orange?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAADSSURBVHgBjVKBEYIwDLzUAXQENwFHYARHwE1gBEZwE3USHUFHqBvoBjpCL/RC+UAw9y9NSbo0lwQADCGEKedcgTHGVwCe5zkRkaK1Pmmtn4yxN6XUJoqiE+ccnHMoigLGGBRFAaUUoihCmqYgIvi+D9/3IaVEkiTI8xxlWaKuayyHRl3XoKZpsN/vB5Gm6cayrKjrukjTNBqPx/B9/x0RAiklqqrCbrdD27Z3xA0Rm6ZB27bwPA9CiL+ICU0IISzLghACQgjQZaIRnQI4/k684Ip8AezHNMDHGitjAAAAAElFTkSuQmCC)](https://ping.pub/cosmos/staking/cosmosvaloper12syxdn3qs7fxua3khvewsvdvrx6xw8cjlsknnm) 20 | 21 |
22 | 23 | Your delegation helps us continue our mission of making the Cosmos ecosystem more secure, accessible, and user-friendly through AI innovation. 24 | 25 | ## Other Ways to Contribute 26 | - 🐛 Report bugs 27 | - 💡 Suggest features 28 | - 🔀 Submit pull requests 29 | - ⭐ Star the repository 30 | - 📢 Share with others 31 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Quasar 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |
2 | 3 | # 🛡️ Cosmos Validator Monitoring Tools 4 | 5 | Cosmos Validator Monitoring 6 | 7 | *Professional monitoring solution for Cosmos Validators* 8 | 9 | > We deliberately chose to use a combination of Bash and Python for this monitoring solution. This hybrid approach ensures both simplicity and reliability, while providing robust endpoint failover capabilities. The system automatically switches to backup RPC endpoints when needed, ensuring continuous monitoring even when primary endpoints fail. 10 | 11 | [![Cosmos](https://img.shields.io/badge/Cosmos-1E88E5?style=flat&logo=cosmos&logoColor=white)](https://cosmos.network) 12 | [![Telegram](https://img.shields.io/badge/Telegram-2CA5E0?style=flat&logo=telegram&logoColor=white)](https://telegram.org) 13 | [![Python](https://img.shields.io/badge/python-3670A0?style=flat&logo=python&logoColor=ffdd54)](https://www.python.org/) 14 | [![Bash](https://img.shields.io/badge/bash-%23121011.svg?style=flat&logo=gnu-bash&logoColor=white)](https://www.gnu.org/software/bash/) 15 | [![License](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE) 16 | 17 | [🚀 Features](#-features) • 18 | [📋 Requirements](#-requirements) • 19 | [🛠️ Installation](#️-quick-start) • 20 | [🤝 Contributing](CONTRIBUTING.md) 21 | 22 |
23 | 24 |
25 | 26 | ## 🎯 Why Choose This Tool? 27 | 28 |
29 | 30 | 31 | 43 | 53 | 54 |
32 | 33 | ### 🔍 Real-time Monitoring 34 | - ⚡ Instant block validation checks 35 | - 🔄 Automatic RPC endpoint failover 36 | - 🌐 Multi-chain support 37 | - 🤖 Automated alert system 38 | - 💽 Regular disk space monitoring 39 | - 🎮 Active RAM usage tracking 40 | - 🤖 Automated alert system 41 | 42 | 44 | 45 | ### 🛡️ Proactive Management 46 | - 🔄 Automatic server reboot on high RAM 47 | - 🔍 Smart endpoint selection 48 | - ⚙️ Customizable alert thresholds 49 | - 📱 Telegram notifications 50 | - 📊 Detailed logging system 51 | 52 |
55 |
56 | 57 | ## 🔄 How It Works 58 | 59 | ```mermaid 60 | %%{init: { 61 | 'theme': 'base', 62 | 'themeVariables': { 63 | 'primaryColor': 'rgba(33, 38, 45, 0.8)', 64 | 'primaryTextColor': '#C9D1D9', 65 | 'primaryBorderColor': 'rgba(48, 54, 61, 0.8)', 66 | 'lineColor': '#2ebd11', 67 | 'secondaryColor': 'rgba(33, 38, 45, 0.8)', 68 | 'tertiaryColor': 'rgba(33, 38, 45, 0.8)', 69 | 'noteTextColor': '#C9D1D9', 70 | 'noteBkgColor': 'rgba(33, 38, 45, 0.8)', 71 | 'noteBorderColor': 'rgba(48, 54, 61, 0.8)', 72 | 'labelTextColor': '#C9D1D9', 73 | 'labelBoxBkgColor': 'rgba(33, 38, 45, 0.8)', 74 | 'labelBoxBorderColor': 'rgba(48, 54, 61, 0.8)', 75 | 'stateBkg': 'rgba(33, 38, 45, 0.8)', 76 | 'stateLabelColor': '#C9D1D9', 77 | 'compositeBackground': 'transparent', 78 | 'altBackground': 'rgba(33, 38, 45, 0.8)' 79 | } 80 | }}%% 81 | 82 | stateDiagram-v2 83 | [*] --> CronScheduler: 🚀 Start Monitoring 84 | 85 | state CronScheduler { 86 | state "📊 check.sh" as BlocksMonitor 87 | state "🎮 ram_check.sh" as RAMMonitor 88 | state "💾 disk_check.sh" as DiskMonitor 89 | 90 | note right of BlocksMonitor 91 | Every 5 minutes 92 | Checks last 10 blocks 93 | end note 94 | note right of RAMMonitor 95 | Every 5 minutes 96 | Threshold: 15GB 97 | end note 98 | note right of DiskMonitor 99 | Every 6 hours 100 | Threshold: <100GB free 101 | end note 102 | } 103 | 104 | state "System Checks" as Checks { 105 | state "🔍 Check Last 10 Blocks" as Blocks 106 | state "⚡ Check RAM Usage" as RAM 107 | state "📀 Check Free Space" as Disk 108 | } 109 | 110 | BlocksMonitor --> Blocks: Monitor 111 | RAMMonitor --> RAM: Track 112 | DiskMonitor --> Disk: Analyze 113 | 114 | state "Alert Process" as Alerts { 115 | state "📝 Prepare Message" as PrepareMsg 116 | state "📱 Send Telegram" as SendTG 117 | state "📋 Write Logs" as Log 118 | } 119 | 120 | Blocks --> PrepareMsg: ❌ Missed in last 10 121 | RAM --> PrepareMsg: ⚠️ Usage ≥ 15GB 122 | Disk --> PrepareMsg: 💾 Free < 100GB 123 | 124 | PrepareMsg --> SendTG: 📤 Notify 125 | SendTG --> Log: 📥 Record 126 | 127 | state "🔄 System Reboot" as Reboot 128 | 129 | RAM --> Reboot: After alert sent 130 | 131 | note right of Alerts 132 | All alerts are sent to 133 | Telegram and logged in 134 | system logs 135 | end note 136 | ``` 137 | 138 | ## ✨ Features 139 | 140 |
141 | 142 | | Feature | Description | Check Interval | 143 | |---------|-------------|----------------| 144 | | 🔍 Missed Blocks Detection | Real-time validator performance tracking | Every 5 minutes | 145 | | 🎮 RAM Management | Proactive memory management | Every 5 minutes | 146 | | 💾 Disk Space Monitoring | Regular storage monitoring | Every 6 hours | 147 | | 🔔 Telegram Alerts | Instant notifications | On events | 148 | | 📊 Performance Logs | Comprehensive system logs | Continuous | 149 | | ⚡ Auto-Recovery | Smart reboot system | On RAM alert | 150 | 151 |
152 | 153 | ## 📋 Requirements 154 | 155 |
156 | 157 | | Requirement | Minimum | Recommended | 158 | |-------------|---------|-------------| 159 | | 🐧 Linux | Ubuntu 20.04 | Ubuntu 22.04 | 160 | | 🔄 curl | 7.68.0 | Latest | 161 | | 🔄 Python | 3.8+ | Latest | 162 | | 🔧 jq | 1.6 | Latest | 163 | | 📱 Telegram Bot | Any | Latest | 164 | | 💻 Bash | 5.0 | Latest | 165 | 166 |
167 | 168 | Required Python packages: 169 | ```bash 170 | pip3 install requests 171 | ``` 172 | 173 | ## 🤖 Telegram Bot Setup 174 | 175 | 1. Create a new bot: 176 | - Message [@BotFather](https://t.me/BotFather) on Telegram 177 | - Use `/newbot` command 178 | - Follow instructions to create bot 179 | - Copy the provided API token 180 | 181 | 2. Get Chat ID: 182 | - Add bot to your group 183 | - Make bot admin 184 | - Send any message to group 185 | - Visit: `https://api.telegram.org/bot/getUpdates` 186 | - Find "chat" -> "id" in the response 187 | 188 | 3. Get Thread ID (if using topics): 189 | - Enable topics in group settings 190 | - Create new topic 191 | - Send message in topic 192 | - Find "message_thread_id" in the getUpdates response 193 | 194 | 4. Update script variables: 195 | ```bash 196 | # Edit check.sh 197 | BOT_TOKEN="YOUR_BOT_TOKEN" 198 | CHAT_ID="YOUR_CHAT_ID" 199 | THREAD_ID="YOUR_THREAD_ID" # Optional, for topic messages 200 | ``` 201 | 202 | ## ⚡ Quick Start 203 | 204 | ```bash 205 | # Get the tools 206 | git clone https://github.com/Quasar-hub/quasar-server-sentinel.git 207 | 208 | # Go to directory 209 | cd quasar-server-sentinel 210 | 211 | # Make executable 212 | chmod +x *.sh 213 | 214 | # Update script variables 215 | nano check.sh 216 | nano ram_check.sh 217 | nano disk_check.sh 218 | 219 | # Set up crontab 220 | crontab -e 221 | 222 | # Add these lines: 223 | */5 * * * * $HOME/quasar-server-sentinel/check.sh [LOG_DIR] 224 | */5 * * * * $HOME/quasar-server-sentinel/ram_check.sh 225 | 0 */6 * * * $HOME/quasar-server-sentinel/disk_check.sh 226 | 227 | # Verify crontab 228 | crontab -l 229 | 230 | # Example: 231 | */5 * * * * $HOME/quasar-server-sentinel/check.sh http://localhost:26657 76A8A9A8151255E9E69E89499CFE9CB86FD cosmoshub-4 cosmos ./logs 232 | ``` 233 | 234 | ## 📝 Monitoring Logs 235 | 236 | Check your monitoring logs using these commands: 237 | 238 | ```bash 239 | # Check RAM monitoring logs 240 | cat /var/log/syslog | grep "RAM Monitor" 241 | 242 | # Check Disk monitoring logs 243 | cat /var/log/syslog | grep "Disk Monitor" 244 | ``` 245 | 246 | ## ⚙️ Configuration 247 | 248 |
249 | 📜 Missed Blocks Settings (click to expand) 250 | 251 | ```bash 252 | # Script parameters 253 | RPC_ENDPOINT="$1" # Primary RPC endpoint (usually local) 254 | VALIDATOR_ADDRESS="$2" # Validator address in hex format 849ASD..... 255 | CHAIN_ID="$3" # Chain ID (e.g., cosmoshub-4, osmosis-1) 256 | CHAIN_NAME="$4" # Chain name for endpoint lookup (e.g., cosmoshub, osmosis) 257 | LOG_DIR="${5:-./logs}" # Directory for log files, default to ./logs if not provided 258 | ``` 259 |
260 | 261 |
262 | 💾 Disk Space Settings (click to expand) 263 | 264 | ```bash 265 | # disk_check.sh configuration 266 | FREE_SPACE=100 # Alert when free space is below 100GB 267 | ``` 268 |
269 | 270 |
271 | 🎮 RAM Settings (click to expand) 272 | 273 | ```bash 274 | # ram_check.sh configuration 275 | USED_RAM=15 # Alert and reboot when RAM usage exceeds 15GB 276 | ``` 277 |
278 | 279 | ## 🔔 Alert Examples 280 | 281 | 282 | 283 | 294 | 310 | 311 |
284 | 285 | ### 🚫 Missed Blocks 286 | ``` 287 | ⚠️ Alert: Validator Issues 288 | • Missed: 10 consecutive blocks 289 | • Chain: your_chain_id 290 | • Action: Check validator 291 | ``` 292 | 293 | 295 | 296 | ### 💾 Storage Alert 297 | ``` 298 | ⚠️ Alert: Low Storage 299 | • Free: 95GB 300 | • Status: Warning 301 | • Action: Cleanup needed 302 | ``` 303 | ### 🔌 Endpoint Failover 304 | ``` 305 | 🔴 cosmoshub-4 - no response from primary or backup endpoints 306 | (tried 5 additional endpoints) on hostname 307 | ``` 308 | 309 |
312 | 313 | ## 📊 System Status 314 | 315 | ``` 316 | 📊 Monitoring Schedule 317 | ┌──────────────┬─────────────┬──────────────┐ 318 | │ Service │ Frequency │ Next Check │ 319 | ├──────────────┼─────────────┼──────────────┤ 320 | │ Blocks Check │ 5 minutes │ Auto │ 321 | │ RAM Check │ 5 minutes │ Auto │ 322 | │ Disk Check │ 6 hours │ Auto │ 323 | └──────────────┴─────────────┴──────────────┘ 324 | ``` 325 | ## 💡 Additional Resources 326 | - [Security Settings](SECURITY.md) - Basic firewall and SSH configuration 327 | 328 | ## 🤝 Contributing 329 | 330 | We welcome contributions! Check our [Contributing Guide](CONTRIBUTING.md) for details. 331 | 332 |
333 | 334 | | 🐛 [Report Bug](../../issues/new?template=bug_report.md) | 🔀 [Pull Request](../../pulls) | 💡 [Suggest Feature](../../issues/new?template=feature_request.md) | 335 | |-------------------|-----------------|------------------------| 336 | 337 |
338 | 339 | ## 📈 Resource Usage 340 | 341 | ``` 342 | System Impact: 343 | CPU: < 1% average usage 344 | RAM: ~ 50MB memory usage 345 | Disk: ~ 10MB for logs 346 | Network: ~ 1MB/hour 347 | ``` 348 | 349 | ## ⭐ Support 350 | 351 | If you find this useful: 352 | - 🌟 Star this repository 353 | - 🔄 Fork and contribute 354 | - 📢 Share with others 355 | 356 | ## 📜 License 357 | 358 |
359 | 360 | [MIT License](LICENSE) • Free and Open Source 361 | 362 | --- 363 | 364 |
365 | 366 | Made with ❤️ by Quasar 🤖🧬🧠 validator 367 | 368 | [🔝 Back to Top](#top) 369 | 370 |
371 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | # 🔒 Basic Security Settings 2 | 3 | ### Firewall Configuration 4 | ```bash 5 | # Reset and configure UFW 6 | sudo ufw disable 7 | sudo ufw reset 8 | 9 | # Allow essential ports 10 | sudo ufw allow 22/tcp # SSH 11 | sudo ufw allow 26656/tcp # Cosmos P2P 12 | 13 | # Set default policy and enable 14 | sudo ufw default deny incoming 15 | sudo ufw enable 16 | ``` 17 | 18 | ### SSH & User Security 19 | 20 | First, you need to add your SSH public key to authorize access to the server: 21 | ```bash 22 | # Create .ssh directory if it doesn't exist 23 | mkdir -p ~/.ssh 24 | 25 | # Create or edit authorized_keys file 26 | nano ~/.ssh/authorized_keys 27 | 28 | # Add your public key (example format): 29 | # ssh-rsa AAAAB3NzaC1yc2EAAAADA... your.email@example.com 30 | 31 | # Set correct permissions 32 | chmod 700 ~/.ssh 33 | chmod 600 ~/.ssh/authorized_keys 34 | ``` 35 | 36 | After adding your SSH key, secure the SSH configuration: 37 | ```bash 38 | # Disable password authentication completely (only SSH keys allowed) 39 | # This prevents brute force attacks by requiring SSH key authentication 40 | sudo sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config 41 | 42 | # Restrict root login to only allow SSH key authentication 43 | # Prevents direct root login attempts and adds additional security layer 44 | sudo sed -i 's/PermitRootLogin yes/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config 45 | 46 | # Restart SSH service to apply changes 47 | sudo systemctl restart ssh 48 | 49 | # Edit sudoers file to prevent user 'ubuntu' from using 'sudo su' 50 | sudo visudo 51 | # Add line: 52 | ubuntu ALL=(ALL:ALL) !/bin/su 53 | ``` 54 | 55 | ### Check Settings 56 | ```bash 57 | # Check UFW status 58 | sudo ufw status 59 | 60 | # Verify SSH configuration 61 | sudo sshd -T | grep -E 'passwordauthentication|permitrootlogin' 62 | 63 | # Check current user 64 | whoami 65 | ``` 66 | -------------------------------------------------------------------------------- /assets/logo.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 15 | 16 | 17 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 29 | 30 | 31 | 36 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 50 | 51 | 52 | 🛡️ 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 64 | 65 | RAM 66 | 67 | 68 | 69 | 70 | 71 | 75 | 76 | DISK 77 | 78 | 79 | 80 | 81 | 82 | 86 | 87 | BLOCKS 88 | 89 | 90 | 91 | 97 | COSMOS VALIDATOR MONITORING 98 | 99 | 100 | -------------------------------------------------------------------------------- /chain_endpoints.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import requests 3 | import json 4 | from typing import List, Dict 5 | import logging 6 | import os 7 | from endpoint_storage import EndpointStorage 8 | import time 9 | 10 | logging.basicConfig(level=logging.INFO) 11 | logger = logging.getLogger(__name__) 12 | 13 | class ChainEndpoints: 14 | """Manager for chain endpoints with file storage""" 15 | 16 | GITHUB_RAW_BASE = "https://raw.githubusercontent.com/ping-pub/ping.pub/main/chains/mainnet/" 17 | 18 | def __init__(self): 19 | self.headers = { 20 | 'Accept': 'application/vnd.github.v3+json' 21 | } 22 | self.storage = EndpointStorage() 23 | 24 | def _extract_api_endpoints(self, api_data) -> List[str]: 25 | """ 26 | Extract API endpoints from different JSON formats 27 | 28 | Handles formats: 29 | 1. List of strings: ["url1", "url2"] 30 | 2. List of dicts with address: [{"address": "url1"}, {"address": "url2"}] 31 | 3. Mixed list: ["url1", {"address": "url2", "provider": "name"}] 32 | 4. Single string: "url" 33 | """ 34 | endpoints = [] 35 | 36 | if isinstance(api_data, list): 37 | for item in api_data: 38 | if isinstance(item, dict) and 'address' in item: 39 | endpoints.append(item['address']) 40 | elif isinstance(item, str): 41 | endpoints.append(item) 42 | elif isinstance(api_data, str): 43 | endpoints.append(api_data) 44 | 45 | return endpoints 46 | 47 | def _check_endpoint(self, endpoint: str) -> bool: 48 | """Check if endpoint is working""" 49 | try: 50 | response = requests.get( 51 | f"{endpoint}/status", # RPC endpoint check 52 | timeout=5 53 | ) 54 | return response.status_code == 200 55 | except: 56 | return False 57 | 58 | def get_endpoints(self, chain_id: str) -> List[str]: 59 | """Get list of API endpoints for a specific chain""" 60 | # Try getting from local storage first 61 | endpoints = self.storage.get_endpoints(chain_id) 62 | if endpoints: 63 | # Verify first endpoint is working 64 | if self._check_endpoint(endpoints[0]): 65 | return endpoints 66 | logger.info(f"Stored endpoints for {chain_id} not working, fetching new ones") 67 | 68 | # If not in storage or not working, get from GitHub 69 | try: 70 | # First try direct chain_id 71 | logger.info(f"Fetching config for {chain_id}") 72 | response = requests.get(f"{self.GITHUB_RAW_BASE}{chain_id}.json") 73 | 74 | if response.status_code == 404: 75 | # If not found, try to find by registry_name in all files 76 | logger.info("Chain not found, searching in all configurations...") 77 | all_chains = requests.get("https://api.github.com/repos/ping-pub/ping.pub/contents/chains/mainnet") 78 | if all_chains.status_code == 200: 79 | for item in all_chains.json(): 80 | if item['name'].endswith('.json'): 81 | config_response = requests.get(item['download_url']) 82 | if config_response.status_code == 200: 83 | config = config_response.json() 84 | if (config.get('registry_name') == chain_id or 85 | config.get('chain_name') == chain_id): 86 | response = config_response 87 | break 88 | 89 | if response.status_code == 200: 90 | config = response.json() 91 | rpc_data = config.get('rpc', []) # Get RPC endpoints instead of API 92 | endpoints = self._extract_api_endpoints(rpc_data) # Extract RPC endpoints 93 | 94 | # Filter working endpoints 95 | working_endpoints = [ 96 | endpoint for endpoint in endpoints 97 | if self._check_endpoint(endpoint) 98 | ] 99 | 100 | if working_endpoints: 101 | logger.info(f"Found {len(working_endpoints)} working endpoints for {chain_id}") 102 | # Save to local storage 103 | self.storage.save_endpoints(chain_id, working_endpoints) 104 | return working_endpoints 105 | else: 106 | logger.warning(f"No working endpoints found for {chain_id}") 107 | else: 108 | logger.error(f"Failed to get config for {chain_id}: {response.status_code}") 109 | except Exception as e: 110 | logger.error(f"Error fetching config for {chain_id}: {e}") 111 | 112 | return [] 113 | 114 | # Singleton instance 115 | chain_endpoints = ChainEndpoints() 116 | 117 | def test_endpoints(): 118 | """Test function to demonstrate functionality""" 119 | 120 | # Test 1: Get all available chain configs 121 | print("\n=== Test 1: Available Chains ===") 122 | try: 123 | response = requests.get("https://api.github.com/repos/ping-pub/ping.pub/contents/chains/mainnet") 124 | if response.status_code == 200: 125 | chains = [ 126 | item['name'].replace('.json', '') 127 | for item in response.json() 128 | if item['name'].endswith('.json') and not item['name'].endswith('.disabled') 129 | ] 130 | print(f"Total chains available: {len(chains)}") 131 | print("\nFirst 10 chains:") 132 | for chain in sorted(chains[:10]): 133 | print(f" - {chain}") 134 | print("...") 135 | except Exception as e: 136 | print(f"Error getting chain list: {e}") 137 | 138 | # Test 2: Test popular chains RPC endpoints 139 | print("\n=== Test 2: Popular Chains RPC Endpoints ===") 140 | test_chains = ['cosmoshub', 'osmosis', 'celestia', 'juno', 'stargaze'] 141 | for chain in test_chains: 142 | print(f"\nTesting {chain}:") 143 | endpoints = chain_endpoints.get_endpoints(chain) 144 | print(f"Found {len(endpoints)} working RPC endpoints:") 145 | for endpoint in endpoints: 146 | print(f" - {endpoint}") 147 | 148 | # Test 3: Test cache functionality 149 | print("\n=== Test 3: Cache Test ===") 150 | chain = 'cosmoshub' 151 | print(f"First request for {chain} (might fetch from GitHub):") 152 | start_time = time.time() 153 | endpoints1 = chain_endpoints.get_endpoints(chain) 154 | time1 = time.time() - start_time 155 | 156 | print(f"Second request for {chain} (should use cache):") 157 | start_time = time.time() 158 | endpoints2 = chain_endpoints.get_endpoints(chain) 159 | time2 = time.time() - start_time 160 | 161 | print(f"\nFirst request time: {time1:.2f}s") 162 | print(f"Second request time: {time2:.2f}s") 163 | print(f"Cache speedup: {time1/time2:.1f}x") 164 | 165 | # Test 4: Test RPC endpoint health check 166 | print("\n=== Test 4: RPC Endpoint Health Check ===") 167 | chain = 'osmosis' 168 | endpoints = chain_endpoints.get_endpoints(chain) 169 | print(f"\nChecking {len(endpoints)} RPC endpoints for {chain}:") 170 | for endpoint in endpoints: 171 | start_time = time.time() 172 | is_working = chain_endpoints._check_endpoint(endpoint) 173 | response_time = time.time() - start_time 174 | status = "✅" if is_working else "❌" 175 | print(f"{status} {endpoint} - Response time: {response_time:.2f}s") 176 | 177 | def test_endpoint_extraction(): 178 | """Test different JSON formats for endpoint extraction""" 179 | print("\n=== Test 6: JSON Format Handling ===") 180 | 181 | # Test Format 1: Mixed list (strings and objects) - Axelar example 182 | print("\nTesting Format 1 (Axelar-style):") 183 | axelar_format = [ 184 | "https://rpc-axelar.imperator.co:443", 185 | "https://axelar-rpc.quickapi.com:443", 186 | { 187 | "address": "https://tm.axelar.lava.build", 188 | "provider": "Lava network" 189 | } 190 | ] 191 | endpoints = chain_endpoints._extract_api_endpoints(axelar_format) 192 | print(f"Found {len(endpoints)} RPC endpoints:") 193 | for endpoint in endpoints: 194 | print(f" - {endpoint}") 195 | 196 | # Test Format 2: List of objects only - Andromeda example 197 | print("\nTesting Format 2 (Andromeda-style):") 198 | andromeda_format = [ 199 | { 200 | "address": "https://rpc.andromeda-1.andromeda.aviaone.com", 201 | "provider": "AVIAONE" 202 | }, 203 | { 204 | "address": "https://andromeda-rpc.lavenderfive.com:443", 205 | "provider": "Lavender.Five Nodes 🐝" 206 | } 207 | ] 208 | endpoints = chain_endpoints._extract_api_endpoints(andromeda_format) 209 | print(f"Found {len(endpoints)} RPC endpoints:") 210 | for endpoint in endpoints: 211 | print(f" - {endpoint}") 212 | 213 | # Test Format 3: List of objects with registry name - Cosmos example 214 | print("\nTesting Format 3 (Cosmos-style):") 215 | cosmos_format = [ 216 | {"provider": "cosmos.directory", "address": "https://rpc.cosmos.directory/cosmoshub"}, 217 | {"provider": "Lava network", "address": "https://cosmoshub.tendermintrpc.lava.build"} 218 | ] 219 | endpoints = chain_endpoints._extract_api_endpoints(cosmos_format) 220 | print(f"Found {len(endpoints)} RPC endpoints:") 221 | for endpoint in endpoints: 222 | print(f" - {endpoint}") 223 | 224 | # Test real chains with different formats 225 | print("\nTesting real chains with different formats:") 226 | test_chains = ['axelar', 'andromeda', 'cosmoshub'] 227 | for chain in test_chains: 228 | print(f"\n{chain}:") 229 | endpoints = chain_endpoints.get_endpoints(chain) 230 | print(f"Found {len(endpoints)} working RPC endpoints:") 231 | for endpoint in endpoints: 232 | print(f" - {endpoint}") 233 | 234 | if __name__ == "__main__": 235 | import sys 236 | import time 237 | 238 | if len(sys.argv) > 1: 239 | # Get endpoints for specific chain 240 | chain_id = sys.argv[1] 241 | logger.info(f"Getting endpoints for {chain_id}...") 242 | endpoints = chain_endpoints.get_endpoints(chain_id) 243 | if endpoints: 244 | # Print endpoints space-separated for bash script 245 | print(" ".join(endpoints)) 246 | else: 247 | logger.warning("No working endpoints found") 248 | print("") # Print empty line for bash script 249 | else: 250 | # Run tests 251 | test_endpoints() 252 | test_endpoint_extraction() -------------------------------------------------------------------------------- /check.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Get script directory 4 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 5 | 6 | # Source configuration file 7 | source "${SCRIPT_DIR}/config.sh" 8 | 9 | # Check if all required parameters are provided 10 | # Usage: ./check.sh [LOG_DIR] 11 | if [ "$#" -lt 4 ]; then 12 | echo "Usage: $0 [LOG_DIR]" 13 | echo "Example: $0 http://localhost:26657 76A8A9A8151255E9E69E89499CFE9CB86F cosmoshub-4 cosmos" 14 | echo "Example: $0 http://localhost:26657 76A8A9A8151255E9E69E89499CFE9CB86F cosmoshub-4 cosmos ./logs" 15 | exit 1 16 | fi 17 | 18 | # Script parameters 19 | RPC_ENDPOINT="$1" # Primary RPC endpoint (usually local) 20 | VALIDATOR_ADDRESS="$2" # Validator address in hex format 849ASD..... 21 | CHAIN_ID="$3" # Chain ID (e.g., cosmoshub-4, osmosis-1) 22 | CHAIN_NAME="$4" # Chain name for endpoint lookup (e.g., cosmos, osmosis) 23 | LOG_DIR="${5:-$LOG_DIR}" # Use provided LOG_DIR or default from config 24 | 25 | # Create log directory and file if they don't exist 26 | mkdir -p "${LOG_DIR}" # Create logs directory if it doesn't exist 27 | touch "${LOG_FILE}" # Create log file if it doesn't exist 28 | 29 | # Function to send messages to Telegram 30 | send_tg_message() { 31 | local message=$1 32 | local response=$(curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \ 33 | -d "chat_id=${CHAT_ID}" \ 34 | -d "message_thread_id=${THREAD_ID}" \ 35 | -d "text=${message}" \ 36 | -d "parse_mode=HTML") 37 | 38 | if echo "$response" | grep -q '"ok":true'; then 39 | echo "Message sent successfully to Telegram" 40 | else 41 | echo "Failed to send message to Telegram. Error response:" 42 | echo "$response" 43 | fi 44 | } 45 | 46 | # Main function to check missed blocks 47 | # Checks last N blocks and counts how many were missed by the validator 48 | check_missed_blocks_in_last_n() { 49 | local N=10 # Number of blocks to check 50 | local missed_blocks=0 # Counter for missed blocks 51 | local current_endpoint="$RPC_ENDPOINT" # Current working endpoint 52 | local failed_attempts=0 # Counter for failed endpoint attempts 53 | 54 | # Get latest block height from current endpoint 55 | LATEST_HEIGHT=$(curl -s "${current_endpoint}/status" | jq -r '.result.sync_info.latest_block_height') 56 | 57 | # If primary endpoint fails, get and try backup endpoints 58 | if [[ -z "$LATEST_HEIGHT" || "$LATEST_HEIGHT" == "null" ]]; then 59 | echo "Local endpoint not responding, searching for backup endpoints..." 60 | 61 | # Get backup endpoints from chain_endpoints.py 62 | echo "Running: python3 ${SCRIPT_DIR}/chain_endpoints.py $CHAIN_NAME" 63 | if ! BACKUP_ENDPOINTS=$(python3 "${SCRIPT_DIR}/chain_endpoints.py" "$CHAIN_NAME" 2>/dev/null); then 64 | echo "Error running chain_endpoints.py" 65 | BACKUP_ENDPOINTS="" 66 | fi 67 | SCRIPT_EXIT_CODE=$? 68 | echo "Script exit code: $SCRIPT_EXIT_CODE" 69 | echo "Backup endpoints: $BACKUP_ENDPOINTS" 70 | 71 | if [ $SCRIPT_EXIT_CODE -eq 0 ] && [ ! -z "$BACKUP_ENDPOINTS" ]; then 72 | IFS=' ' read -r -a endpoints <<< "$BACKUP_ENDPOINTS" 73 | echo "Loaded ${#endpoints[@]} backup endpoints for ${CHAIN_NAME}" 74 | 75 | # Try each backup endpoint 76 | for endpoint in "${endpoints[@]}"; do 77 | echo "Trying endpoint: $endpoint" 78 | LATEST_HEIGHT=$(curl -s "${endpoint}/status" | jq -r '.result.sync_info.latest_block_height') 79 | if [[ ! -z "$LATEST_HEIGHT" && "$LATEST_HEIGHT" != "null" ]]; then 80 | current_endpoint="$endpoint" 81 | echo "Found working endpoint: $endpoint" 82 | break 83 | fi 84 | ((failed_attempts++)) 85 | echo "Endpoint failed: $endpoint" 86 | done 87 | else 88 | echo "Failed to get backup endpoints" 89 | declare -a endpoints=() 90 | fi 91 | fi 92 | 93 | # Handle case when all endpoints failed 94 | if [[ -z "$LATEST_HEIGHT" || "$LATEST_HEIGHT" == "null" ]]; then 95 | local msg="🔴 $CHAIN_ID - no response from $RPC_ENDPOINT or backup endpoints (tried ${failed_attempts} additional endpoints) on $(hostname)" 96 | local log_msg="$(date): $CHAIN_ID - all endpoints failed after ${failed_attempts} attempts!" 97 | echo "$log_msg" >> "$LOG_FILE" 98 | send_tg_message "$msg" 99 | return 1 100 | fi 101 | 102 | # Check last N blocks for missed signatures 103 | for ((i=0; i= THRESHOLD )); then 120 | LOG_ENTRY="$(date): Validator missed ${missed_blocks} blocks on $(hostname)! for ${CHAIN_ID} (using endpoint: ${current_endpoint})" 121 | echo "$LOG_ENTRY" | tee -a "$LOG_FILE" 122 | 123 | send_tg_message "$ALERT_MSG" 124 | fi 125 | } 126 | 127 | # Run the check 128 | check_missed_blocks_in_last_n 129 | -------------------------------------------------------------------------------- /config.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Telegram configuration 4 | export BOT_TOKEN="" 5 | export CHAT_ID="" 6 | export THREAD_ID="" 7 | 8 | # Monitoring thresholds 9 | export DISK_THRESHOLD=300 # Free space threshold in GB 10 | export RAM_THRESHOLD=100 # RAM usage threshold in GB 11 | export BLOCKS_THRESHOLD=10 # Missed blocks threshold 12 | 13 | # Log configuration 14 | export LOG_DIR="./logs" 15 | export LOG_FILE="${LOG_DIR}/cosmos_monitor.log" 16 | 17 | # Alert messages 18 | export DISK_ALERT_MSG="⚠️ Critical low disk space on $(hostname)! 19 | Free space: %dGB 20 | Threshold: ${DISK_THRESHOLD}GB 21 | 22 | ❗️ Disk cleanup required!" 23 | 24 | export RAM_ALERT_MSG="⚠️ High RAM usage on $(hostname)! 25 | Used: %dGB 26 | Threshold: ${RAM_THRESHOLD}GB 27 | 28 | 🔄 Server restart required." 29 | 30 | export BLOCKS_ALERT_MSG="⚠️ Validator for %s missed more than ${BLOCKS_THRESHOLD} blocks on $(hostname)." 31 | -------------------------------------------------------------------------------- /disk_check.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Get the directory where the script is located 4 | # This ensures we can find the config file regardless of where the script is called from 5 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 6 | 7 | # Load configuration variables from config.sh (thresholds, Telegram settings, etc.) 8 | source "${SCRIPT_DIR}/config.sh" 9 | 10 | # Add after source command 11 | if [ -z "$DISK_THRESHOLD" ] || [ -z "$BOT_TOKEN" ] || [ -z "$CHAT_ID" ] || [ -z "$THREAD_ID" ]; then 12 | logger "Disk Monitor: Configuration error - missing required variables" 13 | exit 1 14 | fi 15 | 16 | # Function to send notifications via Telegram 17 | # Parameters: 18 | # $1 - message text to send 19 | send_tg_message() { 20 | local message=$1 21 | curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \ 22 | -d chat_id="${CHAT_ID}" \ 23 | -d message_thread_id="${THREAD_ID}" \ 24 | -d text="${message}" \ 25 | -d parse_mode="HTML" 26 | } 27 | 28 | # Get available disk space in gigabytes 29 | # df -BG - show disk space in GB units 30 | # awk 'NR==2 {print $4}' - get the 4th column (free space) from the second line 31 | # sed 's/G//' - remove the 'G' suffix from the number 32 | FREE_SPACE=$(df -BG / | awk 'NR==2 {print $4}' | sed 's/G//') 33 | 34 | # Add after FREE_SPACE definition 35 | if [ -z "$FREE_SPACE" ]; then 36 | logger "Disk Monitor: Error getting disk space information" 37 | exit 1 38 | fi 39 | 40 | # Check if free space is below the threshold 41 | # If free space is less than DISK_THRESHOLD (defined in config.sh), send alert 42 | if [ ${FREE_SPACE} -lt ${DISK_THRESHOLD} ]; then 43 | # Format alert message with current free space value 44 | FORMATTED_ALERT=$(printf "$DISK_ALERT_MSG" "$FREE_SPACE") 45 | 46 | # Send notification to Telegram channel/group 47 | send_tg_message "$FORMATTED_ALERT" 48 | 49 | # Write event to system log for future reference 50 | logger "Disk Monitor: Free space is below ${DISK_THRESHOLD}GB (Current: ${FREE_SPACE}GB)" 51 | fi 52 | -------------------------------------------------------------------------------- /endpoint_storage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import json 3 | import os 4 | from datetime import datetime, timedelta 5 | 6 | class EndpointStorage: 7 | """ 8 | Local storage manager for chain endpoints 9 | """ 10 | def __init__(self, cache_dir="cache"): 11 | self.cache_dir = cache_dir 12 | self.cache_file = os.path.join(cache_dir, "endpoints_cache.json") 13 | self.cache_duration = timedelta(hours=24) # Cache valid for 24 hours 14 | 15 | # Create cache directory if it doesn't exist 16 | os.makedirs(cache_dir, exist_ok=True) 17 | 18 | # Create cache file with empty JSON object if it doesn't exist 19 | if not os.path.exists(self.cache_file): 20 | with open(self.cache_file, 'w') as f: 21 | json.dump({}, f) 22 | 23 | def save_endpoints(self, chain_id: str, endpoints: list): 24 | """Save endpoints to storage""" 25 | cache_data = self.load_cache() 26 | cache_data[chain_id] = { 27 | "endpoints": endpoints, 28 | "timestamp": datetime.now().isoformat() 29 | } 30 | 31 | with open(self.cache_file, 'w') as f: 32 | json.dump(cache_data, f, indent=2) 33 | 34 | def get_endpoints(self, chain_id: str) -> list: 35 | """Get endpoints from storage if they're still valid""" 36 | cache_data = self.load_cache() 37 | 38 | if chain_id in cache_data: 39 | cached = cache_data[chain_id] 40 | cached_time = datetime.fromisoformat(cached["timestamp"]) 41 | 42 | # Return cached endpoints if they're still valid 43 | if datetime.now() - cached_time < self.cache_duration: 44 | return cached["endpoints"] 45 | 46 | return [] 47 | 48 | def load_cache(self) -> dict: 49 | """Load storage from file""" 50 | try: 51 | with open(self.cache_file) as f: 52 | data = f.read().strip() 53 | if not data: # If file is empty 54 | return {} 55 | return json.loads(data) 56 | except json.JSONDecodeError: # If JSON is invalid 57 | return {} 58 | except Exception as e: 59 | print(f"Error loading storage: {e}") 60 | return {} -------------------------------------------------------------------------------- /ram_check.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Get the directory where the script is located 4 | # This ensures we can find the config file regardless of where the script is called from 5 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 6 | 7 | # Load configuration variables from config.sh (thresholds, Telegram settings, etc.) 8 | source "${SCRIPT_DIR}/config.sh" 9 | 10 | # Add after source command 11 | if [ -z "$RAM_THRESHOLD" ] || [ -z "$BOT_TOKEN" ] || [ -z "$CHAT_ID" ] || [ -z "$THREAD_ID" ]; then 12 | logger "RAM Monitor: Configuration error - missing required variables" 13 | exit 1 14 | fi 15 | 16 | # Function to send notifications via Telegram 17 | # Parameters: 18 | # $1 - message text to send 19 | send_tg_message() { 20 | local message=$1 21 | curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \ 22 | -d chat_id="${CHAT_ID}" \ 23 | -d message_thread_id="${THREAD_ID}" \ 24 | -d text="${message}" \ 25 | -d parse_mode="HTML" 26 | } 27 | 28 | # Get current RAM usage in gigabytes 29 | # free -g - show memory info in GB 30 | # grep Mem - get the line with RAM info 31 | # awk '{print $3}' - get the 3rd column (used memory) 32 | USED_RAM=$(free -g | grep Mem | awk '{print $3}') 33 | 34 | # Add after USED_RAM definition 35 | if [ -z "$USED_RAM" ]; then 36 | logger "RAM Monitor: Error getting RAM usage information" 37 | exit 1 38 | fi 39 | 40 | # Check if RAM usage is above the threshold 41 | # If used RAM is greater or equal to RAM_THRESHOLD (defined in config.sh) 42 | if [ ${USED_RAM} -ge ${RAM_THRESHOLD} ]; then 43 | # Format alert message with current RAM usage 44 | FORMATTED_ALERT=$(printf "$RAM_ALERT_MSG" "$USED_RAM") 45 | 46 | # Send notification to Telegram channel/group 47 | send_tg_message "$FORMATTED_ALERT" 48 | 49 | # Write event to system log for future reference 50 | logger "RAM Monitor: Memory usage exceeded ${RAM_THRESHOLD}GB (Current: ${USED_RAM}GB)" 51 | 52 | # Reboot the system when RAM usage is too high 53 | # Using full path to reboot command for security 54 | # Requires sudo privileges for the user running the script 55 | sudo /sbin/reboot 56 | fi 57 | --------------------------------------------------------------------------------