├── README.md ├── media ├── ch_activity_insert_mergetree.png ├── ch_activity_insert_replicatedmergetree.png ├── ch_information1_general.png ├── ch_information2_settings.png ├── ch_information3_users.png ├── ch_information4_tables.png ├── ch_information5_replica_queries.png ├── ch_information6_activity.png └── ch_information_metrics.png └── scripts ├── clickhouse_activity.sh ├── clickhouse_activity_refresh.sh ├── clickhouse_client.sh ├── clickhouse_information.sh ├── clickhouse_information_metrics.sh ├── clickhouse_logs.sh ├── clickhouse_start.sh ├── clickhouse_status.sh ├── clickhouse_stop.sh ├── settings.txt ├── zookeeper_logs.sh ├── zookeeper_start.sh ├── zookeeper_status.sh └── zookeeper_stop.sh /README.md: -------------------------------------------------------------------------------- 1 | ## ch_dba_scripts - ClickHouse DBA scripts 2 | 3 | A collection of shell scripts for ClickHouse database administrator (DBA). Tested on ClickHouse version 21.3.2.5-lts. 4 | 5 | #### [scripts/clickhouse_activity.sh](#clickhouse_activity). ClickHouse monitoring script, all information is displayed on one page. 6 | 7 | General information: 8 | - Displays ClickHouse version, hostname and IP address, CPU and Disks load; 9 | - Apache ZooKeeper cluster status; 10 | - ClickHouse disks; 11 | - Databases information; 12 | - Cluster information and status. 13 | 14 | Replication: 15 | - Replicated tables located on the local server; 16 | - Replication queue; 17 | - Replicated fetches; 18 | - Distribution queue; 19 | - Merges and part mutations currently in process for tables in the MergeTree family; 20 | - Mutations of MergeTree tables and their progress. 21 | 22 | Activity: 23 | - Queries that is being processed; 24 | - At the end, the last entries of the ClickHouse log file are displayed. 25 | 26 | [scripts/clickhouse_activity_refresh.sh](#clickhouse_activity). Fast refresh of the **clickhouse_activity.sh** script every 5 seconds. 27 | 28 | 29 | #### [scripts/clickhouse_information.sh](#clickhouse_information). ClickHouse information script, a single tape with details about ClickHouse work. 30 | 31 | General information: 32 | - ClickHouse version; 33 | - ClickHouse disks; 34 | - Storage policies and volumes; 35 | - Databases information; 36 | - Cluster information; 37 | - Macros settings for local cluster; 38 | - External dictionaries; 39 | - Zookeeper cluster information. 40 | 41 | Settings: 42 | - Settings that have been changed from its default values; 43 | - Settings profiles. Properties of configured setting profiles; 44 | - Settings profile elements. Content of the settings profile; 45 | - Error codes with the number of times they have been triggered. 46 | 47 | Users, roles, profiles, quotas, grants: 48 | - Users; 49 | - Grants; 50 | - Roles; 51 | - Role grants; 52 | - Settings profiles; 53 | - Row policies for the specified table; 54 | - Quotas; 55 | - Quota consumption for all users; 56 | - Quota limits. Information about maximums for all intervals of all quotas. 57 | 58 | Tables: 59 | - Tables compress ratio; 60 | - Top tables by size; 61 | - Parts of tables MergeTree families; 62 | - Tables in memory (Memory engine); 63 | - Detached parts of MergeTree tables. 64 | 65 | Replication: 66 | - Replicated tables located on the local server; 67 | - Replication queue; 68 | - Replicated fetches; 69 | - Distribution queue; 70 | - Merges and part mutations currently in process for tables in the MergeTree family; 71 | - Mutations of MergeTree tables and their progress. 72 | 73 | Queries: 74 | - Longest running queries; 75 | - Queries by memory consumption. 76 | 77 | System activity (from system.metric_log table): 78 | - last 14 days; 79 | - today by hours; 80 | - today by minute. 81 | 82 | Activity: 83 | - Queries that is being processed. Ordered by elapsed time; 84 | - Queries that is being processed. Ordered by memory usage. 85 | 86 | 87 | #### [scripts/clickhouse_information_metrics.sh](#clickhouse_information_metrics). Information script ClickHouse about metrics and events. 88 | 89 | Additional information about metrics and events: 90 | - System metrics; 91 | - Asynchronous metrics; 92 | - System events. 93 | 94 | 95 | #### Small scripts to manage ClickHouse and Apache ZooKeeper: 96 | - [scripts/clickhouse_start.sh](#clickhouse_start). Start ClickHouse systemctl service, confirmation is required. 97 | - [scripts/clickhouse_stop.sh](#clickhouse_stop). Stop ClickHouse systemctl service, confirmation is required. 98 | - [scripts/clickhouse_status.sh](#clickhouse_status). ClickHouse status. Additionally, ClickHouse processes and network connection are displayed. 99 | - [scripts/clickhouse_logs.sh](#clickhouse_logs). Shows the ClickHouse log file with auto-update. 100 | - [scripts/clickhouse_client.sh](#clickhouse_client). Launching clickhouse-client with current settings from **settings.txt** file. 101 | 102 | - [scripts/zookeeper_start.sh](#zookeeper_start). Start Apache ZooKeeper systemctl service, confirmation is required. 103 | - [scripts/zookeeper_stop.sh](#zookeeper_stop). Stop Apache ZooKeeper systemctl service, confirmation is required. 104 | - [scripts/zookeeper_status.sh](#zookeeper_status). Apache ZooKeeper status. Additionally, ZooKeeper processes, network connection and cluster status are displayed. 105 | - [scripts/zookeeper_logs.sh](#zookeeper_logs). Shows the Apache ZooKeeper log file with auto-update. 106 | 107 | 108 | ## Installation 109 | 110 | As user **root**, download the latest version of the scripts collection (see [Releases](https://github.com/Azmodey/ch_dba_scripts/releases) page): 111 | ``` 112 | # wget https://github.com/Azmodey/ch_dba_scripts/archive/1.3.0.tar.gz 113 | ``` 114 | 115 | Extract script files to separate directory (for example **~scripts/**) and grant the necessary execution rights: 116 | ``` 117 | # tar xvzf 1.3.0.tar.gz 118 | # mv ch_dba_scripts-1.3.0/scripts ~/scripts 119 | # chmod 700 ~/scripts/*.sh 120 | # chmod 600 ~/scripts/settings.txt 121 | ``` 122 | 123 | 124 | ## Setup 125 | 126 | Modify file **settings.txt**. Uncomment and correct the entries for your current ClickHouse connection settings. 127 | ``` 128 | CH_HOST="localhost" # Host name. Default: localhost 129 | CH_PORT="9000" # Port. Default: 9000 130 | 131 | CH_LOGIN="" # Administrative user login 132 | CH_PASSWORD="" # Administrative user password 133 | ``` 134 | 135 | If you plan to use Apache ZooKeeper cluster health monitoring, you should add this feature to the configuration file. 136 | Add the following lines to your ZooKeeper config file like **/opt/zookeeper/conf/zoo.cfg** : 137 | ``` 138 | # Whitelist for statistics 139 | 4lw.commands.whitelist=stat, mntr 140 | ``` 141 | 142 | 143 | --- 144 | ### clickhouse_activity 145 | 146 | ClickHouse monitoring script, all information is displayed on one page. 147 | 148 | General information: 149 | - Displays ClickHouse version, hostname and IP address, CPU and Disks load; 150 | - Apache ZooKeeper cluster status; 151 | - ClickHouse disks; 152 | - Databases information; 153 | - Cluster information and status. 154 | 155 | Replication: 156 | - Replicated tables located on the local server; 157 | - Replication queue; 158 | - Replicated fetches; 159 | - Distribution queue; 160 | - Merges and part mutations currently in process for tables in the MergeTree family; 161 | - Mutations of MergeTree tables and their progress. 162 | 163 | Activity: 164 | - Queries that is being processed; 165 | - At the end, the last entries of the ClickHouse log file are displayed. 166 | 167 | #### Setup: 168 | 169 | Change the value of the LOG_LINES parameter in the script, which is responsible for displaying the number of last lines of the ClickHouse log file. 170 | ``` 171 | LOG_LINES=0 # Number of ClickHouse log lines to display. 0 - disable output 172 | ``` 173 | 174 | Change the value of the ZooKeeperHosts parameter in the script, which is responsible for displaying Apache ZooKeeper cluster status. 175 | ``` 176 | # Apache ZooKeeper hosts 177 | ZooKeeperHosts="" # "" - disable output 178 | #ZooKeeperHosts="zoo_server_1 zoo_server_2 zoo_server_3" # Servers list, hostnames. Format: "server_1" "server_2" ... 179 | ``` 180 | 181 | #### Examples of work: 182 | 183 | Inserting data into tutorial.hits_v1 table with MergeTree engine. In active **Queries**, the INSERT operation is observed, the progress of the **Merge** operation is visible for tables of the MergeTree family. 184 | 185 | ![clickhouse_activity1](media/ch_activity_insert_mergetree.png) 186 | 187 | 188 | Inserting data into the tutorial.hits_replica table with the ReplicatedMergeTree engine. In active **Queries**, the INSERT operation is observed, the progress of the **Merge** operation for tables of the MergeTree family is visible, and in the **Replication queue** data is transferred to remote servers for tables of the ReplicatedMergeTree family. 189 | 190 | ![clickhouse_activity2](media/ch_activity_insert_replicatedmergetree.png) 191 | 192 | 193 | --- 194 | ### clickhouse_information 195 | 196 | ClickHouse information script, a single tape with details about ClickHouse work. 197 | 198 | General information: 199 | - ClickHouse version; 200 | - ClickHouse disks; 201 | - Storage policies and volumes; 202 | - Databases information; 203 | - Cluster information; 204 | - Macros settings for local cluster; 205 | - External dictionaries; 206 | - Zookeeper cluster information. 207 | 208 | Settings: 209 | - Settings that have been changed from its default values; 210 | - Settings profiles. Properties of configured setting profiles; 211 | - Settings profile elements. Content of the settings profile; 212 | - Error codes with the number of times they have been triggered. 213 | 214 | Users, roles, profiles, quotas, grants: 215 | - Users; 216 | - Grants; 217 | - Roles; 218 | - Role grants; 219 | - Settings profiles; 220 | - Row policies for the specified table; 221 | - Quotas; 222 | - Quota consumption for all users; 223 | - Quota limits. Information about maximums for all intervals of all quotas. 224 | 225 | Tables: 226 | - Tables compress ratio; 227 | - Top tables by size; 228 | - Parts of tables MergeTree families; 229 | - Tables in memory (Memory engine); 230 | - Detached parts of MergeTree tables. 231 | 232 | Replication: 233 | - Replicated tables located on the local server; 234 | - Replication queue; 235 | - Replicated fetches; 236 | - Distribution queue; 237 | - Merges and part mutations currently in process for tables in the MergeTree family; 238 | - Mutations of MergeTree tables and their progress. 239 | 240 | Queries: 241 | - Longest running queries; 242 | - Queries by memory consumption. 243 | 244 | System activity (from system.metric_log table): 245 | - last 14 days; 246 | - today by hours; 247 | - today by minute. 248 | 249 | Activity: 250 | - Queries that is being processed. Ordered by elapsed time; 251 | - Queries that is being processed. Ordered by memory usage. 252 | 253 | #### Examples of work: 254 | 255 | General information. 256 | 257 | ![clickhouse_information1](media/ch_information1_general.png) 258 | 259 | 260 | Settings. 261 | 262 | ![clickhouse_information2](media/ch_information2_settings.png) 263 | 264 | 265 | Users, roles, profiles, quotas, grants. 266 | 267 | ![clickhouse_information3](media/ch_information3_users.png) 268 | 269 | 270 | Tables. 271 | 272 | ![clickhouse_information4](media/ch_information4_tables.png) 273 | 274 | 275 | Replication and Queries. 276 | 277 | ![clickhouse_information5](media/ch_information5_replica_queries.png) 278 | 279 | 280 | System activity history and Queries that is being processed. 281 | 282 | ![clickhouse_information6](media/ch_information6_activity.png) 283 | 284 | 285 | --- 286 | #### clickhouse_information_metrics 287 | 288 | Additional information about metrics and events: 289 | - System metrics; 290 | - Asynchronous metrics; 291 | - System events. 292 | 293 | #### Examples of work: 294 | 295 | ![clickhouse_information_metrics1](media/ch_information_metrics.png) 296 | 297 | 298 | --- 299 | ### clickhouse_start 300 | 301 | Start ClickHouse systemctl service, confirmation is required. 302 | 303 | ``` 304 | Start ClickHouse (Y/N)? y 305 | 306 | ClickHouse status: 307 | ? clickhouse-server.service - ClickHouse Server (analytic DBMS for big data) 308 | Loaded: loaded (/etc/systemd/system/clickhouse-server.service; enabled; vendor preset: disabled) 309 | Active: active (running) since Thu 2021-03-18 13:07:45 MSK; 9ms ago 310 | Main PID: 8309 (clickhouse-serv) 311 | CGroup: /system.slice/clickhouse-server.service 312 | L-8309 /usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid 313 | 314 | Mar 18 13:07:45 c7clickhouse-1.local systemd[1]: Started ClickHouse Server (analytic DBMS for big data). 315 | ``` 316 | 317 | --- 318 | ### clickhouse_stop 319 | 320 | Stop ClickHouse systemctl service, confirmation is required. 321 | 322 | ``` 323 | Stop ClickHouse (Y/N)? y 324 | 325 | ClickHouse status: 326 | ? clickhouse-server.service - ClickHouse Server (analytic DBMS for big data) 327 | Loaded: loaded (/etc/systemd/system/clickhouse-server.service; enabled; vendor preset: disabled) 328 | Active: inactive (dead) since Thu 2021-03-18 13:07:24 MSK; 5ms ago 329 | Process: 6605 ExecStart=/usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid (code=exited, status=0/SUCCESS) 330 | Main PID: 6605 (code=exited, status=0/SUCCESS) 331 | 332 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Merging configuration file '/etc/clickhouse-server/config.d/connections.xml'. 333 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Merging configuration file '/etc/clickhouse-server/config.d/logger.xml'. 334 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Merging configuration file '/etc/clickhouse-server/config.d/macros_shard_and_replica_names.xml'. 335 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Merging configuration file '/etc/clickhouse-server/config.d/remote_servers_perftest_2shards_2replicas.xml'. 336 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Merging configuration file '/etc/clickhouse-server/config.d/settings_misc.xml'. 337 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Merging configuration file '/etc/clickhouse-server/config.d/zookeeper.xml'. 338 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Logging information to /var/log/clickhouse-server/clickhouse-server.log 339 | Mar 18 12:57:57 c7clickhouse-1.local clickhouse-server[6605]: Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log 340 | Mar 18 13:07:22 c7clickhouse-1.local systemd[1]: Stopping ClickHouse Server (analytic DBMS for big data)... 341 | Mar 18 13:07:24 c7clickhouse-1.local systemd[1]: Stopped ClickHouse Server (analytic DBMS for big data). 342 | ``` 343 | 344 | 345 | --- 346 | ### clickhouse_status 347 | 348 | ClickHouse status. Additionally, ClickHouse processes and network connection are displayed. 349 | 350 | ``` 351 | ClickHouse processes: 352 | UID PID PPID C STIME TTY TIME CMD 353 | root 6471 1671 0 12:55 pts/1 00:00:00 /bin/bash ./clickhouse_status.sh 354 | clickho+ 1379 1 0 11:39 ? 00:00:00 clickhouse-watchdog --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid 355 | clickho+ 1380 1379 1 11:39 ? 00:01:12 \_ /usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid 356 | 357 | ClickHouse version: 358 | ClickHouse server version 21.3.2.5 (official build). 359 | 360 | ClickHouse network connection: 361 | 8123 - HTTP Client, 9000 - TCP/IP Native Client, 9004 - communicating using MySQL protocol, 9009 - Inter-Server Replication 362 | tcp 0 0 0.0.0.0:8123 0.0.0.0:* LISTEN 1380/clickhouse-ser 363 | tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 1380/clickhouse-ser 364 | tcp 0 0 0.0.0.0:9004 0.0.0.0:* LISTEN 1380/clickhouse-ser 365 | tcp 0 0 0.0.0.0:9009 0.0.0.0:* LISTEN 1380/clickhouse-ser 366 | 367 | ClickHouse status: 368 | ? clickhouse-server.service - ClickHouse Server (analytic DBMS for big data) 369 | Loaded: loaded (/etc/systemd/system/clickhouse-server.service; enabled; vendor preset: disabled) 370 | Active: active (running) since Thu 2021-03-18 11:39:00 MSK; 1h 16min ago 371 | Main PID: 1379 (clckhouse-watch) 372 | CGroup: /system.slice/clickhouse-server.service 373 | +-1379 clickhouse-watchdog --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid 374 | L-1380 /usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid 375 | 376 | Mar 18 11:39:00 c7clickhouse-1.local systemd[1]: Started ClickHouse Server (analytic DBMS for big data). 377 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Processing configuration file '/etc/clickhouse-server/config.xml'. 378 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Merging configuration file '/etc/clickhouse-server/config.d/connections.xml'. 379 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Merging configuration file '/etc/clickhouse-server/config.d/logger.xml'. 380 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Merging configuration file '/etc/clickhouse-server/config.d/macros_shard_and_replica_names.xml'. 381 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Merging configuration file '/etc/clickhouse-server/config.d/remote_servers_perftest_2shards_2replicas.xml'. 382 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Merging configuration file '/etc/clickhouse-server/config.d/settings_misc.xml'. 383 | Mar 18 11:39:00 c7clickhouse-1.local clickhouse-server[1379]: Merging configuration file '/etc/clickhouse-server/config.d/zookeeper.xml'. 384 | Mar 18 11:39:01 c7clickhouse-1.local clickhouse-server[1379]: Logging information to /var/log/clickhouse-server/clickhouse-server.log 385 | Mar 18 11:39:01 c7clickhouse-1.local clickhouse-server[1379]: Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log 386 | ``` 387 | 388 | 389 | --- 390 | ### clickhouse_logs 391 | 392 | Shows the ClickHouse log file with auto-update. 393 | 394 | ``` 395 | ClickHouse log: /var/log/clickhouse-server/clickhouse-server.log 396 | 2021.03.18 11:39:21.429835 [ 1398 ] {} tutorial.hits_replica (8ed1a592-271d-4cc9-93fb-cddd16f78609): Became leader 397 | 2021.03.18 11:45:01.026999 [ 1382 ] {4d7cd6f0-ba91-4f16-a94f-559131e20919} executeQuery: Read 14 rows, 1.72 KiB in 0.004285805 sec., 3266 rows/sec., 400.35 KiB/sec. 398 | 2021.03.18 11:45:01.107899 [ 1382 ] {81906254-a729-47d2-b34f-5357aa891cd7} executeQuery: Read 1 rows, 1.00 B in 0.007399404 sec., 135 rows/sec., 135.15 B/sec. 399 | 2021.03.18 11:45:01.428841 [ 1382 ] {e60b79b0-dcf5-4a37-98d9-f747fc9d0064} executeQuery: Read 1 rows, 83.00 B in 0.015618538 sec., 64 rows/sec., 5.19 KiB/sec. 400 | 2021.03.18 11:45:01.462329 [ 1382 ] {e6eb7f62-0d7d-490a-9158-84690bcf109b} executeQuery: Read 106 rows, 2.79 KiB in 0.00530687 sec., 19974 rows/sec., 525.74 KiB/sec. 401 | ``` 402 | 403 | 404 | --- 405 | ### clickhouse_client 406 | 407 | Launching clickhouse-client with current settings from **settings.txt** file. 408 | 409 | ``` 410 | ClickHouse client version 21.3.2.5 (official build). 411 | Connecting to localhost:9000 as user default. 412 | Connected to ClickHouse server version 21.3.2 revision 54447. 413 | 414 | c7clickhouse-1.local :) 415 | ``` 416 | 417 | 418 | --- 419 | ### zookeeper_start 420 | 421 | Start Apache ZooKeeper systemctl service, confirmation is required. 422 | 423 | ``` 424 | Start Apache ZooKeeper (Y/N)? y 425 | 426 | Apache ZooKeeper service: 427 | ? zookeeper.service - ZooKeeper Service 428 | Loaded: loaded (/usr/lib/systemd/system/zookeeper.service; enabled; vendor preset: disabled) 429 | Active: active (running) since Thu 2021-03-18 13:06:46 MSK; 8ms ago 430 | Docs: http://zookeeper.apache.org 431 | Process: 7997 ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS) 432 | Process: 8047 ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS) 433 | Main PID: 8063 (java) 434 | CGroup: /system.slice/zookeeper.service 435 | L-8063 java -Dzookeeper.log.dir=/opt/zookeeper/bin/../logs -Dzookeeper.log.file=zookeeper-zookeeper-server-c7clickhouse-1.local.log -Dzookeeper.root.logger=INFO,CONSOLE -XX:+HeapDu... 436 | 437 | Mar 18 13:06:45 c7clickhouse-1.local systemd[1]: Starting ZooKeeper Service... 438 | Mar 18 13:06:45 c7clickhouse-1.local zkServer.sh[8047]: /usr/bin/java 439 | Mar 18 13:06:45 c7clickhouse-1.local zkServer.sh[8047]: ZooKeeper JMX enabled by default 440 | Mar 18 13:06:45 c7clickhouse-1.local zkServer.sh[8047]: Using config: /opt/zookeeper/conf/zoo.cfg 441 | Mar 18 13:06:46 c7clickhouse-1.local systemd[1]: Started ZooKeeper Service. 442 | ``` 443 | 444 | 445 | --- 446 | ### zookeeper_stop 447 | 448 | Stop Apache ZooKeeper systemctl service, confirmation is required. 449 | 450 | ``` 451 | Stop Apache ZooKeeper (Y/N)? y 452 | 453 | Apache ZooKeeper service: 454 | ? zookeeper.service - ZooKeeper Service 455 | Loaded: loaded (/usr/lib/systemd/system/zookeeper.service; enabled; vendor preset: disabled) 456 | Active: failed (Result: exit-code) since Thu 2021-03-18 13:07:02 MSK; 5ms ago 457 | Docs: http://zookeeper.apache.org 458 | Process: 8139 ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS) 459 | Process: 8047 ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS) 460 | Main PID: 8063 (code=exited, status=143) 461 | 462 | Mar 18 13:06:45 c7clickhouse-1.local zkServer.sh[8047]: Using config: /opt/zookeeper/conf/zoo.cfg 463 | Mar 18 13:06:46 c7clickhouse-1.local systemd[1]: Started ZooKeeper Service. 464 | Mar 18 13:07:01 c7clickhouse-1.local systemd[1]: Stopping ZooKeeper Service... 465 | Mar 18 13:07:01 c7clickhouse-1.local zkServer.sh[8139]: /usr/bin/java 466 | Mar 18 13:07:01 c7clickhouse-1.local zkServer.sh[8139]: ZooKeeper JMX enabled by default 467 | Mar 18 13:07:01 c7clickhouse-1.local zkServer.sh[8139]: Using config: /opt/zookeeper/conf/zoo.cfg 468 | Mar 18 13:07:01 c7clickhouse-1.local systemd[1]: zookeeper.service: main process exited, code=exited, status=143/n/a 469 | Mar 18 13:07:02 c7clickhouse-1.local systemd[1]: Stopped ZooKeeper Service. 470 | Mar 18 13:07:02 c7clickhouse-1.local systemd[1]: Unit zookeeper.service entered failed state. 471 | Mar 18 13:07:02 c7clickhouse-1.local systemd[1]: zookeeper.service failed. 472 | ``` 473 | 474 | --- 475 | ### zookeeper_status 476 | 477 | Apache ZooKeeper status. Additionally, ZooKeeper processes, network connection and cluster status are displayed. 478 | 479 | #### Setup: 480 | 481 | Change the value of the ZooKeeperHosts parameter in the script, which is responsible for displaying Apache ZooKeeper cluster status. 482 | ``` 483 | # Apache ZooKeeper hosts 484 | ZooKeeperHosts="" # "" - disable output 485 | #ZooKeeperHosts="zoo_server_1 zoo_server_2 zoo_server_3" # Servers list, hostnames. Format: "server_1" "server_2" ... 486 | ``` 487 | 488 | #### Examples of work: 489 | 490 | ``` 491 | Apache ZooKeeper processes: 492 | UID PID PPID C STIME TTY TIME CMD 493 | zookeep+ 979 1 0 11:38 ? 00:00:18 java -Dzookeeper.log.dir=/opt/zookeeper/bin/../logs -Dzookeeper.log.file=zookeeper-zookeeper-server-c7clickhouse-1.local.log -Dzookeeper.root.lo 494 | 495 | Apache ZooKeeper network connection: 496 | client port 2181. Cluster ports 2888 and 3888 (for leader), 3888 (for follower) 497 | tcp 0 0 0.0.0.0:2181 0.0.0.0:* LISTEN 979/java 498 | tcp 0 0 192.168.1.190:3888 0.0.0.0:* LISTEN 979/java 499 | 500 | Apache ZooKeeper service: 501 | ? zookeeper.service - ZooKeeper Service 502 | Loaded: loaded (/usr/lib/systemd/system/zookeeper.service; enabled; vendor preset: disabled) 503 | Active: active (running) since Thu 2021-03-18 11:38:20 MSK; 1h 21min ago 504 | Docs: http://zookeeper.apache.org 505 | Process: 932 ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS) 506 | Main PID: 979 (java) 507 | CGroup: /system.slice/zookeeper.service 508 | L-979 java -Dzookeeper.log.dir=/opt/zookeeper/bin/../logs -Dzookeeper.log.file=zookeeper-zookeeper-server-c7clickhouse-1.local.log -Dzookeeper.root.logger=INFO,CONSOLE -XX:+HeapDum... 509 | 510 | Mar 18 11:38:18 c7clickhouse-1.local systemd[1]: Starting ZooKeeper Service... 511 | Mar 18 11:38:18 c7clickhouse-1.local zkServer.sh[932]: /usr/bin/java 512 | Mar 18 11:38:18 c7clickhouse-1.local zkServer.sh[932]: ZooKeeper JMX enabled by default 513 | Mar 18 11:38:18 c7clickhouse-1.local zkServer.sh[932]: Using config: /opt/zookeeper/conf/zoo.cfg 514 | Mar 18 11:38:20 c7clickhouse-1.local systemd[1]: Started ZooKeeper Service. 515 | 516 | Apache ZooKeeper local node status: 517 | ZooKeeper JMX enabled by default 518 | Using config: /opt/zookeeper/bin/../conf/zoo.cfg 519 | Client port found: 2181. Client address: localhost. 520 | Mode: follower 521 | 522 | Apache ZooKeeper cluster status: 523 | [c7clickhouse-1] Mode: follower 524 | [c7clickhouse-2] Mode: follower 525 | [c7clickhouse-3] Mode: leader 526 | [c7clickhouse-4] Mode: follower 527 | ``` 528 | 529 | 530 | --- 531 | ### zookeeper_logs 532 | 533 | Shows the Apache ZooKeeper log file with auto-update. 534 | 535 | ``` 536 | Apache ZooKeeper log: /opt/zookeeper/logs/zookeeper-zookeeper-server-c7clickhouse-1.local.out 537 | 2021-03-18 11:38:41,114 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1255] - maxSessionTimeout set to 40000 538 | 2021-03-18 11:38:41,116 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ResponseCache@45] - Response cache size is initialized with value 400. 539 | 2021-03-18 11:38:41,116 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ResponseCache@45] - Response cache size is initialized with value 400. 540 | 2021-03-18 11:38:41,117 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@111] - zookeeper.pathStats.slotCapacity = 60 541 | 2021-03-18 11:38:41,118 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@112] - zookeeper.pathStats.slotDuration = 15 542 | 2021-03-18 11:38:41,118 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@113] - zookeeper.pathStats.maxDepth = 6 543 | 2021-03-18 11:38:41,118 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@114] - zookeeper.pathStats.initialDelay = 5 544 | 2021-03-18 11:38:41,118 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@115] - zookeeper.pathStats.delay = 5 545 | 2021-03-18 11:38:41,118 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@116] - zookeeper.pathStats.enabled = false 546 | 2021-03-18 11:38:41,120 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1471] - The max bytes for all large requests are set to 104857600 547 | 2021-03-18 11:38:41,120 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1485] - The large request threshold is set to -1 548 | ``` 549 | -------------------------------------------------------------------------------- /media/ch_activity_insert_mergetree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_activity_insert_mergetree.png -------------------------------------------------------------------------------- /media/ch_activity_insert_replicatedmergetree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_activity_insert_replicatedmergetree.png -------------------------------------------------------------------------------- /media/ch_information1_general.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information1_general.png -------------------------------------------------------------------------------- /media/ch_information2_settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information2_settings.png -------------------------------------------------------------------------------- /media/ch_information3_users.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information3_users.png -------------------------------------------------------------------------------- /media/ch_information4_tables.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information4_tables.png -------------------------------------------------------------------------------- /media/ch_information5_replica_queries.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information5_replica_queries.png -------------------------------------------------------------------------------- /media/ch_information6_activity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information6_activity.png -------------------------------------------------------------------------------- /media/ch_information_metrics.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azmodey/ch_dba_scripts/fe5faadd512a0afbed74920240b4c261cdd94d98/media/ch_information_metrics.png -------------------------------------------------------------------------------- /scripts/clickhouse_activity.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse activity 4 | 5 | 6 | # ClickHouse log file 7 | LOG_LINES=0 # Number of ClickHouse log lines to display. 0 - disable output 8 | LOG_FILENAME="/var/log/clickhouse-server/clickhouse-server.log" # ClickHouse log file name 9 | 10 | # Apache ZooKeeper hosts 11 | ZooKeeperHosts="" # "" - disable output 12 | #ZooKeeperHosts="zoo_server_1 zoo_server_2 zoo_server_3" # Servers list, hostnames. Format: "server_1" "server_2" ... 13 | 14 | 15 | # ------------------------------------------------ 16 | # ClickHouse client 17 | source ./settings.txt 18 | 19 | if [[ $CH_HOST && $CH_PORT ]]; then 20 | CH_CLIENT="clickhouse-client --host=$CH_HOST --port=$CH_PORT" 21 | else 22 | CH_CLIENT="clickhouse-client" 23 | fi 24 | 25 | if [[ $CH_LOGIN && $CH_PASSWORD ]]; then 26 | CH_CLIENT="$CH_CLIENT --user=$CH_LOGIN --password=$CH_PASSWORD" 27 | fi 28 | 29 | 30 | # Colors 31 | GREYDARK='\033[1;30m' 32 | RED='\033[0;31m' 33 | REDLIGHT='\033[1;31m' 34 | GREEN='\033[0;32m' 35 | GREENLIGHT='\033[1;32m' 36 | ORANGE='\033[0;33m' 37 | YELLOW='\033[1;33m' 38 | BLUE='\033[0;34m' 39 | BLUELIGHT='\033[1;34m' 40 | PURPLE='\033[0;35m' 41 | PURPLELIGHT='\033[1;35m' 42 | CYAN='\033[0;36m' 43 | CYANLIGHT='\033[1;36m' 44 | WHITE='\033[1;37m' 45 | # 46 | NC='\033[0m' # No Color 47 | BOLD='\033[1m' 48 | UNDERLINE='\033[4m' 49 | 50 | 51 | # ------------------------------------------------ 52 | 53 | # System 54 | PLATFORM=`awk -F= '/^NAME/{print $2}' /etc/os-release | tr -d '"'` # Red Hat Enterprise Linux Server / CentOS Linux / Debian GNU/Linux / Ubuntu 55 | 56 | 57 | # Apache ZooKeeper 58 | ZOO_STATUS="| " 59 | for val in $ZooKeeperHosts; do 60 | ZooKeeperMode=`echo stat | nc $val 2181 | grep Mode` 61 | 62 | if [[ $ZooKeeperMode = *Mode* ]]; then 63 | ZOO_STATUS="$ZOO_STATUS[$val] ${GREENLIGHT}$ZooKeeperMode${NC} | " 64 | else 65 | ZOO_STATUS="$ZOO_STATUS[$val] ${REDLIGHT}Fail${NC} | " 66 | fi 67 | 68 | done 69 | 70 | 71 | # ClickHouse clusters 72 | CLICKHOUSE_CLUSTERS=`$CH_CLIENT --query "SELECT distinct host_name FROM system.clusters WHERE host_name not in ('127.0.0.1', '127.0.0.2', 'localhost')"` 73 | 74 | CLICKHOUSE_CLUSTER_STATUS="| " 75 | for cluster in $CLICKHOUSE_CLUSTERS ; do 76 | 77 | if [[ $CH_HOST && $CH_PORT ]]; then 78 | CH_CLIENT_CL="clickhouse-client --host=$cluster --port=$CH_PORT" 79 | else 80 | CH_CLIENT_CL="clickhouse-client --host=$cluster" 81 | fi 82 | 83 | if [[ $CH_LOGIN && $CH_PASSWORD ]]; then 84 | CH_CLIENT_CL="$CH_CLIENT_CL --user=$CH_LOGIN --password=$CH_PASSWORD" 85 | fi 86 | 87 | CLUSTER_VER=`$CH_CLIENT_CL --query "SELECT version()" 2> /dev/null` 88 | #echo "[$cluster] ver [$CLUSTER_VER]" 89 | 90 | re='^[0-9]+([.][0-9]+)+([.][0-9]+)+([.][0-9]+)$' 91 | if ! [[ $CLUSTER_VER =~ $re ]] ; then 92 | # Not a number 93 | CLICKHOUSE_CLUSTER_STATUS="$CLICKHOUSE_CLUSTER_STATUS[$cluster] ${REDLIGHT}Fail${NC} | " 94 | else 95 | # Number 96 | CLICKHOUSE_CLUSTER_STATUS="$CLICKHOUSE_CLUSTER_STATUS[$cluster] ${GREENLIGHT}Active${NC} | " 97 | fi 98 | 99 | done 100 | 101 | 102 | # Title (1st line) 103 | DATE=$(date '+%d.%m.%Y %H:%M:%S') 104 | HOST=`hostname --short` 105 | HOSTIP=`hostname -I | xargs` 106 | UPTIME=`uptime` 107 | UPTIME=${UPTIME#*load average: } 108 | 109 | if [[ $PLATFORM == "Red Hat Enterprise Linux Server" || $PLATFORM == "CentOS Linux" ]]; then 110 | IOSTAT_AWAIT=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 11` 111 | IOSTAT_UTIL=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 15` 112 | fi 113 | if [[ $PLATFORM == "Debian GNU/Linux" ]]; then 114 | IOSTAT_R_AWAIT=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 11 | sed 's/,/./g'` 115 | IOSTAT_W_AWAIT=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 12 | sed 's/,/./g'` 116 | IOSTAT_UTIL=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 17 | sed 's/,/./g'` 117 | 118 | IOSTAT_AWAIT=`awk "BEGIN {print ($IOSTAT_R_AWAIT+$IOSTAT_W_AWAIT)/2}"` 119 | fi 120 | if [[ $PLATFORM == "Ubuntu" ]]; then 121 | IOSTAT_R_AWAIT=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 7 | sed 's/,/./g'` 122 | IOSTAT_W_AWAIT=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 13 | sed 's/,/./g'` 123 | IOSTAT_UTIL=`iostat -d -x -g ALL | grep ALL | tr -s " " | cut -d " " -f 22 | sed 's/,/./g'` 124 | 125 | IOSTAT_AWAIT=`awk "BEGIN {print ($IOSTAT_R_AWAIT+$IOSTAT_W_AWAIT)/2}"` 126 | fi 127 | 128 | CLICKHOUSE_VER=`clickhouse-server -V` 129 | CLICKHOUSE_VER=${CLICKHOUSE_VER#"ClickHouse server version "} 130 | CLICKHOUSE_VER=`echo $CLICKHOUSE_VER | rev | cut -c 2- | rev` 131 | 132 | STATUS="${GREENLIGHT}[$HOST ($HOSTIP) / ClickHouse $CLICKHOUSE_VER]${YELLOW}" 133 | 134 | 135 | # ------------------------------------------------ 136 | 137 | # Title (1st line) 138 | echo -e "${YELLOW}[$DATE] $STATUS [CPU load (1/5/15 min): $UPTIME] [Disk load: util $IOSTAT_UTIL %, await $IOSTAT_AWAIT ms] ${NC}" 139 | 140 | # Title (2nd line) 141 | if [[ $ZooKeeperHosts ]]; then 142 | echo -e "${GREENLIGHT}Apache ZooKeeper cluster:${NC} $ZOO_STATUS" 143 | fi 144 | 145 | echo 146 | 147 | 148 | # ------------------------------------------------ 149 | 150 | echo -e "${GREENLIGHT}ClickHouse disks:${NC}" 151 | $CH_CLIENT --query "SELECT name, path, formatReadableSize(sum(free_space)) as free_space, formatReadableSize(sum(total_space)) as total_space, 152 | formatReadableSize(sum(keep_free_space)) as keep_free_space, type 153 | FROM system.disks 154 | GROUP BY name, path, type FORMAT PrettyCompact" 155 | echo 156 | 157 | echo -e "${GREENLIGHT}Databases:${NC}" 158 | $CH_CLIENT --query "SELECT d.name, formatReadableSize(sum(p.bytes_on_disk)) as size, d.engine, d.data_path, d.metadata_path, d.uuid 159 | FROM system.databases d LEFT JOIN system.parts p ON d.name = p.database 160 | GROUP BY d.name, d.engine, d.data_path, d.metadata_path, d.uuid 161 | ORDER BY sum(p.bytes_on_disk) DESC FORMAT PrettyCompact" 162 | echo 163 | 164 | echo -e "${GREENLIGHT}Clusters:${NC}" 165 | $CH_CLIENT --query " 166 | SELECT cluster, shard_num, shard_weight, replica_num, host_name, host_address, port, is_local, errors_count, estimated_recovery_time 167 | FROM system.clusters 168 | WHERE host_name not in ('127.0.0.1', '127.0.0.2', 'localhost') 169 | FORMAT PrettyCompact" 170 | echo -e "${GREENLIGHT}Cluster status:${NC} $CLICKHOUSE_CLUSTER_STATUS" 171 | echo 172 | 173 | 174 | replicated_tables=`$CH_CLIENT --query "SELECT table FROM system.replicas"` 175 | if [[ ${#replicated_tables} >0 ]]; then 176 | echo -e "${GREENLIGHT}Replicated tables located on the local server:${NC}" 177 | $CH_CLIENT --query " 178 | SELECT database, table, engine, is_leader as leader, is_readonly as readonly, parts_to_check, 179 | queue_size, inserts_in_queue as queue_insert, merges_in_queue as queue_merge, -- очередь 180 | log_max_index, log_pointer, total_replicas as tot_repl, active_replicas as act_repl -- ZooKeeper 181 | FROM system.replicas FORMAT PrettyCompact" 182 | echo 183 | LOG_LINES=$((LOG_LINES-5)) 184 | fi 185 | 186 | 187 | replication_problems=`$CH_CLIENT --query "SELECT table FROM system.replicas 188 | WHERE is_readonly OR is_session_expired OR future_parts > 20 OR parts_to_check > 10 OR queue_size > 20 189 | OR inserts_in_queue > 10 OR log_max_index - log_pointer > 10 OR total_replicas < 2 190 | OR active_replicas < total_replicas"` 191 | if [[ ${#replication_problems} >0 ]]; then 192 | echo -e "${REDLIGHT}Replication problems:${NC}" 193 | $CH_CLIENT --query " 194 | SELECT database, table, is_leader, total_replicas, active_replicas 195 | FROM system.replicas 196 | WHERE is_readonly 197 | OR is_session_expired 198 | OR future_parts > 20 199 | OR parts_to_check > 10 200 | OR queue_size > 20 201 | OR inserts_in_queue > 10 202 | OR log_max_index - log_pointer > 10 203 | OR total_replicas < 2 204 | OR active_replicas < total_replicas FORMAT PrettyCompact" 205 | echo 206 | LOG_LINES=$((LOG_LINES-5)) 207 | fi 208 | 209 | 210 | replication_queue=`$CH_CLIENT --query "SELECT table FROM system.replication_queue"` 211 | if [[ ${#replication_queue} >0 ]]; then 212 | echo -e "${GREENLIGHT}Replication queue.${NC} Tasks from replication queues stored in ZooKeeper for tables in the ReplicatedMergeTree family:" 213 | $CH_CLIENT --query " 214 | SELECT database, table, replica_name, position, node_name, type, create_time, num_tries, last_attempt_time, num_postponed, last_postpone_time 215 | FROM system.replication_queue FORMAT PrettyCompact" 216 | echo 217 | LOG_LINES=$((LOG_LINES-5)) 218 | fi 219 | 220 | 221 | replicated_fetches=`$CH_CLIENT --query "SELECT table FROM system.replicated_fetches"` 222 | if [[ ${#replicated_fetches} >0 ]]; then 223 | echo -e "${GREENLIGHT}Replicated fetches.${NC} Currently running background fetches:" 224 | $CH_CLIENT --query " 225 | SELECT database, table, elapsed, progress, result_part_name, partition_id, source_replica_hostname, source_replica_port, interserver_scheme, to_detached, thread_id 226 | FROM system.replicated_fetches ORDER BY database, table FORMAT PrettyCompact" 227 | echo 228 | LOG_LINES=$((LOG_LINES-5)) 229 | fi 230 | 231 | 232 | distribution_queue=`$CH_CLIENT --query "SELECT table FROM system.distribution_queue"` 233 | if [[ ${#distribution_queue} >0 ]]; then 234 | echo -e "${GREENLIGHT}Distribution queue.${NC} Local files that are in the queue to be sent to the shards. Contain new parts that are created by inserting new data into the Distributed table:" 235 | $CH_CLIENT --query "SELECT database, table, is_blocked, error_count, data_files, data_compressed_bytes, last_exception FROM system.distribution_queue ORDER BY database, table FORMAT PrettyCompact" 236 | echo 237 | LOG_LINES=$((LOG_LINES-5)) 238 | fi 239 | 240 | 241 | merges=`$CH_CLIENT --query "SELECT table FROM system.merges"` 242 | if [[ ${#merges} >0 ]]; then 243 | echo -e "${YELLOW}Merges and part mutations currently in process for tables in the MergeTree family:${NC}" 244 | $CH_CLIENT --query "SELECT database, table, elapsed, progress, num_parts, rows_read, rows_written, memory_usage, merge_type, merge_algorithm FROM system.merges FORMAT PrettyCompact" 245 | echo 246 | LOG_LINES=$((LOG_LINES-5)) 247 | fi 248 | 249 | 250 | table_mutations=`$CH_CLIENT --query "SELECT table FROM system.mutations"` 251 | if [[ ${#table_mutations} >0 ]]; then 252 | echo -e "${YELLOW}Mutations of MergeTree tables and their progress (ALTER command):${NC}" 253 | $CH_CLIENT --query "SELECT database, table, mutation_id, command, create_time, parts_to_do, is_done, latest_failed_part, latest_fail_time, latest_fail_reason FROM system.mutations FORMAT PrettyCompact" 254 | echo 255 | LOG_LINES=$((LOG_LINES-5)) 256 | fi 257 | 258 | 259 | echo -e "${GREENLIGHT}Queries that is being processed.${NC} Full SQL text by query_id: SELECT distinct query FROM system.query_log WHERE query_id='' FORMAT TabSeparatedRaw" 260 | $CH_CLIENT --query "SELECT user, address as client, elapsed as time_seconds, formatReadableSize(memory_usage) as memory, formatReadableSize(read_bytes) as read, read_rows, total_rows_approx as total_rows, query_id, substring(query,1,40) as query 261 | FROM system.processes 262 | ORDER BY elapsed desc FORMAT PrettyCompact" 263 | echo 264 | 265 | 266 | # show ClickHouse log 267 | if [[ $LOG_LINES -gt 0 ]]; then 268 | echo -e "${GREENLIGHT}ClickHouse log:${NC} $LOG_FILENAME" 269 | tail --lines=$LOG_LINES $LOG_FILENAME 270 | fi 271 | -------------------------------------------------------------------------------- /scripts/clickhouse_activity_refresh.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Refresh clickhouse_activity.sh script 4 | 5 | i=0 6 | while [ i==0 ] 7 | do 8 | ./clickhouse_activity.sh > clickhouse_activity.txt 9 | clear 10 | cat ./clickhouse_activity.txt 11 | sleep 5 12 | done 13 | -------------------------------------------------------------------------------- /scripts/clickhouse_client.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse client 4 | 5 | # ------------------------------------------------ 6 | source ./settings.txt 7 | 8 | if [[ $CH_HOST && $CH_PORT ]]; then 9 | CH_CLIENT="clickhouse-client --host=$CH_HOST --port=$CH_PORT" 10 | else 11 | CH_CLIENT="clickhouse-client" 12 | fi 13 | 14 | if [[ $CH_LOGIN && $CH_PASSWORD ]]; then 15 | CH_CLIENT="$CH_CLIENT --user=$CH_LOGIN --password=$CH_PASSWORD" 16 | fi 17 | 18 | 19 | # ------------------------------------------------ 20 | 21 | $CH_CLIENT 22 | #$CH_CLIENT --multiline 23 | -------------------------------------------------------------------------------- /scripts/clickhouse_information.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse information 4 | 5 | 6 | # ------------------------------------------------ 7 | # ClickHouse client 8 | source ./settings.txt 9 | 10 | if [[ $CH_HOST && $CH_PORT ]]; then 11 | CH_CLIENT="clickhouse-client --host=$CH_HOST --port=$CH_PORT" 12 | else 13 | CH_CLIENT="clickhouse-client" 14 | fi 15 | 16 | if [[ $CH_LOGIN && $CH_PASSWORD ]]; then 17 | CH_CLIENT="$CH_CLIENT --user=$CH_LOGIN --password=$CH_PASSWORD" 18 | fi 19 | 20 | 21 | # Colors 22 | GREYDARK='\033[1;30m' 23 | RED='\033[0;31m' 24 | REDLIGHT='\033[1;31m' 25 | GREEN='\033[0;32m' 26 | GREENLIGHT='\033[1;32m' 27 | ORANGE='\033[0;33m' 28 | YELLOW='\033[1;33m' 29 | BLUE='\033[0;34m' 30 | BLUELIGHT='\033[1;34m' 31 | PURPLE='\033[0;35m' 32 | PURPLELIGHT='\033[1;35m' 33 | CYAN='\033[0;36m' 34 | CYANLIGHT='\033[1;36m' 35 | WHITE='\033[1;37m' 36 | # 37 | NC='\033[0m' # No Color 38 | BOLD='\033[1m' 39 | UNDERLINE='\033[4m' 40 | 41 | 42 | # ------------------------------------------------ 43 | 44 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 45 | 46 | clickhouse-server -V 47 | $CH_CLIENT --query="SELECT 'ClickHouse ' || version() || ' at ' || hostName() || '. Start time: ' || toString(now()-uptime())" --format=TSVRaw 48 | echo 49 | 50 | echo "ClickHouse disks:" 51 | $CH_CLIENT --query "SELECT name, path, formatReadableSize(sum(free_space)) as free_space, formatReadableSize(sum(total_space)) as total_space, 52 | formatReadableSize(sum(keep_free_space)) as keep_free_space, type 53 | FROM system.disks 54 | GROUP BY name, path, type FORMAT PrettyCompact" 55 | echo 56 | 57 | echo "Storage policies and volumes:" 58 | $CH_CLIENT --query "select * from system.storage_policies FORMAT PrettyCompact" 59 | echo 60 | 61 | echo "Databases:" 62 | $CH_CLIENT --query "SELECT d.name, formatReadableSize(sum(p.bytes_on_disk)) as size, d.engine, d.data_path, d.metadata_path, d.uuid 63 | FROM system.databases d LEFT JOIN system.parts p ON d.name = p.database 64 | GROUP BY d.name, d.engine, d.data_path, d.metadata_path, d.uuid 65 | ORDER BY sum(p.bytes_on_disk) DESC FORMAT PrettyCompact" 66 | echo 67 | 68 | echo "Clusters:" 69 | $CH_CLIENT --query "select cluster, shard_num, shard_weight, replica_num, host_name, host_address, port, is_local, errors_count, estimated_recovery_time as recovery_time from system.clusters FORMAT PrettyCompact" 70 | echo 71 | 72 | echo "Macros settings for local cluster:" 73 | $CH_CLIENT --query "SELECT * FROM system.macros FORMAT PrettyCompact" 74 | echo 75 | 76 | echo "External dictionaries:" 77 | $CH_CLIENT --query " 78 | SELECT database, name, status, origin, type, key, formatReadableSize(bytes_allocated) as size, query_count, hit_rate, element_count, load_factor, source, last_exception 79 | FROM system.dictionaries FORMAT PrettyCompact" 80 | echo 81 | 82 | echo "Zookeeper:" 83 | $CH_CLIENT --query="SELECT name, value, czxid, mzxid, ctime, mtime, version as ver, cversion as cver, aversion as aver, ephemeralOwner as ephOwner, dataLength, numChildren, pzxid, path FROM system.zookeeper WHERE path='/'" --format=PrettyCompactMonoBlock 84 | echo 85 | 86 | 87 | 88 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 89 | 90 | echo "Settings that have been changed:" 91 | $CH_CLIENT --query="SELECT name, value, changed, substring(description,1,80) as description, min, max, readonly, type FROM system.settings WHERE changed FORMAT PrettyCompact" 92 | echo 93 | 94 | echo "Settings profiles. Properties of configured setting profiles:" 95 | $CH_CLIENT --query="SELECT * FROM system.settings_profiles FORMAT PrettyCompact" 96 | echo 97 | 98 | echo "Settings profile elements. Content of the settings profile:" 99 | $CH_CLIENT --query="SELECT * FROM system.settings_profile_elements FORMAT PrettyCompact" 100 | echo 101 | 102 | echo "Error codes with the number of times they have been triggered:" 103 | $CH_CLIENT --query="SELECT * FROM system.errors FORMAT PrettyCompact" 104 | echo 105 | 106 | 107 | 108 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 109 | 110 | echo "Users:" 111 | $CH_CLIENT --query "SELECT name, storage, auth_type, auth_params, host_ip, host_names, default_roles_all, default_roles_list, default_roles_except FROM system.users FORMAT PrettyCompact" 112 | echo 113 | 114 | echo "Grants. Privileges granted to ClickHouse user accounts:" 115 | $CH_CLIENT --query "SELECT * FROM system.grants FORMAT PrettyCompact" 116 | echo 117 | 118 | echo "Roles:" 119 | $CH_CLIENT --query "SELECT * FROM system.roles FORMAT PrettyCompact" 120 | echo 121 | 122 | echo "Role grants:" 123 | $CH_CLIENT --query "SELECT * FROM system.role_grants FORMAT PrettyCompact" 124 | echo 125 | 126 | echo "Settings profiles:" 127 | $CH_CLIENT --query "SELECT * FROM system.settings_profiles FORMAT PrettyCompact" 128 | echo 129 | 130 | echo "Row policies for the specified table:" 131 | $CH_CLIENT --query "SELECT * FROM system.row_policies FORMAT PrettyCompact" 132 | echo 133 | 134 | echo "Quotas:" 135 | $CH_CLIENT --query "SELECT * FROM system.quotas FORMAT PrettyCompact" 136 | echo 137 | 138 | echo "Quota consumption for all users:" 139 | $CH_CLIENT --query "SELECT quota_name, quota_key, is_current, queries, max_queries, errors, result_rows, result_bytes, read_rows, read_bytes, execution_time FROM system.quotas_usage FORMAT PrettyCompact" 140 | echo 141 | 142 | echo "Quota limits. Information about maximums for all intervals of all quotas:" 143 | $CH_CLIENT --query " 144 | SELECT quota_name, duration, is_randomized_interval as rand_interval, max_queries as queries, max_query_selects as query_selects, 145 | max_query_inserts as query_inserts, max_errors, max_result_rows as result_rows, max_result_bytes as result_bytes, 146 | max_read_rows as read_rows, max_read_bytes as read_bytes, max_execution_time as exec_time 147 | FROM system.quota_limits FORMAT PrettyCompact" 148 | echo 149 | 150 | echo "SHOW ACCESS - all users, roles, profiles, etc. and all their grants:" 151 | $CH_CLIENT --query "SHOW ACCESS FORMAT PrettyCompact" 152 | echo 153 | 154 | 155 | 156 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 157 | 158 | echo "Tables compress ratio:" 159 | $CH_CLIENT --query " 160 | SELECT database, table, count(*) AS parts, 161 | uniq(partition) AS partitions, 162 | sum(marks) AS marks, 163 | sum(rows) AS rows, 164 | formatReadableSize(sum(data_compressed_bytes)) AS compressed, 165 | formatReadableSize(sum(data_uncompressed_bytes)) AS uncompressed, 166 | round((sum(data_compressed_bytes) / sum(data_uncompressed_bytes)) * 100., 2) AS percentage 167 | FROM system.parts WHERE active='1' 168 | GROUP BY database, table 169 | ORDER BY rows DESC 170 | LIMIT 50 FORMAT PrettyCompact" 171 | echo 172 | 173 | echo "Top tables by size:" 174 | $CH_CLIENT --query " 175 | SELECT database, name, engine, storage_policy, total_rows as rows, formatReadableSize(sum(total_bytes)) as size, is_temporary, metadata_modification_time 176 | FROM system.tables 177 | --WHERE database != 'system' 178 | WHERE total_rows > 0 179 | GROUP BY database, name, engine, is_temporary, metadata_modification_time, storage_policy, total_rows, total_bytes 180 | ORDER BY sum(total_bytes) DESC 181 | LIMIT 20 FORMAT PrettyCompact" 182 | echo 183 | 184 | echo "Parts of tables MergeTree families:" 185 | $CH_CLIENT --query " 186 | SELECT database, table, engine, disk_name, countIf(active) data_parts_active, countIf(not active) data_parts_not_active 187 | FROM system.parts 188 | GROUP BY database, table, engine, disk_name 189 | ORDER BY database, table DESC 190 | LIMIT 20 FORMAT PrettyCompact" 191 | echo 192 | 193 | echo "Tables in memory (Memory engine):" 194 | $CH_CLIENT --query " 195 | SELECT database, name, formatReadableSize(total_bytes) 196 | FROM system.tables 197 | WHERE engine = 'Memory' 198 | ORDER BY total_bytes DESC 199 | LIMIT 10 FORMAT PrettyCompact" 200 | echo 201 | 202 | echo "Detached parts of MergeTree tables:" 203 | $CH_CLIENT --query "SELECT * FROM system.detached_parts ORDER BY database, table FORMAT PrettyCompact" 204 | echo 205 | 206 | 207 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 208 | 209 | echo "Replicated tables located on the local server:" 210 | $CH_CLIENT --query " 211 | SELECT database, table, engine, is_leader as leader, is_readonly as readonly, parts_to_check, 212 | queue_size, inserts_in_queue as queue_insert, merges_in_queue as queue_merge, -- очередь 213 | log_max_index, log_pointer, total_replicas as tot_repl, active_replicas as act_repl -- ZooKeeper 214 | FROM system.replicas FORMAT PrettyCompact" 215 | echo 216 | 217 | echo "Replication queue. Tasks from replication queues stored in ZooKeeper for tables in the ReplicatedMergeTree family:" 218 | $CH_CLIENT --query " 219 | SELECT database, table, replica_name, position, node_name, type, create_time, num_tries, last_attempt_time, num_postponed, last_postpone_time 220 | FROM system.replication_queue FORMAT PrettyCompact" 221 | echo 222 | 223 | echo "Replicated fetches. Currently running background fetches:" 224 | $CH_CLIENT --query " 225 | SELECT database, table, elapsed, progress, result_part_name, partition_id, source_replica_hostname, source_replica_port, interserver_scheme, to_detached, thread_id 226 | FROM system.replicated_fetches ORDER BY database, table FORMAT PrettyCompact" 227 | echo 228 | 229 | echo "Distribution queue. Local files that are in the queue to be sent to the shards. Contain new parts that are created by inserting new data into the Distributed table:" 230 | $CH_CLIENT --query "SELECT database, table, is_blocked, error_count, data_files, data_compressed_bytes, last_exception FROM system.distribution_queue ORDER BY database, table FORMAT PrettyCompact" 231 | echo 232 | 233 | echo "Merges and part mutations currently in process for tables in the MergeTree family:" 234 | $CH_CLIENT --query "SELECT database, table, elapsed, progress, num_parts, rows_read, rows_written, memory_usage, merge_type, merge_algorithm FROM system.merges FORMAT PrettyCompact" 235 | echo 236 | 237 | echo "Mutations of MergeTree tables and their progress (ALTER command):" 238 | echo "- kill: KILL MUTATION WHERE database = 'default' AND table = 'table'" 239 | $CH_CLIENT --query "SELECT database, table, mutation_id, command, create_time, parts_to_do, is_done, latest_failed_part, latest_fail_time, latest_fail_reason FROM system.mutations FORMAT PrettyCompact" 240 | echo 241 | 242 | 243 | 244 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 245 | 246 | echo "Longest running queries:" 247 | $CH_CLIENT --query " 248 | SELECT user, 249 | --client_hostname AS host, 250 | --client_name AS client, 251 | --formatDateTime(query_start_time, '%T') AS started, 252 | query_duration_ms / 1000 AS dur_sec, 253 | round(memory_usage / 1048576) AS mem_mb, 254 | result_rows, 255 | formatReadableSize(result_bytes) AS result_size, 256 | read_rows, 257 | formatReadableSize(read_bytes) AS read_size, 258 | written_rows as write_rows, 259 | formatReadableSize(written_bytes) AS write_size, 260 | query_id, 261 | substring(query,1,30) as query 262 | FROM system.query_log 263 | WHERE type = 2 264 | ORDER BY query_duration_ms DESC 265 | LIMIT 10 FORMAT PrettyCompact" 266 | echo 267 | 268 | 269 | echo "Queries by memory consumption:" 270 | $CH_CLIENT --query " 271 | SELECT user, 272 | --client_hostname AS host, 273 | --client_name AS client, 274 | --formatDateTime(query_start_time, '%T') AS started, 275 | query_duration_ms / 1000 AS dur_sec, 276 | round(memory_usage / 1048576) AS mem_mb, 277 | result_rows, 278 | formatReadableSize(result_bytes) AS result_size, 279 | read_rows, 280 | formatReadableSize(read_bytes) AS read_size, 281 | written_rows as write_rows, 282 | formatReadableSize(written_bytes) AS write_size, 283 | query_id, 284 | substring(query,1,30) as query 285 | FROM system.query_log 286 | WHERE type = 2 287 | ORDER BY mem_mb DESC 288 | LIMIT 10 FORMAT PrettyCompact" 289 | echo 290 | 291 | 292 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 293 | 294 | echo "System activity:" 295 | 296 | MAX_USER_TIME=`$CH_CLIENT --query " 297 | SELECT user_time FROM ( 298 | SELECT toStartOfDay(event_time) AS time, sum(ProfileEvent_UserTimeMicroseconds) AS user_time 299 | FROM system.metric_log WHERE event_date >= date_sub(DAY, 14, today()) GROUP BY time ORDER BY user_time DESC LIMIT 1 300 | )"` 301 | 302 | MAX_INSERTED_ROWS=`$CH_CLIENT --query " 303 | SELECT inserted_rows FROM ( 304 | SELECT toStartOfDay(event_time) AS time, sum(ProfileEvent_InsertedRows) as inserted_rows 305 | FROM system.metric_log WHERE event_date >= date_sub(DAY, 14, today()) GROUP BY time ORDER BY inserted_rows DESC LIMIT 1 306 | )"` 307 | 308 | MAX_SELECTED_ROWS=`$CH_CLIENT --query " 309 | SELECT selected_rows FROM ( 310 | SELECT toStartOfDay(event_time) AS time, sum(ProfileEvent_SelectedRows) as selected_rows 311 | FROM system.metric_log WHERE event_date >= date_sub(DAY, 14, today()) GROUP BY time ORDER BY selected_rows DESC LIMIT 1 312 | )"` 313 | 314 | MAX_MERGED_ROWS=`$CH_CLIENT --query " 315 | SELECT merged_rows FROM ( 316 | SELECT toStartOfDay(event_time) AS time, sum(ProfileEvent_MergedRows) as merged_rows 317 | FROM system.metric_log WHERE event_date >= date_sub(DAY, 14, today()) GROUP BY time ORDER BY merged_rows DESC LIMIT 1 318 | )"` 319 | 320 | # 14 days 321 | $CH_CLIENT --query " 322 | SELECT toStartOfDay(event_time) AS time, 323 | sum(ProfileEvent_UserTimeMicroseconds) AS user_time, 324 | bar(user_time, 0, $MAX_USER_TIME, 15) AS user_time_bar, 325 | sum(ProfileEvent_InsertedRows) as inserted_rows, 326 | bar(inserted_rows, 0, $MAX_INSERTED_ROWS, 15) AS insert_rows_bar, 327 | formatReadableSize(sum(ProfileEvent_InsertedBytes)) as inserted, 328 | sum(ProfileEvent_SelectedRows) as selected_rows, 329 | bar(selected_rows, 0, $MAX_SELECTED_ROWS, 15) AS selected_rows_bar, 330 | formatReadableSize(sum(ProfileEvent_SelectedBytes)) as selected, 331 | sum(ProfileEvent_MergedRows) as merged_rows, 332 | bar(merged_rows, 0, $MAX_MERGED_ROWS, 15) AS merged_rows_bar 333 | FROM system.metric_log 334 | WHERE event_date >= date_sub(DAY, 14, today()) 335 | GROUP BY time ORDER BY time FORMAT PrettyCompact" 336 | 337 | # today by hours 338 | $CH_CLIENT --query " 339 | SELECT toStartOfHour(event_time) AS time, 340 | sum(ProfileEvent_UserTimeMicroseconds) AS user_time, 341 | bar(user_time, 0, $MAX_USER_TIME, 15) AS user_time_bar, 342 | sum(ProfileEvent_InsertedRows) as inserted_rows, 343 | bar(inserted_rows, 0, $MAX_INSERTED_ROWS, 15) AS insert_rows_bar, 344 | formatReadableSize(sum(ProfileEvent_InsertedBytes)) as inserted, 345 | sum(ProfileEvent_SelectedRows) as selected_rows, 346 | bar(selected_rows, 0, $MAX_SELECTED_ROWS, 15) AS selected_rows_bar, 347 | formatReadableSize(sum(ProfileEvent_SelectedBytes)) as selected, 348 | sum(ProfileEvent_MergedRows) as merged_rows, 349 | bar(merged_rows, 0, $MAX_MERGED_ROWS, 15) AS merged_rows_bar 350 | FROM system.metric_log 351 | WHERE event_date >= date_sub(HOUR, 48, today()) 352 | GROUP BY time ORDER BY time FORMAT PrettyCompact" 353 | 354 | # today by minute (last 60 minutes) 355 | $CH_CLIENT --query " 356 | SELECT toStartOfMinute(event_time) AS time, 357 | sum(ProfileEvent_UserTimeMicroseconds) AS user_time, 358 | bar(user_time, 0, $MAX_USER_TIME, 15) AS user_time_bar, 359 | sum(ProfileEvent_InsertedRows) as inserted_rows, 360 | bar(inserted_rows, 0, $MAX_INSERTED_ROWS, 15) AS insert_rows_bar, 361 | formatReadableSize(sum(ProfileEvent_InsertedBytes)) as inserted, 362 | sum(ProfileEvent_SelectedRows) as selected_rows, 363 | bar(selected_rows, 0, $MAX_SELECTED_ROWS, 15) AS selected_rows_bar, 364 | formatReadableSize(sum(ProfileEvent_SelectedBytes)) as selected, 365 | sum(ProfileEvent_MergedRows) as merged_rows, 366 | bar(merged_rows, 0, $MAX_MERGED_ROWS, 15) AS merged_rows_bar 367 | FROM system.metric_log 368 | WHERE event_time >= timestamp_sub(minute, 60, now()) 369 | GROUP BY time ORDER BY time FORMAT PrettyCompact" 370 | echo 371 | 372 | 373 | 374 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 375 | 376 | echo "Queries that is being processed. Ordered by elapsed time:" 377 | echo "- full SQL text by query_id: SELECT distinct query FROM system.query_log WHERE query_id='' FORMAT TabSeparatedRaw" 378 | echo "- kill by query_id: KILL QUERY WHERE query_id=''" 379 | echo "- kill all queries by user: KILL QUERY WHERE user='' SYNC" 380 | $CH_CLIENT --query "SELECT elapsed as time_seconds, formatReadableSize(memory_usage) as memory, formatReadableSize(read_bytes) as read, read_rows, total_rows_approx as total_rows, user, address as client, query_id, substring(query,1,48) as query 381 | FROM system.processes 382 | ORDER BY elapsed desc FORMAT PrettyCompact" 383 | echo 384 | 385 | echo "Queries that is being processed. Ordered by memory usage:" 386 | $CH_CLIENT --query "SELECT elapsed as time_seconds, formatReadableSize(memory_usage) as memory, formatReadableSize(read_bytes) as read, read_rows, total_rows_approx as total_rows, user, address as client, query_id, substring(query,1,48) as query 387 | FROM system.processes 388 | ORDER BY memory_usage desc FORMAT PrettyCompact" 389 | echo 390 | 391 | echo -e "${CYANLIGHT}---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------${NC}" 392 | 393 | -------------------------------------------------------------------------------- /scripts/clickhouse_information_metrics.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse information metrics 4 | 5 | # ------------------------------------------------ 6 | # ClickHouse client 7 | source ./settings.txt 8 | 9 | if [[ $CH_HOST && $CH_PORT ]]; then 10 | CH_CLIENT="clickhouse-client --host=$CH_HOST --port=$CH_PORT" 11 | else 12 | CH_CLIENT="clickhouse-client" 13 | fi 14 | 15 | if [[ $CH_LOGIN && $CH_PASSWORD ]]; then 16 | CH_CLIENT="$CH_CLIENT --user=$CH_LOGIN --password=$CH_PASSWORD" 17 | fi 18 | 19 | 20 | # ------------------------------------------------ 21 | 22 | clickhouse-server -V 23 | $CH_CLIENT --query="SELECT 'ClickHouse ' || version() || ' at ' || hostName() || '. Start time: ' || toString(now()-uptime())" --format=TSVRaw 24 | echo 25 | 26 | # ------------------------------------------------ 27 | echo "System metrics:" 28 | $CH_CLIENT --query "SELECT metric, value, substring(description,1,120) as description FROM system.metrics order by metric FORMAT PrettyCompact" 29 | echo 30 | 31 | echo "Asynchronous metrics:" 32 | $CH_CLIENT --query "SELECT metric, value FROM system.asynchronous_metrics ORDER BY metric FORMAT PrettyCompact" 33 | echo 34 | 35 | echo "System events:" 36 | $CH_CLIENT --query "SELECT event, value, substring(description,1,119) as description FROM system.events ORDER BY event FORMAT PrettyCompact" 37 | 38 | 39 | # ------------------------------------------------ 40 | -------------------------------------------------------------------------------- /scripts/clickhouse_logs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse log file 4 | LOG_LINES=100 # Number of ClickHouse log lines to display. 0 - disable output 5 | LOG_FILENAME="/var/log/clickhouse-server/clickhouse-server.log" # ClickHouse log file name 6 | 7 | 8 | # show ClickHouse log 9 | echo -e "ClickHouse log: $LOG_FILENAME" 10 | tail -f --lines=$LOG_LINES $LOG_FILENAME 11 | -------------------------------------------------------------------------------- /scripts/clickhouse_start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse start 4 | 5 | read -p "Start ClickHouse (Y/N)? " -n 1 -r 6 | echo 7 | if [[ $REPLY =~ ^[Yy]$ ]] 8 | then 9 | systemctl start clickhouse-server 10 | 11 | echo 12 | echo "ClickHouse status:" 13 | systemctl status clickhouse-server 14 | fi 15 | -------------------------------------------------------------------------------- /scripts/clickhouse_status.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse status 4 | 5 | echo "ClickHouse processes:" 6 | ps -afH --forest -u clickhouse | grep -v 'ps -afH' | grep -v '/usr/bin/mc' 7 | echo 8 | 9 | echo "ClickHouse version:" 10 | clickhouse-server -V 11 | echo 12 | 13 | echo "ClickHouse network connection:" 14 | echo "8123 - HTTP Client, 9000 - TCP/IP Native Client, 9004 - communicating using MySQL protocol, 9009 - Inter-Server Replication" 15 | netstat -tulpn | grep clickhouse 16 | echo 17 | 18 | echo "ClickHouse status:" 19 | systemctl status clickhouse-server 20 | -------------------------------------------------------------------------------- /scripts/clickhouse_stop.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # ClickHouse stop 4 | 5 | read -p "Stop ClickHouse (Y/N)? " -n 1 -r 6 | echo 7 | if [[ $REPLY =~ ^[Yy]$ ]] 8 | then 9 | systemctl stop clickhouse-server 10 | 11 | echo 12 | echo "ClickHouse status:" 13 | systemctl status clickhouse-server 14 | fi 15 | -------------------------------------------------------------------------------- /scripts/settings.txt: -------------------------------------------------------------------------------- 1 | # ClickHouse settings 2 | 3 | CH_HOST="localhost" # Host name. Default: localhost 4 | CH_PORT="9000" # Port. Default: 9000 5 | 6 | CH_LOGIN="" # Administrative user login 7 | CH_PASSWORD="" # Administrative user password 8 | -------------------------------------------------------------------------------- /scripts/zookeeper_logs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Show Apache ZooKeeper log file 4 | 5 | LOG_LINES=100 # log lines to show 6 | LOG_DIR="/opt/zookeeper/logs" # log direcrory 7 | LOG_FILENAME=`ls -t $LOG_DIR/zookeeper-*.out | head -n1` # newest log file in log directory 8 | 9 | 10 | # show Apache ZooKeeper log 11 | echo -e "Apache ZooKeeper log: $LOG_FILENAME" 12 | tail -f --lines=$LOG_LINES $LOG_FILENAME 13 | -------------------------------------------------------------------------------- /scripts/zookeeper_start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Apache ZooKeeper start 4 | 5 | read -p "Start Apache ZooKeeper (Y/N)? " -n 1 -r 6 | echo 7 | if [[ $REPLY =~ ^[Yy]$ ]] 8 | then 9 | systemctl start zookeeper 10 | 11 | echo 12 | echo "Apache ZooKeeper service:" 13 | systemctl status zookeeper 14 | fi 15 | -------------------------------------------------------------------------------- /scripts/zookeeper_status.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Apache ZooKeeper status 4 | 5 | 6 | # Apache ZooKeeper hosts 7 | ZooKeeperHosts="" # "" - disable output 8 | #ZooKeeperHosts="zoo_server_1 zoo_server_2 zoo_server_3" # Servers list, hostnames. Format: "server_1" "server_2" ... 9 | 10 | 11 | # --------------------------------------------------- 12 | 13 | echo "Apache ZooKeeper processes:" 14 | ps -afH --forest -u zookeeper 15 | echo 16 | 17 | echo "Apache ZooKeeper network connection:" 18 | echo "client port 2181. Cluster ports 2888 and 3888 (for leader), 3888 (for follower)" 19 | netstat -tulpn | grep '2181\|2888\|3888' 20 | echo 21 | 22 | echo "Apache ZooKeeper service:" 23 | systemctl status zookeeper 24 | echo 25 | 26 | echo "Apache ZooKeeper local node status:" 27 | /opt/zookeeper/bin/zkServer.sh status 28 | 29 | # cluster 30 | if [[ $ZooKeeperHosts ]]; then 31 | echo 32 | echo "Apache ZooKeeper cluster status:" 33 | for val in $ZooKeeperHosts; do 34 | ZooKeeperMode=`echo stat | nc $val 2181 | grep Mode` 35 | echo [$val] $ZooKeeperMode 36 | done 37 | fi 38 | -------------------------------------------------------------------------------- /scripts/zookeeper_stop.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Apache ZooKeeper stop 4 | 5 | read -p "Stop Apache ZooKeeper (Y/N)? " -n 1 -r 6 | echo 7 | if [[ $REPLY =~ ^[Yy]$ ]] 8 | then 9 | systemctl stop zookeeper 10 | 11 | echo 12 | echo "Apache ZooKeeper service:" 13 | systemctl status zookeeper 14 | fi 15 | --------------------------------------------------------------------------------