├── README.md └── server.cnf /README.md: -------------------------------------------------------------------------------- 1 | # Zabbix MYSQL tuned for 40000 values per second. 2 | This is repo for tuned mysql configuration. This mysql configuration can take 40k values per second. Server is VM in VMWare. HW: 12cpu, 48GB RAM, 400GB SAS HDD. SW: Zabbix 4, CentOS 7, MariaDB 10.3. Server is only for DB. 3 | 4 | 5 | How i have configured server: 6 | ## 1) MariaDB 7 | ``` 8 | vi /etc/yum.repos.d/mariadb.repo 9 | ``` 10 | ``` 11 | [mariadb] 12 | name = MariaDB 13 | baseurl = http://yum.mariadb.org/10.3/rhel7-amd64 14 | gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB 15 | gpgcheck=1 16 | ``` 17 | ``` 18 | yum install MariaDB-server MariaDB-client 19 | ``` 20 | ``` 21 | systemctl enable mysql.service 22 | ``` 23 | ``` 24 | systemctl start mysql.service 25 | ``` 26 | **Securing DB** 27 | ``` 28 | /usr/bin/mysql_secure_installation 29 | ``` 30 | ``` 31 | NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB 32 | SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY! 33 | 34 | In order to log into MariaDB to secure it, we'll need the current 35 | password for the root user. If you've just installed MariaDB, and 36 | you haven't set the root password yet, the password will be blank, 37 | so you should just press enter here. 38 | 39 | Enter current password for root (enter for none): 40 | OK, successfully used password, moving on... 41 | 42 | Setting the root password ensures that nobody can log into the MariaDB 43 | root user without the proper authorisation. 44 | 45 | Set root password? [Y/n] y 46 | New password: 47 | Re-enter new password: 48 | Password updated successfully! 49 | Reloading privilege tables.. 50 | ... Success! 51 | 52 | 53 | By default, a MariaDB installation has an anonymous user, allowing anyone 54 | to log into MariaDB without having to have a user account created for 55 | them. This is intended only for testing, and to make the installation 56 | go a bit smoother. You should remove them before moving into a 57 | production environment. 58 | 59 | Remove anonymous users? [Y/n] y 60 | ... Success! 61 | 62 | Normally, root should only be allowed to connect from 'localhost'. This 63 | ensures that someone cannot guess at the root password from the network. 64 | 65 | Disallow root login remotely? [Y/n] y 66 | ... Success! 67 | 68 | By default, MariaDB comes with a database named 'test' that anyone can 69 | access. This is also intended only for testing, and should be removed 70 | before moving into a production environment. 71 | 72 | Remove test database and access to it? [Y/n] y 73 | - Dropping test database... 74 | ... Success! 75 | - Removing privileges on test database... 76 | ... Success! 77 | 78 | Reloading the privilege tables will ensure that all changes made so far 79 | will take effect immediately. 80 | 81 | Reload privilege tables now? [Y/n] y 82 | ... Success! 83 | 84 | Cleaning up... 85 | 86 | All done! If you've completed all of the above steps, your MariaDB 87 | installation should now be secure. 88 | 89 | Thanks for using MariaDB! 90 | ``` 91 | ## 2) Set partioning 92 | **I use this howto for setting of partitionings** 93 | https://zabbix.org/wiki/Docs/howto/mysql_partition 94 | 95 | **After this i create task for automatic recreating tables and deleting this:** 96 | ``` 97 | mysql -p 98 | ``` 99 | ``` 100 | MariaDB [(none)]> use zabbix; 101 | ``` 102 | ``` 103 | CREATE EVENT Partitions ON SCHEDULE EVERY 1 DAY DO CALL partition_maintenance_all('zabbix'); 104 | ``` 105 | 106 | **Check partitions:** 107 | ``` 108 | mysql -p 109 | ``` 110 | ``` 111 | MariaDB [(none)]> use zabbix; 112 | ``` 113 | ``` 114 | CALL partition_maintenance_all('zabbix'); 115 | ``` 116 | ``` 117 | SELECT PARTITION_NAME, TABLE_ROWS FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_NAME='history_uint'; 118 | ``` 119 | 120 | ## 3) Huge memory 121 | **I use this howto for counting my momory** 122 | https://www.linkedin.com/pulse/configuring-huge-pages-mysql-server-red-hat-linux-juan-soto/ 123 | ``` 124 | vi /etc/security/limits.conf 125 | ``` 126 | ``` 127 | # /etc/security/limits.conf 128 | # 129 | #This file sets the resource limits for the users logged in via PAM. 130 | #It does not affect resource limits of the system services. 131 | # 132 | #Also note that configuration files in /etc/security/limits.d directory, 133 | #which are read in alphabetical order, override the settings in this 134 | #file in case the domain is the same or more specific. 135 | #That means for example that setting a limit for wildcard domain here 136 | #can be overriden with a wildcard setting in a config file in the 137 | #subdirectory, but a user specific setting here can be overriden only 138 | #with a user specific setting in the subdirectory. 139 | # 140 | #Each line describes a limit for a user in the form: 141 | # 142 | # 143 | # 144 | #Where: 145 | # can be: 146 | # - a user name 147 | # - a group name, with @group syntax 148 | # - the wildcard *, for default entry 149 | # - the wildcard %, can be also used with %group syntax, 150 | # for maxlogin limit 151 | # 152 | # can have the two values: 153 | # - "soft" for enforcing the soft limits 154 | # - "hard" for enforcing hard limits 155 | # 156 | # can be one of the following: 157 | # - core - limits the core file size (KB) 158 | # - data - max data size (KB) 159 | # - fsize - maximum filesize (KB) 160 | # - memlock - max locked-in-memory address space (KB) 161 | # - nofile - max number of open file descriptors 162 | # - rss - max resident set size (KB) 163 | # - stack - max stack size (KB) 164 | # - cpu - max CPU time (MIN) 165 | # - nproc - max number of processes 166 | # - as - address space limit (KB) 167 | # - maxlogins - max number of logins for this user 168 | # - maxsyslogins - max number of logins on the system 169 | # - priority - the priority to run user process with 170 | # - locks - max number of file locks the user can hold 171 | # - sigpending - max number of pending signals 172 | # - msgqueue - max memory used by POSIX message queues (bytes) 173 | # - nice - max nice priority allowed to raise to values: [-20, 19] 174 | # - rtprio - max realtime priority 175 | # 176 | # 177 | # 178 | 179 | #* soft core 0 180 | #* hard rss 10000 181 | #@student hard nproc 20 182 | #@faculty soft nproc 20 183 | #@faculty hard nproc 50 184 | #ftp hard nproc 0 185 | #@student - maxlogins 4 186 | 187 | @mysql soft memlock unlimited 188 | @mysql hard memlock unlimited 189 | 190 | # End of file 191 | ``` 192 | 193 | ## 4) Use tuned my.cnf 194 | **Backup old config** 195 | ``` 196 | mv /etc/my.cnf.d/server.cnf /etc/my.cnf.d/server.cnf_ORIG 197 | ``` 198 | **Copy new configuration** 199 | ``` 200 | wget -O /etc/my.cnf.d/server.cnf https://raw.githubusercontent.com/hermanekt/Zabbix_MYSQL_tunned_for_40k/master/server.cnf 201 | ``` 202 | **Create log path** 203 | ``` 204 | mkdir /var/log/mariadb 205 | ``` 206 | ``` 207 | chown mysql:mysql /var/log/mariadb/ 208 | ``` 209 | **Restart mariaDB and check log** 210 | ``` 211 | systemctl restart mysql.service 212 | ``` 213 | ``` 214 | less /var/log/mariadb/mariadb-error.log 215 | ``` 216 | 217 | ## Authors 218 | 219 | * **Tomas Hermanek** - *Initial work* - [hermanekt](https://github.com/hermanekt) 220 | 221 | ## Acknowledgments 222 | 223 | ### This is Alfa version, please give me feadback if you find bug or need some another check. Email is info"@"tomashermanek.cz, twitter is: hermanekt. 224 | [![paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=GEH7YJEBWTFWE¤cy_code=USD&source=url) 225 | -------------------------------------------------------------------------------- /server.cnf: -------------------------------------------------------------------------------- 1 | # Tuned for 25K zabbix info@tomashermanek.cz 2 | # https://github.com/hermanekt/Zabbix_MYSQL_tunned_for_25k 3 | 4 | # These groups are read by MariaDB server. 5 | # Use it for options that only the server (but not clients) should see 6 | # 7 | # See the examples of server my.cnf files in /usr/share/mysql/ 8 | # 9 | 10 | # this is read by the standalone daemon and embedded servers 11 | [server] 12 | 13 | # this is only for the mysqld standalone daemon 14 | [mysqld] 15 | #Tuned by CoreIT s.r.o. 16 | open_files_limit = 16364 17 | max_connections = 500 18 | event_scheduler = 1 19 | wait_timeout = 86400 20 | default_storage_engine = InnoDB 21 | 22 | log_error = /var/log/mariadb/mariadb-error.log 23 | slow_query_log_file = /var/log/mariadb/mariadb-slow.log 24 | general_log_file = /var/log/mariadb/mariadb.log 25 | general_log = 0 26 | skip-name-resolve = 1 27 | performance_schema = ON 28 | 29 | innodb_autoinc_lock_mode = 2 30 | innodb_flush_log_at_trx_commit = 0 31 | innodb_autoextend_increment = 256 32 | innodb_buffer_pool_instances = 12 33 | innodb_buffer_pool_size = 32G 34 | innodb_change_buffer_max_size = 50 35 | innodb_concurrency_tickets = 5000 36 | innodb_file_per_table = 1 37 | innodb_flush_method = O_DIRECT 38 | innodb_log_file_size = 512M 39 | innodb_log_files_in_group = 4 40 | innodb_old_blocks_time = 1000 41 | innodb_open_files = 2048 42 | innodb_stats_on_metadata = OFF 43 | innodb_lock_wait_timeout = 50 44 | innodb_io_capacity = 2000 45 | 46 | large-pages 47 | binlog-row-event-max-size = 8192 48 | character_set_server = utf8 49 | collation_server = utf8_bin 50 | expire_logs_days = 1 51 | join_buffer_size = 262144 52 | max_allowed_packet = 32M 53 | query_cache_type = 0 54 | query_cache_size = 0 55 | slow-query-log = ON 56 | table_open_cache = 2048 57 | thread_cache_size = 64 58 | tmp_table_size = 134217728 59 | thread_pool_size = 12 60 | # This is needed for best performance, this is bug with select BY ORDER 61 | optimizer_switch = 'index_condition_pushdown=off' 62 | 63 | # 64 | # * Galera-related settings 65 | # 66 | [galera] 67 | # Mandatory settings 68 | #wsrep_on=ON 69 | #wsrep_provider= 70 | #wsrep_cluster_address= 71 | #binlog_format=row 72 | #default_storage_engine=InnoDB 73 | #innodb_autoinc_lock_mode=2 74 | # 75 | # Allow server to accept connections on all interfaces. 76 | # 77 | bind-address=0.0.0.0 78 | # 79 | # Optional setting 80 | #wsrep_slave_threads=1 81 | #innodb_flush_log_at_trx_commit=0 82 | 83 | # this is only for embedded server 84 | [embedded] 85 | 86 | # This group is only read by MariaDB servers, not by MySQL. 87 | # If you use the same .cnf file for MySQL and MariaDB, 88 | # you can put MariaDB-only options here 89 | [mariadb] 90 | 91 | # This group is only read by MariaDB-10.3 servers. 92 | # If you use the same .cnf file for MariaDB of different versions, 93 | # use this group for options that older servers don't understand 94 | [mariadb-10.3] 95 | 96 | --------------------------------------------------------------------------------