├── .gitignore ├── Dockerfile ├── README.md ├── clean-up.sh ├── demographic.png ├── docker-compose.yml ├── master ├── init.sql └── my.cnf ├── proxysql └── proxysql.cnf ├── sbtest-app ├── .gitignore ├── .mvn │ └── wrapper │ │ ├── MavenWrapperDownloader.java │ │ ├── maven-wrapper.jar │ │ └── maven-wrapper.properties ├── mvnw ├── mvnw.cmd ├── pom.xml └── src │ ├── main │ ├── java │ │ └── com │ │ │ └── poc │ │ │ └── proxysql │ │ │ ├── ProxysqlPocApplication.java │ │ │ ├── config │ │ │ └── OpenApiConfiguration.java │ │ │ ├── controller │ │ │ └── UserController.java │ │ │ ├── models │ │ │ ├── UserCreateRequest.java │ │ │ ├── UserDTO.java │ │ │ └── UserUpdateRequest.java │ │ │ ├── repository │ │ │ ├── UserRepository.java │ │ │ └── entity │ │ │ │ └── UserEntity.java │ │ │ └── service │ │ │ ├── UserMapper.java │ │ │ ├── UserService.java │ │ │ └── UserServiceImpl.java │ └── resources │ │ └── application.yml │ └── test │ └── java │ └── com │ └── poc │ └── proxysql │ ├── ProxysqlPocApplicationTests.java │ └── contrioller │ └── UserControllerTests.java ├── slave ├── init.sql ├── my-slave1.cnf └── my-slave2.cnf └── user_api_postman_collection.json /.gitignore: -------------------------------------------------------------------------------- 1 | master/data 2 | slave/data 3 | proxysql/data 4 | 5 | ### VS Code ### 6 | .vscode/ -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM amazoncorretto:11 2 | 3 | ARG JAR_FILE=./sbtest-app/target/*.jar 4 | ARG APP_DIR=/opt/app 5 | 6 | COPY ${JAR_FILE} ${APP_DIR}/application.jar 7 | WORKDIR ${APP_DIR} 8 | ENTRYPOINT ["java", "-Dspring.profiles.active=${ACTIVE_PROFILE}", "-jar", "application.jar"] -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MySQL Replication and ProxySQL 2 | 3 | In this documentation, we will cover 4 | 5 | 1. Problem Discussion 6 | - Common Database problems and solutions. 7 | - What is Database Replication? 8 | - When do you need this? 9 | 2. MySQL DB Replication (1 master and 2 slaves) on Docker 10 | 3. Load Balancing them using ProxySQL on Docker 11 | 4. Sending Request from our Spring Boot Application. 12 | 5. Observe the Query Execution in the Master and Slave's General log 13 | 14 | # Problem Discussion 15 | Let's say, you have a single relational databases instance and you are probably building a web application. So you need a web server and a Web Developement Framework such as Spring Boot, django, nodejs bla bla. Your web application that essentially speaks to HTTP protocol that talks to the browser or some HTTP clients to execute the api endpoints and those apis eventually executing Read/Write operation to the database. 16 | 17 | Assume, your database is getting reads/write requests from the client. For example a read request: `SELECT * FROM products WHERE price < 5000`. Your application was working smoothly but as the time passed the you have noticed your query is performing slow, it's not much faster as earlier. It's taking some times which may reduce the user-experience. 18 | 19 | But what's the problem here ? 20 | Your *product* tables is growing larger in course of time and when you are executing the query above, it's doing a `Full Table Scan` which basically a Linear Search operation ( Time Complexity `O(n)` ). So you can say, ok fine I'm gonna add an **Index** to `price` column and my beautiful databse will arrange it in a `Balanced Binary Tree` or a `B-Tree` where the Tree *Balances* itself when a new record *Inserted*, *Deleted* or *Updated* the existing column value each time, to make the *Read* operations faster in `O(log n)` Time Complexity where **“n” is the total number of elements in the B-Tree**. But the balancing tree has a cost, besides renge query (`SELECT job FROM products WHERE price BETWEEN 5000 AND 10000`) is not effecient in `B-Tree`, hence `B+ Tree` comes into the picture. But still, what if you have 1 millions records in the `products` table and you just have inserted a new record and bam!! Your DB is doing so much work to `re-balance` the large Tree. 21 | 22 | So, What can you do now ? Ok, You can do table **Partitioning** based on `id` let's say, because the `id` use-case fits best here. **Partitioning is a technique to subdivide objects into smaller pieces.** It actually breaks the huge table into many different partiotion tables by range depending on the partiotion key and these partition tables are again mapped by the `` pair where the *partition_key* as the KEY and *partition_table_reference* as the VALUE of the map. 23 | For example: your page size = 20000 so the ids from 1 to 20000 falls into *partition_1* and ids from 20000 - 40000 goes into *partition_2* and so on and so forth. But don't worry guys, these partitions are *managed implicitly* by the Database itself. It knows, to get result for the for the Specific *Read* query (`SELECT * FROM products WHERE id = 18`) in which partition it needs to look for. So it can be a solution to reduce the Tree Balancing Cost, because as you can feel the Search space is much smaller than before so the `Balanced B+ Tree` cost has optimized. Great, peorblem solved. But as your business grew, your user-base also grown. Now you have a thousands of concurrent users who are reading millions of records from your (Read Heavy) Database and the Single Database server is dealing with a huge number of concurrent TCP connections. Your single Database instance is junked up with these enourmous number of concurrent requestes and it might be ran out of it's `QPS (Query Per Second)` limit too. Here's **Replication** comes into the solution space. 24 | 25 | ## What is DB Replication? 26 | DB Replication is a kind of Horizontal Scaling making the same copy of the Full Database and Distribute them in **Master/Slave** architecture where the **Master** deals with all the **Write** operations and *Periodically Updates* it's **Slave** Replicas, who will Handle only the **Read** queries. So your Database load is Distributed now. But remember, the *Slaves* must be **Consistent** with the *Master* so there must be a Replication Strategy. 27 | 28 | ### How the Replication works from Master to Slaves? 29 | After any Write Query (Insert. Update, Delete) executed in the Master, the DB somehow Replicate the changes to the Slaves. The Master triggers a change event and the Slaves pull the Changes from the Event and Update themselves. Let's generate some ideas on this. 30 | 31 | Idea 1: Can we Stream the SQL Statements? 32 | 33 | So Basically, We will be Streaming the SQL Query Statements and the Slaves will pull them from the channel and Execute those SQL Statements inside themselves. Well, this can make the Replication **Inconsistent**. Let's see how. 34 | Assume, you are Creating a new Product 35 | ```sql 36 | INSERT INTO products (product_name, product_status, price, created_at) 37 | VALUES('TP-Link Archar C60', 'AVAILABLE', 3500, sysdate(3)) 38 | ``` 39 | - Firstly, The Query will be executed at *Master* when the value of *sysdate(3) = 2022-01-07 12:04:59.114* 40 | 41 | - Secondly, The Query will be executed at *Slave 1* when the value of *sysdate(3) = 2022-01-07 12:05:00.100* 42 | 43 | - Thirdly, The Query will be executed at *Slave 2* when the value of *sysdate(3) = 2022-01-07 12:05:00.405* 44 | 45 | Epic Fail!! Right? This will certainly create inconsitancy problem. So we need to drop this idea. 46 | 47 | Idea 2: How about Transfering [Bin Log](https://dev.mysql.com/doc/internals/en/binary-log-overview.html) files? 48 | 49 | > The binary log is a set of log files that contain information about data modifications made to a MySQL server instance. Simply it saves the Database States 50 | 51 | So, When any Write Query executes in the Master Replica, the change is saved into the Bin Log. After that the Master will transfer these log files towards the Slave Databases **asynchronusly** and the Slaves will pull the change and update their states according to the bin logs.There are also other replication staratagies like Synchronus, Asynchronus and Semi-Asynchronus, but Mysql do Asynchronus replication by default so we are gonna use this for now. 52 | 53 | But another problem is knocking at the door. How will you distribute the traffics to the appropriate DB Replicas? 54 | Since, you have a multiple instances of the same database depending on *Read/Write* purpose, how your application can differenfiate when to go to the Read Replica and when to the Master Replica. 55 | - DB connection information can change on the way 56 | - It is troublesome (but complicated) to use DB properly in the Read/Write logic of the application. 57 | 58 | So we need a Reverse Proxy for sure to solve this problem. The proxy sits in-between of the Application and Database and Load Balance the Request to the different DB instances based on our operation types (Read/Write). But how the Proxy will distinguish the Request Type ? The answere is the Proxy must be **SQLAware**. We are going to use `ProxySQL` here. 59 | 60 | ## What is ProxySql? 61 | 62 | ![Blank Diagram](./demographic.png) 63 | 64 | `ProxySQL` is an *Opensource SQLAware Reverse Proxy* unlike other HTTP or TCP proxy (Nginx, HAProxy etc) it can distinguish the Read/Write operations and deliver the packet to the Specific Replica either it's Master or Slave. 65 | ProxySQL goes between the application and the DB and does the following: 66 | - Automatic Proxify to the Master/Slave depending on query 67 | - Load Distribution 68 | - Change seamless connection settings 69 | By the way, ProxySQL can be used with other DBs like Postgres as well. 70 | 71 | ## MySQL Replica Configuration 72 | Here is a good read to get the insight of [Mysql Replica Configuration](https://www.digitalocean.com/community/tutorials/how-to-set-up-replication-in-mysql). I used MySQL 5.7 to prepare one master and two slaves, and set up replication settings. Here is the `docker-compose.yml` file I'm using 73 | 74 | ```yaml 75 | version: '3' 76 | services: 77 | mysql-master: 78 | image: mysql:5.7 79 | container_name: proxysql-mysql-replication-master 80 | environment: 81 | MYSQL_ROOT_PASSWORD: password 82 | MYSQL_DATABASE: sbtest 83 | volumes: 84 | - ./master/my.cnf:/etc/mysql/my.cnf 85 | - ./master/data:/var/lib/mysql 86 | - ./master/init.sql:/docker-entrypoint-initdb.d/init.sql 87 | ports: 88 | - 3306:3306 89 | networks: 90 | - mysql_cluster_net 91 | 92 | mysql-slave1: 93 | image: mysql:5.7 94 | container_name: proxysql-mysql-replication-slave1 95 | environment: 96 | MYSQL_ROOT_PASSWORD: password 97 | MYSQL_DATABASE: sbtest 98 | volumes: 99 | - ./slave/my-slave1.cnf:/etc/mysql/my.cnf 100 | - ./slave/data/slave1:/var/lib/mysql 101 | - ./slave/init.sql:/docker-entrypoint-initdb.d/init.sql 102 | ports: 103 | - 3307:3306 104 | depends_on: 105 | - mysql-master 106 | networks: 107 | - mysql_cluster_net 108 | 109 | mysql-slave2: 110 | image: mysql:5.7 111 | container_name: proxysql-mysql-replication-slave2 112 | environment: 113 | MYSQL_ROOT_PASSWORD: password 114 | MYSQL_DATABASE: sbtest 115 | volumes: 116 | - ./slave/my-slave2.cnf:/etc/mysql/my.cnf 117 | - ./slave/data/slave2:/var/lib/mysql 118 | - ./slave/init.sql:/docker-entrypoint-initdb.d/init.sql 119 | ports: 120 | - 3308:3306 121 | depends_on: 122 | - mysql-master 123 | networks: 124 | - mysql_cluster_net 125 | 126 | networks: 127 | mysql_cluster_net: 128 | driver: bridge 129 | 130 | ``` 131 | 132 | Let's bring up the docker containers and check the Master and Slave's status. `docker-compose up -d` 133 | 134 | ### Check Master's Status 135 | ``` 136 | docker-compose exec mysql-master sh -c "export MYSQL_PWD=password; mysql -u root sbtest -e 'show master status\G'" 137 | ``` 138 | Expected Output: 139 | ``` 140 | *************************** 1. row *************************** 141 | File: mysql-bin.000003 142 | Position: 194 143 | Binlog_Do_DB: sbtest 144 | Binlog_Ignore_DB: 145 | Executed_Gtid_Set: 9618dc00-6f2a-11ec-a895-0242ac120002:1-9 146 | ``` 147 | ### Check Slave 1 Status 148 | ``` 149 | docker-compose exec mysql-slave1 sh -c "export MYSQL_PWD=password; mysql -u root sbtest -e 'show slave status\G'" 150 | ``` 151 | Expected Output: 152 | ``` 153 | *************************** 1. row *************************** 154 | Slave_IO_State: Waiting for master to send event 155 | Master_Host: mysql-master 156 | Master_User: slave_user 157 | Master_Port: 3306 158 | Connect_Retry: 60 159 | Master_Log_File: mysql-bin.000003 160 | Read_Master_Log_Pos: 194 161 | Relay_Log_File: mysql-relay-bin.000004 162 | Relay_Log_Pos: 407 163 | Relay_Master_Log_File: mysql-bin.000003 164 | Slave_IO_Running: Yes 165 | Slave_SQL_Running: Yes 166 | ... 167 | Master_Server_Id: 1 168 | Master_UUID: 9618dc00-6f2a-11ec-a895-0242ac120002 169 | Master_Info_File: /var/lib/mysql/master.info 170 | SQL_Delay: 0 171 | SQL_Remaining_Delay: NULL 172 | Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates 173 | Master_Retry_Count: 86400 174 | Master_Bind: 175 | Last_IO_Error_Timestamp: 176 | Last_SQL_Error_Timestamp: 177 | Master_SSL_Crl: 178 | Master_SSL_Crlpath: 179 | 180 | Retrieved_Gtid_Set: 9618dc00-6f2a-11ec-a895-0242ac120002:1-9 181 | Executed_Gtid_Set: 9618dc00-6f2a-11ec-a895-0242ac120002:1-9, 962ec4d2-6f2a-11ec-8a4d-0242ac120004:1-5 182 | ... 183 | ``` 184 | As you can se the `Slave_IO_Running: Yes, Slave_SQL_Running: Yes` means the slave is started properly. 185 | 186 | Also `Master_UUID: 9618dc00-6f2a-11ec-a895-0242ac120002` means it's connected successfully with the Master. If the Slaves fails to connect to the master then run the ./clean-up.sh. it will gracefully shutdown the containers and clean the master, slave data directories and start the contains in `-d` demon mode 187 | 188 | ### Check Slave 2 Status 189 | ``` 190 | docker-compose exec mysql-slave2 sh -c "export MYSQL_PWD=password; mysql -u root sbtest -e 'show slave status\G'" 191 | ``` 192 | Expected Output: 193 | ``` 194 | *************************** 1. row *************************** 195 | Slave_IO_State: Waiting for master to send event 196 | Master_Host: mysql-master 197 | Master_User: slave_user 198 | Master_Port: 3306 199 | Connect_Retry: 60 200 | Master_Log_File: mysql-bin.000003 201 | Read_Master_Log_Pos: 194 202 | Relay_Log_File: mysql-relay-bin.000004 203 | Relay_Log_Pos: 407 204 | Relay_Master_Log_File: mysql-bin.000003 205 | Slave_IO_Running: Yes 206 | Slave_SQL_Running: Yes 207 | ... 208 | Master_Server_Id: 1 209 | Master_UUID: 9618dc00-6f2a-11ec-a895-0242ac120002 210 | Master_Info_File: /var/lib/mysql/master.info 211 | SQL_Delay: 0 212 | SQL_Remaining_Delay: NULL 213 | Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates 214 | Master_Retry_Count: 86400 215 | Master_Bind: 216 | Last_IO_Error_Timestamp: 217 | Last_SQL_Error_Timestamp: 218 | Master_SSL_Crl: 219 | Master_SSL_Crlpath: 220 | Retrieved_Gtid_Set: 9618dc00-6f2a-11ec-a895-0242ac120002:1-9 221 | Executed_Gtid_Set: 9618dc00-6f2a-11ec-a895-0242ac120002:1-9, 9633aafa-6f2a-11ec-ba14-0242ac120003:1-5 222 | ... 223 | ``` 224 | 225 | Looks Good, Now it's time to configure our `ProxySQL` 226 | 227 | ## ProxySQL Configuration 228 | The configuration file is as follows. 229 | ```ruby 230 | datadir="/var/lib/proxysql" 231 | 232 | # ProxySQL Admin Configuration 233 | admin_variables= 234 | { 235 | admin_credentials="admin:admin;admin2:pass2" 236 | mysql_ifaces="0.0.0.0:6032" 237 | refresh_interval=2000 238 | stats_credentials="stats:admin" 239 | } 240 | 241 | # ProxySQL configuration for MySQL Cluster 242 | mysql_variables= 243 | { 244 | threads=4 245 | max_connections=2048 246 | default_query_delay=0 247 | default_query_timeout=36000000 248 | have_compress=true 249 | poll_timeout=2000 250 | #Where the clinet application will be connected 251 | interfaces="0.0.0.0:6033;/tmp/proxysql.sock" 252 | default_schema="information_schema" 253 | stacksize=1048576 254 | server_version="5.7" 255 | connect_timeout_server=10000 256 | monitor_history=60000 257 | monitor_connect_interval=200000 258 | monitor_ping_interval=200000 259 | ping_interval_server_msec=10000 260 | ping_timeout_server=200 261 | commands_stats=true 262 | sessions_sort=true 263 | # setting up mysql cluster monitoring credentials 264 | monitor_username="monitor" 265 | monitor_password="monitor" 266 | } 267 | 268 | # Host Group 10 = Master Group for Write 269 | # Host Group 20 = Slave Group for Read 270 | mysql_replication_hostgroups = 271 | ( 272 | { writer_hostgroup=10 , reader_hostgroup=20 , comment="host groups" } 273 | ) 274 | 275 | # replication_lag, checks if the servers are alive or not. 276 | # replication_lag = 5 mean if any slave replica is unable to catch the the master change event within 5 sec, proxySQL will mark it as SHUNNED (kind of Banned) 277 | mysql_servers = 278 | ( 279 | { address="mysql-master" , port=3306 , hostgroup=10, max_connections=100 , max_replication_lag = 5 }, 280 | { address="mysql-slave1" , port=3306 , hostgroup=20, max_connections=100 , max_replication_lag = 5 }, 281 | { address="mysql-slave2" , port=3306 , hostgroup=20, max_connections=100 , max_replication_lag = 5 } 282 | ) 283 | 284 | # The SQL Awareness Rules 285 | mysql_query_rules = 286 | ( 287 | { 288 | rule_id=100 289 | active=1 290 | match_pattern="^SELECT .* FOR UPDATE" 291 | destination_hostgroup=10 292 | apply=1 293 | }, 294 | { 295 | rule_id=200 296 | active=1 297 | match_pattern="^SELECT .*" 298 | destination_hostgroup=20 299 | apply=1 300 | }, 301 | { 302 | rule_id=300 303 | active=1 304 | match_pattern=".*" 305 | destination_hostgroup=10 306 | apply=1 307 | } 308 | ) 309 | # ProxySql to Mysql Connection Credential. This credential will be used by our Spring Boot Application or any application you want to develop 310 | mysql_users = 311 | ( 312 | { username = "root" , password = "password" , default_hostgroup = 10 , active = 1 } 313 | ) 314 | 315 | ``` 316 | Now Let's add proxySql into our existing docker-compose file 317 | 318 | ```yaml 319 | ... 320 | proxysql: 321 | image: proxysql/proxysql:2.0.12 322 | container_name: proxysql-mysql-replication-proxysql 323 | ports: 324 | - 6032:6032 325 | - 6033:6033 326 | volumes: 327 | - ./proxysql/proxysql.cnf:/etc/proxysql.cnf 328 | - ./proxysql/data:/var/lib/proxysql 329 | networks: 330 | - mysql_cluster_net 331 | depends_on: 332 | - mysql-master 333 | - mysql-slave1 334 | - mysql-slave2 335 | 336 | networks: 337 | mysql_cluster_net: 338 | driver: bridge 339 | ``` 340 | 341 | ## Credentials 342 | - ProxySQL [SQL Aware LB] 343 | - `:6032` (admin, `user`: admin2, `pass`: pass2) 344 | - `:6033` (MySQL endpoint, `user`: root, `pass`: password) 345 | - MySQL replication 346 | - master x 1 347 | - slave x 2 348 | - `user`: root, `pass`: password 349 | 350 | ## Getting Started 351 | Turn up all the docker containers `docker-compose up -d` and check docker processes 352 | 353 | ``` 354 | docker ps 355 | ``` 356 | Output: 357 | ``` 358 | 359 | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 360 | 361 | 5e5e850339d3 proxysql/proxysql:2.0.12 "proxysql -f -D /var…" 30 seconds ago Up 28 seconds 0.0.0.0:6032-6033->6032-6033/tcp, :::6032-6033->6032-6033/tcp proxysql-mysql-replication-proxysql 362 | 363 | b6a5296c7c27 mysql:5.7 "docker-entrypoint.s…" 31 seconds ago Up 30 seconds 33060/tcp, 0.0.0.0:3307->3306/tcp, :::3307->3306/tcp proxysql-mysql-replication-slave1 364 | 365 | ef6d0cb4249b mysql:5.7 "docker-entrypoint.s…" 31 seconds ago Up 30 seconds 33060/tcp, 0.0.0.0:3308->3306/tcp, :::3308->3306/tcp proxysql-mysql-replication-slave2 366 | 367 | d64d42b53e41 mysql:5.7 "docker-entrypoint.s…" 32 seconds ago Up 31 seconds 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp proxysql-mysql-replication-master 368 | 369 | ``` 370 | All containers are up and running. 371 | 372 | ## Check Replication States now 373 | Incase you don't have mysql client installed in your machine then install it first, the execute the command bellow. 374 | - Mysql Installation (Ubuntu 20.04) (Optional) 375 | ``` 376 | $ wget https://dev.mysql.com/get/mysql-apt-config_0.8.20-1_all.deb 377 | $ dpkg -i mysql-apt-config_0.8.20-1_all.deb # and select mysql-8.0 378 | $ sudo apt install mysql-server-8.0 379 | ``` 380 | - ProxySQL 381 | ``` 382 | $ mysql -h 0.0.0.0 -P 6032 -u admin2 -p -e 'select * from mysql_servers' 383 | ``` 384 | Enter password: pass2 385 | ``` 386 | +--------------+--------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ 387 | | hostgroup_id | hostname | port | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment | 388 | +--------------+--------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ 389 | | 10 | mysql-master | 3306 | 0 | ONLINE | 1 | 0 | 100 | 5 | 0 | 0 | | 390 | | 20 | mysql-slave2 | 3306 | 0 | ONLINE | 1 | 0 | 100 | 5 | 0 | 0 | | 391 | | 20 | mysql-slave1 | 3306 | 0 | ONLINE | 1 | 0 | 100 | 5 | 0 | 0 | | 392 | +--------------+--------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ 393 | ``` 394 | Looks Good, All the Master and Slaves are Online and Synced up. 395 | 396 | Now Open a new Terminal and Try to run some query and Monitor the General Logs of Master and Slaves 397 | 398 | ## Showtime 399 | All our tedious configuration is done, now lets open 3 terminals 1 for master and other 2 for slaves and place them side by side so that you can monitor all of them together. Try to run some Read/Write query from your Spring Boot application or any Database Client like Mysql Workbench and Monitor the General Logs of Master and Slaves 400 | 401 | 1. Read ALL Users 402 | > URL: http://localhost:8080/users 403 | > Method: GET 404 | 405 | **Output from SLAVE 1 Console** 406 | ``` 407 | docker-compose exec mysql-slave1 sh -c 'tail -f /var/log/mysql/*.log' 408 | ``` 409 | Output: 410 | ``` 411 | ==> /var/log/mysql/general.log <== 412 | 2022-01-07T10:26:42.237593Z 4025 Query SHOW SLAVE STATUS 413 | 2022-01-07T10:26:42.237716Z 4025 Quit 414 | 415 | 2022-01-07T10:26:43.007045Z 3194 Query select userentity0_.id as id1_0_, userentity0_.name as name2_0_ from users userentity0_ 416 | 417 | 2022-01-07T10:26:43.074505Z 4026 Connect monitor@proxysql-mysql-replication-proxysql.mysql-replication-poc_my on using TCP/IP 418 | 2022-01-07T10:26:43.074741Z 4026 Query SELECT @@global.read_only read_only 419 | 2022-01-07T10:26:43.075023Z 4026 Query SET wait_timeout=2000 420 | ``` 421 | 422 | **Execute the Same API again and Check the output in SLAVE 2 Console** 423 | ``` 424 | docker-compose exec mysql-slave2 sh -c 'tail -f /var/log/mysql/*.log' 425 | ``` 426 | Output: 427 | ``` 428 | 2022-01-07T10:42:42.119645Z 20 Query SELECT @@global.read_only read_only 429 | 2022-01-07T10:42:42.120000Z 20 Query SET wait_timeout=2000 430 | 2022-01-07T10:42:43.128917Z 20 Query SELECT @@global.read_only read_only 431 | 2022-01-07T10:42:44.143227Z 20 Query SELECT @@global.read_only read_only 432 | 2022-01-07T10:42:44.252141Z 21 Connect root@proxysql-mysql-replication-proxysql.mysql-replication-poc_my on sbtest using TCP/IP 433 | 2022-01-07T10:42:44.252377Z 21 Query SET character_set_results=NULL 434 | 435 | 2022-01-07T10:42:44.252534Z 21 Query select userentity0_.id as id1_0_, userentity0_.name as name2_0_ from users userentity0_ 436 | 437 | 2022-01-07T10:42:45.128786Z 20 Query SELECT @@global.read_only read_only 438 | 2 439 | ``` 440 | 441 | 2. CREATE NEW USER 442 | > URL: http://localhost:8080/users 443 | > Method: POST 444 | > body: 445 | ```json 446 | { 447 | "name": "Jhon Doe" 448 | } 449 | ``` 450 | **Check MASTER Status:** 451 | ``` 452 | docker-compose exec mysql-master sh -c 'tail -f /var/log/mysql/*.log' 453 | ``` 454 | **Output from Master Console:** 455 | ``` 456 | 2022-01-07T11:05:11.305574Z 312 Query SELECT @@global.read_only read_only 457 | 2022-01-07T11:05:11.582792Z 35 Query SET autocommit=0 458 | 2022-01-07T11:05:11.583025Z 35 Query SET character_set_results=NULL 459 | 2022-01-07T11:05:11.583181Z 35 Query insert into users (name) values ('Jhon Doe') 460 | 2022-01-07T11:05:11.695636Z 35 Query commit 461 | 2022-01-07T11:05:12.117764Z 312 Query SHOW SLAVE STATUS 462 | ``` 463 | **Output from SLAVE 1 Console:** 464 | ``` 465 | 2022-01-07T11:05:11.326982Z 163 Query SELECT @@global.read_only read_only 466 | 2022-01-07T11:05:11.702399Z 2 Query BEGIN 467 | 2022-01-07T11:05:11.702534Z 2 Query COMMIT /* implicit, from Xid_log_event */ 468 | 2022-01-07T11:05:12.122072Z 163 Query SHOW SLAVE STATUS 469 | 2022-01-07T11:05:12.122218Z 163 Quit 470 | ``` 471 | **Output from SLAVE 2 Console:** 472 | ``` 473 | 2022-01-07T11:05:10.331996Z 162 Query SELECT @@global.read_only read_only 474 | 2022-01-07T11:05:11.316285Z 162 Query SELECT @@global.read_only read_only 475 | 2022-01-07T11:05:11.702399Z 2 Query BEGIN 476 | 2022-01-07T11:05:11.702534Z 2 Query COMMIT /* implicit, from Xid_log_event */ 477 | 2022-01-07T11:05:12.120590Z 162 Query SHOW SLAVE STATUS 478 | 2022-01-07T11:05:12.120696Z 162 Quit 479 | ``` 480 | As you can see the **Write** query has executed in **Master** and the **Bin Log** has been *Replicated from the Master to the Slave Replicas* 481 | 482 | ## Try it yourself 483 | 1. [Download](./user_api_postman_collection.json) the Postman Collection 484 | 2. Run the Spring Boot Application 485 | 3. Try Executing the API endpoints 486 | 487 | ## Refernces 488 | - B-Tree 489 | - https://www.geeksforgeeks.org/introduction-of-b-tree-2/ 490 | - B+ Tree 491 | - https://www.geeksforgeeks.org/introduction-of-b-tree/ 492 | - B Tree vs B+ Tree 493 | - https://stackoverflow.com/questions/870218/what-are-the-differences-between-b-trees-and-b-trees 494 | - http://www.differencebetween.info/difference-between-b-tree-and-b-plus-tree 495 | - Database Partitioning 496 | - https://docs.oracle.com/en/database/oracle/oracle-database/19/vldbg/partition-concepts.html#GUID-EA7EF5CB-DD49-43AF-889A-F83AAC0D7D51 497 | - Disadvantages of Horizontal Partitioning 498 | - https://www.relationaldbdesign.com/database-analysis/module6/disadvantages-ofHorizontal-partitioning.php 499 | - MySQL Bin Log 500 | - https://dev.mysql.com/doc/internals/en/binary-log-overview.html 501 | - MySQL Replication Stratagy 502 | - https://severalnines.com/resources/database-management-tutorials/mysql-replication-high-availability-tutorial 503 | - ProxySQL Configuration 504 | - https://proxysql.com/documentation/global-variables/mysql-variables/ 505 | - Replication Lag 506 | - https://proxysql.com/documentation/backend-monitoring/ 507 | - https://proxysql.com/documentation/backend-server-configuration/ 508 | - ProxySQL Rules 509 | - https://proxysql.com/documentation/proxysql-configuration/ 510 | -------------------------------------------------------------------------------- /clean-up.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | sudo rm -rf master/data 3 | sudo rm -rf slave/data 4 | sudo rm -rf proxysql/data 5 | -------------------------------------------------------------------------------- /demographic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dipanjal/mysql-replication-poc/74fa72d9f27cf1ea2cb4838a10ea385b4ef772ba/demographic.png -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | 3 | services: 4 | 5 | mysql-master: 6 | image: mysql:5.7 7 | container_name: proxysql-mysql-replication-master 8 | environment: 9 | MYSQL_ROOT_PASSWORD: password 10 | MYSQL_DATABASE: sbtest 11 | volumes: 12 | - ./master/my.cnf:/etc/mysql/my.cnf 13 | - ./master/data:/var/lib/mysql 14 | - ./master/init.sql:/docker-entrypoint-initdb.d/init.sql 15 | ports: 16 | - 3306:3306 17 | networks: 18 | - mysql_cluster_net 19 | 20 | mysql-slave1: 21 | image: mysql:5.7 22 | container_name: proxysql-mysql-replication-slave1 23 | environment: 24 | MYSQL_ROOT_PASSWORD: password 25 | MYSQL_DATABASE: sbtest 26 | volumes: # 27 | - ./slave/my-slave1.cnf:/etc/mysql/my.cnf 28 | - ./slave/data/slave1:/var/lib/mysql 29 | - ./slave/init.sql:/docker-entrypoint-initdb.d/init.sql 30 | ports: 31 | - 3307:3306 32 | depends_on: 33 | - mysql-master 34 | networks: 35 | - mysql_cluster_net 36 | 37 | mysql-slave2: 38 | image: mysql:5.7 39 | container_name: proxysql-mysql-replication-slave2 40 | environment: 41 | MYSQL_ROOT_PASSWORD: password 42 | MYSQL_DATABASE: sbtest 43 | volumes: 44 | - ./slave/my-slave2.cnf:/etc/mysql/my.cnf 45 | - ./slave/data/slave2:/var/lib/mysql 46 | - ./slave/init.sql:/docker-entrypoint-initdb.d/init.sql 47 | ports: 48 | - 3308:3306 49 | depends_on: 50 | - mysql-master 51 | networks: 52 | - mysql_cluster_net 53 | 54 | proxysql: 55 | image: proxysql/proxysql:2.0.12 56 | container_name: proxysql-mysql-replication-proxysql 57 | ports: 58 | - 6032:6032 59 | - 6033:6033 60 | volumes: 61 | - ./proxysql/proxysql.cnf:/etc/proxysql.cnf 62 | - ./proxysql/data:/var/lib/proxysql 63 | networks: 64 | - mysql_cluster_net 65 | depends_on: # the proxy will start after all the mysql instance has started 66 | - mysql-master 67 | - mysql-slave1 68 | - mysql-slave2 69 | 70 | ## if you want to dockrize your Spring Boot Application 71 | # proxysql-test-app: 72 | # build: 73 | # context: . 74 | # image: dipanjalmaitra/proxysql-test-app:latest 75 | # ports: 76 | # - 8080:8080 77 | # restart: on-failure 78 | # # networks: 79 | # # - mysql_cluster_net 80 | # # depends_on: 81 | # # - proxysql 82 | # environment: 83 | # DB_PASSWORD: password 84 | # DB_URL: jdbc:mysql://proxysql:6033/sbtest?useSSL=false 85 | # ACTIVE_PROFILE: default 86 | 87 | #creating a common L2 Bridge network for the Docker containers internal communication 88 | networks: 89 | mysql_cluster_net: 90 | driver: bridge -------------------------------------------------------------------------------- /master/init.sql: -------------------------------------------------------------------------------- 1 | /* proxysql user */ 2 | CREATE USER IF NOT EXISTS 'monitor'@'%' IDENTIFIED BY 'monitor'; 3 | 4 | /* mysql exporter user */ 5 | CREATE USER IF NOT EXISTS 'exporter'@'%' IDENTIFIED BY 'password' WITH MAX_USER_CONNECTIONS 3; 6 | GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'%'; 7 | 8 | /* slave user */ 9 | CREATE USER IF NOT EXISTS 'slave_user'@'%' IDENTIFIED BY 'password'; 10 | GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' WITH GRANT OPTION; 11 | 12 | FLUSH PRIVILEGES; 13 | 14 | 15 | create table users 16 | ( 17 | id int auto_increment, 18 | name varchar(255) null, 19 | constraint users_pk 20 | primary key (id) 21 | ); 22 | 23 | INSERT INTO users VALUES (1, 'dipanjal'), (2, 'shohan'); 24 | 25 | -------------------------------------------------------------------------------- /master/my.cnf: -------------------------------------------------------------------------------- 1 | [mysqld] 2 | pid-file = /var/run/mysqld/mysqld.pid 3 | socket = /var/run/mysqld/mysqld.sock 4 | datadir = /var/lib/mysql 5 | secure-file-priv= NULL 6 | symbolic-links=0 7 | default_authentication_plugin=mysql_native_password 8 | lower_case_table_names = 1 9 | 10 | bind-address = 0.0.0.0 11 | server-id = 1 12 | log_bin = /var/run/mysqld/mysql-bin.log 13 | binlog_do_db = sbtest 14 | 15 | gtid_mode = on 16 | enforce_gtid_consistency = on 17 | log_slave_updates = on 18 | 19 | 20 | general_log=1 21 | general_log_file=/var/log/mysql/general.log 22 | 23 | # Slow query settings: 24 | slow_query_log=1 25 | slow_query_log_file=/var/log/mysql/slow.log 26 | long_query_time=0.5 27 | -------------------------------------------------------------------------------- /proxysql/proxysql.cnf: -------------------------------------------------------------------------------- 1 | datadir="/var/lib/proxysql" 2 | 3 | admin_variables= 4 | { 5 | admin_credentials="admin:admin;admin2:pass2" 6 | mysql_ifaces="0.0.0.0:6032" 7 | refresh_interval=2000 8 | stats_credentials="stats:admin" 9 | } 10 | 11 | mysql_variables= 12 | { 13 | threads=4 14 | max_connections=2048 15 | default_query_delay=0 16 | default_query_timeout=36000000 17 | have_compress=true 18 | poll_timeout=2000 19 | interfaces="0.0.0.0:6033;/tmp/proxysql.sock" 20 | default_schema="information_schema" 21 | stacksize=1048576 22 | server_version="8.0" 23 | connect_timeout_server=10000 24 | monitor_history=60000 25 | monitor_connect_interval=200000 26 | monitor_ping_interval=200000 27 | ping_interval_server_msec=10000 28 | ping_timeout_server=200 29 | commands_stats=true 30 | sessions_sort=true 31 | monitor_username="monitor" 32 | monitor_password="monitor" 33 | } 34 | 35 | mysql_replication_hostgroups = 36 | ( 37 | { writer_hostgroup=10 , reader_hostgroup=20 , comment="host groups" } 38 | ) 39 | 40 | mysql_servers = 41 | ( 42 | { address="mysql-master" , port=3306 , hostgroup=10, max_connections=100 , max_replication_lag = 5 }, 43 | { address="mysql-slave1" , port=3306 , hostgroup=20, max_connections=100 , max_replication_lag = 5 }, 44 | { address="mysql-slave2" , port=3306 , hostgroup=20, max_connections=100 , max_replication_lag = 5 } 45 | ) 46 | 47 | mysql_query_rules = 48 | ( 49 | { 50 | rule_id=100 51 | active=1 52 | match_pattern="^SELECT .* FOR UPDATE" 53 | destination_hostgroup=10 54 | apply=1 55 | }, 56 | { 57 | rule_id=200 58 | active=1 59 | match_pattern="^SELECT .*" 60 | destination_hostgroup=20 61 | apply=1 62 | }, 63 | { 64 | rule_id=300 65 | active=1 66 | match_pattern=".*" 67 | destination_hostgroup=10 68 | apply=1 69 | } 70 | ) 71 | 72 | mysql_users = 73 | ( 74 | { username = "root" , password = "password" , default_hostgroup = 10 , active = 1 } 75 | ) 76 | -------------------------------------------------------------------------------- /sbtest-app/.gitignore: -------------------------------------------------------------------------------- 1 | HELP.md 2 | target/ 3 | !.mvn/wrapper/maven-wrapper.jar 4 | !**/src/main/**/target/ 5 | !**/src/test/**/target/ 6 | 7 | ### STS ### 8 | .apt_generated 9 | .classpath 10 | .factorypath 11 | .project 12 | .settings 13 | .springBeans 14 | .sts4-cache 15 | 16 | ### IntelliJ IDEA ### 17 | .idea 18 | *.iws 19 | *.iml 20 | *.ipr 21 | 22 | ### NetBeans ### 23 | /nbproject/private/ 24 | /nbbuild/ 25 | /dist/ 26 | /nbdist/ 27 | /.nb-gradle/ 28 | build/ 29 | !**/src/main/**/build/ 30 | !**/src/test/**/build/ 31 | 32 | ### VS Code ### 33 | .vscode/ 34 | -------------------------------------------------------------------------------- /sbtest-app/.mvn/wrapper/MavenWrapperDownloader.java: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright 2007-present the original author or authors. 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); 5 | * you may not use this file except in compliance with the License. 6 | * You may obtain a copy of the License at 7 | * 8 | * https://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, 12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | * See the License for the specific language governing permissions and 14 | * limitations under the License. 15 | */ 16 | import java.net.*; 17 | import java.io.*; 18 | import java.nio.channels.*; 19 | import java.util.Properties; 20 | 21 | public class MavenWrapperDownloader { 22 | 23 | private static final String WRAPPER_VERSION = "0.5.6"; 24 | /** 25 | * Default URL to download the maven-wrapper.jar from, if no 'downloadUrl' is provided. 26 | */ 27 | private static final String DEFAULT_DOWNLOAD_URL = "https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/" 28 | + WRAPPER_VERSION + "/maven-wrapper-" + WRAPPER_VERSION + ".jar"; 29 | 30 | /** 31 | * Path to the maven-wrapper.properties file, which might contain a downloadUrl property to 32 | * use instead of the default one. 33 | */ 34 | private static final String MAVEN_WRAPPER_PROPERTIES_PATH = 35 | ".mvn/wrapper/maven-wrapper.properties"; 36 | 37 | /** 38 | * Path where the maven-wrapper.jar will be saved to. 39 | */ 40 | private static final String MAVEN_WRAPPER_JAR_PATH = 41 | ".mvn/wrapper/maven-wrapper.jar"; 42 | 43 | /** 44 | * Name of the property which should be used to override the default download url for the wrapper. 45 | */ 46 | private static final String PROPERTY_NAME_WRAPPER_URL = "wrapperUrl"; 47 | 48 | public static void main(String args[]) { 49 | System.out.println("- Downloader started"); 50 | File baseDirectory = new File(args[0]); 51 | System.out.println("- Using base directory: " + baseDirectory.getAbsolutePath()); 52 | 53 | // If the maven-wrapper.properties exists, read it and check if it contains a custom 54 | // wrapperUrl parameter. 55 | File mavenWrapperPropertyFile = new File(baseDirectory, MAVEN_WRAPPER_PROPERTIES_PATH); 56 | String url = DEFAULT_DOWNLOAD_URL; 57 | if(mavenWrapperPropertyFile.exists()) { 58 | FileInputStream mavenWrapperPropertyFileInputStream = null; 59 | try { 60 | mavenWrapperPropertyFileInputStream = new FileInputStream(mavenWrapperPropertyFile); 61 | Properties mavenWrapperProperties = new Properties(); 62 | mavenWrapperProperties.load(mavenWrapperPropertyFileInputStream); 63 | url = mavenWrapperProperties.getProperty(PROPERTY_NAME_WRAPPER_URL, url); 64 | } catch (IOException e) { 65 | System.out.println("- ERROR loading '" + MAVEN_WRAPPER_PROPERTIES_PATH + "'"); 66 | } finally { 67 | try { 68 | if(mavenWrapperPropertyFileInputStream != null) { 69 | mavenWrapperPropertyFileInputStream.close(); 70 | } 71 | } catch (IOException e) { 72 | // Ignore ... 73 | } 74 | } 75 | } 76 | System.out.println("- Downloading from: " + url); 77 | 78 | File outputFile = new File(baseDirectory.getAbsolutePath(), MAVEN_WRAPPER_JAR_PATH); 79 | if(!outputFile.getParentFile().exists()) { 80 | if(!outputFile.getParentFile().mkdirs()) { 81 | System.out.println( 82 | "- ERROR creating output directory '" + outputFile.getParentFile().getAbsolutePath() + "'"); 83 | } 84 | } 85 | System.out.println("- Downloading to: " + outputFile.getAbsolutePath()); 86 | try { 87 | downloadFileFromURL(url, outputFile); 88 | System.out.println("Done"); 89 | System.exit(0); 90 | } catch (Throwable e) { 91 | System.out.println("- Error downloading"); 92 | e.printStackTrace(); 93 | System.exit(1); 94 | } 95 | } 96 | 97 | private static void downloadFileFromURL(String urlString, File destination) throws Exception { 98 | if (System.getenv("MVNW_USERNAME") != null && System.getenv("MVNW_PASSWORD") != null) { 99 | String username = System.getenv("MVNW_USERNAME"); 100 | char[] password = System.getenv("MVNW_PASSWORD").toCharArray(); 101 | Authenticator.setDefault(new Authenticator() { 102 | @Override 103 | protected PasswordAuthentication getPasswordAuthentication() { 104 | return new PasswordAuthentication(username, password); 105 | } 106 | }); 107 | } 108 | URL website = new URL(urlString); 109 | ReadableByteChannel rbc; 110 | rbc = Channels.newChannel(website.openStream()); 111 | FileOutputStream fos = new FileOutputStream(destination); 112 | fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE); 113 | fos.close(); 114 | rbc.close(); 115 | } 116 | 117 | } 118 | -------------------------------------------------------------------------------- /sbtest-app/.mvn/wrapper/maven-wrapper.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dipanjal/mysql-replication-poc/74fa72d9f27cf1ea2cb4838a10ea385b4ef772ba/sbtest-app/.mvn/wrapper/maven-wrapper.jar -------------------------------------------------------------------------------- /sbtest-app/.mvn/wrapper/maven-wrapper.properties: -------------------------------------------------------------------------------- 1 | distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.3/apache-maven-3.8.3-bin.zip 2 | wrapperUrl=https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar 3 | -------------------------------------------------------------------------------- /sbtest-app/mvnw: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # ---------------------------------------------------------------------------- 3 | # Licensed to the Apache Software Foundation (ASF) under one 4 | # or more contributor license agreements. See the NOTICE file 5 | # distributed with this work for additional information 6 | # regarding copyright ownership. The ASF licenses this file 7 | # to you under the Apache License, Version 2.0 (the 8 | # "License"); you may not use this file except in compliance 9 | # with the License. You may obtain a copy of the License at 10 | # 11 | # https://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, 14 | # software distributed under the License is distributed on an 15 | # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 16 | # KIND, either express or implied. See the License for the 17 | # specific language governing permissions and limitations 18 | # under the License. 19 | # ---------------------------------------------------------------------------- 20 | 21 | # ---------------------------------------------------------------------------- 22 | # Maven Start Up Batch script 23 | # 24 | # Required ENV vars: 25 | # ------------------ 26 | # JAVA_HOME - location of a JDK home dir 27 | # 28 | # Optional ENV vars 29 | # ----------------- 30 | # M2_HOME - location of maven2's installed home dir 31 | # MAVEN_OPTS - parameters passed to the Java VM when running Maven 32 | # e.g. to debug Maven itself, use 33 | # set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 34 | # MAVEN_SKIP_RC - flag to disable loading of mavenrc files 35 | # ---------------------------------------------------------------------------- 36 | 37 | if [ -z "$MAVEN_SKIP_RC" ] ; then 38 | 39 | if [ -f /etc/mavenrc ] ; then 40 | . /etc/mavenrc 41 | fi 42 | 43 | if [ -f "$HOME/.mavenrc" ] ; then 44 | . "$HOME/.mavenrc" 45 | fi 46 | 47 | fi 48 | 49 | # OS specific support. $var _must_ be set to either true or false. 50 | cygwin=false; 51 | darwin=false; 52 | mingw=false 53 | case "`uname`" in 54 | CYGWIN*) cygwin=true ;; 55 | MINGW*) mingw=true;; 56 | Darwin*) darwin=true 57 | # Use /usr/libexec/java_home if available, otherwise fall back to /Library/Java/Home 58 | # See https://developer.apple.com/library/mac/qa/qa1170/_index.html 59 | if [ -z "$JAVA_HOME" ]; then 60 | if [ -x "/usr/libexec/java_home" ]; then 61 | export JAVA_HOME="`/usr/libexec/java_home`" 62 | else 63 | export JAVA_HOME="/Library/Java/Home" 64 | fi 65 | fi 66 | ;; 67 | esac 68 | 69 | if [ -z "$JAVA_HOME" ] ; then 70 | if [ -r /etc/gentoo-release ] ; then 71 | JAVA_HOME=`java-config --jre-home` 72 | fi 73 | fi 74 | 75 | if [ -z "$M2_HOME" ] ; then 76 | ## resolve links - $0 may be a link to maven's home 77 | PRG="$0" 78 | 79 | # need this for relative symlinks 80 | while [ -h "$PRG" ] ; do 81 | ls=`ls -ld "$PRG"` 82 | link=`expr "$ls" : '.*-> \(.*\)$'` 83 | if expr "$link" : '/.*' > /dev/null; then 84 | PRG="$link" 85 | else 86 | PRG="`dirname "$PRG"`/$link" 87 | fi 88 | done 89 | 90 | saveddir=`pwd` 91 | 92 | M2_HOME=`dirname "$PRG"`/.. 93 | 94 | # make it fully qualified 95 | M2_HOME=`cd "$M2_HOME" && pwd` 96 | 97 | cd "$saveddir" 98 | # echo Using m2 at $M2_HOME 99 | fi 100 | 101 | # For Cygwin, ensure paths are in UNIX format before anything is touched 102 | if $cygwin ; then 103 | [ -n "$M2_HOME" ] && 104 | M2_HOME=`cygpath --unix "$M2_HOME"` 105 | [ -n "$JAVA_HOME" ] && 106 | JAVA_HOME=`cygpath --unix "$JAVA_HOME"` 107 | [ -n "$CLASSPATH" ] && 108 | CLASSPATH=`cygpath --path --unix "$CLASSPATH"` 109 | fi 110 | 111 | # For Mingw, ensure paths are in UNIX format before anything is touched 112 | if $mingw ; then 113 | [ -n "$M2_HOME" ] && 114 | M2_HOME="`(cd "$M2_HOME"; pwd)`" 115 | [ -n "$JAVA_HOME" ] && 116 | JAVA_HOME="`(cd "$JAVA_HOME"; pwd)`" 117 | fi 118 | 119 | if [ -z "$JAVA_HOME" ]; then 120 | javaExecutable="`which javac`" 121 | if [ -n "$javaExecutable" ] && ! [ "`expr \"$javaExecutable\" : '\([^ ]*\)'`" = "no" ]; then 122 | # readlink(1) is not available as standard on Solaris 10. 123 | readLink=`which readlink` 124 | if [ ! `expr "$readLink" : '\([^ ]*\)'` = "no" ]; then 125 | if $darwin ; then 126 | javaHome="`dirname \"$javaExecutable\"`" 127 | javaExecutable="`cd \"$javaHome\" && pwd -P`/javac" 128 | else 129 | javaExecutable="`readlink -f \"$javaExecutable\"`" 130 | fi 131 | javaHome="`dirname \"$javaExecutable\"`" 132 | javaHome=`expr "$javaHome" : '\(.*\)/bin'` 133 | JAVA_HOME="$javaHome" 134 | export JAVA_HOME 135 | fi 136 | fi 137 | fi 138 | 139 | if [ -z "$JAVACMD" ] ; then 140 | if [ -n "$JAVA_HOME" ] ; then 141 | if [ -x "$JAVA_HOME/jre/sh/java" ] ; then 142 | # IBM's JDK on AIX uses strange locations for the executables 143 | JAVACMD="$JAVA_HOME/jre/sh/java" 144 | else 145 | JAVACMD="$JAVA_HOME/bin/java" 146 | fi 147 | else 148 | JAVACMD="`which java`" 149 | fi 150 | fi 151 | 152 | if [ ! -x "$JAVACMD" ] ; then 153 | echo "Error: JAVA_HOME is not defined correctly." >&2 154 | echo " We cannot execute $JAVACMD" >&2 155 | exit 1 156 | fi 157 | 158 | if [ -z "$JAVA_HOME" ] ; then 159 | echo "Warning: JAVA_HOME environment variable is not set." 160 | fi 161 | 162 | CLASSWORLDS_LAUNCHER=org.codehaus.plexus.classworlds.launcher.Launcher 163 | 164 | # traverses directory structure from process work directory to filesystem root 165 | # first directory with .mvn subdirectory is considered project base directory 166 | find_maven_basedir() { 167 | 168 | if [ -z "$1" ] 169 | then 170 | echo "Path not specified to find_maven_basedir" 171 | return 1 172 | fi 173 | 174 | basedir="$1" 175 | wdir="$1" 176 | while [ "$wdir" != '/' ] ; do 177 | if [ -d "$wdir"/.mvn ] ; then 178 | basedir=$wdir 179 | break 180 | fi 181 | # workaround for JBEAP-8937 (on Solaris 10/Sparc) 182 | if [ -d "${wdir}" ]; then 183 | wdir=`cd "$wdir/.."; pwd` 184 | fi 185 | # end of workaround 186 | done 187 | echo "${basedir}" 188 | } 189 | 190 | # concatenates all lines of a file 191 | concat_lines() { 192 | if [ -f "$1" ]; then 193 | echo "$(tr -s '\n' ' ' < "$1")" 194 | fi 195 | } 196 | 197 | BASE_DIR=`find_maven_basedir "$(pwd)"` 198 | if [ -z "$BASE_DIR" ]; then 199 | exit 1; 200 | fi 201 | 202 | ########################################################################################## 203 | # Extension to allow automatically downloading the maven-wrapper.jar from Maven-central 204 | # This allows using the maven wrapper in projects that prohibit checking in binary data. 205 | ########################################################################################## 206 | if [ -r "$BASE_DIR/.mvn/wrapper/maven-wrapper.jar" ]; then 207 | if [ "$MVNW_VERBOSE" = true ]; then 208 | echo "Found .mvn/wrapper/maven-wrapper.jar" 209 | fi 210 | else 211 | if [ "$MVNW_VERBOSE" = true ]; then 212 | echo "Couldn't find .mvn/wrapper/maven-wrapper.jar, downloading it ..." 213 | fi 214 | if [ -n "$MVNW_REPOURL" ]; then 215 | jarUrl="$MVNW_REPOURL/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar" 216 | else 217 | jarUrl="https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar" 218 | fi 219 | while IFS="=" read key value; do 220 | case "$key" in (wrapperUrl) jarUrl="$value"; break ;; 221 | esac 222 | done < "$BASE_DIR/.mvn/wrapper/maven-wrapper.properties" 223 | if [ "$MVNW_VERBOSE" = true ]; then 224 | echo "Downloading from: $jarUrl" 225 | fi 226 | wrapperJarPath="$BASE_DIR/.mvn/wrapper/maven-wrapper.jar" 227 | if $cygwin; then 228 | wrapperJarPath=`cygpath --path --windows "$wrapperJarPath"` 229 | fi 230 | 231 | if command -v wget > /dev/null; then 232 | if [ "$MVNW_VERBOSE" = true ]; then 233 | echo "Found wget ... using wget" 234 | fi 235 | if [ -z "$MVNW_USERNAME" ] || [ -z "$MVNW_PASSWORD" ]; then 236 | wget "$jarUrl" -O "$wrapperJarPath" 237 | else 238 | wget --http-user=$MVNW_USERNAME --http-password=$MVNW_PASSWORD "$jarUrl" -O "$wrapperJarPath" 239 | fi 240 | elif command -v curl > /dev/null; then 241 | if [ "$MVNW_VERBOSE" = true ]; then 242 | echo "Found curl ... using curl" 243 | fi 244 | if [ -z "$MVNW_USERNAME" ] || [ -z "$MVNW_PASSWORD" ]; then 245 | curl -o "$wrapperJarPath" "$jarUrl" -f 246 | else 247 | curl --user $MVNW_USERNAME:$MVNW_PASSWORD -o "$wrapperJarPath" "$jarUrl" -f 248 | fi 249 | 250 | else 251 | if [ "$MVNW_VERBOSE" = true ]; then 252 | echo "Falling back to using Java to download" 253 | fi 254 | javaClass="$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.java" 255 | # For Cygwin, switch paths to Windows format before running javac 256 | if $cygwin; then 257 | javaClass=`cygpath --path --windows "$javaClass"` 258 | fi 259 | if [ -e "$javaClass" ]; then 260 | if [ ! -e "$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.class" ]; then 261 | if [ "$MVNW_VERBOSE" = true ]; then 262 | echo " - Compiling MavenWrapperDownloader.java ..." 263 | fi 264 | # Compiling the Java class 265 | ("$JAVA_HOME/bin/javac" "$javaClass") 266 | fi 267 | if [ -e "$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.class" ]; then 268 | # Running the downloader 269 | if [ "$MVNW_VERBOSE" = true ]; then 270 | echo " - Running MavenWrapperDownloader.java ..." 271 | fi 272 | ("$JAVA_HOME/bin/java" -cp .mvn/wrapper MavenWrapperDownloader "$MAVEN_PROJECTBASEDIR") 273 | fi 274 | fi 275 | fi 276 | fi 277 | ########################################################################################## 278 | # End of extension 279 | ########################################################################################## 280 | 281 | export MAVEN_PROJECTBASEDIR=${MAVEN_BASEDIR:-"$BASE_DIR"} 282 | if [ "$MVNW_VERBOSE" = true ]; then 283 | echo $MAVEN_PROJECTBASEDIR 284 | fi 285 | MAVEN_OPTS="$(concat_lines "$MAVEN_PROJECTBASEDIR/.mvn/jvm.config") $MAVEN_OPTS" 286 | 287 | # For Cygwin, switch paths to Windows format before running java 288 | if $cygwin; then 289 | [ -n "$M2_HOME" ] && 290 | M2_HOME=`cygpath --path --windows "$M2_HOME"` 291 | [ -n "$JAVA_HOME" ] && 292 | JAVA_HOME=`cygpath --path --windows "$JAVA_HOME"` 293 | [ -n "$CLASSPATH" ] && 294 | CLASSPATH=`cygpath --path --windows "$CLASSPATH"` 295 | [ -n "$MAVEN_PROJECTBASEDIR" ] && 296 | MAVEN_PROJECTBASEDIR=`cygpath --path --windows "$MAVEN_PROJECTBASEDIR"` 297 | fi 298 | 299 | # Provide a "standardized" way to retrieve the CLI args that will 300 | # work with both Windows and non-Windows executions. 301 | MAVEN_CMD_LINE_ARGS="$MAVEN_CONFIG $@" 302 | export MAVEN_CMD_LINE_ARGS 303 | 304 | WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain 305 | 306 | exec "$JAVACMD" \ 307 | $MAVEN_OPTS \ 308 | -classpath "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.jar" \ 309 | "-Dmaven.home=${M2_HOME}" "-Dmaven.multiModuleProjectDirectory=${MAVEN_PROJECTBASEDIR}" \ 310 | ${WRAPPER_LAUNCHER} $MAVEN_CONFIG "$@" 311 | -------------------------------------------------------------------------------- /sbtest-app/mvnw.cmd: -------------------------------------------------------------------------------- 1 | @REM ---------------------------------------------------------------------------- 2 | @REM Licensed to the Apache Software Foundation (ASF) under one 3 | @REM or more contributor license agreements. See the NOTICE file 4 | @REM distributed with this work for additional information 5 | @REM regarding copyright ownership. The ASF licenses this file 6 | @REM to you under the Apache License, Version 2.0 (the 7 | @REM "License"); you may not use this file except in compliance 8 | @REM with the License. You may obtain a copy of the License at 9 | @REM 10 | @REM https://www.apache.org/licenses/LICENSE-2.0 11 | @REM 12 | @REM Unless required by applicable law or agreed to in writing, 13 | @REM software distributed under the License is distributed on an 14 | @REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 15 | @REM KIND, either express or implied. See the License for the 16 | @REM specific language governing permissions and limitations 17 | @REM under the License. 18 | @REM ---------------------------------------------------------------------------- 19 | 20 | @REM ---------------------------------------------------------------------------- 21 | @REM Maven Start Up Batch script 22 | @REM 23 | @REM Required ENV vars: 24 | @REM JAVA_HOME - location of a JDK home dir 25 | @REM 26 | @REM Optional ENV vars 27 | @REM M2_HOME - location of maven2's installed home dir 28 | @REM MAVEN_BATCH_ECHO - set to 'on' to enable the echoing of the batch commands 29 | @REM MAVEN_BATCH_PAUSE - set to 'on' to wait for a keystroke before ending 30 | @REM MAVEN_OPTS - parameters passed to the Java VM when running Maven 31 | @REM e.g. to debug Maven itself, use 32 | @REM set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 33 | @REM MAVEN_SKIP_RC - flag to disable loading of mavenrc files 34 | @REM ---------------------------------------------------------------------------- 35 | 36 | @REM Begin all REM lines with '@' in case MAVEN_BATCH_ECHO is 'on' 37 | @echo off 38 | @REM set title of command window 39 | title %0 40 | @REM enable echoing by setting MAVEN_BATCH_ECHO to 'on' 41 | @if "%MAVEN_BATCH_ECHO%" == "on" echo %MAVEN_BATCH_ECHO% 42 | 43 | @REM set %HOME% to equivalent of $HOME 44 | if "%HOME%" == "" (set "HOME=%HOMEDRIVE%%HOMEPATH%") 45 | 46 | @REM Execute a user defined script before this one 47 | if not "%MAVEN_SKIP_RC%" == "" goto skipRcPre 48 | @REM check for pre script, once with legacy .bat ending and once with .cmd ending 49 | if exist "%HOME%\mavenrc_pre.bat" call "%HOME%\mavenrc_pre.bat" 50 | if exist "%HOME%\mavenrc_pre.cmd" call "%HOME%\mavenrc_pre.cmd" 51 | :skipRcPre 52 | 53 | @setlocal 54 | 55 | set ERROR_CODE=0 56 | 57 | @REM To isolate internal variables from possible post scripts, we use another setlocal 58 | @setlocal 59 | 60 | @REM ==== START VALIDATION ==== 61 | if not "%JAVA_HOME%" == "" goto OkJHome 62 | 63 | echo. 64 | echo Error: JAVA_HOME not found in your environment. >&2 65 | echo Please set the JAVA_HOME variable in your environment to match the >&2 66 | echo location of your Java installation. >&2 67 | echo. 68 | goto error 69 | 70 | :OkJHome 71 | if exist "%JAVA_HOME%\bin\java.exe" goto init 72 | 73 | echo. 74 | echo Error: JAVA_HOME is set to an invalid directory. >&2 75 | echo JAVA_HOME = "%JAVA_HOME%" >&2 76 | echo Please set the JAVA_HOME variable in your environment to match the >&2 77 | echo location of your Java installation. >&2 78 | echo. 79 | goto error 80 | 81 | @REM ==== END VALIDATION ==== 82 | 83 | :init 84 | 85 | @REM Find the project base dir, i.e. the directory that contains the folder ".mvn". 86 | @REM Fallback to current working directory if not found. 87 | 88 | set MAVEN_PROJECTBASEDIR=%MAVEN_BASEDIR% 89 | IF NOT "%MAVEN_PROJECTBASEDIR%"=="" goto endDetectBaseDir 90 | 91 | set EXEC_DIR=%CD% 92 | set WDIR=%EXEC_DIR% 93 | :findBaseDir 94 | IF EXIST "%WDIR%"\.mvn goto baseDirFound 95 | cd .. 96 | IF "%WDIR%"=="%CD%" goto baseDirNotFound 97 | set WDIR=%CD% 98 | goto findBaseDir 99 | 100 | :baseDirFound 101 | set MAVEN_PROJECTBASEDIR=%WDIR% 102 | cd "%EXEC_DIR%" 103 | goto endDetectBaseDir 104 | 105 | :baseDirNotFound 106 | set MAVEN_PROJECTBASEDIR=%EXEC_DIR% 107 | cd "%EXEC_DIR%" 108 | 109 | :endDetectBaseDir 110 | 111 | IF NOT EXIST "%MAVEN_PROJECTBASEDIR%\.mvn\jvm.config" goto endReadAdditionalConfig 112 | 113 | @setlocal EnableExtensions EnableDelayedExpansion 114 | for /F "usebackq delims=" %%a in ("%MAVEN_PROJECTBASEDIR%\.mvn\jvm.config") do set JVM_CONFIG_MAVEN_PROPS=!JVM_CONFIG_MAVEN_PROPS! %%a 115 | @endlocal & set JVM_CONFIG_MAVEN_PROPS=%JVM_CONFIG_MAVEN_PROPS% 116 | 117 | :endReadAdditionalConfig 118 | 119 | SET MAVEN_JAVA_EXE="%JAVA_HOME%\bin\java.exe" 120 | set WRAPPER_JAR="%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.jar" 121 | set WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain 122 | 123 | set DOWNLOAD_URL="https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar" 124 | 125 | FOR /F "tokens=1,2 delims==" %%A IN ("%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.properties") DO ( 126 | IF "%%A"=="wrapperUrl" SET DOWNLOAD_URL=%%B 127 | ) 128 | 129 | @REM Extension to allow automatically downloading the maven-wrapper.jar from Maven-central 130 | @REM This allows using the maven wrapper in projects that prohibit checking in binary data. 131 | if exist %WRAPPER_JAR% ( 132 | if "%MVNW_VERBOSE%" == "true" ( 133 | echo Found %WRAPPER_JAR% 134 | ) 135 | ) else ( 136 | if not "%MVNW_REPOURL%" == "" ( 137 | SET DOWNLOAD_URL="%MVNW_REPOURL%/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar" 138 | ) 139 | if "%MVNW_VERBOSE%" == "true" ( 140 | echo Couldn't find %WRAPPER_JAR%, downloading it ... 141 | echo Downloading from: %DOWNLOAD_URL% 142 | ) 143 | 144 | powershell -Command "&{"^ 145 | "$webclient = new-object System.Net.WebClient;"^ 146 | "if (-not ([string]::IsNullOrEmpty('%MVNW_USERNAME%') -and [string]::IsNullOrEmpty('%MVNW_PASSWORD%'))) {"^ 147 | "$webclient.Credentials = new-object System.Net.NetworkCredential('%MVNW_USERNAME%', '%MVNW_PASSWORD%');"^ 148 | "}"^ 149 | "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $webclient.DownloadFile('%DOWNLOAD_URL%', '%WRAPPER_JAR%')"^ 150 | "}" 151 | if "%MVNW_VERBOSE%" == "true" ( 152 | echo Finished downloading %WRAPPER_JAR% 153 | ) 154 | ) 155 | @REM End of extension 156 | 157 | @REM Provide a "standardized" way to retrieve the CLI args that will 158 | @REM work with both Windows and non-Windows executions. 159 | set MAVEN_CMD_LINE_ARGS=%* 160 | 161 | %MAVEN_JAVA_EXE% %JVM_CONFIG_MAVEN_PROPS% %MAVEN_OPTS% %MAVEN_DEBUG_OPTS% -classpath %WRAPPER_JAR% "-Dmaven.multiModuleProjectDirectory=%MAVEN_PROJECTBASEDIR%" %WRAPPER_LAUNCHER% %MAVEN_CONFIG% %* 162 | if ERRORLEVEL 1 goto error 163 | goto end 164 | 165 | :error 166 | set ERROR_CODE=1 167 | 168 | :end 169 | @endlocal & set ERROR_CODE=%ERROR_CODE% 170 | 171 | if not "%MAVEN_SKIP_RC%" == "" goto skipRcPost 172 | @REM check for post script, once with legacy .bat ending and once with .cmd ending 173 | if exist "%HOME%\mavenrc_post.bat" call "%HOME%\mavenrc_post.bat" 174 | if exist "%HOME%\mavenrc_post.cmd" call "%HOME%\mavenrc_post.cmd" 175 | :skipRcPost 176 | 177 | @REM pause the script if MAVEN_BATCH_PAUSE is set to 'on' 178 | if "%MAVEN_BATCH_PAUSE%" == "on" pause 179 | 180 | if "%MAVEN_TERMINATE_CMD%" == "on" exit %ERROR_CODE% 181 | 182 | exit /B %ERROR_CODE% 183 | -------------------------------------------------------------------------------- /sbtest-app/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 4 | 4.0.0 5 | 6 | org.springframework.boot 7 | spring-boot-starter-parent 8 | 2.6.2 9 | 10 | 11 | com.poc.proxysql 12 | proxysql-poc 13 | 0.0.1-SNAPSHOT 14 | proxysql-poc 15 | ProxySQL PoC Project 16 | 17 | 11 18 | 19 | 20 | 21 | 22 | org.springframework.boot 23 | spring-boot-starter-data-jpa 24 | 25 | 26 | org.springframework.boot 27 | spring-boot-starter-web 28 | 29 | 30 | 31 | org.springframework.boot 32 | spring-boot-devtools 33 | runtime 34 | true 35 | 36 | 37 | mysql 38 | mysql-connector-java 39 | runtime 40 | 41 | 42 | org.projectlombok 43 | lombok 44 | true 45 | 46 | 47 | org.springframework.boot 48 | spring-boot-starter-test 49 | test 50 | 51 | 52 | 53 | 54 | org.springdoc 55 | springdoc-openapi-ui 56 | 1.6.3 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | org.springframework.boot 66 | spring-boot-maven-plugin 67 | 68 | 69 | 70 | org.projectlombok 71 | lombok 72 | 73 | 74 | 75 | 76 | 77 | org.apache.maven.plugins 78 | maven-surefire-plugin 79 | 2.22.2 80 | 81 | true 82 | 83 | 84 | 85 | 86 | 87 | 88 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/ProxysqlPocApplication.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql; 2 | 3 | import org.springframework.boot.SpringApplication; 4 | import org.springframework.boot.autoconfigure.SpringBootApplication; 5 | 6 | @SpringBootApplication 7 | public class ProxysqlPocApplication { 8 | 9 | public static void main(String[] args) { 10 | SpringApplication.run(ProxysqlPocApplication.class, args); 11 | } 12 | 13 | } 14 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/config/OpenApiConfiguration.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.config; 2 | 3 | import io.swagger.v3.oas.models.OpenAPI; 4 | import io.swagger.v3.oas.models.info.Contact; 5 | import io.swagger.v3.oas.models.info.Info; 6 | import io.swagger.v3.oas.models.info.License; 7 | import org.springframework.context.annotation.Bean; 8 | import org.springframework.context.annotation.Configuration; 9 | 10 | /** 11 | * @author dipanjal 12 | * @since 0.0.1 13 | */ 14 | 15 | @Configuration 16 | public class OpenApiConfiguration { 17 | 18 | @Bean 19 | public OpenAPI customOpenAPI() { 20 | return new OpenAPI() 21 | .info(this.getApiInfo()); 22 | } 23 | 24 | private Info getApiInfo() { 25 | return new Info() 26 | .title("User Management REST Api Service") 27 | .version("1.0") 28 | .description("Rest Api server for Mysql Replication Test") 29 | .contact(getContactInfo()) 30 | .termsOfService("http://swagger.io/terms/") 31 | .license(new License().name("Apache 2.0").url("http://springdoc.org")); 32 | } 33 | 34 | private Contact getContactInfo() { 35 | return new Contact() 36 | .name("Dipanjal Maitra") 37 | .email("dipanjalmaitra@gmail.com") 38 | .url("https://github.com/dipanjal"); 39 | } 40 | } 41 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/controller/UserController.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.controller; 2 | 3 | import com.poc.proxysql.models.UserCreateRequest; 4 | import com.poc.proxysql.models.UserDTO; 5 | import com.poc.proxysql.models.UserUpdateRequest; 6 | import com.poc.proxysql.service.UserServiceImpl; 7 | import io.swagger.v3.oas.annotations.Operation; 8 | import lombok.RequiredArgsConstructor; 9 | import org.springframework.http.ResponseEntity; 10 | import org.springframework.web.bind.annotation.*; 11 | 12 | import java.util.List; 13 | 14 | /** 15 | * @author dipanjal 16 | * @since 0.0.1 17 | */ 18 | 19 | @RestController 20 | @RequiredArgsConstructor 21 | public class UserController { 22 | 23 | private final UserServiceImpl userService; 24 | 25 | @GetMapping("/users") 26 | @Operation(description = "Read all Users") 27 | public ResponseEntity> fetchAllUsers() { 28 | return ResponseEntity.ok(userService.getAllUsers()); 29 | } 30 | 31 | @GetMapping("/users/{id}") 32 | @Operation(description = "Read Specific User by Id") 33 | public ResponseEntity fetchUserById(@PathVariable long id){ 34 | return ResponseEntity.ok(userService.getUserById(id)); 35 | } 36 | 37 | @PostMapping("/users") 38 | @Operation(description = "Create new User") 39 | public ResponseEntity createUser(@RequestBody UserCreateRequest request){ 40 | return ResponseEntity.ok(userService.createUser(request)); 41 | } 42 | 43 | @PutMapping("/users") 44 | @Operation(description = "Update User") 45 | public ResponseEntity updateUser(@RequestBody UserUpdateRequest request){ 46 | return ResponseEntity.ok(userService.updateUser(request)); 47 | } 48 | 49 | @DeleteMapping("/users/{id}") 50 | @Operation(description = "Delete User by Id") 51 | public ResponseEntity deleteUser(@PathVariable long id){ 52 | return ResponseEntity.ok(userService.deleteUser(id)); 53 | } 54 | } 55 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/models/UserCreateRequest.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.models; 2 | 3 | import lombok.AllArgsConstructor; 4 | import lombok.Getter; 5 | import lombok.NoArgsConstructor; 6 | import lombok.Setter; 7 | 8 | /** 9 | * @author dipanjal 10 | * @since 0.0.1 11 | */ 12 | 13 | @Getter 14 | @Setter 15 | @AllArgsConstructor 16 | @NoArgsConstructor 17 | public class UserCreateRequest { 18 | private String name; 19 | } 20 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/models/UserDTO.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.models; 2 | 3 | import lombok.Builder; 4 | import lombok.Getter; 5 | import lombok.Setter; 6 | 7 | import java.io.Serializable; 8 | 9 | /** 10 | * @author dipanjal 11 | * @since 0.0.1 12 | */ 13 | 14 | @Getter 15 | @Setter 16 | @Builder 17 | public class UserDTO implements Serializable { 18 | private long id; 19 | private String name; 20 | } 21 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/models/UserUpdateRequest.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.models; 2 | 3 | import lombok.AllArgsConstructor; 4 | import lombok.Getter; 5 | import lombok.NoArgsConstructor; 6 | import lombok.Setter; 7 | 8 | /** 9 | * @author dipanjal 10 | * @since 0.0.1 11 | */ 12 | 13 | @Getter 14 | @Setter 15 | @AllArgsConstructor 16 | @NoArgsConstructor 17 | public class UserUpdateRequest { 18 | private long id; 19 | private String name; 20 | } 21 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/repository/UserRepository.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.repository; 2 | 3 | import com.poc.proxysql.repository.entity.UserEntity; 4 | import org.springframework.data.jpa.repository.JpaRepository; 5 | 6 | /** 7 | * @author dipanjal 8 | * @since 0.0.1 9 | */ 10 | 11 | public interface UserRepository extends JpaRepository { 12 | } 13 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/repository/entity/UserEntity.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.repository.entity; 2 | 3 | import lombok.Data; 4 | import javax.persistence.*; 5 | 6 | /** 7 | * @author dipanjal 8 | * @since 0.0.1 9 | */ 10 | 11 | @Entity 12 | @Table(name = "users") 13 | @Data 14 | public class UserEntity { 15 | @Id 16 | @GeneratedValue(strategy = GenerationType.IDENTITY) 17 | private long id; 18 | private String name; 19 | } 20 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/service/UserMapper.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.service; 2 | 3 | import com.poc.proxysql.models.UserCreateRequest; 4 | import com.poc.proxysql.models.UserDTO; 5 | import com.poc.proxysql.models.UserUpdateRequest; 6 | import com.poc.proxysql.repository.entity.UserEntity; 7 | import org.springframework.stereotype.Component; 8 | 9 | import java.util.List; 10 | import java.util.stream.Collectors; 11 | 12 | /** 13 | * @author dipanjal 14 | * @since 0.0.1 15 | */ 16 | 17 | @Component 18 | public class UserMapper { 19 | 20 | public UserDTO mapToDto(UserEntity entity) { 21 | return UserDTO.builder() 22 | .id(entity.getId()) 23 | .name(entity.getName()) 24 | .build(); 25 | } 26 | 27 | public List mapToDto(List entities){ 28 | return entities 29 | .stream() 30 | .map(this::mapToDto) 31 | .collect(Collectors.toList()); 32 | } 33 | 34 | public UserEntity mapToEntity(UserCreateRequest request){ 35 | UserEntity entity = new UserEntity(); 36 | entity.setName(request.getName()); 37 | return entity; 38 | } 39 | 40 | public UserEntity mapToEntity(UserEntity entity, UserUpdateRequest request){ 41 | entity.setName(request.getName()); 42 | return entity; 43 | } 44 | } 45 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/service/UserService.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.service; 2 | 3 | 4 | import com.poc.proxysql.models.UserCreateRequest; 5 | import com.poc.proxysql.models.UserDTO; 6 | import com.poc.proxysql.models.UserUpdateRequest; 7 | 8 | import java.util.List; 9 | 10 | public interface UserService { 11 | List getAllUsers(); 12 | UserDTO getUserById(long id); 13 | UserDTO createUser(UserCreateRequest request); 14 | UserDTO updateUser(UserUpdateRequest request); 15 | UserDTO deleteUser(long id); 16 | } 17 | -------------------------------------------------------------------------------- /sbtest-app/src/main/java/com/poc/proxysql/service/UserServiceImpl.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.service; 2 | 3 | import com.poc.proxysql.models.UserCreateRequest; 4 | import com.poc.proxysql.models.UserDTO; 5 | import com.poc.proxysql.models.UserUpdateRequest; 6 | import com.poc.proxysql.repository.UserRepository; 7 | import com.poc.proxysql.repository.entity.UserEntity; 8 | import lombok.RequiredArgsConstructor; 9 | import org.springframework.stereotype.Service; 10 | 11 | import java.util.List; 12 | 13 | /** 14 | * @author dipanjal 15 | * @since 0.0.1 16 | */ 17 | 18 | @Service 19 | @RequiredArgsConstructor 20 | public class UserServiceImpl implements UserService { 21 | 22 | private final UserRepository userRepository; 23 | private final UserMapper mapper; 24 | 25 | @Override 26 | public List getAllUsers() { 27 | return mapper.mapToDto(userRepository.findAll()); 28 | } 29 | 30 | @Override 31 | public UserDTO getUserById(long id) { 32 | return userRepository.findById(id) 33 | .map(mapper::mapToDto) 34 | .orElseThrow(() -> new RuntimeException("User Not Found")); 35 | } 36 | 37 | @Override 38 | public UserDTO createUser(UserCreateRequest request) { 39 | UserEntity entity = userRepository.save( 40 | mapper.mapToEntity(request) 41 | ); 42 | return mapper.mapToDto(entity); 43 | } 44 | 45 | @Override 46 | public UserDTO updateUser(UserUpdateRequest request) { 47 | UserEntity entity = userRepository.save( 48 | mapper.mapToEntity(userRepository 49 | .findById(request.getId()) 50 | .orElseThrow(() -> new RuntimeException("Updatable User Not Found")), 51 | request 52 | ) 53 | ); 54 | return mapper.mapToDto(entity); 55 | } 56 | 57 | @Override 58 | public UserDTO deleteUser(long id) { 59 | UserEntity entityToDelete = userRepository 60 | .findById(id) 61 | .orElseThrow(() -> new RuntimeException("User Not Found to Delete")); 62 | userRepository.delete(entityToDelete); 63 | return mapper.mapToDto(entityToDelete); 64 | } 65 | } 66 | -------------------------------------------------------------------------------- /sbtest-app/src/main/resources/application.yml: -------------------------------------------------------------------------------- 1 | spring: 2 | datasource: 3 | url: ${DB_URL:jdbc:mysql://13.233.178.172:6033/sbtest?useSSL=false} 4 | username: ${DB_USERNAME:root} 5 | password: ${DB_PASSWORD:password} 6 | flyway: 7 | enabled: false 8 | 9 | application-test: 10 | testFetchAllUsers: 11 | repeat: 10 #set test repeat time 12 | delay: 5000 #set delays in milli 13 | -------------------------------------------------------------------------------- /sbtest-app/src/test/java/com/poc/proxysql/ProxysqlPocApplicationTests.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql; 2 | 3 | import org.junit.jupiter.api.Test; 4 | import org.springframework.boot.test.context.SpringBootTest; 5 | 6 | @SpringBootTest 7 | class ProxysqlPocApplicationTests { 8 | 9 | @Test 10 | void contextLoads() { 11 | } 12 | 13 | 14 | 15 | } 16 | -------------------------------------------------------------------------------- /sbtest-app/src/test/java/com/poc/proxysql/contrioller/UserControllerTests.java: -------------------------------------------------------------------------------- 1 | package com.poc.proxysql.contrioller; 2 | 3 | 4 | import com.poc.proxysql.models.UserCreateRequest; 5 | import com.poc.proxysql.models.UserDTO; 6 | import com.poc.proxysql.models.UserUpdateRequest; 7 | import lombok.extern.slf4j.Slf4j; 8 | import org.junit.jupiter.api.Assertions; 9 | import org.junit.jupiter.api.Disabled; 10 | import org.junit.jupiter.api.Test; 11 | import org.springframework.beans.factory.annotation.Autowired; 12 | import org.springframework.boot.test.context.SpringBootTest; 13 | import org.springframework.boot.test.web.client.TestRestTemplate; 14 | import org.springframework.boot.web.server.LocalServerPort; 15 | import org.springframework.core.ParameterizedTypeReference; 16 | import org.springframework.core.env.Environment; 17 | import org.springframework.http.HttpEntity; 18 | import org.springframework.http.HttpMethod; 19 | import org.springframework.http.HttpStatus; 20 | import org.springframework.http.ResponseEntity; 21 | 22 | import java.util.List; 23 | 24 | @Slf4j 25 | @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) 26 | public class UserControllerTests { 27 | 28 | @Autowired 29 | private TestRestTemplate restTemplate; 30 | 31 | @Autowired 32 | private Environment env; 33 | 34 | @LocalServerPort 35 | private int port; 36 | 37 | 38 | private String getBaseUrl() { 39 | return "http://localhost:"+port; 40 | } 41 | 42 | 43 | @Test 44 | @Disabled 45 | public void testFetchAllUsers() throws InterruptedException { 46 | int limit = env.getProperty("application-test.testFetchAllUsers.repeat", Integer.TYPE, 1); 47 | long interval = env.getProperty("application-test.testFetchAllUsers.delay", Long.TYPE, 2000L); 48 | 49 | ParameterizedTypeReference> typeReference = new ParameterizedTypeReference<>() {}; 50 | 51 | for(int i = 0; i < limit; i++) { 52 | log.info("[{}] -> {}", i+1, getBaseUrl()+"/users"); 53 | ResponseEntity> response = restTemplate.exchange(getBaseUrl()+"/users", HttpMethod.POST, null, typeReference); 54 | Assertions.assertEquals(response.getStatusCode(), HttpStatus.OK); 55 | Thread.sleep(interval); 56 | } 57 | } 58 | 59 | 60 | @Test 61 | @Disabled 62 | public void createNewUserTest() { 63 | ParameterizedTypeReference typeReference = new ParameterizedTypeReference<>() {}; 64 | UserCreateRequest request = new UserCreateRequest("Jhon Doe"); 65 | ResponseEntity response = restTemplate 66 | .exchange( 67 | getBaseUrl()+"/users", 68 | HttpMethod.POST, 69 | new HttpEntity<>(request), 70 | typeReference 71 | ); 72 | Assertions.assertEquals(response.getStatusCode(), HttpStatus.OK); 73 | } 74 | 75 | @Test 76 | @Disabled 77 | public void updateUserTest() { 78 | ParameterizedTypeReference typeReference = new ParameterizedTypeReference<>() {}; 79 | UserUpdateRequest request = new UserUpdateRequest(1, "Dipanjal"); 80 | ResponseEntity response = restTemplate 81 | .exchange( 82 | getBaseUrl()+"/users", 83 | HttpMethod.PUT, 84 | new HttpEntity<>(request), 85 | typeReference 86 | ); 87 | Assertions.assertEquals(response.getStatusCode(), HttpStatus.OK); 88 | } 89 | } 90 | -------------------------------------------------------------------------------- /slave/init.sql: -------------------------------------------------------------------------------- 1 | /* proxysql user */ 2 | CREATE USER IF NOT EXISTS 'monitor'@'%' IDENTIFIED BY 'monitor'; 3 | 4 | /* mysql exporter user */ 5 | CREATE USER IF NOT EXISTS 'exporter'@'%' IDENTIFIED BY 'password' WITH MAX_USER_CONNECTIONS 3; 6 | GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'%'; 7 | 8 | FLUSH PRIVILEGES; 9 | 10 | /* start replication */ 11 | CHANGE MASTER TO MASTER_HOST='mysql-master',MASTER_USER='slave_user',MASTER_PASSWORD='password',MASTER_AUTO_POSITION=1; 12 | START SLAVE; 13 | -------------------------------------------------------------------------------- /slave/my-slave1.cnf: -------------------------------------------------------------------------------- 1 | [mysqld] 2 | pid-file = /var/run/mysqld/mysqld.pid 3 | socket = /var/run/mysqld/mysqld.sock 4 | datadir = /var/lib/mysql 5 | secure-file-priv= NULL 6 | symbolic-links=0 7 | default_authentication_plugin=mysql_native_password 8 | lower_case_table_names = 1 9 | 10 | bind-address = 0.0.0.0 11 | server-id = 2 12 | relay-log = /var/run/mysqld/mysql-relay-bin.log 13 | log_bin = /var/run/mysqld/mysql-bin.log 14 | binlog_do_db = sbtest 15 | 16 | read_only = on 17 | 18 | gtid_mode = on 19 | enforce_gtid_consistency = on 20 | log_slave_updates = on 21 | 22 | 23 | general_log=1 24 | general_log_file=/var/log/mysql/general.log 25 | 26 | # Slow query settings: 27 | slow_query_log=1 28 | slow_query_log_file=/var/log/mysql/slow.log 29 | long_query_time=0.5 30 | -------------------------------------------------------------------------------- /slave/my-slave2.cnf: -------------------------------------------------------------------------------- 1 | [mysqld] 2 | pid-file = /var/run/mysqld/mysqld.pid 3 | socket = /var/run/mysqld/mysqld.sock 4 | datadir = /var/lib/mysql 5 | secure-file-priv= NULL 6 | symbolic-links=0 7 | default_authentication_plugin=mysql_native_password 8 | lower_case_table_names = 1 9 | 10 | bind-address = 0.0.0.0 11 | server-id = 3 12 | relay-log = /var/run/mysqld/mysql-relay-bin.log 13 | log_bin = /var/run/mysqld/mysql-bin.log 14 | binlog_do_db = sbtest 15 | 16 | read_only = on 17 | 18 | gtid_mode = on 19 | enforce_gtid_consistency = on 20 | log_slave_updates = on 21 | 22 | general_log=1 23 | general_log_file=/var/log/mysql/general.log 24 | 25 | # Slow query settings: 26 | slow_query_log=1 27 | slow_query_log_file=/var/log/mysql/slow.log 28 | long_query_time=0.5 29 | -------------------------------------------------------------------------------- /user_api_postman_collection.json: -------------------------------------------------------------------------------- 1 | { 2 | "info": { 3 | "_postman_id": "4e99db56-9fa9-4330-96c5-7be2196d0b32", 4 | "name": "User API Collection", 5 | "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json" 6 | }, 7 | "item": [ 8 | { 9 | "name": "READ ALL", 10 | "request": { 11 | "method": "GET", 12 | "header": [], 13 | "url": null 14 | }, 15 | "response": [] 16 | }, 17 | { 18 | "name": "READ by ID", 19 | "request": { 20 | "method": "GET", 21 | "header": [], 22 | "url": { 23 | "raw": "http://localhost:8080/users/1", 24 | "protocol": "http", 25 | "host": [ 26 | "localhost" 27 | ], 28 | "port": "8080", 29 | "path": [ 30 | "users", 31 | "1" 32 | ] 33 | } 34 | }, 35 | "response": [] 36 | }, 37 | { 38 | "name": "CREATE NEW", 39 | "request": { 40 | "method": "POST", 41 | "header": [], 42 | "body": { 43 | "mode": "raw", 44 | "raw": "{\r\n \"name\": \"Jhon Doe\"\r\n}", 45 | "options": { 46 | "raw": { 47 | "language": "json" 48 | } 49 | } 50 | }, 51 | "url": { 52 | "raw": "http://localhost:8080/users", 53 | "protocol": "http", 54 | "host": [ 55 | "localhost" 56 | ], 57 | "port": "8080", 58 | "path": [ 59 | "users" 60 | ] 61 | } 62 | }, 63 | "response": [] 64 | }, 65 | { 66 | "name": "UPDATE", 67 | "request": { 68 | "method": "PUT", 69 | "header": [], 70 | "body": { 71 | "mode": "raw", 72 | "raw": "{\r\n \"id\": 1,\r\n \"name\": \"Dipanjal\"\r\n}", 73 | "options": { 74 | "raw": { 75 | "language": "json" 76 | } 77 | } 78 | }, 79 | "url": { 80 | "raw": "http://localhost:8080/users", 81 | "protocol": "http", 82 | "host": [ 83 | "localhost" 84 | ], 85 | "port": "8080", 86 | "path": [ 87 | "users" 88 | ] 89 | } 90 | }, 91 | "response": [] 92 | }, 93 | { 94 | "name": "DELETE", 95 | "request": { 96 | "method": "DELETE", 97 | "header": [], 98 | "url": { 99 | "raw": "http://localhost:8080/users/1", 100 | "protocol": "http", 101 | "host": [ 102 | "localhost" 103 | ], 104 | "port": "8080", 105 | "path": [ 106 | "users", 107 | "1" 108 | ] 109 | } 110 | }, 111 | "response": [] 112 | } 113 | ] 114 | } --------------------------------------------------------------------------------