├── .gitignore ├── PGHAHA.png ├── README.md ├── config.py ├── config.py.example ├── configs └── .gitignore ├── create_haproxy_check.py ├── haproxy-example.cfg ├── screenshots └── hastats.png └── template ├── redirect.template ├── standby-multi.template ├── standby-partial.template └── standby.template /.gitignore: -------------------------------------------------------------------------------- 1 | config.pyc 2 | __pycache__ 3 | -------------------------------------------------------------------------------- /PGHAHA.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gplv2/haproxy-postgresql/ee3bd25893f37b166c9b0db645de7b146f0f2db1/PGHAHA.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # haproxy-postgresql 2 | 3 | Use this to determine who's the master server in a postgresql cluster setup. this sends a `select pg_is_in_recovery()` command to the psql servers to see if it is in standby more or not. haproxy will only mark masters as up. The standby node is in backup mode, so haproxy will not try to write to it unless the master node is down AND the standby node is promoted to a master 4 | 5 | You need to setup a trust connection between haproxy and postgresql. In this example there are 2 pgbouncers in between haproxy and postgresql. This means you need to install the postgresql nodes at a different port (here 6432). Trust connection suggestions will be shown when you run the script. 6 | 7 | calculates byte length so the tcp package is well constructed, when changing length of username, you need to adjust. 8 | 9 | This is tested in conjunction with repmgrd, pgbouncer, keepalived/haproxy architected setup, but without a witness server. 10 | 11 | The latest version will block all connections when it detects more or less than a single master server. You do not want to write your data in just any server. In the case you have 2 disconnected masters servers (meaning 2 servers that aren't standby/slaves) they both will pass this check. Hence a built-in ACL will prevent writing to any servers, which as a DBA is what you want. It's better to block this than to accept , also this way you don't show bias towards a node when using the backup directive for example for your second node. It's not a good idea when a former master comes back after a failure, it will be the primary candidate to write to, but in such cases, your standby server should already be promoted and the old primary should not be written to anymore without resyncing it with the freshly promoted standby. 12 | 13 | ## Test this with vagrant 14 | 15 | use this Vagrant setup to see it in action : https://github.com/gplv2/vagrant-postgres 16 | 17 | ### Screenshot 18 | ![alt text][haproxy1] 19 | 20 | 21 | ## Prepare 22 | 23 | - setup a cluster first (atleast the master) 24 | - use repmgr to create a managed cluster 25 | - use pgbouncer in front of the DB in production setups, point haproxy to the bouncer but check directly on the pg servers 26 | 27 | ## generate a config 28 | 29 | ``` 30 | HA_MASTER_NAME = "node1" 31 | HA_MASTER_DSN = "192.168.1.144:5432" 32 | HA_STANDBY_NAME = "node2" 33 | HA_STANDBY_DSN = "192.168.1.145:5432" 34 | HA_VIP_IP = "192.168.1.141" 35 | HA_CHECK_USER = "pgcheck" 36 | HA_CHECK_PORT = "6432" 37 | HA_LISTEN_PORT = "5432" 38 | HA_STATS_USER = "hapsql" 39 | HA_STATS_PASSWORD = "snowball1" 40 | ``` 41 | 42 | - edit config.py, set vars 43 | 44 | - run it : 45 | ./create_haproxy_check.py standby mystandby 46 | 47 | - alternatively, you can also test the redirect config : 48 | ./create_haproxy_check.py redirect myredirect 49 | 50 | There are 2 templates now, standby will make haproxy mark slaves as bad candidates. the redirect template will allow you to filter out a rogue client to redirect it towards the correct master server while letting legitimate connections (monitoring, admin etc) pass based on ip address ACL's 51 | 52 | ## the results 53 | 54 | Creating haproxy project mytest 55 | Creating configs/mytest/haproxy-mytest.cnf 56 | 57 | ## check the pg_hba suggestions and implement these for passwordless checks 58 | 59 | ### pg_hba user check additions (for balancer access to db) 60 | 61 | Add the following lines to pg_hba.conf: 62 | # special loadbalancer account in trust 63 | host template1 pgcheck 192.168.1.141/32 trust 64 | host template1 pgcheck 192.168.1.144/32 trust 65 | host template1 pgcheck 192.168.1.145/32 trust 66 | 67 | 68 | ### pg_hba repmgr additions 69 | 70 | Add the following lines to pg_hba.conf: 71 | # repmgr account 72 | local replication repmgr trust 73 | host replication repmgr 127.0.0.1/32 trust 74 | host replication repmgr 192.168.1.144/32 trust 75 | host replication repmgr 192.168.1.145/32 trust 76 | local repmgr repmgr trust 77 | host repmgr repmgr 127.0.0.1/32 trust 78 | host repmgr repmgr 192.168.1.144/32 trust 79 | host repmgr repmgr 192.168.1.145/32 trust 80 | 81 | 82 | ## suggestions 83 | - pull requests/ comments welcome here on github 84 | 85 | ## 86 | 87 | [haproxy1]: https://github.com/gplv2/haproxy-postgresql/raw/master/screenshots/hastats.png "Stats example of normal DB situation" 88 | 89 | -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- 1 | HA_MASTER_NAME = "node1" 2 | HA_MASTER_DSN = "192.168.88.11:7432" 3 | #HA_STANDBY_NAME = "node2" 4 | #HA_STANDBY_DSN = "192.168.88.12:7432" 5 | HA_STANDBY_NAME = "node2;node3;node4" 6 | HA_STANDBY_DSN = "192.168.88.12:7432;192.168.88.13:7432;192.168.88.14:7432" 7 | HA_CHECK_USER = "pgc" 8 | HA_CHECK_PORT = "6432" 9 | HA_LISTEN_PORT = "5432" 10 | HA_STATS_USER = "pgadmin" 11 | HA_STATS_PASSWORD = "pgsecret" 12 | HA_VIP_IP = "192.168.88.5\/24" 13 | #HA_VIP_IP = "192.168.88.5/24" 14 | -------------------------------------------------------------------------------- /config.py.example: -------------------------------------------------------------------------------- 1 | HA_MASTER_NAME = "node1" 2 | HA_MASTER_DSN = "192.168.88.11:7432" 3 | #HA_STANDBY_NAME = "node2" 4 | #HA_STANDBY_DSN = "192.168.88.12:7432" 5 | HA_STANDBY_NAME = "node2;node3;node4" 6 | HA_STANDBY_DSN = "192.168.88.12:7432;192.168.88.13:7432;192.168.88.14:7432" 7 | HA_CHECK_USER = "pgc" 8 | HA_CHECK_PORT = "6432" 9 | HA_LISTEN_PORT = "5432" 10 | HA_STATS_USER = "pgadmin" 11 | HA_STATS_PASSWORD = "pgsecret" 12 | HA_VIP_IP = "192.168.88.5\/24" 13 | #HA_VIP_IP = "192.168.88.5/24" 14 | -------------------------------------------------------------------------------- /configs/.gitignore: -------------------------------------------------------------------------------- 1 | # Ignore everything in this directory 2 | * 3 | # Except this file 4 | !.gitignore 5 | -------------------------------------------------------------------------------- /create_haproxy_check.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python3 2 | import config 3 | import os, errno 4 | import sys 5 | from io import StringIO 6 | #from pprint import pprint 7 | 8 | BASEDIR = "configs" 9 | 10 | APPLICATION_PATH = "." 11 | 12 | def utf8len(s): 13 | return len(s.encode('utf-8')) 14 | 15 | def help_exit(exit_status): 16 | if exit_status != 0: 17 | print("Error: Wrong arguments in call", file=sys.stderr) 18 | help_msg = """Usage: 19 | 20 | %s 21 | 22 | Options: 23 | templatename name of a template in the template dir 24 | project project name 25 | """ % sys.argv[0] 26 | print(help_msg, file=sys.stderr) 27 | os.system('ls -altr template/*') 28 | sys.exit(exit_status) 29 | 30 | def replace(source_name, props, output_name): 31 | output = open(output_name, 'w') 32 | source = open(source_name, 'r') 33 | for line in source: 34 | newline = line 35 | for prop in props: 36 | newline = newline.replace(prop, props[prop]) 37 | output.write(newline) 38 | output.close() 39 | source.close() 40 | 41 | def new_haproxy_conf(props, slavelist): 42 | project = props["<%= @bn.project %>"] 43 | new_conf = "%s/%s/haproxy-%s.cnf" % (BASEDIR, project, project) 44 | print("Creating %s" % new_conf, file=sys.stderr) 45 | 46 | if(slavelist): 47 | print("Config.py has multiple slave definitions, using -multi template instead : template/%s-multi.template" % sys.argv[1], file=sys.stderr) 48 | if os.path.isfile("template/%s-multi.template" % sys.argv[1]): 49 | print("Looking for partial template for slave entries: template/%s-partial.template" % sys.argv[1], file=sys.stderr) 50 | if os.path.isfile("template/%s-partial.template" % sys.argv[1]): 51 | #load partial template in variable (expect this to be a one liner) 52 | with open("template/%s-partial.template" % sys.argv[1], 'r') as templatefile: 53 | slavetemplate = templatefile.read().rstrip() 54 | 55 | slaves = StringIO() 56 | #pprint(locals()) 57 | for name,dsn in slavelist.items(): 58 | format_dictionary = {'name': name, 'dsn': dsn, 'checkport': props["<%= @bn.checkport %>"] } 59 | print(" " + slavetemplate.format(**format_dictionary), file=slaves) 60 | 61 | props["<%= @SLAVELIST %>"] = slaves.getvalue() 62 | replace("template/%s-multi.template" % sys.argv[1], props, new_conf) 63 | else: 64 | print("Partial template does not exist : template/%s-partial.template" % sys.argv[1], file=sys.stderr) 65 | sys.exit(0) 66 | else: 67 | print("Template does not exist : template/%s-multi.template" % sys.argv[1], file=sys.stderr) 68 | sys.exit(0) 69 | else: 70 | if os.path.isfile("template/%s.template" % sys.argv[1]): 71 | replace("template/%s.template" % sys.argv[1], props, new_conf) 72 | else: 73 | print("Template does not exist : template/%s.template" % sys.argv[1], file=sys.stderr) 74 | sys.exit(0) 75 | 76 | def add_hba_checkuser(props,extras): 77 | print('') 78 | #print("Add the following lines to pg_hba.conf:", file=sys.stderr) 79 | print("### master/slaves are only a concept at start, any database node can later have a different role") 80 | print("### this is just to give some clear pointers to the ones using this") 81 | print("### special loadbalancer account in trust") 82 | print("") 83 | print("## vip / haproxy check user") 84 | print("host template1 %s %s/32 trust" % (props["<%= @bn.checkuser %>"], props["<%= @bn.vipip %>"])) 85 | print("# master") 86 | print("host template1 %s %s/32 trust" % (props["<%= @bn.checkuser %>"], props["<%= @bn.masterdsn %>"].split(':')[0])) 87 | print("# slaves") 88 | if(extras['checkuser'] == "#"): 89 | print("host template1 %s %s/32 trust" % (props["<%= @bn.checkuser %>"], props["<%= @bn.standbydsn %>"].split(':')[0])) 90 | 91 | if(extras): 92 | print(extras['checkuser']) 93 | 94 | def add_hba_repmgr(props,extras): 95 | #print("Add the following lines to pg_hba.conf:", file=sys.stderr) 96 | print("### replication user") 97 | print("local replication repmgr trust") 98 | print("host replication repmgr 127.0.0.1/32 trust") 99 | print("# master") 100 | print("host replication repmgr %s/32 trust" % props["<%= @bn.masterdsn %>"].split(':')[0]) 101 | print("# slaves") 102 | if(extras['repmgr'] == "#"): 103 | print("host replication repmgr %s/32 trust" % props["<%= @bn.standbydsn %>"].split(':')[0]) 104 | 105 | if(extras): 106 | print(extras['repl']) 107 | print('') 108 | 109 | print("### repmgr user") 110 | print("local repmgr repmgr trust") 111 | print("host repmgr repmgr 127.0.0.1/32 trust") 112 | print("# master") 113 | print("host repmgr repmgr %s/32 trust" % props["<%= @bn.masterdsn %>"].split(':')[0]) 114 | print("# slaves") 115 | if(extras['repl'] == "#"): 116 | print("host repmgr repmgr %s/32 trust" % props["<%= @bn.standbydsn %>"].split(':')[0]) 117 | 118 | if(extras): 119 | print(extras['repmgr']) 120 | print('') 121 | 122 | def main(): 123 | args = len(sys.argv) 124 | if args == 2: 125 | if sys.argv[1] == "help": 126 | help_exit(0) 127 | else: 128 | help_exit(1) 129 | if args != 3: 130 | help_exit(1) 131 | 132 | mastername = config.HA_MASTER_NAME 133 | masterdsn = config.HA_MASTER_DSN 134 | standbyname = config.HA_STANDBY_NAME 135 | standbydsn = config.HA_STANDBY_DSN 136 | checkport = config.HA_CHECK_PORT 137 | checkuser = config.HA_CHECK_USER 138 | listenport = config.HA_LISTEN_PORT 139 | statsuser = config.HA_STATS_USER 140 | statspassword = config.HA_STATS_PASSWORD 141 | vipip = config.HA_VIP_IP 142 | 143 | d = utf8len(checkuser) + 33 + 1; 144 | 145 | multiple_slaves = False 146 | extras = { "repl": "#", "repmgr": "#" , "checkuser": "#" } 147 | 148 | #print("D %s" % d) 149 | #print("H %s" % hex(d).split('x')[-1]) 150 | 151 | # clean up when receiving a cidr block from config (\ escaped or not) 152 | if vipip.find('/'): 153 | vipip = vipip.split('/')[0].strip('\\') 154 | 155 | # the props 156 | props = { 157 | "<%= @bn.template %>": sys.argv[1], 158 | "<%= @bn.project %>": sys.argv[2], 159 | "<%= @bn.mastername %>": mastername, 160 | "<%= @bn.standbyname %>": standbyname, 161 | "<%= @bn.masterdsn %>": masterdsn, 162 | "<%= @bn.masterip %>": masterdsn.split(':')[0], 163 | "<%= @bn.standbydsn %>": standbydsn, 164 | "<%= @bn.checkuserhex %>": checkuser.encode("utf-8").hex() + "00", 165 | "<%= @bn.checkport %>": checkport, 166 | "<%= @bn.stats_user %>": statsuser, 167 | "<%= @bn.stats_password %>": statspassword, 168 | "<%= @bn.checkuser %>": checkuser, 169 | "<%= @bn.listenport %>": listenport, 170 | "<%= @bn.checkuserlen %>": str(utf8len(checkuser)+1), 171 | "<%= @bn.totalsize %>": str(d), 172 | "<%= @bn.vipip %>": vipip, 173 | "<%= @bn.totalbytes %>": str(hex(d).split('x')[-1]), 174 | "<%= @bn.path %>": APPLICATION_PATH 175 | } 176 | 177 | if standbyname.find(";")!=-1: 178 | print("Found multiple slave names.", file=sys.stderr) 179 | names = standbyname.split(";") 180 | multiple_slaves = True 181 | for name in names: 182 | print(name, file=sys.stderr) 183 | else: 184 | print("No multiple standy servers found.", file=sys.stderr) 185 | 186 | if standbydsn.find(";")!=-1: 187 | print("Found multiple slave dsn.", file=sys.stderr) 188 | dsns = standbydsn.split(";") 189 | multiple_slaves = True 190 | for ip in dsns: 191 | print(ip, file=sys.stderr) 192 | else: 193 | print("No multiple standy ips found.", file=sys.stderr) 194 | 195 | slavelist = False 196 | if multiple_slaves == True: 197 | if len(dsns) != len(names): 198 | print("dsn and names do not have the same number of entries", file=sys.stderr) 199 | sys.exit(0) 200 | slavelist = dict(zip(names,dsns)) 201 | hba_repl = StringIO() 202 | hba_repmgr = StringIO() 203 | hba_extra_checkuser = StringIO() 204 | 205 | for dsn in dsns: 206 | print("host replication repmgr %s/32 trust" % dsn.split(':')[0], file=hba_repl) 207 | print("host repmgr repmgr %s/32 trust" % dsn.split(':')[0], file=hba_repmgr) 208 | print("host template1 %s %s/32 trust" % (props["<%= @bn.checkuser %>"], dsn.split(':')[0]), file=hba_extra_checkuser) 209 | #hba_repl.seek(0) 210 | #hba_repmgr.seek(0) 211 | #hba_extra_checkuser.seek(0) 212 | extras['repl'] = hba_repl.getvalue() 213 | extras['repmgr'] = hba_repmgr.getvalue() 214 | extras['checkuser'] = hba_extra_checkuser.getvalue() 215 | 216 | project = props["<%= @bn.project %>"] 217 | directory = ('%s/%s' % (BASEDIR, project)) 218 | 219 | if not os.path.isfile("template/%s.template" % sys.argv[1]): 220 | print("Template does not exist : %s" % sys.argv[1], file=sys.stderr) 221 | sys.exit(0) 222 | 223 | try: 224 | print("Creating haproxy project %s" % (project), file=sys.stderr) 225 | os.makedirs(directory) 226 | except OSError as e: 227 | if e.errno != errno.EEXIST: 228 | raise 229 | 230 | new_haproxy_conf(props, slavelist) 231 | add_hba_checkuser(props, extras) 232 | add_hba_repmgr(props, extras) 233 | 234 | print("Done!", file=sys.stderr) 235 | 236 | if __name__ == '__main__': 237 | main() 238 | -------------------------------------------------------------------------------- /haproxy-example.cfg: -------------------------------------------------------------------------------- 1 | # haproxy postgresql master check 2 | # 3 | # haproxy listen on: 5432 4 | 5 | # Setup architecture : haproxy (2) <-> pgbouncer(2) <-> postgresql (1 + 1) 6 | 7 | # DB, remote instance #1 listen: 6432 (master node) ( 8 | # DB, remote instance #2 listen: 6432 (replica node) 9 | 10 | # passwordless auth for check user pgc 11 | # The check is performed directly, without consulting the pgbouncer, hence checking 12 | # happens on port 6432 , bouncers listen on 5432 13 | 14 | # external failover, promoting replica to master in case of failure: using repmgr 15 | # template1 database is accessible by user pgc 16 | # 17 | # haproxy will pass connection to postgresql master node: 18 | # $ psql -h -p 5432 -U pgc template1 19 | # 20 | 21 | #--------------------------------------------------------------------- 22 | # Global settings 23 | #--------------------------------------------------------------------- 24 | global 25 | log 127.0.0.1 local2 26 | 27 | chroot /var/lib/haproxy 28 | pidfile /var/run/haproxy.pid 29 | maxconn 4000 30 | user haproxy 31 | group haproxy 32 | daemon 33 | spread-checks 5 34 | 35 | # turn on stats unix socket 36 | stats socket /var/lib/haproxy/stats 37 | 38 | #--------------------------------------------------------------------- 39 | # common defaults that all the 'listen' and 'backend' sections will 40 | # use if not designated in their block 41 | #--------------------------------------------------------------------- 42 | defaults 43 | mode tcp 44 | log global 45 | option dontlognull 46 | option redispatch 47 | retries 3 48 | timeout queue 1m 49 | timeout connect 1s 50 | timeout client 3600s 51 | timeout server 3600s 52 | timeout check 2s 53 | maxconn 500 54 | #--------------------------------------------------------------------- 55 | # statistics 56 | #--------------------------------------------------------------------- 57 | # Host HA-Proxy's web stats on Port 8182. 58 | 59 | listen HAProxy-Statistics *:8182 60 | mode http 61 | option httplog 62 | stats enable 63 | stats uri /haproxy?stats 64 | stats refresh 20s 65 | stats realm PSQL Haproxy\ Statistics # Title text for popup window 66 | stats show-node 67 | stats show-legends 68 | stats show-desc PSQL load balancer stats (master) 69 | stats auth pgadmin:pgsecret 70 | 71 | 72 | #--------------------------------------------------------------------- 73 | # main frontend which proxys to the backends 74 | #--------------------------------------------------------------------- 75 | 76 | frontend front_pg 77 | mode tcp 78 | bind *:5432 79 | 80 | acl pg_single_master nbsrv(backend_pg) eq 1 81 | tcp-request connection reject if !pg_single_master 82 | 83 | default_backend backend_pg 84 | 85 | #--------------------------------------------------------------------- 86 | # the postgresql cluster backend (master + standby) 87 | #--------------------------------------------------------------------- 88 | 89 | backend backend_pg 90 | option tcp-check 91 | tcp-check connect 92 | 93 | # user: pgc 94 | # database: template1 95 | # 96 | tcp-check send-binary 00000025 # packet length 97 | tcp-check send-binary 00030000 # protocol version 98 | tcp-check send-binary 7573657200 # "user" ( 5 bytes ) 99 | tcp-check send-binary 70676300 # "pgc" ( 4 bytes ) 100 | tcp-check send-binary 646174616261736500 # "database" ( 9 bytes ) 101 | tcp-check send-binary 74656d706c6174653100 # "template1" ( 10 bytes ) 102 | tcp-check send-binary 00 # terminator 103 | 104 | # expect: Auth 105 | # 106 | tcp-check expect binary 52 # Auth request 107 | tcp-check expect binary 00000008 # packet length ( 8 bytes ) 108 | tcp-check expect binary 00000000 # auth response ok 109 | 110 | # write: run simple query 111 | # "select pg_is_in_recovery();" 112 | # 113 | tcp-check send-binary 51 # simple query 114 | tcp-check send-binary 00000020 # packet length ( 4 bytes) 115 | tcp-check send-binary 73656c65637420 # "select " ( 7 bytes ) 116 | # "pg_is_in_recovery();" 117 | tcp-check send-binary 70675f69735f696e5f7265636f7665727928293b # ( 20 bytes ) 118 | tcp-check send-binary 00 # terminator ( 1 byte ) 119 | 120 | 121 | # write: terminate session 122 | tcp-check send-binary 58 # Termination packet 123 | tcp-check send-binary 00000004 # packet length: 4 (no body) 124 | # avoids : LOG: could not receive data from client: Connection reset by peer 125 | 126 | # expect: Row description packet 127 | # 128 | tcp-check expect binary 54 # row description packet (1 byte) 129 | tcp-check expect binary 0000002a # packet length: 42 (0x2a) 130 | tcp-check expect binary 0001 # field count: 1 131 | tcp-check expect binary 70675f69735f696e5f7265636f7665727900 # field name: pg_is_in_recovery 132 | tcp-check expect binary 00000000 # table oid: 0 133 | tcp-check expect binary 0000 # column index: 0 134 | tcp-check expect binary 00000010 # type oid: 16 135 | tcp-check expect binary 0001 # column length: 1 136 | tcp-check expect binary ffffffff # type modifier: -1 137 | tcp-check expect binary 0000 # format: text 138 | 139 | # expect: query result data 140 | # 141 | # "f" means node in master mode 142 | # "t" means node in standby mode (read-only) 143 | # 144 | tcp-check expect binary 44 # data row packet 145 | tcp-check expect binary 0000000b # packet lenght: 11 (0x0b) 146 | tcp-check expect binary 0001 # field count: 1 147 | tcp-check expect binary 00000001 # column length in bytes: 1 148 | tcp-check expect binary 66 # column data, "f" 149 | 150 | # write: terminate session 151 | tcp-check send-binary 58 # Termination packet 152 | tcp-check send-binary 00000004 # packet length: 4 (no body) 153 | 154 | # close open sessions in case the downed server is still running but is out of sync with the master 155 | default-server on-marked-down shutdown-sessions 156 | 157 | # server list to check 158 | server pgnode1 192.168.1.100:5432 check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port 6432 # this defaults to the initial master 159 | server pgnode2 192.168.1.101:5432 check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port 6432 # this defaults to the initial standby 160 | 161 | # These 2 above can change role depending on their role 162 | 163 | -------------------------------------------------------------------------------- /screenshots/hastats.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gplv2/haproxy-postgresql/ee3bd25893f37b166c9b0db645de7b146f0f2db1/screenshots/hastats.png -------------------------------------------------------------------------------- /template/redirect.template: -------------------------------------------------------------------------------- 1 | # haproxy postgresql master check 2 | # 3 | # haproxy listen on: <%= @bn.listenport %> # 5432 4 | 5 | # Setup architecture : haproxy (2) <-> pgbouncer(2) <-> postgresql (1 + 1) 6 | 7 | # DB, ex remote instance #1 listen: 5432 ( master node ) 8 | # DB, ex remote instance #2 listen: 5432 ( standby node ) 9 | 10 | # passwordless auth for check user <%= @bn.checkuser %> 11 | # The check is performed directly, without consulting the pgbouncer, hence checking 12 | # happens on port 6432 , bouncers listen on 5432 13 | 14 | # external failover, promoting replica to master in case of failure: using repmgr 15 | # template1 database is accessible by user <%= @bn.checkuser %> 16 | # 17 | # haproxy will pass connection to postgresql master node: 18 | # $ psql -h -p 5432 -U <%= @bn.checkuser %> template1 19 | 20 | #--------------------------------------------------------------------- 21 | # Global settings 22 | #--------------------------------------------------------------------- 23 | global 24 | log 127.0.0.1 local2 25 | 26 | chroot /var/lib/haproxy 27 | pidfile /var/run/haproxy.pid 28 | maxconn 4000 29 | user haproxy 30 | group haproxy 31 | daemon 32 | spread-checks 5 33 | 34 | # turn on stats unix socket 35 | stats socket /var/lib/haproxy/stats 36 | 37 | #--------------------------------------------------------------------- 38 | # common defaults that all the 'listen' and 'backend' sections will 39 | # use if not designated in their block 40 | #--------------------------------------------------------------------- 41 | defaults 42 | mode tcp 43 | log global 44 | option dontlognull 45 | option redispatch 46 | retries 3 47 | timeout queue 1m 48 | timeout connect 1s 49 | timeout client 3600s 50 | timeout server 3600s 51 | timeout check 2s 52 | maxconn 500 53 | #--------------------------------------------------------------------- 54 | # statistics 55 | #--------------------------------------------------------------------- 56 | # Host HA-Proxy's web stats on Port 8182. 57 | 58 | listen HAProxy-Statistics 59 | bind *:8182 60 | mode http 61 | option httplog 62 | stats enable 63 | stats uri /haproxy?stats 64 | stats refresh 20s 65 | stats realm PSQL Haproxy\ Statistics # Title text for popup window 66 | stats show-node 67 | stats show-legends 68 | stats show-desc PSQL load balancer stats 69 | stats auth pgadmin:pgsecret 70 | #stats auth <%= @bn.stats_user %>:<%= @bn.stats_password %> 71 | 72 | #--------------------------------------------------------------------- 73 | # main frontend which proxys to the backends 74 | #--------------------------------------------------------------------- 75 | 76 | frontend front_pg 77 | mode tcp 78 | bind *:<%= @bn.listenport %> # 5432 79 | acl client_redirect src <%= @bn.masterip %> 80 | 81 | if client_redirect use good_backend_pg 82 | 83 | # server list to check 84 | default_backend backend_pg 85 | 86 | #--------------------------------------------------------------------- 87 | # the postgresql cluster backend (master + standby) 88 | #--------------------------------------------------------------------- 89 | 90 | backend backend_pg 91 | option pgsql-check user <%= @bn.checkuser %> 92 | 93 | server <%= @bn.standbyname %> <%= @bn.standbydsn %> check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port <%= @bn.checkport %> backup # this defaults to the initial standby 94 | 95 | backend good_backend_pg 96 | option tcp-check 97 | tcp-check connect 98 | 99 | # user: <%= @bn.checkuser %> 100 | # database: template1 101 | # 102 | tcp-check send-binary 000000<%= @bn.totalbytes %> # packet length: <%= @bn.totalsize %> bytes in ( 4 bytes ) 103 | tcp-check send-binary 00030000 # protocol version ( 4 bytes ) 104 | tcp-check send-binary 7573657200 # "user" ( 5 bytes ) 105 | tcp-check send-binary 70676300 # "<%= @bn.checkuser %>" ( <%= @bn.checkuserlen %> bytes ) 106 | tcp-check send-binary 646174616261736500 # "database" ( 9 bytes ) 107 | tcp-check send-binary 74656d706c6174653100 # "template1" ( 10 bytes ) 108 | tcp-check send-binary 00 # terminator ( 1 byte ) 109 | ## TOTAL ( <%= @bn.totalsize %> bytes ) 110 | 111 | # expect: Auth 112 | # 113 | tcp-check expect binary 52 # Auth request 114 | tcp-check expect binary 00000008 # packet length : 8 bytes in ( 4 bytes ) 115 | tcp-check expect binary 00000000 # auth response ok ( 4 bytes ) 116 | ## TOTAL ( 8 bytes ) 117 | 118 | # write: run simple query 119 | # "select pg_is_in_recovery();" 120 | # 121 | tcp-check send-binary 51 # simple query 122 | tcp-check send-binary 00000020 # packet length: 32 bytes in ( 4 bytes ) 123 | tcp-check send-binary 73656c65637420 # "select " ( 7 bytes ) 124 | tcp-check send-binary 70675f69735f696e5f7265636f7665727928293b 125 | # "pg_is_in_recovery(); ( 20 bytes ) 126 | tcp-check send-binary 00 # terminator ( 1 byte ) 127 | ## TOTAL ( 32 bytes ) 128 | 129 | # expect: Row description packet 130 | # 131 | tcp-check expect binary 54 # row description packet 132 | tcp-check expect binary 0000002a # packet length: 42 bytes (0x2a) ( 4 bytes ) 133 | tcp-check expect binary 0001 # field count: 1 ( 1 byte ) 134 | tcp-check expect binary 70675f69735f696e5f7265636f7665727900 135 | # field name: pg_is_in_recovery ( 19 bytes ) 136 | tcp-check expect binary 00000000 # table oid: 0 ( 4 bytes ) 137 | tcp-check expect binary 0000 # column index: 0 ( 2 bytes ) 138 | tcp-check expect binary 00000010 # type oid: 16 ( 4 bytes ) 139 | tcp-check expect binary 0001 # column length: 1 ( 2 bytes ) 140 | tcp-check expect binary ffffffff # type modifier: -1 ( 4 bytes ) 141 | tcp-check expect binary 0000 # format: text ( 2 bytes ) 142 | ## TOTAL ( 42 bytes ) 143 | 144 | # expect: query result data 145 | # 146 | # "f" means node in master mode 147 | # "t" means node in standby mode (read-only) 148 | # 149 | tcp-check expect binary 44 # data row packet 150 | tcp-check expect binary 0000000b # packet length: 11 (0x0b) ( 4 bytes ) 151 | tcp-check expect binary 0001 # field count: 1 ( 2 bytes ) 152 | tcp-check expect binary 00000001 # column length in bytes: 1 ( 4 bytes ) 153 | tcp-check expect binary 66 # column data, "f" ( 1 byte ) 154 | ## TOTAL ( 11 bytes ) 155 | # write: terminate session 156 | tcp-check send-binary 58 # Termination packet 157 | tcp-check send-binary 00000004 # packet length: 4 (no body) 158 | 159 | acl client_redirected src <%= @bn.masterip %> 160 | if client_redirected use new_backend 161 | 162 | # server list to check 163 | server <%= @bn.mastername %> <%= @bn.masterdsn %> check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port <%= @bn.checkport %> # this defaults to the initial master 164 | 165 | # These 2 above can change role depending on their role 166 | 167 | -------------------------------------------------------------------------------- /template/standby-multi.template: -------------------------------------------------------------------------------- 1 | # haproxy postgresql master check 2 | # 3 | # haproxy listen on: <%= @bn.listenport %> # 5432 4 | 5 | # Setup architecture : haproxy (2) <-> pgbouncer(2) <-> postgresql (1 + 1) 6 | 7 | # DB, ex remote instance #1 listen: 5432 ( master node ) 8 | # DB, ex remote instance #2 listen: 5432 ( standby node ) 9 | 10 | # passwordless auth for check user <%= @bn.checkuser %> 11 | # The check is performed directly, without consulting the pgbouncer, hence checking 12 | # happens on port 6432 , bouncers listen on 5432 13 | 14 | # external failover, promoting replica to master in case of failure: using repmgr 15 | # template1 database is accessible by user <%= @bn.checkuser %> 16 | # 17 | # haproxy will pass connection to postgresql master node: 18 | # $ psql -h -p 5432 -U <%= @bn.checkuser %> template1 19 | 20 | #--------------------------------------------------------------------- 21 | # Global settings 22 | #--------------------------------------------------------------------- 23 | global 24 | log 127.0.0.1 local2 25 | 26 | chroot /var/lib/haproxy 27 | pidfile /var/run/haproxy.pid 28 | maxconn 4000 29 | user haproxy 30 | group haproxy 31 | daemon 32 | spread-checks 5 33 | 34 | # turn on stats unix socket 35 | stats socket /var/lib/haproxy/stats 36 | 37 | #--------------------------------------------------------------------- 38 | # common defaults that all the 'listen' and 'backend' sections will 39 | # use if not designated in their block 40 | #--------------------------------------------------------------------- 41 | defaults 42 | mode tcp 43 | log global 44 | option dontlognull 45 | option redispatch 46 | retries 3 47 | timeout queue 1m 48 | timeout connect 1s 49 | timeout client 3600s 50 | timeout server 3600s 51 | timeout check 2s 52 | maxconn 500 53 | #--------------------------------------------------------------------- 54 | # statistics 55 | #--------------------------------------------------------------------- 56 | # Host HA-Proxy's web stats on Port 8182. 57 | 58 | listen HAProxy-Statistics 59 | bind *:8182 60 | mode http 61 | option httplog 62 | stats enable 63 | stats uri /haproxy?stats 64 | stats refresh 20s 65 | stats realm PSQL Haproxy\ Statistics # Title text for popup window 66 | stats show-node 67 | stats show-legends 68 | stats show-desc PSQL load balancer stats 69 | stats auth pgadmin:pgsecret 70 | #stats auth <%= @bn.stats_user %>:<%= @bn.stats_password %> 71 | 72 | #--------------------------------------------------------------------- 73 | # main frontend which proxys to the backends 74 | #--------------------------------------------------------------------- 75 | 76 | frontend front_pg 77 | mode tcp 78 | bind <%= @bn.vipip %>:<%= @bn.listenport %> # 5432 79 | 80 | acl pg_single_master nbsrv(backend_pg) eq 1 81 | tcp-request connection reject if !pg_single_master 82 | 83 | default_backend backend_pg 84 | 85 | #--------------------------------------------------------------------- 86 | # the postgresql cluster backend (master + standby) 87 | #--------------------------------------------------------------------- 88 | 89 | backend backend_pg 90 | option tcp-check 91 | tcp-check connect 92 | 93 | # user: <%= @bn.checkuser %> 94 | # database: template1 95 | # 96 | tcp-check send-binary 000000<%= @bn.totalbytes %> # packet length: <%= @bn.totalsize %> bytes in ( 4 bytes ) 97 | tcp-check send-binary 00030000 # protocol version ( 4 bytes ) 98 | tcp-check send-binary 7573657200 # "user" ( 5 bytes ) 99 | tcp-check send-binary <%= @bn.checkuserhex %> # "<%= @bn.checkuser %>" ( <%= @bn.checkuserlen %> bytes ) 100 | tcp-check send-binary 646174616261736500 # "database" ( 9 bytes ) 101 | tcp-check send-binary 74656d706c6174653100 # "template1" ( 10 bytes ) 102 | tcp-check send-binary 00 # terminator ( 1 byte ) 103 | ## TOTAL ( <%= @bn.totalsize %> bytes ) 104 | 105 | # expect: Auth 106 | # 107 | tcp-check expect binary 52 # Auth request 108 | tcp-check expect binary 00000008 # packet length : 8 bytes in ( 4 bytes ) 109 | tcp-check expect binary 00000000 # auth response ok ( 4 bytes ) 110 | ## TOTAL ( 8 bytes ) 111 | 112 | # write: run simple query 113 | # "select pg_is_in_recovery();" 114 | # 115 | tcp-check send-binary 51 # simple query 116 | tcp-check send-binary 00000020 # packet length: 32 bytes in ( 4 bytes ) 117 | tcp-check send-binary 73656c65637420 # "select " ( 7 bytes ) 118 | tcp-check send-binary 70675f69735f696e5f7265636f7665727928293b 119 | # "pg_is_in_recovery(); ( 20 bytes ) 120 | tcp-check send-binary 00 # terminator ( 1 byte ) 121 | ## TOTAL ( 32 bytes ) 122 | 123 | # write: terminate session (this fixes #issue1) 124 | tcp-check send-binary 58 # Termination packet 125 | tcp-check send-binary 00000004 # packet length: 4 (no body) 126 | 127 | # expect: Row description packet 128 | # 129 | tcp-check expect binary 54 # row description packet 130 | tcp-check expect binary 0000002a # packet length: 42 bytes (0x2a) ( 4 bytes ) 131 | tcp-check expect binary 0001 # field count: 1 ( 1 byte ) 132 | tcp-check expect binary 70675f69735f696e5f7265636f7665727900 133 | # field name: pg_is_in_recovery ( 19 bytes ) 134 | tcp-check expect binary 00000000 # table oid: 0 ( 4 bytes ) 135 | tcp-check expect binary 0000 # column index: 0 ( 2 bytes ) 136 | tcp-check expect binary 00000010 # type oid: 16 ( 4 bytes ) 137 | tcp-check expect binary 0001 # column length: 1 ( 2 bytes ) 138 | tcp-check expect binary ffffffff # type modifier: -1 ( 4 bytes ) 139 | tcp-check expect binary 0000 # format: text ( 2 bytes ) 140 | ## TOTAL ( 42 bytes ) 141 | 142 | # expect: query result data 143 | # 144 | # "f" means node in master mode 145 | # "t" means node in standby mode (read-only) 146 | # 147 | tcp-check expect binary 44 # data row packet 148 | tcp-check expect binary 0000000b # packet length: 11 (0x0b) ( 4 bytes ) 149 | tcp-check expect binary 0001 # field count: 1 ( 2 bytes ) 150 | tcp-check expect binary 00000001 # column length in bytes: 1 ( 4 bytes ) 151 | tcp-check expect binary 66 # column data, "f" ( 1 byte ) 152 | ## TOTAL ( 11 bytes ) 153 | # write: terminate session 154 | tcp-check send-binary 58 # Termination packet 155 | tcp-check send-binary 00000004 # packet length: 4 (no body) 156 | 157 | # close open sessions in case the downed server is still running but is out of sync with the master 158 | default-server on-marked-down shutdown-sessions 159 | 160 | # server list to check 161 | server <%= @bn.mastername %> <%= @bn.masterdsn %> check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port <%= @bn.checkport %> # this defaults to the initial master 162 | <%= @SLAVELIST %> 163 | 164 | -------------------------------------------------------------------------------- /template/standby-partial.template: -------------------------------------------------------------------------------- 1 | server {name} {dsn} check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port {checkport} 2 | -------------------------------------------------------------------------------- /template/standby.template: -------------------------------------------------------------------------------- 1 | # haproxy postgresql master check 2 | # 3 | # haproxy listen on: <%= @bn.listenport %> # 5432 4 | 5 | # Setup architecture : haproxy (2) <-> pgbouncer(2) <-> postgresql (1 + 1) 6 | 7 | # DB, ex remote instance #1 listen: 5432 ( master node ) 8 | # DB, ex remote instance #2 listen: 5432 ( standby node ) 9 | 10 | # passwordless auth for check user <%= @bn.checkuser %> 11 | # The check is performed directly, without consulting the pgbouncer, hence checking 12 | # happens on port 6432 , bouncers listen on 5432 13 | 14 | # external failover, promoting replica to master in case of failure: using repmgr 15 | # template1 database is accessible by user <%= @bn.checkuser %> 16 | # 17 | # haproxy will pass connection to postgresql master node: 18 | # $ psql -h -p 5432 -U <%= @bn.checkuser %> template1 19 | 20 | #--------------------------------------------------------------------- 21 | # Global settings 22 | #--------------------------------------------------------------------- 23 | global 24 | log 127.0.0.1 local2 25 | 26 | chroot /var/lib/haproxy 27 | pidfile /var/run/haproxy.pid 28 | maxconn 4000 29 | user haproxy 30 | group haproxy 31 | daemon 32 | spread-checks 5 33 | 34 | # turn on stats unix socket 35 | stats socket /var/lib/haproxy/stats 36 | 37 | #--------------------------------------------------------------------- 38 | # common defaults that all the 'listen' and 'backend' sections will 39 | # use if not designated in their block 40 | #--------------------------------------------------------------------- 41 | defaults 42 | mode tcp 43 | log global 44 | option dontlognull 45 | option redispatch 46 | retries 3 47 | timeout queue 1m 48 | timeout connect 1s 49 | timeout client 3600s 50 | timeout server 3600s 51 | timeout check 2s 52 | maxconn 500 53 | #--------------------------------------------------------------------- 54 | # statistics 55 | #--------------------------------------------------------------------- 56 | # Host HA-Proxy's web stats on Port 8182. 57 | 58 | listen HAProxy-Statistics 59 | bind *:8182 60 | mode http 61 | option httplog 62 | stats enable 63 | stats uri /haproxy?stats 64 | stats refresh 20s 65 | stats realm PSQL Haproxy\ Statistics # Title text for popup window 66 | stats show-node 67 | stats show-legends 68 | stats show-desc PSQL load balancer stats 69 | stats auth pgadmin:pgsecret 70 | #stats auth <%= @bn.stats_user %>:<%= @bn.stats_password %> 71 | 72 | #--------------------------------------------------------------------- 73 | # main frontend which proxys to the backends 74 | #--------------------------------------------------------------------- 75 | 76 | frontend front_pg 77 | mode tcp 78 | bind <%= @bn.vipip %>:<%= @bn.listenport %> # 5432 79 | 80 | acl pg_single_master nbsrv(backend_pg) eq 1 81 | tcp-request connection reject if !pg_single_master 82 | 83 | default_backend backend_pg 84 | 85 | #--------------------------------------------------------------------- 86 | # the postgresql cluster backend (master + standby) 87 | #--------------------------------------------------------------------- 88 | 89 | backend backend_pg 90 | option tcp-check 91 | tcp-check connect 92 | 93 | # user: <%= @bn.checkuser %> 94 | # database: template1 95 | # 96 | tcp-check send-binary 000000<%= @bn.totalbytes %> # packet length: <%= @bn.totalsize %> bytes in ( 4 bytes ) 97 | tcp-check send-binary 00030000 # protocol version ( 4 bytes ) 98 | tcp-check send-binary 7573657200 # "user" ( 5 bytes ) 99 | tcp-check send-binary <%= @bn.checkuserhex %> # "<%= @bn.checkuser %>" ( <%= @bn.checkuserlen %> bytes ) 100 | tcp-check send-binary 646174616261736500 # "database" ( 9 bytes ) 101 | tcp-check send-binary 74656d706c6174653100 # "template1" ( 10 bytes ) 102 | tcp-check send-binary 00 # terminator ( 1 byte ) 103 | ## TOTAL ( <%= @bn.totalsize %> bytes ) 104 | 105 | # expect: Auth 106 | # 107 | tcp-check expect binary 52 # Auth request 108 | tcp-check expect binary 00000008 # packet length : 8 bytes in ( 4 bytes ) 109 | tcp-check expect binary 00000000 # auth response ok ( 4 bytes ) 110 | ## TOTAL ( 8 bytes ) 111 | 112 | # write: run simple query 113 | # "select pg_is_in_recovery();" 114 | # 115 | tcp-check send-binary 51 # simple query 116 | tcp-check send-binary 00000020 # packet length: 32 bytes in ( 4 bytes ) 117 | tcp-check send-binary 73656c65637420 # "select " ( 7 bytes ) 118 | tcp-check send-binary 70675f69735f696e5f7265636f7665727928293b 119 | # "pg_is_in_recovery(); ( 20 bytes ) 120 | tcp-check send-binary 00 # terminator ( 1 byte ) 121 | ## TOTAL ( 32 bytes ) 122 | 123 | # write: terminate session (this fixes #issue1) 124 | tcp-check send-binary 58 # Termination packet 125 | tcp-check send-binary 00000004 # packet length: 4 (no body) 126 | 127 | # expect: Row description packet 128 | # 129 | tcp-check expect binary 54 # row description packet 130 | tcp-check expect binary 0000002a # packet length: 42 bytes (0x2a) ( 4 bytes ) 131 | tcp-check expect binary 0001 # field count: 1 ( 1 byte ) 132 | tcp-check expect binary 70675f69735f696e5f7265636f7665727900 133 | # field name: pg_is_in_recovery ( 19 bytes ) 134 | tcp-check expect binary 00000000 # table oid: 0 ( 4 bytes ) 135 | tcp-check expect binary 0000 # column index: 0 ( 2 bytes ) 136 | tcp-check expect binary 00000010 # type oid: 16 ( 4 bytes ) 137 | tcp-check expect binary 0001 # column length: 1 ( 2 bytes ) 138 | tcp-check expect binary ffffffff # type modifier: -1 ( 4 bytes ) 139 | tcp-check expect binary 0000 # format: text ( 2 bytes ) 140 | ## TOTAL ( 42 bytes ) 141 | 142 | # expect: query result data 143 | # 144 | # "f" means node in master mode 145 | # "t" means node in standby mode (read-only) 146 | # 147 | tcp-check expect binary 44 # data row packet 148 | tcp-check expect binary 0000000b # packet length: 11 (0x0b) ( 4 bytes ) 149 | tcp-check expect binary 0001 # field count: 1 ( 2 bytes ) 150 | tcp-check expect binary 00000001 # column length in bytes: 1 ( 4 bytes ) 151 | tcp-check expect binary 66 # column data, "f" ( 1 byte ) 152 | ## TOTAL ( 11 bytes ) 153 | # write: terminate session 154 | tcp-check send-binary 58 # Termination packet 155 | tcp-check send-binary 00000004 # packet length: 4 (no body) 156 | 157 | # close open sessions in case the downed server is still running but is out of sync with the master 158 | default-server on-marked-down shutdown-sessions 159 | 160 | # server list to check 161 | server <%= @bn.mastername %> <%= @bn.masterdsn %> check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port <%= @bn.checkport %> # this defaults to the initial master 162 | server <%= @bn.standbyname %> <%= @bn.standbydsn %> check inter 5000 fastinter 2000 downinter 5000 rise 2 fall 3 port <%= @bn.checkport %> # this defaults to the initial standby 163 | 164 | # These 2 above can change role depending on their role 165 | 166 | --------------------------------------------------------------------------------