├── README.md ├── blog ├── building_service_mesh │ ├── README.md │ ├── authorize.lua │ ├── consul-server │ │ └── consul.d │ │ │ └── basic_config.json │ ├── controller.sh │ ├── docker-compose.yml │ ├── redis │ │ ├── .dockerignore │ │ ├── Dockerfile │ │ ├── authorize.lua │ │ ├── controller.sh │ │ └── start.sh │ └── www │ │ ├── .dockerignore │ │ ├── Dockerfile │ │ ├── app.js │ │ ├── authorize.lua │ │ ├── controller.sh │ │ ├── package.json │ │ └── start.sh ├── haproxy_docker_dns_link │ ├── blog_haproxy_dns │ │ ├── Dockerfile │ │ ├── haproxy.cfg │ │ └── haproxy.sh │ └── blog_rsyslogd │ │ ├── Dockerfile │ │ ├── rsyslogd.conf │ │ └── rsyslogd.sh ├── integration_with_consul │ ├── haproxy.conf.by_consul_template │ ├── haproxy.conf.tmpl │ └── haproxy_reload.sh └── ssl_client_certificate_management_at_application_level │ ├── ca.sh │ ├── ca2.sh │ ├── pem_creation.sh │ └── revocation_list_creation.sh ├── configuration_templates ├── exchange2010_https_services_forward.tpl ├── exchange2010_mapi_services.tpl ├── exchange2013_simple_https_load-balancing.tpl ├── genconf.ps1 ├── genconf.sh ├── microsoft_rds_load-balancing.tpl └── tomcat_jsessionid_persistence.tpl ├── logs ├── logrotate.haproxy.conf ├── srv-4xx ├── srv-5xx ├── srv-requests ├── srv-response-time ├── syslog-ng_full_http_url.conf ├── url-404 ├── url-errors ├── url-requests └── url-response-time ├── monitoring └── haproxy.monit └── scripts ├── errorfile_content_length ├── halog_installation ├── haproxy.init.debian ├── haproxy.init.debian7 └── haproxy_update_from_git /README.md: -------------------------------------------------------------------------------- 1 | haproxy 2 | ======= 3 | 4 | HAProxy related stuff: scripts, configs, etc... provided by HAProxy Technologies: http://www.haproxy.com/ 5 | 6 | -------------------------------------------------------------------------------- /blog/building_service_mesh/README.md: -------------------------------------------------------------------------------- 1 | Building a Service Mesh with HAProxy and Consul 2 | =============================================== 3 | 4 | This repo contains a PoC of connecting HAProxy with Consul Connect version 1.3.0 and above. 5 | 6 | The solution is composed by 2 scripts: 7 | * controller.sh: used to monitor events in consul and to generate HAProxy's configuration accordingly. 8 | * authorize.lua: used perform the authorization calls to consul for any new incoming connection. 9 | 10 | In order to work, you must of course have a consul server in the infrastructure, that will maintain the service mesh configuration. 11 | 12 | Then, you need to prepare your application servers with the following components: 13 | * your application 14 | * consul agent 15 | * haproxy + the scripts above 16 | 17 | As an example, please find below an integration of the components in a Docker container executing a nodejs application: 18 | 19 | ``` 20 | FROM alpine:latest 21 | 22 | ARG CONSUL_VER= 23 | 24 | # consul related 25 | RUN echo "Installing consul v${CONSUL_VER}" \ 26 | && apk add --no-cache unzip openssl util-linux su-exec curl jq drill \ 27 | && mkdir /usr/src \ 28 | && cd /usr/src \ 29 | && wget -q https://releases.hashicorp.com/consul/${CONSUL_VER}/consul_${CONSUL_VER}_linux_amd64.zip \ 30 | && unzip -o consul_${CONSUL_VER}_linux_amd64.zip \ 31 | && mv consul /usr/bin/ \ 32 | && mkdir /tmp/consul /etc/consul.d \ 33 | && chown nobody:nobody /tmp/consul \ 34 | && rm -rf /usr/src/* \ 35 | && apk del unzip openssl 36 | 37 | # haproxy related 38 | RUN echo "@edge http://nl.alpinelinux.org/alpine/edge/main" >> /etc/apk/repositories \ 39 | && apk add --no-cache haproxy@edge \ 40 | && apk add --no-cache lua5.3-ossl lua5.3-socket socat lua5.3-cjson openssl iproute2 bash \ 41 | && chown -R nobody:nobody /etc/haproxy \ 42 | && mkdir /var/run/haproxy \ 43 | && chown nobody:nobody /var/run/haproxy 44 | COPY controller.sh authorize.lua / 45 | 46 | # application related 47 | RUN mkdir /www \ 48 | && apk add --no-cache nodejs 49 | COPY www.js /www/www.js 50 | COPY node_modules /www/node_modules/ 51 | 52 | # entrypoint 53 | COPY start.sh / 54 | ENTRYPOINT [ "/start.sh" ] 55 | ``` 56 | 57 | The start.sh script introduce above is also very important, since it configures consul and starts up all the services: 58 | * the application 59 | * consul 60 | * the HAProxy controller 61 | 62 | An example of such script is included in the repo too. 63 | It embeds configuration for a sidecar which: 64 | * expose the 'www' local application on the external network 65 | * expose a remote 'redis' service on the loopback, to be consumed by the local 'www' service 66 | 67 | Docker-compose usage 68 | ==================== 69 | 70 | Run the following commands in that order: 71 | ``` 72 | docker-compose up -d consul-server 73 | sleep 10 74 | docker-compose exec consul-server curl --request PUT --header "X-Consul-Token: mastertoken" --data '{ "ID": "agenttoken", "Name": "Agent Token", "Type": "client", "Rules": "node \"\" { policy = \"write\" } service \"\" { policy = \"write\" }" }' http://localhost:8500/v1/acl/create 75 | sleep 1 76 | docker-compose up -d www redis 77 | ``` 78 | 79 | By default, the ACL will deny the traffic. In order to allow 'www' to contact 'redis', you must create the relevant intention in consul-server UI. 80 | 81 | Usage 82 | ===== 83 | 84 | controller.sh takes a single argument: '-sidecar-for='. 85 | It is mainly a loop, with a blocking query on the consul connect API endpoint waiting for events (timeout set to 10s for retries). 86 | Once an event happens, the controller will parse the JSON returner by Consul and generate the relevant HAProxy configuration. It then parses the Upstream list and add the relevant frontend/backend corresponding to the remote service this local service has to access. 87 | The main loop, also ensure that the SSL certificate for the local service is still valid, if not, it will update it. 88 | 89 | 90 | -------------------------------------------------------------------------------- /blog/building_service_mesh/authorize.lua: -------------------------------------------------------------------------------- 1 | -- 2 | -- authorize.lua 3 | -- 4 | -- Connection / request authorization when HAProxy is used as a "proxy" in 5 | -- a service mesh managed by consul connect 6 | -- 7 | -- 8 | -- Copyright (c) 2017-2018. Baptiste Assmann 9 | -- Copyright (c) 2017-2018. HAProxy Technologies, LLC. 10 | -- 11 | -- This program is free software: you can redistribute it and/or modify 12 | -- it under the terms of the GNU General Public License as published by 13 | -- the Free Software Foundation, version 2 of the License 14 | 15 | --package.path = package.path .. ';/opt/hapee-1.8/usr/lib/lua/?.lua' 16 | 17 | local http = require("socket.http") 18 | local ltn12 = require("ltn12") 19 | local json = require("cjson") 20 | local x509 = require("openssl.x509") 21 | 22 | cache = {} 23 | 24 | -- return true if the key is in the cache, return false otherwise 25 | local function checkCache(key) 26 | if cache[key] == nil then 27 | return false 28 | end 29 | return true 30 | end 31 | 32 | -- return true if the key in the cache has expired 33 | local function hasExpired(key) 34 | if not checkCache(key) then 35 | return true 36 | end 37 | 38 | local now = core.now()["sec"] 39 | 40 | if now - cache[key]["timestamp"] < 1 then 41 | return false 42 | end 43 | 44 | return true 45 | end 46 | 47 | local function setCache(spiffie_url, auth_status, reason) 48 | cache[spiffie_url] = {} 49 | cache[spiffie_url]["authorized"] = auth_status 50 | cache[spiffie_url]["reason"] = reason 51 | cache[spiffie_url]["timestamp"] = core.now()["sec"] 52 | end 53 | 54 | core.register_action("authorize", { "http-req", "tcp-req" }, function(txn) 55 | local serial = txn.f:ssl_c_serial() 56 | local der = txn.sf:ssl_c_der() 57 | local client_cert_url = '' 58 | local headers = {} 59 | local request_body = '' 60 | local response_body = {} 61 | local target = txn.sf:fe_name() 62 | 63 | -- this is due to a 'second' call, need to troubleshoot why 64 | -- this function is called twice in TCP mode 65 | if type(der) == 'nil' then 66 | txn.set_var(txn, "txn.SpiffeUrl", "-") 67 | txn.set_var(txn, "txn.Authorized", "false") 68 | txn.set_var(txn, "txn.Reason", "Error parsing client certificate") 69 | return 70 | end 71 | 72 | -- collecting client cert spiffe URL and client certificate serial 73 | local mycert = x509.new(der, "DER") 74 | if mycert ~= nil then 75 | local myext = mycert:getExtension('X509v3 Subject Alternative Name') 76 | client_cert_url = string.gsub(myext:text(), 'URI:', '') 77 | txn.set_var(txn, "txn.SpiffeUrl", client_cert_url) 78 | end 79 | serial = txn.c:hex(serial) 80 | 81 | if not hasExpired(client_cert_url) then 82 | txn.set_var(txn, "txn.Authorized", cache[client_cert_url]["authorized"]) 83 | txn.set_var(txn, "txn.Reason", cache[client_cert_url]["reason"]) 84 | return 85 | end 86 | 87 | -- prepate and run the HTTP request to consul connect 88 | -- update the target to remove the 'f_' prefix 89 | target = string.sub(target, 3) 90 | request_body = json.encode({ Target = target, ClientCertURI = client_cert_url, ClientCertSerial = serial }) 91 | headers['Content-Type'] = 'application/json' 92 | headers['Content-Length'] = #request_body 93 | 94 | local b, c, h = http.request { 95 | url = "http://127.0.0.1:8500/v1/agent/connect/authorize", 96 | create = core.tcp, 97 | method = "POST", 98 | headers = headers, 99 | source = ltn12.source.string(request_body), 100 | sink = ltn12.sink.table(response_body) 101 | } 102 | 103 | -- analyze the feedback from consul connect 104 | if c == nil or c ~= 200 then 105 | setCache(client_cert_url, "false", "Not authorized by consul") 106 | txn.set_var(txn, "txn.Authorized", "false") 107 | txn.set_var(txn, "txn.Reason", 'Not authorized by consul') 108 | return 109 | end 110 | 111 | local myjson = json.decode(table.concat(response_body)) 112 | if myjson['Authorized'] == nil or myjson['Authorized'] ~= true then 113 | txn.set_var(txn, "txn.Authorized", "false") 114 | if myjson['Reason'] ~= nil then 115 | setCache(client_cert_url, "false", myjson['Reason']) 116 | txn.set_var(txn, "txn.Reason", myjson['Reason']) 117 | else 118 | setCache(client_cert_url, "false", 'Not authorized by consul') 119 | txn.set_var(txn, "txn.Reason", 'Not authorized by consul') 120 | end 121 | return 122 | end 123 | 124 | setCache(client_cert_url, "true", myjson['Reason']) 125 | 126 | txn.set_var(txn, "txn.Authorized", "true") 127 | txn.set_var(txn, "txn.Reason", myjson['Reason']) 128 | end) 129 | -------------------------------------------------------------------------------- /blog/building_service_mesh/consul-server/consul.d/basic_config.json: -------------------------------------------------------------------------------- 1 | { 2 | "dns_config": { 3 | "enable_truncate": true, 4 | "udp_answer_limit": 100 5 | }, 6 | "connect": { 7 | "enabled": true 8 | }, 9 | "acl_datacenter":"dc1", 10 | "acl_default_policy":"deny", 11 | "acl_down_policy":"extend-cache", 12 | "acl_master_token":"mastertoken" 13 | } 14 | -------------------------------------------------------------------------------- /blog/building_service_mesh/controller.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # "optimized" for bash/Ubuntu 4 | 5 | set -x 6 | #VERBOSE="--verbose" 7 | 8 | LOGFILE=/tmp/controller.log 9 | 10 | declare -A ARGS 11 | ARGS["SIDECAR_FOR"]="" 12 | 13 | ERROR="false" 14 | for arg in "$@" 15 | do 16 | case ${arg} in 17 | -sidecar-for=*) 18 | ARGS["SIDECAR_FOR"]=${arg##*=} 19 | shift 20 | ;; 21 | *) 22 | echo "unknown argument: $arg" >>$LOGFILE 23 | ERROR="true" 24 | ;; 25 | esac 26 | done 27 | 28 | if [ -z "${ARGS["SIDECAR_FOR"]}" ]; then 29 | echo "Need at least one argument: \"-sidecar-for=\"" 30 | exit 1 31 | fi 32 | 33 | if [ ${ERROR} = "true" ]; then 34 | cat $LOGFILE 35 | exit 1 36 | fi 37 | 38 | # HAProxy Enterprise related variables 39 | #CONF_TMP_DIR=/tmp/cfg 40 | #CONF_PROD_DIR=/etc/hapee-1.8 41 | #CONF_CERTS_DIRNAME=certs 42 | #CONF_HAP_FILENAME=hapee-lb.cfg 43 | #HAP_BINARY_PATH=/opt/hapee-1.8/sbin/ 44 | #HAP_BINARY_NAME=hapee-lb 45 | #HAP_BINARY=${HAP_BINARY_PATH}/${HAP_BINARY_NAME} 46 | #HAP_PIDFILE=/var/run/hapee-1.8/hapee-lb.pid 47 | #HAP_SOCKET=/var/run/hapee-1.8/hapee-lb.sock 48 | #HAP_USER=hapee-lb 49 | #HAP_GROUP=hapee 50 | #HAP_MODULES="module-path /opt/hapee-1.8/modules" 51 | 52 | # HAPCE alpine 53 | CONF_TMP_DIR=/tmp/cfg 54 | CONF_PROD_DIR=/etc/haproxy 55 | CONF_CERTS_DIRNAME=certs 56 | CONF_HAP_FILENAME=haproxy.cfg 57 | HAP_BINARY_PATH=/usr/sbin 58 | HAP_BINARY_NAME=haproxy 59 | HAP_BINARY=${HAP_BINARY_PATH}/${HAP_BINARY_NAME} 60 | HAP_PIDFILE=/var/run/haproxy/haproxy.pid 61 | HAP_SOCKET=/var/run/haproxy/haproxy.sock 62 | HAP_USER=nobody 63 | HAP_GROUP=nobody 64 | HAP_MODULES= 65 | 66 | # common URLs 67 | CURL='curl --silent ' 68 | CONSUL_SERVICE_URL="http://localhost:8500/v1/agent/service/${ARGS["SIDECAR_FOR"]}-sidecar-proxy" 69 | CONSUL_ROOTCA_URL="http://localhost:8500/v1/agent/connect/ca/roots" 70 | CONSUL_LEAF_CERT_URL="http://localhost:8500/v1/agent/connect/ca/leaf" 71 | 72 | # this function triggers a reload of HAProxy 73 | function do-reload() { 74 | pidof ${HAP_BINARY_NAME} 75 | CHECK=$? 76 | if [ $CHECK -eq 1 ]; then 77 | echo "Starting required" >>$LOGFILE 78 | ${HAP_BINARY} -f ${CONF_PROD_DIR}/${CONF_HAP_FILENAME} -W 2>>$LOGFILE 79 | else 80 | echo "Reload required" >>$LOGFILE 81 | kill -SIGUSR2 $(cat ${HAP_PIDFILE}) 82 | fi 83 | } 84 | 85 | # this function checks haproxy's configuration file integrity (including certificates) 86 | # it returns 1 in case of error and 0 otherwise 87 | function do-check() { 88 | ${HAP_BINARY} -f ${CONF_TMP_DIR}/${CONF_HAP_FILENAME} -c 89 | return $? 90 | } 91 | 92 | # this function move haproxy's configuration from a temporary dir to the production one 93 | function move-cfg() { 94 | SRC=$1 95 | DST=$2 96 | 97 | [ -z "${SRC}" ] && return 1 98 | [ -z "${DST}" ] && return 1 99 | 100 | #mv -f ${DST} ${DST}.old 2>>$LOGFILE 101 | mv -f ${SRC}/${CONF_HAP_FILENAME} ${DST}/${CONF_HAP_FILENAME} 2>>$LOGFILE 102 | [ ! -d ${DST}/${CONF_CERTS_DIRNAME} ] && mkdir -p ${DST}/${CONF_CERTS_DIRNAME} 103 | mv -f ${SRC}/${CONF_CERTS_DIRNAME}/* ${DST}/${CONF_CERTS_DIRNAME}/ 2>>$LOGFILE 104 | sed -i -e "s!${SRC}!${DST}!g" ${DST}/${CONF_HAP_FILENAME} 105 | 106 | return 0 107 | } 108 | 109 | ### CERTIFICATES ### 110 | declare -A CERTS 111 | 112 | # this function watch for LEAF certificates changes and update local storage 113 | # when required. 114 | # it returns an integer with the number of certs that has been updated locally. 115 | function check-certs() { 116 | local ret=0 117 | 118 | for cert in "${!CERTS[@]}" 119 | do 120 | CONSUL_LEAFCERT_JSON=$(${CURL} ${CONSUL_LEAF_CERT_URL}/${CERTNAME} 2>>$LOGFILE) 121 | SERIAL=$(jq -r .SerialNumber <<<${CONSUL_LEAFCERT_JSON}) 122 | if [ "${CERTS[$cert]}" != "$SERIAL" ] || [ ! -s "${CONF_PROD_DIR}/${CONF_CERTS_DIRNAME}/${cert}.pem" ]; then 123 | save-leaf-cert $cert ${CONF_PROD_DIR}/${CONF_CERTS_DIRNAME} 124 | ret+=1 125 | fi 126 | done 127 | 128 | return $ret 129 | } 130 | 131 | function save-leaf-cert() { 132 | CERTNAME=$1 133 | DESTINATION_DIR=$2 134 | 135 | [ -z "${CERTNAME}" ] && return 0 136 | [ -z "${DESTINATION_DIR}" ] && return 0 137 | 138 | CONSUL_LEAFCERT_JSON=$(${CURL} ${CONSUL_LEAF_CERT_URL}/${CERTNAME} 2>>$LOGFILE) 139 | SERIAL=$(jq -r .SerialNumber <<<${CONSUL_LEAFCERT_JSON}) 140 | [ $? -ne 0 ] && return 1 141 | jq -r .CertPEM <<<${CONSUL_LEAFCERT_JSON} > ${DESTINATION_DIR}/${CERTNAME}.pem 142 | jq -r .PrivateKeyPEM <<<${CONSUL_LEAFCERT_JSON} >> ${DESTINATION_DIR}/${CERTNAME}.pem 143 | sed -i -e 's/\\n/\n/g' -e '/^$/d' ${DESTINATION_DIR}/${CERTNAME}.pem 144 | 145 | # save cert serial in the list 146 | CERTS[${CERTNAME}]=$SERIAL 147 | 148 | return 0 149 | } 150 | ### END OF CERTIFICATES ### 151 | 152 | 153 | ### CONFIG ### 154 | declare -A CONFIG 155 | # initialize a new CONFIG array 156 | function config-init() { 157 | # global variables 158 | CONFIG["TARGET_SERVICE_NAME"]=${ARGS["SIDECAR_FOR"]} 159 | CONFIG["PIDFILE"]=${HAP_PIDFILE} 160 | CONFIG["STATS_SOCKET"]=${HAP_SOCKET} 161 | CONFIG["CERTS_FOLDER"]="${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME}" 162 | CONFIG["ACL_UNSECURED"]="acl UNSECURED dst_port 0" 163 | CONFIG["USER"]=${HAP_USER} 164 | CONFIG["GROUP"]=${HAP_GROUP} 165 | CONFIG["MODULES"]=${HAP_MODULES} 166 | 167 | # mandatory parameters provided by the consul API 168 | CONFIG["BIND_ADDRESS"]="" 169 | CONFIG["BIND_PORT"]="" 170 | CONFIG["LOCAL_SERVICE_ADDRESS"]="" 171 | CONFIG["LOCAL_SERVICE_PORT"]="" 172 | 173 | # custom parameters 174 | CONFIG["UNSECURED_BIND_PORT"]="" 175 | CONFIG["LOCAL_SERVICE_MODE"]="tcp" 176 | CONFIG["LOCAL_BACKEND_MODE"]="mode tcp" 177 | CONFIG["SYSLOG_SERVER"]="" 178 | CONFIG["LOG_FORMAT"]="option tcplog" 179 | CONFIG["DEFAULTS_LOG"]="" 180 | } 181 | 182 | # collect data from consul to build the configuration 183 | function config-getdata() { 184 | CONSUL_PROXY_JSON=$($CURL ${CONSUL_SERVICE_URL} 2>>$LOGFILE) 185 | echo $CONSUL_PROXY_JSON >>$LOGFILE 186 | 187 | CONFIG["BIND_ADDRESS"]=$(jq -r .Address <<<${CONSUL_PROXY_JSON}) 188 | CONFIG["BIND_PORT"]=$(jq -r .Port <<<${CONSUL_PROXY_JSON}) 189 | CONFIG["LOCAL_SERVICE_ADDRESS"]=$(jq -r .Proxy.LocalServiceAddress <<<${CONSUL_PROXY_JSON}) 190 | CONFIG["LOCAL_SERVICE_PORT"]=$(jq -r .Proxy.LocalServicePort <<<${CONSUL_PROXY_JSON}) 191 | 192 | # custom parameters 193 | #CONFIG["AUTHORIZATIONFREE_BIND_PORT=$(jq -r .Config.authorizationfree_bind_port <<<${CONSUL_PROXY_JSON}) 194 | #if [ "${AUTHORIZATIONFREE_BIND_PORT}" = "null" ]; then 195 | # AUTHORIZATIONFREE_BIND_PORT= 196 | #fi 197 | UNSECURED_BIND_PORT=$(jq -r .Proxy.Config.unsecured_bind_port <<<${CONSUL_PROXY_JSON}) 198 | if [ "${UNSECURED_BIND_PORT}" != "null" ]; then 199 | CONFIG["UNSECURED_BIND_PORT"]="bind :${UNSECURED_BIND_PORT} name unsecured" 200 | CONFIG["ACL_UNSECURED"]="${CONFIG["ACL_UNSECURED"]} ${UNSECURED_BIND_PORT}" 201 | fi 202 | 203 | LOCAL_SERVICE_MODE=$(jq -r .Proxy.Config.local_service_mode <<<${CONSUL_PROXY_JSON}) 204 | case ${LOCAL_SERVICE_MODE} in 205 | http) 206 | CONFIG["LOCAL_SERVICE_MODE"]="mode http" 207 | CONFIG["LOCAL_BACKEND_MODE"]="mode http" 208 | CONFIG["LOG_FORMAT"]="log-format \"%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs {%[var(txn.SpiffeUrl)]|%[var(txn.Authorized)]|%[var(txn.Reason)]} {%sslv|%sslc|%[ssl_fc_session_id,hex]|%[ssl_fc_is_resumed]|%[ssl_fc_sni]} %{+Q}r\"" 209 | ;; 210 | tcp|*) 211 | CONFIG["LOCAL_SERVICE_MODE"]="mode tcp" 212 | CONFIG["LOCAL_BACKEND_MODE"]="mode tcp" 213 | CONFIG["LOG_FORMAT"]="log-format \"%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq {%[var(txn.SpiffeUrl)]|%[var(txn.Authorized)]|%[var(txn.Reason)]} {%sslv|%sslc|%[ssl_fc_session_id,hex]|%[ssl_fc_is_resumed]|%[ssl_fc_sni]}\"" 214 | ;; 215 | esac 216 | CONFIG["LOCAL_SERVICE_MODE"]="${CONFIG["LOCAL_SERVICE_MODE"]} 217 | tcp-request inspect-delay 1s 218 | tcp-request content lua.authorize unless UNSECURED 219 | tcp-request content capture var(txn.SpiffeUrl) len 64 if { var(txn.SpiffeUrl) -m found } 220 | tcp-request content capture var(txn.Authorized) len 64 if { var(txn.Authorized) -m found } 221 | tcp-request content capture var(txn.Reason) len 64 if { var(txn.Reason) -m found } 222 | tcp-request content reject unless UNSECURED or { var(txn.Authorized) -m str true } 223 | " 224 | SYSLOG_SERVER=$(jq -r .Proxy.Config.syslog_server <<<${CONSUL_PROXY_JSON}) 225 | if [ "${SYSLOG_SERVER}" != "null" ]; then 226 | CONFIG["SYSLOG_SERVER"]="log ${SYSLOG_SERVER} local0 info 227 | log-tag sidecar-for-${ARGS["SIDECAR_FOR"]}" 228 | CONFIG["DEFAULTS_LOG"]="log global 229 | option httplog" 230 | fi 231 | } 232 | 233 | # write HAProxy Configuration 234 | function config-write() { 235 | cat <${CONF_TMP_DIR}/${CONF_HAP_FILENAME} 236 | global 237 | maxconn 10000 238 | #user ${CONFIG["USER"]} # useless cause we're already started as ${CONFIG["USER"]} user 239 | #group ${CONFIG["GROUP"]} # useless cause we're already started as ${CONFIG["GROUP"]} user 240 | #chroot /var/empty 241 | daemon 242 | tune.ssl.default-dh-param 1024 243 | pidfile ${CONFIG["PIDFILE"]} 244 | stats socket ${CONFIG["STATS_SOCKET"]} user ${CONFIG["USER"]} group ${CONFIG["GROUP"]} mode 660 level admin 245 | stats timeout 10m 246 | ca-base ${CONFIG["CERTS_FOLDER"]} 247 | crt-base ${CONFIG["CERTS_FOLDER"]} 248 | ${CONFIG["MODULES"]} 249 | lua-load /authorize.lua 250 | ${CONFIG["SYSLOG_SERVER"]} 251 | 252 | resolvers consul 253 | nameserver consul 127.0.0.1:8600 254 | 255 | defaults 256 | timeout client 30s 257 | timeout connect 250ms 258 | timeout server 30s 259 | option redispatch 1 260 | retries 3 261 | option socket-stats 262 | option tcplog 263 | option dontlognull 264 | option dontlog-normal 265 | default-server init-addr none 266 | ${CONFIG["DEFAULTS_LOG"]} 267 | 268 | frontend f_stats 269 | bind 0.0.0.0:1936 270 | mode http 271 | http-request set-log-level silent 272 | stats enable 273 | stats uri / 274 | stats show-legends 275 | stats show-desc ${CONFIG["TARGET_SERVICE_NAME"]} on ${CONFIG["PROXY_ID"]} 276 | option httpclose 277 | 278 | # Proxied (local) service 279 | frontend f_${CONFIG["TARGET_SERVICE_NAME"]} 280 | bind :${CONFIG["BIND_PORT"]} ssl ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem verify required name secured 281 | ${CONFIG["AUTHORIZATIONFREE_BIND_PORT"]} 282 | ${CONFIG["UNSECURED_BIND_PORT"]} 283 | ${CONFIG["ACL_UNSECURED"]} 284 | ${CONFIG["LOCAL_SERVICE_MODE"]} 285 | default_backend b_${CONFIG["TARGET_SERVICE_NAME"]} 286 | 287 | backend b_${CONFIG["TARGET_SERVICE_NAME"]} 288 | ${CONFIG["LOCAL_BACKEND_MODE"]} 289 | server localhost ${CONFIG["LOCAL_SERVICE_ADDRESS"]}:${CONFIG["LOCAL_SERVICE_PORT"]} check 290 | 291 | # upstreams: 292 | EOF 293 | } 294 | 295 | # dump content of the CONFIG variable 296 | function config-dump() { 297 | for a in ${!CONFIG[@]} 298 | do 299 | echo "$a: ${CONFIG[$a]}" >>$LOGFILE 300 | done 301 | } 302 | ### END OF CONFIG ### 303 | 304 | 305 | ### UPSTREAM ### 306 | declare -A UPSTREAM 307 | function upstream-init() { 308 | # mandatory parameters provided by the consul API 309 | UPSTREAM["DESTINATION_NAME"]="" 310 | UPSTREAM["DESTINATION_SERVERS"]="" 311 | UPSTREAM["LOCAL_BIND_ADDRESS"]="" 312 | UPSTREAM["LOCAL_BIND_PORT"]="" 313 | 314 | # custom parameters 315 | UPSTREAM["DESTINATION_MODE"]="mode tcp" 316 | UPSTREAM["LOG_FORMAT"]="option tcplog" 317 | UPSTREAM["ADVANCED_CHECK"]="option tcp-check 318 | tcp-check connect ssl" 319 | } 320 | 321 | function upstream-getdata() { 322 | # mandatory parameters provided by the consul API 323 | UPSTREAM["DESTINATION_NAME"]=$(jq -r .DestinationName <<<${1}) 324 | UPSTREAM["LOCAL_BIND_ADDRESS"]=$(jq -r .LocalBindAddress <<<${1}) 325 | UPSTREAM["LOCAL_BIND_PORT"]=$(jq -r .LocalBindPort <<<${1}) 326 | DESTINATION_TYPE=$(jq -r .DestinationType <<<${1}) 327 | case ${DESTINATION_TYPE} in 328 | "service") 329 | UPSTREAM["DESTINATION_SERVERS"]="server-template s_${UPSTREAM["DESTINATION_NAME"]} 20 _${UPSTREAM["DESTINATION_NAME"]}._tcp.service.consul ssl resolvers consul ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem check #no-tls-tickets" 330 | ;; 331 | "connect") 332 | # FIXME: use SRV records instead of catalog API 333 | CONSUL_CATALOG_URL="http://localhost:8500/v1/catalog/service/${UPSTREAM["DESTINATION_NAME"]}-sidecar-proxy" 334 | CATALOG_UPSTREAM_JSON=$($CURL ${CONSUL_CATALOG_URL} 2>>$LOGFILE) 335 | 336 | for row in $(jq -c .[] <<<${CATALOG_UPSTREAM_JSON}) 337 | do 338 | UPSTREAM["SERVICE_PORT"]=$(jq -r .ServicePort <<<${row}) 339 | UPSTREAM["DESTINATION_SERVERS"]="server-template s_${UPSTREAM["DESTINATION_NAME"]} 20 ${UPSTREAM["DESTINATION_NAME"]}.connect.consul:${UPSTREAM["SERVICE_PORT"]} ssl resolvers consul ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem check #no-tls-tickets" 340 | done 341 | ;; 342 | esac 343 | 344 | # custom parameters 345 | DESTINATION_MODE=$(jq -r .destination_mode <<<${1}) 346 | if [ "${DESTINATION_MODE}" = "http" ]; then 347 | UPSTREAM["DESTINATION_MODE"]="mode http" 348 | UPSTREAM["LOG_FORMAT"]="option httplog" 349 | fi 350 | 351 | ADVANCED_CHECK=$(jq -r .advanced_check <<<${1}) 352 | case ${ADVANCED_CHECK} in 353 | redis) 354 | UPSTREAM["ADVANCED_CHECK"]="option redis-check" 355 | ;; 356 | esac 357 | } 358 | 359 | # write the upstream configuration into the config file 360 | function upstream-write() { 361 | cat <>${CONF_TMP_DIR}/${CONF_HAP_FILENAME} 362 | 363 | frontend f_${UPSTREAM["DESTINATION_NAME"]} 364 | bind ${UPSTREAM["LOCAL_BIND_ADDRESS"]}:${UPSTREAM["LOCAL_BIND_PORT"]} 365 | ${UPSTREAM["DESTINATION_MODE"]} 366 | ${UPSTREAM["LOG_FORMAT"]} 367 | default_backend b_${UPSTREAM["DESTINATION_NAME"]} 368 | 369 | backend b_${UPSTREAM["DESTINATION_NAME"]} 370 | ${UPSTREAM["DESTINATION_MODE"]} 371 | ${UPSTREAM["ADVANCED_CHECK"]} 372 | ${UPSTREAM["DESTINATION_SERVERS"]} 373 | EOF 374 | } 375 | 376 | # dump content of the UPSTREAM variable 377 | function upstream-dump() { 378 | for a in ${!UPSTREAM[@]} 379 | do 380 | echo "$a: ${UPSTREAM[$a]}" >>$LOGFILE 381 | done 382 | } 383 | ### END OF UPSTREAM ### 384 | 385 | 386 | ### MAIN ### 387 | 388 | # this function generates an HAProxy configuration in $CONF_TMP_DIR and returns: 389 | # 0 if no error happened 390 | # 1 if anything was wrong 391 | function generate-cfg() { 392 | echo "GENERATING NEW CONFIGURATION" >>$LOGFILE 393 | 394 | # clear CERTS array, to ensure we monitor only certs we need to 395 | for cert in "${!CERTS[@]}" 396 | do 397 | unset -v CERTS[$cert] 398 | done 399 | 400 | config-init 401 | config-getdata 402 | config-dump 403 | 404 | ERROR="false" 405 | echo "" >>$LOGFILE 406 | [ -z "${CONF_TMP_DIR}" ] && return 1 407 | rm -rf ${CONF_TMP_DIR} 408 | mkdir -p ${CONF_TMP_DIR} 409 | mkdir -p ${CONFIG["CERTS_FOLDER"]} 410 | 411 | # get cert and key for target service 412 | save-leaf-cert ${CONFIG["TARGET_SERVICE_NAME"]} ${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME} 413 | [ $? -ne 0 ] && return 1 414 | 415 | # get CA root certificate 416 | ${CURL} ${CONSUL_ROOTCA_URL} | jq -r .Roots[0].RootCert | sed -e 's/\\n/\n/g' > ${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME}/ca.pem 417 | 418 | # Write configuration to file 419 | config-write 420 | 421 | # configuring upstreams 422 | for row in $(jq -c .Proxy.Upstreams[] <<<${CONSUL_PROXY_JSON}) 423 | do 424 | upstream-init 425 | upstream-getdata "${row}" 426 | upstream-dump 427 | upstream-write 428 | done 429 | 430 | return 0 431 | } 432 | 433 | 434 | # blocking loop wait for an update on the connect proxy configuration piece 435 | HASH="0" 436 | OLDHASH="0" 437 | while true; 438 | do 439 | TMPFILE=$(mktemp) 440 | ${CURL} --include --max-time 10 ${VERBOSE} --output ${TMPFILE} ${CONSUL_SERVICE_URL}?hash=${HASH} >${TMPFILE} 441 | 442 | RETCODE=$? 443 | # 28 is the return code when curl wait for 'max-time' 444 | if [ ${RETCODE} -eq 28 ]; then 445 | sleep 1 446 | rm ${TMPFILE} 447 | check-certs 448 | [ $? -gt 0 ] && do-reload 449 | continue 450 | fi 451 | # any other code than 0 may indicates an error, so we don't want to apply anything 452 | if [ ${RETCODE} -ne 0 ]; then 453 | sleep 1 454 | rm ${TMPFILE} 455 | continue 456 | fi 457 | 458 | STATUSCODE=$(head -n 1 ${TMPFILE} | cut -d' ' -f 2) 459 | if [ "${STATUSCODE}" != "200" ]; then 460 | sleep 1 461 | OLDHASH=${HASH} 462 | rm ${TMPFILE} 463 | continue 464 | fi 465 | 466 | HASH=$(sed -n 's/^X-Consul-Contenthash: \(.*\)\r/\1/p' ${TMPFILE}) 467 | if [ -z "${HASH}" ]; then 468 | sleep 1 469 | rm ${TMPFILE} 470 | continue 471 | fi 472 | 473 | # the configuration has not changed 474 | if [ "${OLDHASH}" = "${HASH}" ]; then 475 | sleep 1 476 | rm ${TMPFILE} 477 | continue 478 | fi 479 | 480 | rm ${TMPFILE} 481 | 482 | generate-cfg 483 | if [ $? -ne 0 ]; then 484 | sleep 1 485 | HASH="foo" 486 | continue 487 | fi 488 | do-check 489 | [ $? -ne 0 ] && continue 490 | move-cfg ${CONF_TMP_DIR} ${CONF_PROD_DIR} 491 | [ $? -ne 0 ] && continue 492 | 493 | echo "APPLYING NEW CONFIGURATION" >>$LOGFILE 494 | do-reload 495 | 496 | sleep 1 497 | done 498 | 499 | -------------------------------------------------------------------------------- /blog/building_service_mesh/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3" 2 | services: 3 | 4 | consul-server: 5 | image: consul:1.4.0 6 | environment: 7 | CONSUL_LOCAL_CONFIG: '{ "connect": { "enabled": true }}' 8 | command: ["agent", "-ui", "-client=0.0.0.0", "-bind=0.0.0.0", "-data-dir=/consul/data", "-server", "-bootstrap-expect=1", "-config-dir=/etc/consul.d" ] 9 | volumes: 10 | - ./consul-server/consul.d:/etc/consul.d 11 | ports: 12 | - "8500:8500" 13 | 14 | www: 15 | image: haproxytech/consulproxy 16 | build: 17 | context: ./www 18 | args: 19 | CONSUL_VER: "1.4.0" 20 | environment: 21 | SERVICENAME: www 22 | REDIS_URL: localhost 23 | CONSUL_HTTP_TOKEN: agenttoken 24 | ports: 25 | - "21002:21002" 26 | - "1936:1936" 27 | depends_on: 28 | - consul-server 29 | 30 | redis: 31 | image: haproxytech/redis 32 | build: 33 | context: ./redis 34 | args: 35 | CONSUL_VER: "1.4.0" 36 | environment: 37 | SERVICENAME: redis 38 | CONSUL_HTTP_TOKEN: agenttoken 39 | ports: 40 | - "1937:1936" 41 | depends_on: 42 | - consul-server 43 | -------------------------------------------------------------------------------- /blog/building_service_mesh/redis/.dockerignore: -------------------------------------------------------------------------------- 1 | Dockerfile -------------------------------------------------------------------------------- /blog/building_service_mesh/redis/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM redis:alpine 2 | 3 | ARG CONSUL_VER= 4 | 5 | # consul related 6 | RUN echo "Installing consul v${CONSUL_VER}" \ 7 | && apk add --no-cache unzip wget openssl util-linux su-exec curl jq drill \ 8 | && mkdir -p /usr/src \ 9 | && cd /usr/src \ 10 | && wget -q https://releases.hashicorp.com/consul/${CONSUL_VER}/consul_${CONSUL_VER}_linux_amd64.zip \ 11 | && unzip -o consul_${CONSUL_VER}_linux_amd64.zip \ 12 | && mv consul /usr/bin/ \ 13 | && mkdir /tmp/consul /etc/consul.d \ 14 | && chown nobody:nobody /tmp/consul \ 15 | && rm -rf /usr/src/* \ 16 | && apk del unzip openssl 17 | 18 | # haproxy related 19 | RUN echo "@edge http://nl.alpinelinux.org/alpine/edge/main" >> /etc/apk/repositories \ 20 | && apk add --no-cache haproxy@edge lua5.3-ossl@edge lua5.3-socket@edge lua5.3-cjson@edge openssl@edge \ 21 | && apk add --no-cache socat iproute2 bash \ 22 | && chown -R nobody:nobody /etc/haproxy \ 23 | && mkdir /var/run/haproxy \ 24 | && chown nobody:nobody /var/run/haproxy 25 | COPY start.sh controller.sh authorize.lua / 26 | 27 | # entrypoint 28 | WORKDIR / 29 | RUN chmod +x start.sh 30 | ENTRYPOINT [ "/start.sh" ] 31 | -------------------------------------------------------------------------------- /blog/building_service_mesh/redis/authorize.lua: -------------------------------------------------------------------------------- 1 | -- 2 | -- authorize.lua 3 | -- 4 | -- Connection / request authorization when HAProxy is used as a "proxy" in 5 | -- a service mesh managed by consul connect 6 | -- 7 | -- 8 | -- Copyright (c) 2017-2018. Baptiste Assmann 9 | -- Copyright (c) 2017-2018. HAProxy Technologies, LLC. 10 | -- 11 | -- This program is free software: you can redistribute it and/or modify 12 | -- it under the terms of the GNU General Public License as published by 13 | -- the Free Software Foundation, version 2 of the License 14 | 15 | --package.path = package.path .. ';/opt/hapee-1.8/usr/lib/lua/?.lua' 16 | 17 | local http = require("socket.http") 18 | local ltn12 = require("ltn12") 19 | local json = require("cjson") 20 | local x509 = require("openssl.x509") 21 | 22 | cache = {} 23 | 24 | -- return true if the key is in the cache, return false otherwise 25 | local function checkCache(key) 26 | if cache[key] == nil then 27 | return false 28 | end 29 | return true 30 | end 31 | 32 | -- return true if the key in the cache has expired 33 | local function hasExpired(key) 34 | if not checkCache(key) then 35 | return true 36 | end 37 | 38 | local now = core.now()["sec"] 39 | 40 | if now - cache[key]["timestamp"] < 1 then 41 | return false 42 | end 43 | 44 | return true 45 | end 46 | 47 | local function setCache(spiffie_url, auth_status, reason) 48 | cache[spiffie_url] = {} 49 | cache[spiffie_url]["authorized"] = auth_status 50 | cache[spiffie_url]["reason"] = reason 51 | cache[spiffie_url]["timestamp"] = core.now()["sec"] 52 | end 53 | 54 | core.register_action("authorize", { "http-req", "tcp-req" }, function(txn) 55 | local serial = txn.f:ssl_c_serial() 56 | local der = txn.sf:ssl_c_der() 57 | local client_cert_url = '' 58 | local headers = {} 59 | local request_body = '' 60 | local response_body = {} 61 | local target = txn.sf:fe_name() 62 | 63 | -- this is due to a 'second' call, need to troubleshoot why 64 | -- this function is called twice in TCP mode 65 | if type(der) == 'nil' then 66 | txn.set_var(txn, "txn.SpiffeUrl", "-") 67 | txn.set_var(txn, "txn.Authorized", "false") 68 | txn.set_var(txn, "txn.Reason", "Error parsing client certificate") 69 | return 70 | end 71 | 72 | -- collecting client cert spiffe URL and client certificate serial 73 | local mycert = x509.new(der, "DER") 74 | if mycert ~= nil then 75 | local myext = mycert:getExtension('X509v3 Subject Alternative Name') 76 | client_cert_url = string.gsub(myext:text(), 'URI:', '') 77 | txn.set_var(txn, "txn.SpiffeUrl", client_cert_url) 78 | end 79 | serial = txn.c:hex(serial) 80 | 81 | if not hasExpired(client_cert_url) then 82 | txn.set_var(txn, "txn.Authorized", cache[client_cert_url]["authorized"]) 83 | txn.set_var(txn, "txn.Reason", cache[client_cert_url]["reason"]) 84 | return 85 | end 86 | 87 | -- prepate and run the HTTP request to consul connect 88 | -- update the target to remove the 'f_' prefix 89 | target = string.sub(target, 3) 90 | request_body = json.encode({ Target = target, ClientCertURI = client_cert_url, ClientCertSerial = serial }) 91 | headers['Content-Type'] = 'application/json' 92 | headers['Content-Length'] = #request_body 93 | 94 | local b, c, h = http.request { 95 | url = "http://127.0.0.1:8500/v1/agent/connect/authorize", 96 | create = core.tcp, 97 | method = "POST", 98 | headers = headers, 99 | source = ltn12.source.string(request_body), 100 | sink = ltn12.sink.table(response_body) 101 | } 102 | 103 | -- analyze the feedback from consul connect 104 | if c == nil or c ~= 200 then 105 | setCache(client_cert_url, "false", "Not authorized by consul") 106 | txn.set_var(txn, "txn.Authorized", "false") 107 | txn.set_var(txn, "txn.Reason", 'Not authorized by consul') 108 | return 109 | end 110 | 111 | local myjson = json.decode(table.concat(response_body)) 112 | if myjson['Authorized'] == nil or myjson['Authorized'] ~= true then 113 | txn.set_var(txn, "txn.Authorized", "false") 114 | if myjson['Reason'] ~= nil then 115 | setCache(client_cert_url, "false", myjson['Reason']) 116 | txn.set_var(txn, "txn.Reason", myjson['Reason']) 117 | else 118 | setCache(client_cert_url, "false", 'Not authorized by consul') 119 | txn.set_var(txn, "txn.Reason", 'Not authorized by consul') 120 | end 121 | return 122 | end 123 | 124 | setCache(client_cert_url, "true", myjson['Reason']) 125 | 126 | txn.set_var(txn, "txn.Authorized", "true") 127 | txn.set_var(txn, "txn.Reason", myjson['Reason']) 128 | end) 129 | -------------------------------------------------------------------------------- /blog/building_service_mesh/redis/controller.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # "optimized" for bash/Ubuntu 4 | 5 | set -x 6 | #VERBOSE="--verbose" 7 | 8 | LOGFILE=/tmp/controller.log 9 | 10 | declare -A ARGS 11 | ARGS["SIDECAR_FOR"]="" 12 | 13 | ERROR="false" 14 | for arg in "$@" 15 | do 16 | case ${arg} in 17 | -sidecar-for=*) 18 | ARGS["SIDECAR_FOR"]=${arg##*=} 19 | shift 20 | ;; 21 | *) 22 | echo "unknown argument: $arg" >>$LOGFILE 23 | ERROR="true" 24 | ;; 25 | esac 26 | done 27 | 28 | if [ -z "${ARGS["SIDECAR_FOR"]}" ]; then 29 | echo "Need at least one argument: \"-sidecar-for=\"" 30 | exit 1 31 | fi 32 | 33 | if [ ${ERROR} = "true" ]; then 34 | cat $LOGFILE 35 | exit 1 36 | fi 37 | 38 | # HAProxy Enterprise related variables 39 | #CONF_TMP_DIR=/tmp/cfg 40 | #CONF_PROD_DIR=/etc/hapee-1.8 41 | #CONF_CERTS_DIRNAME=certs 42 | #CONF_HAP_FILENAME=hapee-lb.cfg 43 | #HAP_BINARY_PATH=/opt/hapee-1.8/sbin/ 44 | #HAP_BINARY_NAME=hapee-lb 45 | #HAP_BINARY=${HAP_BINARY_PATH}/${HAP_BINARY_NAME} 46 | #HAP_PIDFILE=/var/run/hapee-1.8/hapee-lb.pid 47 | #HAP_SOCKET=/var/run/hapee-1.8/hapee-lb.sock 48 | #HAP_USER=hapee-lb 49 | #HAP_GROUP=hapee 50 | #HAP_MODULES="module-path /opt/hapee-1.8/modules" 51 | 52 | # HAPCE alpine 53 | CONF_TMP_DIR=/tmp/cfg 54 | CONF_PROD_DIR=/etc/haproxy 55 | CONF_CERTS_DIRNAME=certs 56 | CONF_HAP_FILENAME=haproxy.cfg 57 | HAP_BINARY_PATH=/usr/sbin 58 | HAP_BINARY_NAME=haproxy 59 | HAP_BINARY=${HAP_BINARY_PATH}/${HAP_BINARY_NAME} 60 | HAP_PIDFILE=/var/run/haproxy/haproxy.pid 61 | HAP_SOCKET=/var/run/haproxy/haproxy.sock 62 | HAP_USER=nobody 63 | HAP_GROUP=nobody 64 | HAP_MODULES= 65 | 66 | # common URLs 67 | CURL='curl --silent ' 68 | CONSUL_SERVICE_URL="http://localhost:8500/v1/agent/service/${ARGS["SIDECAR_FOR"]}-sidecar-proxy" 69 | CONSUL_ROOTCA_URL="http://localhost:8500/v1/agent/connect/ca/roots" 70 | CONSUL_LEAF_CERT_URL="http://localhost:8500/v1/agent/connect/ca/leaf" 71 | 72 | # this function triggers a reload of HAProxy 73 | function do-reload() { 74 | pidof ${HAP_BINARY_NAME} 75 | CHECK=$? 76 | if [ $CHECK -eq 1 ]; then 77 | echo "Starting required" >>$LOGFILE 78 | ${HAP_BINARY} -f ${CONF_PROD_DIR}/${CONF_HAP_FILENAME} -W 2>>$LOGFILE 79 | else 80 | echo "Reload required" >>$LOGFILE 81 | kill -SIGUSR2 $(cat ${HAP_PIDFILE}) 82 | fi 83 | } 84 | 85 | # this function checks haproxy's configuration file integrity (including certificates) 86 | # it returns 1 in case of error and 0 otherwise 87 | function do-check() { 88 | ${HAP_BINARY} -f ${CONF_TMP_DIR}/${CONF_HAP_FILENAME} -c 89 | return $? 90 | } 91 | 92 | # this function move haproxy's configuration from a temporary dir to the production one 93 | function move-cfg() { 94 | SRC=$1 95 | DST=$2 96 | 97 | [ -z "${SRC}" ] && return 1 98 | [ -z "${DST}" ] && return 1 99 | 100 | #mv -f ${DST} ${DST}.old 2>>$LOGFILE 101 | mv -f ${SRC}/${CONF_HAP_FILENAME} ${DST}/${CONF_HAP_FILENAME} 2>>$LOGFILE 102 | [ ! -d ${DST}/${CONF_CERTS_DIRNAME} ] && mkdir -p ${DST}/${CONF_CERTS_DIRNAME} 103 | mv -f ${SRC}/${CONF_CERTS_DIRNAME}/* ${DST}/${CONF_CERTS_DIRNAME}/ 2>>$LOGFILE 104 | sed -i -e "s!${SRC}!${DST}!g" ${DST}/${CONF_HAP_FILENAME} 105 | 106 | return 0 107 | } 108 | 109 | ### CERTIFICATES ### 110 | declare -A CERTS 111 | 112 | # this function watch for LEAF certificates changes and update local storage 113 | # when required. 114 | # it returns an integer with the number of certs that has been updated locally. 115 | function check-certs() { 116 | local ret=0 117 | 118 | for cert in "${!CERTS[@]}" 119 | do 120 | CONSUL_LEAFCERT_JSON=$(${CURL} ${CONSUL_LEAF_CERT_URL}/${CERTNAME} 2>>$LOGFILE) 121 | SERIAL=$(jq -r .SerialNumber <<<${CONSUL_LEAFCERT_JSON}) 122 | if [ "${CERTS[$cert]}" != "$SERIAL" ] || [ ! -s "${CONF_PROD_DIR}/${CONF_CERTS_DIRNAME}/${cert}.pem" ]; then 123 | save-leaf-cert $cert ${CONF_PROD_DIR}/${CONF_CERTS_DIRNAME} 124 | ret+=1 125 | fi 126 | done 127 | 128 | return $ret 129 | } 130 | 131 | function save-leaf-cert() { 132 | CERTNAME=$1 133 | DESTINATION_DIR=$2 134 | 135 | [ -z "${CERTNAME}" ] && return 0 136 | [ -z "${DESTINATION_DIR}" ] && return 0 137 | 138 | CONSUL_LEAFCERT_JSON=$(${CURL} ${CONSUL_LEAF_CERT_URL}/${CERTNAME} 2>>$LOGFILE) 139 | SERIAL=$(jq -r .SerialNumber <<<${CONSUL_LEAFCERT_JSON}) 140 | [ $? -ne 0 ] && return 1 141 | jq -r .CertPEM <<<${CONSUL_LEAFCERT_JSON} > ${DESTINATION_DIR}/${CERTNAME}.pem 142 | jq -r .PrivateKeyPEM <<<${CONSUL_LEAFCERT_JSON} >> ${DESTINATION_DIR}/${CERTNAME}.pem 143 | sed -i -e 's/\\n/\n/g' -e '/^$/d' ${DESTINATION_DIR}/${CERTNAME}.pem 144 | 145 | # save cert serial in the list 146 | CERTS[${CERTNAME}]=$SERIAL 147 | 148 | return 0 149 | } 150 | ### END OF CERTIFICATES ### 151 | 152 | 153 | ### CONFIG ### 154 | declare -A CONFIG 155 | # initialize a new CONFIG array 156 | function config-init() { 157 | # global variables 158 | CONFIG["TARGET_SERVICE_NAME"]=${ARGS["SIDECAR_FOR"]} 159 | CONFIG["PIDFILE"]=${HAP_PIDFILE} 160 | CONFIG["STATS_SOCKET"]=${HAP_SOCKET} 161 | CONFIG["CERTS_FOLDER"]="${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME}" 162 | CONFIG["ACL_UNSECURED"]="acl UNSECURED dst_port 0" 163 | CONFIG["USER"]=${HAP_USER} 164 | CONFIG["GROUP"]=${HAP_GROUP} 165 | CONFIG["MODULES"]=${HAP_MODULES} 166 | 167 | # mandatory parameters provided by the consul API 168 | CONFIG["BIND_ADDRESS"]="" 169 | CONFIG["BIND_PORT"]="" 170 | CONFIG["LOCAL_SERVICE_ADDRESS"]="" 171 | CONFIG["LOCAL_SERVICE_PORT"]="" 172 | 173 | # custom parameters 174 | CONFIG["UNSECURED_BIND_PORT"]="" 175 | CONFIG["LOCAL_SERVICE_MODE"]="tcp" 176 | CONFIG["LOCAL_BACKEND_MODE"]="mode tcp" 177 | CONFIG["SYSLOG_SERVER"]="" 178 | CONFIG["LOG_FORMAT"]="option tcplog" 179 | CONFIG["DEFAULTS_LOG"]="" 180 | } 181 | 182 | # collect data from consul to build the configuration 183 | function config-getdata() { 184 | CONSUL_PROXY_JSON=$($CURL ${CONSUL_SERVICE_URL} 2>>$LOGFILE) 185 | echo $CONSUL_PROXY_JSON >>$LOGFILE 186 | 187 | CONFIG["BIND_ADDRESS"]=$(jq -r .Address <<<${CONSUL_PROXY_JSON}) 188 | CONFIG["BIND_PORT"]=$(jq -r .Port <<<${CONSUL_PROXY_JSON}) 189 | CONFIG["LOCAL_SERVICE_ADDRESS"]=$(jq -r .Proxy.LocalServiceAddress <<<${CONSUL_PROXY_JSON}) 190 | CONFIG["LOCAL_SERVICE_PORT"]=$(jq -r .Proxy.LocalServicePort <<<${CONSUL_PROXY_JSON}) 191 | 192 | # custom parameters 193 | #CONFIG["AUTHORIZATIONFREE_BIND_PORT=$(jq -r .Config.authorizationfree_bind_port <<<${CONSUL_PROXY_JSON}) 194 | #if [ "${AUTHORIZATIONFREE_BIND_PORT}" = "null" ]; then 195 | # AUTHORIZATIONFREE_BIND_PORT= 196 | #fi 197 | UNSECURED_BIND_PORT=$(jq -r .Proxy.Config.unsecured_bind_port <<<${CONSUL_PROXY_JSON}) 198 | if [ "${UNSECURED_BIND_PORT}" != "null" ]; then 199 | CONFIG["UNSECURED_BIND_PORT"]="bind :${UNSECURED_BIND_PORT} name unsecured" 200 | CONFIG["ACL_UNSECURED"]="${CONFIG["ACL_UNSECURED"]} ${UNSECURED_BIND_PORT}" 201 | fi 202 | 203 | LOCAL_SERVICE_MODE=$(jq -r .Proxy.Config.local_service_mode <<<${CONSUL_PROXY_JSON}) 204 | case ${LOCAL_SERVICE_MODE} in 205 | http) 206 | CONFIG["LOCAL_SERVICE_MODE"]="mode http" 207 | CONFIG["LOCAL_BACKEND_MODE"]="mode http" 208 | CONFIG["LOG_FORMAT"]="log-format \"%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs {%[var(txn.SpiffeUrl)]|%[var(txn.Authorized)]|%[var(txn.Reason)]} {%sslv|%sslc|%[ssl_fc_session_id,hex]|%[ssl_fc_is_resumed]|%[ssl_fc_sni]} %{+Q}r\"" 209 | ;; 210 | tcp|*) 211 | CONFIG["LOCAL_SERVICE_MODE"]="mode tcp" 212 | CONFIG["LOCAL_BACKEND_MODE"]="mode tcp" 213 | CONFIG["LOG_FORMAT"]="log-format \"%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq {%[var(txn.SpiffeUrl)]|%[var(txn.Authorized)]|%[var(txn.Reason)]} {%sslv|%sslc|%[ssl_fc_session_id,hex]|%[ssl_fc_is_resumed]|%[ssl_fc_sni]}\"" 214 | ;; 215 | esac 216 | CONFIG["LOCAL_SERVICE_MODE"]="${CONFIG["LOCAL_SERVICE_MODE"]} 217 | tcp-request inspect-delay 1s 218 | tcp-request content lua.authorize unless UNSECURED 219 | tcp-request content capture var(txn.SpiffeUrl) len 64 if { var(txn.SpiffeUrl) -m found } 220 | tcp-request content capture var(txn.Authorized) len 64 if { var(txn.Authorized) -m found } 221 | tcp-request content capture var(txn.Reason) len 64 if { var(txn.Reason) -m found } 222 | tcp-request content reject unless UNSECURED or { var(txn.Authorized) -m str true } 223 | " 224 | SYSLOG_SERVER=$(jq -r .Proxy.Config.syslog_server <<<${CONSUL_PROXY_JSON}) 225 | if [ "${SYSLOG_SERVER}" != "null" ]; then 226 | CONFIG["SYSLOG_SERVER"]="log ${SYSLOG_SERVER} local0 info 227 | log-tag sidecar-for-${ARGS["SIDECAR_FOR"]}" 228 | CONFIG["DEFAULTS_LOG"]="log global 229 | option httplog" 230 | fi 231 | } 232 | 233 | # write HAProxy Configuration 234 | function config-write() { 235 | cat <${CONF_TMP_DIR}/${CONF_HAP_FILENAME} 236 | global 237 | maxconn 10000 238 | #user ${CONFIG["USER"]} # useless cause we're already started as ${CONFIG["USER"]} user 239 | #group ${CONFIG["GROUP"]} # useless cause we're already started as ${CONFIG["GROUP"]} user 240 | #chroot /var/empty 241 | daemon 242 | tune.ssl.default-dh-param 1024 243 | pidfile ${CONFIG["PIDFILE"]} 244 | stats socket ${CONFIG["STATS_SOCKET"]} user ${CONFIG["USER"]} group ${CONFIG["GROUP"]} mode 660 level admin 245 | stats timeout 10m 246 | ca-base ${CONFIG["CERTS_FOLDER"]} 247 | crt-base ${CONFIG["CERTS_FOLDER"]} 248 | ${CONFIG["MODULES"]} 249 | lua-load /authorize.lua 250 | ${CONFIG["SYSLOG_SERVER"]} 251 | 252 | resolvers consul 253 | nameserver consul 127.0.0.1:8600 254 | 255 | defaults 256 | timeout client 30s 257 | timeout connect 250ms 258 | timeout server 30s 259 | option redispatch 1 260 | retries 3 261 | option socket-stats 262 | option tcplog 263 | option dontlognull 264 | option dontlog-normal 265 | default-server init-addr none 266 | ${CONFIG["DEFAULTS_LOG"]} 267 | 268 | frontend f_stats 269 | bind 0.0.0.0:1936 270 | mode http 271 | http-request set-log-level silent 272 | stats enable 273 | stats uri / 274 | stats show-legends 275 | stats show-desc ${CONFIG["TARGET_SERVICE_NAME"]} on ${CONFIG["PROXY_ID"]} 276 | option httpclose 277 | 278 | # Proxied (local) service 279 | frontend f_${CONFIG["TARGET_SERVICE_NAME"]} 280 | bind :${CONFIG["BIND_PORT"]} ssl ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem verify required name secured 281 | ${CONFIG["AUTHORIZATIONFREE_BIND_PORT"]} 282 | ${CONFIG["UNSECURED_BIND_PORT"]} 283 | ${CONFIG["ACL_UNSECURED"]} 284 | ${CONFIG["LOCAL_SERVICE_MODE"]} 285 | default_backend b_${CONFIG["TARGET_SERVICE_NAME"]} 286 | 287 | backend b_${CONFIG["TARGET_SERVICE_NAME"]} 288 | ${CONFIG["LOCAL_BACKEND_MODE"]} 289 | server localhost ${CONFIG["LOCAL_SERVICE_ADDRESS"]}:${CONFIG["LOCAL_SERVICE_PORT"]} check 290 | 291 | # upstreams: 292 | EOF 293 | } 294 | 295 | # dump content of the CONFIG variable 296 | function config-dump() { 297 | for a in ${!CONFIG[@]} 298 | do 299 | echo "$a: ${CONFIG[$a]}" >>$LOGFILE 300 | done 301 | } 302 | ### END OF CONFIG ### 303 | 304 | 305 | ### UPSTREAM ### 306 | declare -A UPSTREAM 307 | function upstream-init() { 308 | # mandatory parameters provided by the consul API 309 | UPSTREAM["DESTINATION_NAME"]="" 310 | UPSTREAM["DESTINATION_SERVERS"]="" 311 | UPSTREAM["LOCAL_BIND_ADDRESS"]="" 312 | UPSTREAM["LOCAL_BIND_PORT"]="" 313 | 314 | # custom parameters 315 | UPSTREAM["DESTINATION_MODE"]="mode tcp" 316 | UPSTREAM["LOG_FORMAT"]="option tcplog" 317 | UPSTREAM["ADVANCED_CHECK"]="option tcp-check 318 | tcp-check connect ssl" 319 | } 320 | 321 | function upstream-getdata() { 322 | # mandatory parameters provided by the consul API 323 | UPSTREAM["DESTINATION_NAME"]=$(jq -r .DestinationName <<<${1}) 324 | UPSTREAM["LOCAL_BIND_ADDRESS"]=$(jq -r .LocalBindAddress <<<${1}) 325 | UPSTREAM["LOCAL_BIND_PORT"]=$(jq -r .LocalBindPort <<<${1}) 326 | DESTINATION_TYPE=$(jq -r .DestinationType <<<${1}) 327 | case ${DESTINATION_TYPE} in 328 | "service") 329 | UPSTREAM["DESTINATION_SERVERS"]="server-template s_${UPSTREAM["DESTINATION_NAME"]} 20 _${UPSTREAM["DESTINATION_NAME"]}._tcp.service.consul ssl resolvers consul ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem check #no-tls-tickets" 330 | ;; 331 | "connect") 332 | # FIXME: use SRV records instead of catalog API 333 | CONSUL_CATALOG_URL="http://localhost:8500/v1/catalog/service/${UPSTREAM["DESTINATION_NAME"]}-sidecar-proxy" 334 | CATALOG_UPSTREAM_JSON=$($CURL ${CONSUL_CATALOG_URL} 2>>$LOGFILE) 335 | 336 | for row in $(jq -c .[] <<<${CATALOG_UPSTREAM_JSON}) 337 | do 338 | UPSTREAM["SERVICE_PORT"]=$(jq -r .ServicePort <<<${row}) 339 | UPSTREAM["DESTINATION_SERVERS"]="server-template s_${UPSTREAM["DESTINATION_NAME"]} 20 ${UPSTREAM["DESTINATION_NAME"]}.connect.consul:${UPSTREAM["SERVICE_PORT"]} ssl resolvers consul ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem check #no-tls-tickets" 340 | done 341 | ;; 342 | esac 343 | 344 | # custom parameters 345 | DESTINATION_MODE=$(jq -r .destination_mode <<<${1}) 346 | if [ "${DESTINATION_MODE}" = "http" ]; then 347 | UPSTREAM["DESTINATION_MODE"]="mode http" 348 | UPSTREAM["LOG_FORMAT"]="option httplog" 349 | fi 350 | 351 | ADVANCED_CHECK=$(jq -r .advanced_check <<<${1}) 352 | case ${ADVANCED_CHECK} in 353 | redis) 354 | UPSTREAM["ADVANCED_CHECK"]="option redis-check" 355 | ;; 356 | esac 357 | } 358 | 359 | # write the upstream configuration into the config file 360 | function upstream-write() { 361 | cat <>${CONF_TMP_DIR}/${CONF_HAP_FILENAME} 362 | 363 | frontend f_${UPSTREAM["DESTINATION_NAME"]} 364 | bind ${UPSTREAM["LOCAL_BIND_ADDRESS"]}:${UPSTREAM["LOCAL_BIND_PORT"]} 365 | ${UPSTREAM["DESTINATION_MODE"]} 366 | ${UPSTREAM["LOG_FORMAT"]} 367 | default_backend b_${UPSTREAM["DESTINATION_NAME"]} 368 | 369 | backend b_${UPSTREAM["DESTINATION_NAME"]} 370 | ${UPSTREAM["DESTINATION_MODE"]} 371 | ${UPSTREAM["ADVANCED_CHECK"]} 372 | ${UPSTREAM["DESTINATION_SERVERS"]} 373 | EOF 374 | } 375 | 376 | # dump content of the UPSTREAM variable 377 | function upstream-dump() { 378 | for a in ${!UPSTREAM[@]} 379 | do 380 | echo "$a: ${UPSTREAM[$a]}" >>$LOGFILE 381 | done 382 | } 383 | ### END OF UPSTREAM ### 384 | 385 | 386 | ### MAIN ### 387 | 388 | # this function generates an HAProxy configuration in $CONF_TMP_DIR and returns: 389 | # 0 if no error happened 390 | # 1 if anything was wrong 391 | function generate-cfg() { 392 | echo "GENERATING NEW CONFIGURATION" >>$LOGFILE 393 | 394 | # clear CERTS array, to ensure we monitor only certs we need to 395 | for cert in "${!CERTS[@]}" 396 | do 397 | unset -v CERTS[$cert] 398 | done 399 | 400 | config-init 401 | config-getdata 402 | config-dump 403 | 404 | ERROR="false" 405 | echo "" >>$LOGFILE 406 | [ -z "${CONF_TMP_DIR}" ] && return 1 407 | rm -rf ${CONF_TMP_DIR} 408 | mkdir -p ${CONF_TMP_DIR} 409 | mkdir -p ${CONFIG["CERTS_FOLDER"]} 410 | 411 | # get cert and key for target service 412 | save-leaf-cert ${CONFIG["TARGET_SERVICE_NAME"]} ${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME} 413 | [ $? -ne 0 ] && return 1 414 | 415 | # get CA root certificate 416 | ${CURL} ${CONSUL_ROOTCA_URL} | jq -r .Roots[0].RootCert | sed -e 's/\\n/\n/g' > ${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME}/ca.pem 417 | 418 | # Write configuration to file 419 | config-write 420 | 421 | # configuring upstreams 422 | for row in $(jq -c .Proxy.Upstreams[] <<<${CONSUL_PROXY_JSON}) 423 | do 424 | upstream-init 425 | upstream-getdata "${row}" 426 | upstream-dump 427 | upstream-write 428 | done 429 | 430 | return 0 431 | } 432 | 433 | 434 | # blocking loop wait for an update on the connect proxy configuration piece 435 | HASH="0" 436 | OLDHASH="0" 437 | while true; 438 | do 439 | TMPFILE=$(mktemp) 440 | ${CURL} --include --max-time 10 ${VERBOSE} --output ${TMPFILE} ${CONSUL_SERVICE_URL}?hash=${HASH} >${TMPFILE} 441 | 442 | RETCODE=$? 443 | # 28 is the return code when curl wait for 'max-time' 444 | if [ ${RETCODE} -eq 28 ]; then 445 | sleep 1 446 | rm ${TMPFILE} 447 | check-certs 448 | [ $? -gt 0 ] && do-reload 449 | continue 450 | fi 451 | # any other code than 0 may indicates an error, so we don't want to apply anything 452 | if [ ${RETCODE} -ne 0 ]; then 453 | sleep 1 454 | rm ${TMPFILE} 455 | continue 456 | fi 457 | 458 | STATUSCODE=$(head -n 1 ${TMPFILE} | cut -d' ' -f 2) 459 | if [ "${STATUSCODE}" != "200" ]; then 460 | sleep 1 461 | OLDHASH=${HASH} 462 | rm ${TMPFILE} 463 | continue 464 | fi 465 | 466 | HASH=$(sed -n 's/^X-Consul-Contenthash: \(.*\)\r/\1/p' ${TMPFILE}) 467 | if [ -z "${HASH}" ]; then 468 | sleep 1 469 | rm ${TMPFILE} 470 | continue 471 | fi 472 | 473 | # the configuration has not changed 474 | if [ "${OLDHASH}" = "${HASH}" ]; then 475 | sleep 1 476 | rm ${TMPFILE} 477 | continue 478 | fi 479 | 480 | rm ${TMPFILE} 481 | 482 | generate-cfg 483 | if [ $? -ne 0 ]; then 484 | sleep 1 485 | HASH="foo" 486 | continue 487 | fi 488 | do-check 489 | [ $? -ne 0 ] && continue 490 | move-cfg ${CONF_TMP_DIR} ${CONF_PROD_DIR} 491 | [ $? -ne 0 ] && continue 492 | 493 | echo "APPLYING NEW CONFIGURATION" >>$LOGFILE 494 | do-reload 495 | 496 | sleep 1 497 | done 498 | 499 | -------------------------------------------------------------------------------- /blog/building_service_mesh/redis/start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | set -x 5 | 6 | redis-server & 7 | 8 | MYIP=$(hostname -i) 9 | 10 | [ -z "${SERVICENAME}" ] && exit 1 11 | 12 | cat </etc/consul.d/services.json 13 | { 14 | "service": { 15 | "name": "${SERVICENAME}", 16 | "port": 6379, 17 | "connect": { 18 | "sidecar_service": { 19 | "proxy": { 20 | "upstreams": [], 21 | "config": {} 22 | } 23 | } 24 | } 25 | }, 26 | "acl_datacenter":"dc1", 27 | "acl_default_policy":"deny", 28 | "acl_token":"agenttoken" 29 | } 30 | EOF 31 | 32 | cat /etc/consul.d/services.json 33 | 34 | /controller.sh -sidecar-for=${SERVICENAME} & 35 | 36 | exec su -l nobody -s /bin/bash -c "consul agent -data-dir=/tmp/consul -node=$HOSTNAME -node-id=$(uuidgen) -bind=0.0.0.0 \ 37 | -enable-script-checks \ 38 | -config-dir=/etc/consul.d -retry-join consul-server" 39 | 40 | -------------------------------------------------------------------------------- /blog/building_service_mesh/www/.dockerignore: -------------------------------------------------------------------------------- 1 | Dockerfile -------------------------------------------------------------------------------- /blog/building_service_mesh/www/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM alpine:latest 2 | 3 | ARG CONSUL_VER= 4 | 5 | # consul related 6 | RUN echo "Installing consul v${CONSUL_VER}" \ 7 | && apk add --no-cache unzip openssl util-linux su-exec curl jq drill \ 8 | && mkdir /usr/src \ 9 | && cd /usr/src \ 10 | && wget -q https://releases.hashicorp.com/consul/${CONSUL_VER}/consul_${CONSUL_VER}_linux_amd64.zip \ 11 | && unzip -o consul_${CONSUL_VER}_linux_amd64.zip \ 12 | && mv consul /usr/bin/ \ 13 | && mkdir /tmp/consul /etc/consul.d \ 14 | && chown nobody:nobody /tmp/consul \ 15 | && rm -rf /usr/src/* \ 16 | && apk del unzip openssl 17 | 18 | # haproxy related 19 | RUN echo "@edge http://nl.alpinelinux.org/alpine/edge/main" >> /etc/apk/repositories \ 20 | && apk add --no-cache haproxy@edge lua5.3-ossl@edge lua5.3-socket@edge lua5.3-cjson@edge openssl@edge \ 21 | && apk add --no-cache socat iproute2 bash \ 22 | && chown -R nobody:nobody /etc/haproxy \ 23 | && mkdir /var/run/haproxy \ 24 | && chown nobody:nobody /var/run/haproxy 25 | 26 | # application related 27 | RUN mkdir -p /www \ 28 | && apk add npm \ 29 | && apk add --no-cache nodejs 30 | COPY app.js package.json /www/ 31 | 32 | # entrypoint 33 | WORKDIR / 34 | COPY start.sh controller.sh authorize.lua / 35 | RUN chmod +x start.sh 36 | ENTRYPOINT [ "./start.sh" ] 37 | -------------------------------------------------------------------------------- /blog/building_service_mesh/www/app.js: -------------------------------------------------------------------------------- 1 | const express = require('express') 2 | const bodyParser = require('body-parser') 3 | const redis = require('redis') 4 | const app = express() 5 | const port = 8080 6 | const redisUrl = process.env.REDIS_URL 7 | app.use(bodyParser.urlencoded({extended: true})) 8 | 9 | app.get('/', function(req, res, next) { 10 | var redisClient = null 11 | 12 | var message = "" 13 | redisClient = redis.createClient({ 14 | host: redisUrl, 15 | retry_strategy: function(args) { 16 | return undefined 17 | } 18 | }) 19 | 20 | redisClient.on("error", function(err) { 21 | console.log("Error: " + err) 22 | message = err 23 | res.send(` 24 |

App served by Consul and HAProxy!

25 |
26 | 27 | 28 | 29 |
30 |

Message: ${message}

`) 31 | next() 32 | }) 33 | 34 | redisClient.get('message', function(err, reply) { 35 | if (err != null) { 36 | console.log("Error: " + err) 37 | } 38 | else { 39 | message = reply 40 | res.send(` 41 |

App served by Consul and HAProxy!

42 |
43 | 44 | 45 | 46 |
47 |

Message: ${message}

`) 48 | } 49 | }) 50 | 51 | redisClient.quit() 52 | }) 53 | 54 | app.post('/', function(req, res, next) { 55 | var redisClient = null 56 | var message = req.body.message 57 | 58 | redisClient = redis.createClient({ 59 | host: redisUrl, 60 | retry_strategy: function(args) { 61 | return undefined 62 | } 63 | }) 64 | 65 | redisClient.on("error", function(err) { 66 | console.log("Error: " + err) 67 | next() 68 | }) 69 | 70 | redisClient.set('message', message) 71 | redisClient.quit() 72 | res.redirect(303, '/') 73 | }) 74 | 75 | app.listen(port, function() { 76 | console.log(`Listening on port ${port}`) 77 | }) -------------------------------------------------------------------------------- /blog/building_service_mesh/www/authorize.lua: -------------------------------------------------------------------------------- 1 | -- 2 | -- authorize.lua 3 | -- 4 | -- Connection / request authorization when HAProxy is used as a "proxy" in 5 | -- a service mesh managed by consul connect 6 | -- 7 | -- 8 | -- Copyright (c) 2017-2018. Baptiste Assmann 9 | -- Copyright (c) 2017-2018. HAProxy Technologies, LLC. 10 | -- 11 | -- This program is free software: you can redistribute it and/or modify 12 | -- it under the terms of the GNU General Public License as published by 13 | -- the Free Software Foundation, version 2 of the License 14 | 15 | --package.path = package.path .. ';/opt/hapee-1.8/usr/lib/lua/?.lua' 16 | 17 | local http = require("socket.http") 18 | local ltn12 = require("ltn12") 19 | local json = require("cjson") 20 | local x509 = require("openssl.x509") 21 | 22 | cache = {} 23 | 24 | -- return true if the key is in the cache, return false otherwise 25 | local function checkCache(key) 26 | if cache[key] == nil then 27 | return false 28 | end 29 | return true 30 | end 31 | 32 | -- return true if the key in the cache has expired 33 | local function hasExpired(key) 34 | if not checkCache(key) then 35 | return true 36 | end 37 | 38 | local now = core.now()["sec"] 39 | 40 | if now - cache[key]["timestamp"] < 1 then 41 | return false 42 | end 43 | 44 | return true 45 | end 46 | 47 | local function setCache(spiffie_url, auth_status, reason) 48 | cache[spiffie_url] = {} 49 | cache[spiffie_url]["authorized"] = auth_status 50 | cache[spiffie_url]["reason"] = reason 51 | cache[spiffie_url]["timestamp"] = core.now()["sec"] 52 | end 53 | 54 | core.register_action("authorize", { "http-req", "tcp-req" }, function(txn) 55 | local serial = txn.f:ssl_c_serial() 56 | local der = txn.sf:ssl_c_der() 57 | local client_cert_url = '' 58 | local headers = {} 59 | local request_body = '' 60 | local response_body = {} 61 | local target = txn.sf:fe_name() 62 | 63 | -- this is due to a 'second' call, need to troubleshoot why 64 | -- this function is called twice in TCP mode 65 | if type(der) == 'nil' then 66 | txn.set_var(txn, "txn.SpiffeUrl", "-") 67 | txn.set_var(txn, "txn.Authorized", "false") 68 | txn.set_var(txn, "txn.Reason", "Error parsing client certificate") 69 | return 70 | end 71 | 72 | -- collecting client cert spiffe URL and client certificate serial 73 | local mycert = x509.new(der, "DER") 74 | if mycert ~= nil then 75 | local myext = mycert:getExtension('X509v3 Subject Alternative Name') 76 | client_cert_url = string.gsub(myext:text(), 'URI:', '') 77 | txn.set_var(txn, "txn.SpiffeUrl", client_cert_url) 78 | end 79 | serial = txn.c:hex(serial) 80 | 81 | if not hasExpired(client_cert_url) then 82 | txn.set_var(txn, "txn.Authorized", cache[client_cert_url]["authorized"]) 83 | txn.set_var(txn, "txn.Reason", cache[client_cert_url]["reason"]) 84 | return 85 | end 86 | 87 | -- prepate and run the HTTP request to consul connect 88 | -- update the target to remove the 'f_' prefix 89 | target = string.sub(target, 3) 90 | request_body = json.encode({ Target = target, ClientCertURI = client_cert_url, ClientCertSerial = serial }) 91 | headers['Content-Type'] = 'application/json' 92 | headers['Content-Length'] = #request_body 93 | 94 | local b, c, h = http.request { 95 | url = "http://127.0.0.1:8500/v1/agent/connect/authorize", 96 | create = core.tcp, 97 | method = "POST", 98 | headers = headers, 99 | source = ltn12.source.string(request_body), 100 | sink = ltn12.sink.table(response_body) 101 | } 102 | 103 | -- analyze the feedback from consul connect 104 | if c == nil or c ~= 200 then 105 | setCache(client_cert_url, "false", "Not authorized by consul") 106 | txn.set_var(txn, "txn.Authorized", "false") 107 | txn.set_var(txn, "txn.Reason", 'Not authorized by consul') 108 | return 109 | end 110 | 111 | local myjson = json.decode(table.concat(response_body)) 112 | if myjson['Authorized'] == nil or myjson['Authorized'] ~= true then 113 | txn.set_var(txn, "txn.Authorized", "false") 114 | if myjson['Reason'] ~= nil then 115 | setCache(client_cert_url, "false", myjson['Reason']) 116 | txn.set_var(txn, "txn.Reason", myjson['Reason']) 117 | else 118 | setCache(client_cert_url, "false", 'Not authorized by consul') 119 | txn.set_var(txn, "txn.Reason", 'Not authorized by consul') 120 | end 121 | return 122 | end 123 | 124 | setCache(client_cert_url, "true", myjson['Reason']) 125 | 126 | txn.set_var(txn, "txn.Authorized", "true") 127 | txn.set_var(txn, "txn.Reason", myjson['Reason']) 128 | end) 129 | -------------------------------------------------------------------------------- /blog/building_service_mesh/www/controller.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # "optimized" for bash/Ubuntu 4 | 5 | set -x 6 | #VERBOSE="--verbose" 7 | 8 | LOGFILE=/tmp/controller.log 9 | 10 | declare -A ARGS 11 | ARGS["SIDECAR_FOR"]="" 12 | 13 | ERROR="false" 14 | for arg in "$@" 15 | do 16 | case ${arg} in 17 | -sidecar-for=*) 18 | ARGS["SIDECAR_FOR"]=${arg##*=} 19 | shift 20 | ;; 21 | *) 22 | echo "unknown argument: $arg" >>$LOGFILE 23 | ERROR="true" 24 | ;; 25 | esac 26 | done 27 | 28 | if [ -z "${ARGS["SIDECAR_FOR"]}" ]; then 29 | echo "Need at least one argument: \"-sidecar-for=\"" 30 | exit 1 31 | fi 32 | 33 | if [ ${ERROR} = "true" ]; then 34 | cat $LOGFILE 35 | exit 1 36 | fi 37 | 38 | # HAProxy Enterprise related variables 39 | #CONF_TMP_DIR=/tmp/cfg 40 | #CONF_PROD_DIR=/etc/hapee-1.8 41 | #CONF_CERTS_DIRNAME=certs 42 | #CONF_HAP_FILENAME=hapee-lb.cfg 43 | #HAP_BINARY_PATH=/opt/hapee-1.8/sbin/ 44 | #HAP_BINARY_NAME=hapee-lb 45 | #HAP_BINARY=${HAP_BINARY_PATH}/${HAP_BINARY_NAME} 46 | #HAP_PIDFILE=/var/run/hapee-1.8/hapee-lb.pid 47 | #HAP_SOCKET=/var/run/hapee-1.8/hapee-lb.sock 48 | #HAP_USER=hapee-lb 49 | #HAP_GROUP=hapee 50 | #HAP_MODULES="module-path /opt/hapee-1.8/modules" 51 | 52 | # HAPCE alpine 53 | CONF_TMP_DIR=/tmp/cfg 54 | CONF_PROD_DIR=/etc/haproxy 55 | CONF_CERTS_DIRNAME=certs 56 | CONF_HAP_FILENAME=haproxy.cfg 57 | HAP_BINARY_PATH=/usr/sbin 58 | HAP_BINARY_NAME=haproxy 59 | HAP_BINARY=${HAP_BINARY_PATH}/${HAP_BINARY_NAME} 60 | HAP_PIDFILE=/var/run/haproxy/haproxy.pid 61 | HAP_SOCKET=/var/run/haproxy/haproxy.sock 62 | HAP_USER=nobody 63 | HAP_GROUP=nobody 64 | HAP_MODULES= 65 | 66 | # common URLs 67 | CURL='curl --silent ' 68 | CONSUL_SERVICE_URL="http://localhost:8500/v1/agent/service/${ARGS["SIDECAR_FOR"]}-sidecar-proxy" 69 | CONSUL_ROOTCA_URL="http://localhost:8500/v1/agent/connect/ca/roots" 70 | CONSUL_LEAF_CERT_URL="http://localhost:8500/v1/agent/connect/ca/leaf" 71 | 72 | # this function triggers a reload of HAProxy 73 | function do-reload() { 74 | pidof ${HAP_BINARY_NAME} 75 | CHECK=$? 76 | if [ $CHECK -eq 1 ]; then 77 | echo "Starting required" >>$LOGFILE 78 | ${HAP_BINARY} -f ${CONF_PROD_DIR}/${CONF_HAP_FILENAME} -W 2>>$LOGFILE 79 | else 80 | echo "Reload required" >>$LOGFILE 81 | kill -SIGUSR2 $(cat ${HAP_PIDFILE}) 82 | fi 83 | } 84 | 85 | # this function checks haproxy's configuration file integrity (including certificates) 86 | # it returns 1 in case of error and 0 otherwise 87 | function do-check() { 88 | ${HAP_BINARY} -f ${CONF_TMP_DIR}/${CONF_HAP_FILENAME} -c 89 | return $? 90 | } 91 | 92 | # this function move haproxy's configuration from a temporary dir to the production one 93 | function move-cfg() { 94 | SRC=$1 95 | DST=$2 96 | 97 | [ -z "${SRC}" ] && return 1 98 | [ -z "${DST}" ] && return 1 99 | 100 | #mv -f ${DST} ${DST}.old 2>>$LOGFILE 101 | mv -f ${SRC}/${CONF_HAP_FILENAME} ${DST}/${CONF_HAP_FILENAME} 2>>$LOGFILE 102 | [ ! -d ${DST}/${CONF_CERTS_DIRNAME} ] && mkdir -p ${DST}/${CONF_CERTS_DIRNAME} 103 | mv -f ${SRC}/${CONF_CERTS_DIRNAME}/* ${DST}/${CONF_CERTS_DIRNAME}/ 2>>$LOGFILE 104 | sed -i -e "s!${SRC}!${DST}!g" ${DST}/${CONF_HAP_FILENAME} 105 | 106 | return 0 107 | } 108 | 109 | ### CERTIFICATES ### 110 | declare -A CERTS 111 | 112 | # this function watch for LEAF certificates changes and update local storage 113 | # when required. 114 | # it returns an integer with the number of certs that has been updated locally. 115 | function check-certs() { 116 | local ret=0 117 | 118 | for cert in "${!CERTS[@]}" 119 | do 120 | CONSUL_LEAFCERT_JSON=$(${CURL} ${CONSUL_LEAF_CERT_URL}/${CERTNAME} 2>>$LOGFILE) 121 | SERIAL=$(jq -r .SerialNumber <<<${CONSUL_LEAFCERT_JSON}) 122 | if [ "${CERTS[$cert]}" != "$SERIAL" ] || [ ! -s "${CONF_PROD_DIR}/${CONF_CERTS_DIRNAME}/${cert}.pem" ]; then 123 | save-leaf-cert $cert ${CONF_PROD_DIR}/${CONF_CERTS_DIRNAME} 124 | ret+=1 125 | fi 126 | done 127 | 128 | return $ret 129 | } 130 | 131 | function save-leaf-cert() { 132 | CERTNAME=$1 133 | DESTINATION_DIR=$2 134 | 135 | [ -z "${CERTNAME}" ] && return 0 136 | [ -z "${DESTINATION_DIR}" ] && return 0 137 | 138 | CONSUL_LEAFCERT_JSON=$(${CURL} ${CONSUL_LEAF_CERT_URL}/${CERTNAME} 2>>$LOGFILE) 139 | SERIAL=$(jq -r .SerialNumber <<<${CONSUL_LEAFCERT_JSON}) 140 | [ $? -ne 0 ] && return 1 141 | jq -r .CertPEM <<<${CONSUL_LEAFCERT_JSON} > ${DESTINATION_DIR}/${CERTNAME}.pem 142 | jq -r .PrivateKeyPEM <<<${CONSUL_LEAFCERT_JSON} >> ${DESTINATION_DIR}/${CERTNAME}.pem 143 | sed -i -e 's/\\n/\n/g' -e '/^$/d' ${DESTINATION_DIR}/${CERTNAME}.pem 144 | 145 | # save cert serial in the list 146 | CERTS[${CERTNAME}]=$SERIAL 147 | 148 | return 0 149 | } 150 | ### END OF CERTIFICATES ### 151 | 152 | 153 | ### CONFIG ### 154 | declare -A CONFIG 155 | # initialize a new CONFIG array 156 | function config-init() { 157 | # global variables 158 | CONFIG["TARGET_SERVICE_NAME"]=${ARGS["SIDECAR_FOR"]} 159 | CONFIG["PIDFILE"]=${HAP_PIDFILE} 160 | CONFIG["STATS_SOCKET"]=${HAP_SOCKET} 161 | CONFIG["CERTS_FOLDER"]="${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME}" 162 | CONFIG["ACL_UNSECURED"]="acl UNSECURED dst_port 0" 163 | CONFIG["USER"]=${HAP_USER} 164 | CONFIG["GROUP"]=${HAP_GROUP} 165 | CONFIG["MODULES"]=${HAP_MODULES} 166 | 167 | # mandatory parameters provided by the consul API 168 | CONFIG["BIND_ADDRESS"]="" 169 | CONFIG["BIND_PORT"]="" 170 | CONFIG["LOCAL_SERVICE_ADDRESS"]="" 171 | CONFIG["LOCAL_SERVICE_PORT"]="" 172 | 173 | # custom parameters 174 | CONFIG["UNSECURED_BIND_PORT"]="" 175 | CONFIG["LOCAL_SERVICE_MODE"]="tcp" 176 | CONFIG["LOCAL_BACKEND_MODE"]="mode tcp" 177 | CONFIG["SYSLOG_SERVER"]="" 178 | CONFIG["LOG_FORMAT"]="option tcplog" 179 | CONFIG["DEFAULTS_LOG"]="" 180 | } 181 | 182 | # collect data from consul to build the configuration 183 | function config-getdata() { 184 | CONSUL_PROXY_JSON=$($CURL ${CONSUL_SERVICE_URL} 2>>$LOGFILE) 185 | echo $CONSUL_PROXY_JSON >>$LOGFILE 186 | 187 | CONFIG["BIND_ADDRESS"]=$(jq -r .Address <<<${CONSUL_PROXY_JSON}) 188 | CONFIG["BIND_PORT"]=$(jq -r .Port <<<${CONSUL_PROXY_JSON}) 189 | CONFIG["LOCAL_SERVICE_ADDRESS"]=$(jq -r .Proxy.LocalServiceAddress <<<${CONSUL_PROXY_JSON}) 190 | CONFIG["LOCAL_SERVICE_PORT"]=$(jq -r .Proxy.LocalServicePort <<<${CONSUL_PROXY_JSON}) 191 | 192 | # custom parameters 193 | #CONFIG["AUTHORIZATIONFREE_BIND_PORT=$(jq -r .Config.authorizationfree_bind_port <<<${CONSUL_PROXY_JSON}) 194 | #if [ "${AUTHORIZATIONFREE_BIND_PORT}" = "null" ]; then 195 | # AUTHORIZATIONFREE_BIND_PORT= 196 | #fi 197 | UNSECURED_BIND_PORT=$(jq -r .Proxy.Config.unsecured_bind_port <<<${CONSUL_PROXY_JSON}) 198 | if [ "${UNSECURED_BIND_PORT}" != "null" ]; then 199 | CONFIG["UNSECURED_BIND_PORT"]="bind :${UNSECURED_BIND_PORT} name unsecured" 200 | CONFIG["ACL_UNSECURED"]="${CONFIG["ACL_UNSECURED"]} ${UNSECURED_BIND_PORT}" 201 | fi 202 | 203 | LOCAL_SERVICE_MODE=$(jq -r .Proxy.Config.local_service_mode <<<${CONSUL_PROXY_JSON}) 204 | case ${LOCAL_SERVICE_MODE} in 205 | http) 206 | CONFIG["LOCAL_SERVICE_MODE"]="mode http" 207 | CONFIG["LOCAL_BACKEND_MODE"]="mode http" 208 | CONFIG["LOG_FORMAT"]="log-format \"%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs {%[var(txn.SpiffeUrl)]|%[var(txn.Authorized)]|%[var(txn.Reason)]} {%sslv|%sslc|%[ssl_fc_session_id,hex]|%[ssl_fc_is_resumed]|%[ssl_fc_sni]} %{+Q}r\"" 209 | ;; 210 | tcp|*) 211 | CONFIG["LOCAL_SERVICE_MODE"]="mode tcp" 212 | CONFIG["LOCAL_BACKEND_MODE"]="mode tcp" 213 | CONFIG["LOG_FORMAT"]="log-format \"%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq {%[var(txn.SpiffeUrl)]|%[var(txn.Authorized)]|%[var(txn.Reason)]} {%sslv|%sslc|%[ssl_fc_session_id,hex]|%[ssl_fc_is_resumed]|%[ssl_fc_sni]}\"" 214 | ;; 215 | esac 216 | CONFIG["LOCAL_SERVICE_MODE"]="${CONFIG["LOCAL_SERVICE_MODE"]} 217 | tcp-request inspect-delay 1s 218 | tcp-request content lua.authorize unless UNSECURED 219 | tcp-request content capture var(txn.SpiffeUrl) len 64 if { var(txn.SpiffeUrl) -m found } 220 | tcp-request content capture var(txn.Authorized) len 64 if { var(txn.Authorized) -m found } 221 | tcp-request content capture var(txn.Reason) len 64 if { var(txn.Reason) -m found } 222 | tcp-request content reject unless UNSECURED or { var(txn.Authorized) -m str true } 223 | " 224 | SYSLOG_SERVER=$(jq -r .Proxy.Config.syslog_server <<<${CONSUL_PROXY_JSON}) 225 | if [ "${SYSLOG_SERVER}" != "null" ]; then 226 | CONFIG["SYSLOG_SERVER"]="log ${SYSLOG_SERVER} local0 info 227 | log-tag sidecar-for-${ARGS["SIDECAR_FOR"]}" 228 | CONFIG["DEFAULTS_LOG"]="log global 229 | option httplog" 230 | fi 231 | } 232 | 233 | # write HAProxy Configuration 234 | function config-write() { 235 | cat <${CONF_TMP_DIR}/${CONF_HAP_FILENAME} 236 | global 237 | maxconn 10000 238 | #user ${CONFIG["USER"]} # useless cause we're already started as ${CONFIG["USER"]} user 239 | #group ${CONFIG["GROUP"]} # useless cause we're already started as ${CONFIG["GROUP"]} user 240 | #chroot /var/empty 241 | daemon 242 | tune.ssl.default-dh-param 1024 243 | pidfile ${CONFIG["PIDFILE"]} 244 | stats socket ${CONFIG["STATS_SOCKET"]} user ${CONFIG["USER"]} group ${CONFIG["GROUP"]} mode 660 level admin 245 | stats timeout 10m 246 | ca-base ${CONFIG["CERTS_FOLDER"]} 247 | crt-base ${CONFIG["CERTS_FOLDER"]} 248 | ${CONFIG["MODULES"]} 249 | lua-load /authorize.lua 250 | ${CONFIG["SYSLOG_SERVER"]} 251 | 252 | resolvers consul 253 | nameserver consul 127.0.0.1:8600 254 | 255 | defaults 256 | timeout client 30s 257 | timeout connect 250ms 258 | timeout server 30s 259 | option redispatch 1 260 | retries 3 261 | option socket-stats 262 | option tcplog 263 | option dontlognull 264 | option dontlog-normal 265 | default-server init-addr none 266 | ${CONFIG["DEFAULTS_LOG"]} 267 | 268 | frontend f_stats 269 | bind 0.0.0.0:1936 270 | mode http 271 | http-request set-log-level silent 272 | stats enable 273 | stats uri / 274 | stats show-legends 275 | stats show-desc ${CONFIG["TARGET_SERVICE_NAME"]} on ${CONFIG["PROXY_ID"]} 276 | option httpclose 277 | 278 | # Proxied (local) service 279 | frontend f_${CONFIG["TARGET_SERVICE_NAME"]} 280 | bind :${CONFIG["BIND_PORT"]} ssl ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem verify required name secured 281 | ${CONFIG["AUTHORIZATIONFREE_BIND_PORT"]} 282 | ${CONFIG["UNSECURED_BIND_PORT"]} 283 | ${CONFIG["ACL_UNSECURED"]} 284 | ${CONFIG["LOCAL_SERVICE_MODE"]} 285 | default_backend b_${CONFIG["TARGET_SERVICE_NAME"]} 286 | 287 | backend b_${CONFIG["TARGET_SERVICE_NAME"]} 288 | ${CONFIG["LOCAL_BACKEND_MODE"]} 289 | server localhost ${CONFIG["LOCAL_SERVICE_ADDRESS"]}:${CONFIG["LOCAL_SERVICE_PORT"]} check 290 | 291 | # upstreams: 292 | EOF 293 | } 294 | 295 | # dump content of the CONFIG variable 296 | function config-dump() { 297 | for a in ${!CONFIG[@]} 298 | do 299 | echo "$a: ${CONFIG[$a]}" >>$LOGFILE 300 | done 301 | } 302 | ### END OF CONFIG ### 303 | 304 | 305 | ### UPSTREAM ### 306 | declare -A UPSTREAM 307 | function upstream-init() { 308 | # mandatory parameters provided by the consul API 309 | UPSTREAM["DESTINATION_NAME"]="" 310 | UPSTREAM["DESTINATION_SERVERS"]="" 311 | UPSTREAM["LOCAL_BIND_ADDRESS"]="" 312 | UPSTREAM["LOCAL_BIND_PORT"]="" 313 | 314 | # custom parameters 315 | UPSTREAM["DESTINATION_MODE"]="mode tcp" 316 | UPSTREAM["LOG_FORMAT"]="option tcplog" 317 | UPSTREAM["ADVANCED_CHECK"]="option tcp-check 318 | tcp-check connect ssl" 319 | } 320 | 321 | function upstream-getdata() { 322 | # mandatory parameters provided by the consul API 323 | UPSTREAM["DESTINATION_NAME"]=$(jq -r .DestinationName <<<${1}) 324 | UPSTREAM["LOCAL_BIND_ADDRESS"]=$(jq -r .LocalBindAddress <<<${1}) 325 | UPSTREAM["LOCAL_BIND_PORT"]=$(jq -r .LocalBindPort <<<${1}) 326 | DESTINATION_TYPE=$(jq -r .DestinationType <<<${1}) 327 | case ${DESTINATION_TYPE} in 328 | "service") 329 | UPSTREAM["DESTINATION_SERVERS"]="server-template s_${UPSTREAM["DESTINATION_NAME"]} 20 _${UPSTREAM["DESTINATION_NAME"]}._tcp.service.consul ssl resolvers consul ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem check #no-tls-tickets" 330 | ;; 331 | "connect") 332 | # FIXME: use SRV records instead of catalog API 333 | CONSUL_CATALOG_URL="http://localhost:8500/v1/catalog/service/${UPSTREAM["DESTINATION_NAME"]}-sidecar-proxy" 334 | CATALOG_UPSTREAM_JSON=$($CURL ${CONSUL_CATALOG_URL} 2>>$LOGFILE) 335 | 336 | for row in $(jq -c .[] <<<${CATALOG_UPSTREAM_JSON}) 337 | do 338 | UPSTREAM["SERVICE_PORT"]=$(jq -r .ServicePort <<<${row}) 339 | UPSTREAM["DESTINATION_SERVERS"]="server-template s_${UPSTREAM["DESTINATION_NAME"]} 20 ${UPSTREAM["DESTINATION_NAME"]}.connect.consul:${UPSTREAM["SERVICE_PORT"]} ssl resolvers consul ca-file ca.pem crt ${CONFIG["TARGET_SERVICE_NAME"]}.pem check #no-tls-tickets" 340 | done 341 | ;; 342 | esac 343 | 344 | # custom parameters 345 | DESTINATION_MODE=$(jq -r .destination_mode <<<${1}) 346 | if [ "${DESTINATION_MODE}" = "http" ]; then 347 | UPSTREAM["DESTINATION_MODE"]="mode http" 348 | UPSTREAM["LOG_FORMAT"]="option httplog" 349 | fi 350 | 351 | ADVANCED_CHECK=$(jq -r .advanced_check <<<${1}) 352 | case ${ADVANCED_CHECK} in 353 | redis) 354 | UPSTREAM["ADVANCED_CHECK"]="option redis-check" 355 | ;; 356 | esac 357 | } 358 | 359 | # write the upstream configuration into the config file 360 | function upstream-write() { 361 | cat <>${CONF_TMP_DIR}/${CONF_HAP_FILENAME} 362 | 363 | frontend f_${UPSTREAM["DESTINATION_NAME"]} 364 | bind ${UPSTREAM["LOCAL_BIND_ADDRESS"]}:${UPSTREAM["LOCAL_BIND_PORT"]} 365 | ${UPSTREAM["DESTINATION_MODE"]} 366 | ${UPSTREAM["LOG_FORMAT"]} 367 | default_backend b_${UPSTREAM["DESTINATION_NAME"]} 368 | 369 | backend b_${UPSTREAM["DESTINATION_NAME"]} 370 | ${UPSTREAM["DESTINATION_MODE"]} 371 | ${UPSTREAM["ADVANCED_CHECK"]} 372 | ${UPSTREAM["DESTINATION_SERVERS"]} 373 | EOF 374 | } 375 | 376 | # dump content of the UPSTREAM variable 377 | function upstream-dump() { 378 | for a in ${!UPSTREAM[@]} 379 | do 380 | echo "$a: ${UPSTREAM[$a]}" >>$LOGFILE 381 | done 382 | } 383 | ### END OF UPSTREAM ### 384 | 385 | 386 | ### MAIN ### 387 | 388 | # this function generates an HAProxy configuration in $CONF_TMP_DIR and returns: 389 | # 0 if no error happened 390 | # 1 if anything was wrong 391 | function generate-cfg() { 392 | echo "GENERATING NEW CONFIGURATION" >>$LOGFILE 393 | 394 | # clear CERTS array, to ensure we monitor only certs we need to 395 | for cert in "${!CERTS[@]}" 396 | do 397 | unset -v CERTS[$cert] 398 | done 399 | 400 | config-init 401 | config-getdata 402 | config-dump 403 | 404 | ERROR="false" 405 | echo "" >>$LOGFILE 406 | [ -z "${CONF_TMP_DIR}" ] && return 1 407 | rm -rf ${CONF_TMP_DIR} 408 | mkdir -p ${CONF_TMP_DIR} 409 | mkdir -p ${CONFIG["CERTS_FOLDER"]} 410 | 411 | # get cert and key for target service 412 | save-leaf-cert ${CONFIG["TARGET_SERVICE_NAME"]} ${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME} 413 | [ $? -ne 0 ] && return 1 414 | 415 | # get CA root certificate 416 | ${CURL} ${CONSUL_ROOTCA_URL} | jq -r .Roots[0].RootCert | sed -e 's/\\n/\n/g' > ${CONF_TMP_DIR}/${CONF_CERTS_DIRNAME}/ca.pem 417 | 418 | # Write configuration to file 419 | config-write 420 | 421 | # configuring upstreams 422 | for row in $(jq -c .Proxy.Upstreams[] <<<${CONSUL_PROXY_JSON}) 423 | do 424 | upstream-init 425 | upstream-getdata "${row}" 426 | upstream-dump 427 | upstream-write 428 | done 429 | 430 | return 0 431 | } 432 | 433 | 434 | # blocking loop wait for an update on the connect proxy configuration piece 435 | HASH="0" 436 | OLDHASH="0" 437 | while true; 438 | do 439 | TMPFILE=$(mktemp) 440 | ${CURL} --include --max-time 10 ${VERBOSE} --output ${TMPFILE} ${CONSUL_SERVICE_URL}?hash=${HASH} >${TMPFILE} 441 | 442 | RETCODE=$? 443 | # 28 is the return code when curl wait for 'max-time' 444 | if [ ${RETCODE} -eq 28 ]; then 445 | sleep 1 446 | rm ${TMPFILE} 447 | check-certs 448 | [ $? -gt 0 ] && do-reload 449 | continue 450 | fi 451 | # any other code than 0 may indicates an error, so we don't want to apply anything 452 | if [ ${RETCODE} -ne 0 ]; then 453 | sleep 1 454 | rm ${TMPFILE} 455 | continue 456 | fi 457 | 458 | STATUSCODE=$(head -n 1 ${TMPFILE} | cut -d' ' -f 2) 459 | if [ "${STATUSCODE}" != "200" ]; then 460 | sleep 1 461 | OLDHASH=${HASH} 462 | rm ${TMPFILE} 463 | continue 464 | fi 465 | 466 | HASH=$(sed -n 's/^X-Consul-Contenthash: \(.*\)\r/\1/p' ${TMPFILE}) 467 | if [ -z "${HASH}" ]; then 468 | sleep 1 469 | rm ${TMPFILE} 470 | continue 471 | fi 472 | 473 | # the configuration has not changed 474 | if [ "${OLDHASH}" = "${HASH}" ]; then 475 | sleep 1 476 | rm ${TMPFILE} 477 | continue 478 | fi 479 | 480 | rm ${TMPFILE} 481 | 482 | generate-cfg 483 | if [ $? -ne 0 ]; then 484 | sleep 1 485 | HASH="foo" 486 | continue 487 | fi 488 | do-check 489 | [ $? -ne 0 ] && continue 490 | move-cfg ${CONF_TMP_DIR} ${CONF_PROD_DIR} 491 | [ $? -ne 0 ] && continue 492 | 493 | echo "APPLYING NEW CONFIGURATION" >>$LOGFILE 494 | do-reload 495 | 496 | sleep 1 497 | done 498 | 499 | -------------------------------------------------------------------------------- /blog/building_service_mesh/www/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "www", 3 | "version": "1.0.0", 4 | "description": "Sample application using Consul service mesh and HAProxy", 5 | "main": "app.js", 6 | "scripts": { 7 | "start": "node app.js" 8 | }, 9 | "author": "HAProxy Technologies", 10 | "license": "ISC", 11 | "dependencies": { 12 | "express": "^4.16.4", 13 | "redis": "^2.8.0" 14 | } 15 | } -------------------------------------------------------------------------------- /blog/building_service_mesh/www/start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | set -x 5 | 6 | cd /www 7 | npm install 8 | npm start & 9 | 10 | MYIP=$(hostname -i) 11 | 12 | [ -z "${SERVICENAME}" ] && exit 1 13 | 14 | cat </etc/consul.d/services.json 15 | { 16 | "service": { 17 | "name": "${SERVICENAME}", 18 | "port": 8080, 19 | "address": "${MYIP}", 20 | "connect": { 21 | "sidecar_service": { 22 | "proxy": { 23 | "upstreams": [ 24 | { 25 | "destination_name": "redis", 26 | "destination_type": "connect", 27 | "local_bind_address": "127.0.0.1", 28 | "local_bind_port": 6379 29 | } 30 | ], 31 | "config": { 32 | "unsecured_bind_port": 21002, 33 | "local_service_mode": "http" 34 | } 35 | } 36 | } 37 | } 38 | }, 39 | "acl_datacenter":"dc1", 40 | "acl_default_policy":"deny", 41 | "acl_token":"agenttoken" 42 | } 43 | EOF 44 | 45 | cat /etc/consul.d/services.json 46 | 47 | /controller.sh -sidecar-for=${SERVICENAME} & 48 | 49 | exec su -l nobody -s /bin/bash -c "consul agent -data-dir=/tmp/consul -node=$HOSTNAME -node-id=$(uuidgen) -bind=0.0.0.0 \ 50 | -enable-script-checks \ 51 | -config-dir=/etc/consul.d -retry-join consul-server" 52 | 53 | -------------------------------------------------------------------------------- /blog/haproxy_docker_dns_link/blog_haproxy_dns/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM haproxytech/haproxy-ubuntu:latest 2 | MAINTAINER Baptiste Assmann 3 | 4 | # install third party tools 5 | RUN export DEBIAN_FRONTEND=noninteractive && \ 6 | apt-get update && \ 7 | apt-get install --yes inotify-tools dnsmasq && \ 8 | apt-get clean && \ 9 | rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 10 | 11 | # traffic ports 12 | EXPOSE 80 443 13 | 14 | # administrative ports 15 | # 82: TCP stats socket 16 | # 88: HTTP stats page 17 | EXPOSE 81 82 88 18 | 19 | ADD haproxy.cfg /etc/haproxy/haproxy.cfg 20 | ADD haproxy.sh / 21 | 22 | ENTRYPOINT /haproxy.sh 23 | 24 | -------------------------------------------------------------------------------- /blog/haproxy_docker_dns_link/blog_haproxy_dns/haproxy.cfg: -------------------------------------------------------------------------------- 1 | global 2 | log 172.16.0.1:8514 local0 3 | log 172.16.0.1:8514 local1 notice 4 | daemon 5 | 6 | resolvers docker 7 | nameserver dnsmasq 127.0.0.1:53 8 | resolve_retries 3 9 | timeout retry 1s 10 | hold valid 3s 11 | 12 | defaults 13 | mode http 14 | log global 15 | option httplog 16 | 17 | frontend f_myapp 18 | bind :80 19 | default_backend b_myapp 20 | 21 | backend b_myapp 22 | server appsrv1 appsrv1:80 check resolvers docker resolve-prefer ipv4 23 | 24 | frontend f_stats 25 | bind :88 26 | stats enable 27 | stats uri / 28 | 29 | -------------------------------------------------------------------------------- /blog/haproxy_docker_dns_link/blog_haproxy_dns/haproxy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [ -f /etc/haproxy/haproxy.cfg ]; then 4 | dnsmasq --no-dhcp-interface= --port 53 --interface=lo --cache-size=0 --listen-address=127.0.0.1 --pid-file=/var/run/dnsmasq.pid --user=root --max-cache-ttl=0 --local-ttl=0 5 | while true 6 | do 7 | inotifywait -qq -e modify /etc/hosts 8 | kill -HUP $(cat /var/run/dnsmasq.pid) 9 | done & 10 | haproxy -db -f /etc/haproxy/haproxy.cfg 11 | else 12 | haproxy -vv 13 | fi 14 | 15 | -------------------------------------------------------------------------------- /blog/haproxy_docker_dns_link/blog_rsyslogd/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:trusty 2 | MAINTAINER Baptiste Assmann 3 | 4 | EXPOSE 8514/udp 5 | 6 | ADD rsyslogd.conf / 7 | ADD rsyslogd.sh / 8 | 9 | ENTRYPOINT ["/rsyslogd.sh"] 10 | 11 | -------------------------------------------------------------------------------- /blog/haproxy_docker_dns_link/blog_rsyslogd/rsyslogd.conf: -------------------------------------------------------------------------------- 1 | $ModLoad imudp.so 2 | $UDPServerRun 8514 3 | $template CustomFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" 4 | $ActionFileDefaultTemplate CustomFormat 5 | 6 | /* info */ 7 | if $programname contains 'haproxy' and $syslogseverity == 6 then ( 8 | action(type="omfile" file="/var/log/haproxy/traffic") 9 | ) 10 | if $programname contains 'haproxy' and $syslogseverity-text == 'err' then ( 11 | action(type="omfile" file="/var/log/haproxy/errors") 12 | ) 13 | /* notice */ 14 | if $programname contains 'haproxy' and $syslogseverity <= 5 then ( 15 | action(type="omfile" file="/var/log/haproxy/events") 16 | ) 17 | 18 | *.* /var/log/messages 19 | 20 | -------------------------------------------------------------------------------- /blog/haproxy_docker_dns_link/blog_rsyslogd/rsyslogd.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | rsyslogd -f /rsyslogd.conf 4 | 5 | # haproxy logs 6 | mkdir /var/log/haproxy 7 | touch /var/log/haproxy/traffic /var/log/haproxy/events /var/log/haproxy/errors 8 | 9 | tail -F /var/log/haproxy/* 10 | 11 | -------------------------------------------------------------------------------- /blog/integration_with_consul/haproxy.conf.by_consul_template: -------------------------------------------------------------------------------- 1 | global 2 | daemon 3 | pidfile /tmp/haproxy.pid 4 | stats socket /tmp/haproxy.sock level admin 5 | maxconn 2000 6 | log 10.42.0.1:514 local0 info 7 | server-state-file /tmp/haproxy.serverstates 8 | description HAProxy / consul demo 9 | 10 | resolvers consul 11 | nameserver consul consul_haproxy_1.docker:8600 12 | accepted_payload_size 8192 13 | 14 | defaults 15 | log global 16 | option httplog 17 | option socket-stats 18 | load-server-state-from-file global 19 | default-server init-addr none inter 1s rise 2 fall 2 20 | mode http 21 | 22 | timeout client 30s 23 | timeout connect 4s 24 | timeout http-keep-alive 10s 25 | timeout http-request 5s 26 | timeout server 30s 27 | 28 | frontend http-in 29 | bind *:8000 30 | maxconn 2000 31 | use_backend b_%[req.hdr(Host),lower,word(1,:)] 32 | 33 | 34 | backend b_api.consul.itchy.local 35 | server-template api 10 _api._tcp.service.consul resolvers consul resolve-prefer ipv4 check 36 | 37 | backend b_consul.consul.itchy.local 38 | server-template consul 1 _consul._tcp.service.consul resolvers consul resolve-prefer ipv4 check 39 | 40 | backend b_consul-dashboard.consul.itchy.local 41 | server-template consul-dashboard 10 _consul-dashboard._tcp.service.consul resolvers consul resolve-prefer ipv4 check 42 | 43 | backend b_haproxystats.consul.itchy.local 44 | server-template haproxystats 1 _haproxystats._tcp.service.consul resolvers consul resolve-prefer ipv4 check 45 | 46 | backend b_new.consul.itchy.local 47 | server-template new 10 _new._tcp.service.consul resolvers consul resolve-prefer ipv4 check 48 | 49 | backend b_www.consul.itchy.local 50 | server-template www 10 _www._tcp.service.consul resolvers consul resolve-prefer ipv4 check 51 | 52 | 53 | frontend stats 54 | bind *:1936 55 | mode http 56 | option forceclose 57 | stats enable 58 | stats uri / 59 | stats show-legends 60 | stats show-desc 61 | stats show-node 62 | stats refresh 2 63 | -------------------------------------------------------------------------------- /blog/integration_with_consul/haproxy.conf.tmpl: -------------------------------------------------------------------------------- 1 | global 2 | daemon 3 | # debug 4 | pidfile /haproxy.pid 5 | stats socket /haproxy.sock level admin 6 | maxconn {{key "service/haproxy/maxconn"}} 7 | log 10.42.0.1:514 local0 info 8 | server-state-file /haproxy.serverstates 9 | description HAProxy / consul demo 10 | 11 | resolvers consul 12 | nameserver consul 127.0.0.1:8600 13 | accepted_payload_size 8192 14 | 15 | defaults 16 | log global 17 | option httplog 18 | option socket-stats 19 | load-server-state-from-file global 20 | default-server init-addr none inter 1s rise 2 fall 2 21 | mode {{key "service/haproxy/mode"}} 22 | {{range ls "service/haproxy/timeouts"}} 23 | timeout {{.Key}} {{.Value}}{{end}} 24 | 25 | frontend http-in 26 | bind *:80 27 | maxconn {{key "service/haproxy/maxconn"}} 28 | use_backend b_%[req.hdr(Host),lower,word(1,:)] 29 | 30 | {{range services}}{{$servicename := .Name}}{{$nbsrvkeyname := printf "service/haproxy/backend/%s/nbsrv" $servicename}} 31 | backend b_{{$servicename}}.{{key "service/haproxy/domainname"}} 32 | server-template {{$servicename}} {{keyOrDefault $nbsrvkeyname "10"}} _{{$servicename}}._tcp.service.consul resolvers consul resolve-prefer ipv4 check 33 | {{end}} 34 | 35 | frontend stats 36 | bind *:1936 37 | mode http 38 | option forceclose 39 | stats enable 40 | stats uri / 41 | stats show-legends 42 | stats show-desc 43 | stats show-node 44 | stats refresh 2 45 | -------------------------------------------------------------------------------- /blog/integration_with_consul/haproxy_reload.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | [ -n "${DEBUG}" ] && set -x 4 | 5 | # first start 6 | if [ ! -f /haproxy.conf.previous ]; then 7 | echo "$0: First configuration" 8 | cp /haproxy.conf /haproxy.conf.previous 9 | haproxy -f /haproxy.conf 10 | exit 0 11 | fi 12 | 13 | # configuration update occured 14 | CHECK=$(diff -u -p /haproxy.conf.previous /haproxy.conf | egrep -c "^[-+]backend ") 15 | 16 | # we trigger a reload only when backends have been removed or added 17 | if [ ${CHECK} -gt 0 ]; then 18 | if [ -S /haproxy.sock ]; then 19 | echo "show servers state" | socat /haproxy.sock - > /haproxy.serverstates 20 | fi 21 | echo "$0: Backend(s) has(ve) been added or removed, need to reload the configuration" 22 | haproxy -f /haproxy.conf -sf $(cat /haproxy.pid) 23 | fi 24 | 25 | cp /haproxy.conf /haproxy.conf.previous 26 | -------------------------------------------------------------------------------- /blog/ssl_client_certificate_management_at_application_level/ca.sh: -------------------------------------------------------------------------------- 1 | # certificate authority creation 2 | openssl genrsa -out ca.key 4096 3 | openssl req -new -x509 -days 365 -key ca.key -out ca.crt 4 | 5 | # server certificate creation 6 | openssl genrsa -out server.key 1024 7 | openssl req -new -key server.key -out server.csr 8 | openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt 9 | 10 | # client certificate creation 11 | openssl genrsa -out client1.key 1024 12 | openssl genrsa -out client2.key 1024 13 | openssl req -new -key client1.key -out client1.csr 14 | openssl req -new -key client2.key -out client2.csr 15 | openssl x509 -req -days 365 -in client1.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client1.crt 16 | openssl x509 -req -days 365 -in client2.csr -CA ca.crt -CAkey ca.key -set_serial 03 -out client2.crt 17 | 18 | # expired client certificate creation 19 | sudo date -s "Mon Oct 1 14:22:07 CEST 2012" 20 | openssl genrsa -out client_expired.key 1024 21 | openssl req -new -key client_expired.key -out client_expired.csr 22 | openssl x509 -req -days 1 -in client_expired.csr -CA ca.crt -CAkey ca.key -set_serial 04 -out client_expired.crt 23 | sudo ntpdate pool.ntp.org 24 | 25 | -------------------------------------------------------------------------------- /blog/ssl_client_certificate_management_at_application_level/ca2.sh: -------------------------------------------------------------------------------- 1 | # certificate authority creation 2 | openssl genrsa -out ca2.key 4096 3 | openssl req -new -x509 -days 365 -key ca2.key -out ca2.crt 4 | 5 | # client certificate creation 6 | openssl genrsa -out client_company.key 1024 7 | openssl req -new -key client_company.key -out client_company.csr 8 | openssl x509 -req -days 365 -in client_company.csr -CA ca2.crt -CAkey ca2.key -set_serial 02 -out client_company.crt 9 | 10 | -------------------------------------------------------------------------------- /blog/ssl_client_certificate_management_at_application_level/pem_creation.sh: -------------------------------------------------------------------------------- 1 | # server PEM file creation (order matters) 2 | cat server.crt > server.pem 3 | cat server.key >> server.pem 4 | 5 | # certificate authority concatenation (order doesn't matters) 6 | cat ca.crt > ca.pem 7 | cat ca2.crt >> ca.pem 8 | 9 | -------------------------------------------------------------------------------- /blog/ssl_client_certificate_management_at_application_level/revocation_list_creation.sh: -------------------------------------------------------------------------------- 1 | # preparation 2 | mkdir demoCA 3 | touch demoCA/index.txt 4 | echo 01 > demoCA/crlnumber 5 | 6 | # SSL certificate revocation list creation 7 | openssl ca -gencrl -keyfile ca.key -cert ca.crt -out ca_crl.pem 8 | 9 | # client certificate addition to the crl 10 | openssl ca -revoke client2.crt -keyfile ca.key -cert ca.crt 11 | 12 | # after each addition, we must regenerate the crl 13 | openssl ca -gencrl -keyfile ca.key -cert ca.crt -out ca_crl.pem 14 | 15 | -------------------------------------------------------------------------------- /configuration_templates/exchange2010_https_services_forward.tpl: -------------------------------------------------------------------------------- 1 | # HAProxy configuration for ALOHA Load-Balancer 2 | # Microsoft Exchange 2010 HTTPS services without SSL acceleration 3 | # 4 | # Copyright Exceliance 5 | # v 1.0 - August 6th, 2013 6 | # 7 | # required information: 8 | # : 9 | # : 10 | # : 11 | # : 12 | # : 13 | # : 14 | # : 15 | # : 16 | # : 17 | # end of required information 18 | 19 | ######################################################### 20 | # exchange 2010 HTTPS services with no SSL acceleration # 21 | ######################################################### 22 | defaults 23 | option dontlognull 24 | option redispatch 25 | option contstats 26 | retries 3 27 | timeout client 600s 28 | timeout connect 5s 29 | timeout queue 30s 30 | timeout server 600s 31 | timeout tarpit 60s 32 | backlog 10000 33 | 34 | ### persistence synchronisation ### 35 | peers aloha 36 | peer :1023 37 | peer :1023 38 | 39 | ### client ip based persistence ### 40 | backend sourceip 41 | stick-table type ip size 10k peers aloha 42 | 43 | ### HTTPS services forwarding ### 44 | backend bk_exchange_https 45 | balance leastconn 46 | mode tcp 47 | log global 48 | option tcplog 49 | timeout server 600s 50 | timeout connect 5s 51 | stick on src table sourceip 52 | default-server inter 3s rise 2 fall 3 53 | server :443 check 54 | server :443 check 55 | 56 | frontend ft_exchange_https 57 | bind :443 name https 58 | mode tcp 59 | log global 60 | option tcplog 61 | timeout client 600s 62 | default_backend bk_exchange_https 63 | 64 | ### HTTP configuration to forward users to HTTPS ### 65 | frontend ft_exchange_http 66 | bind :80 name http 67 | mode http 68 | log global 69 | option httplog 70 | timeout client 25s 71 | timeout http-request 15s 72 | timeout http-keep-alive 1s 73 | maxconn 1000 74 | redirect location /owa/ code 302 if { path / } 75 | redirect scheme https code 302 unless { ssl_fc } 76 | 77 | 78 | -------------------------------------------------------------------------------- /configuration_templates/exchange2010_mapi_services.tpl: -------------------------------------------------------------------------------- 1 | # HAProxy configuration for ALOHA Load-Balancer 2 | # Microsoft Exchange 2010 RPC services load-balancing 3 | # 4 | # Copyright Exceliance 5 | # v 1.0 - August 6th, 2013 6 | # 7 | # required information: 8 | # : 9 | # : 10 | # : 11 | # : 12 | # : 13 | # : 14 | # : 15 | # : 16 | # : 17 | # : 18 | # : 19 | # end of required information 20 | 21 | ############################################ 22 | # exchange 2010 TCP RPC mapi configuration # 23 | ############################################ 24 | defaults 25 | option dontlognull 26 | option redispatch 27 | option contstats 28 | retries 3 29 | timeout client 600s 30 | timeout connect 5s 31 | timeout queue 30s 32 | timeout server 600s 33 | timeout tarpit 60s 34 | backlog 10000 35 | 36 | ### persistence synchronisation ### 37 | peers aloha 38 | peer :1023 39 | peer :1023 40 | 41 | ### client ip based persistence ### 42 | backend sourceip 43 | stick-table type ip size 10k peers aloha 44 | 45 | ### 3 ports moniroting for server 46 | listen chk_ 47 | bind 127.0.0.1:1001 48 | mode http 49 | monitor-uri /check 50 | monitor fail if { nbsrv lt 3 } 51 | default-server inter 3s fall 2 rise 2 52 | server _endpointmapper :135 check 53 | server _clientaccess : check 54 | server _addressbook : check 55 | 56 | ### 3 ports moniroting for server 57 | listen chk_ 58 | bind 127.0.0.1:1002 59 | mode http 60 | monitor-uri /check 61 | monitor fail if { nbsrv lt 3 } 62 | default-server inter 3s fall 2 rise 2 63 | server _endpointmapper :135 check 64 | server _clientaccess : check 65 | server _addressbook : check 66 | 67 | ### exchange 2010 endpoint mapper service ### 68 | backend bk_exchange2010_endpointmapper 69 | balance leastconn 70 | mode tcp 71 | log global 72 | option tcplog 73 | timeout server 600s 74 | timeout connect 5s 75 | stick on src table sourceip 76 | option httpchk HEAD /check HTTP/1.0 77 | default-server inter 1s rise 1 fall 1 on-marked-down shutdown-sessions 78 | server :135 check addr 127.0.0.1 port 1001 observe layer4 79 | server :135 check addr 127.0.0.1 port 1002 observe layer4 80 | 81 | frontend ft_endpointmapper 82 | bind :135 name endpointmapper 83 | mode tcp 84 | log global 85 | option tcplog 86 | timeout client 600s 87 | default_backend bk_exchange2010_endpointmapper 88 | 89 | ### exchange 2010 client access service ### 90 | backend bk_exchange2010_clientaccess 91 | balance leastconn 92 | mode tcp 93 | log global 94 | option tcplog 95 | timeout server 600s 96 | timeout connect 5s 97 | option httpchk HEAD /check HTTP/1.0 98 | stick on src table sourceip 99 | default-server inter 1s rise 1 fall 1 on-marked-down shutdown-sessions 100 | server : check addr 127.0.0.1 port 1001 observe layer4 101 | server : check addr 127.0.0.1 port 1002 observe layer4 102 | 103 | frontend ft_clientaccess 104 | bind : name clientaccess 105 | mode tcp 106 | log global 107 | option tcplog 108 | timeout client 600s 109 | default_backend bk_exchange2010_clientaccess 110 | 111 | ### exchange 2010 address book service ### 112 | backend bk_exchange2010_addressbook 113 | balance leastconn 114 | mode tcp 115 | log global 116 | option tcplog 117 | timeout server 600s 118 | timeout connect 5s 119 | option httpchk HEAD /check HTTP/1.0 120 | stick on src table sourceip 121 | default-server inter 1s rise 1 fall 1 on-marked-down shutdown-sessions 122 | server : check addr 127.0.0.1 port 1001 observe layer4 123 | server : check addr 127.0.0.1 port 1002 observe layer4 124 | 125 | frontend ft_addressbook 126 | bind : name addressbook 127 | mode tcp 128 | log global 129 | option tcplog 130 | timeout client 600s 131 | default_backend bk_exchange2010_addressbook 132 | 133 | -------------------------------------------------------------------------------- /configuration_templates/exchange2013_simple_https_load-balancing.tpl: -------------------------------------------------------------------------------- 1 | # HAProxy configuration for ALOHA Load-Balancer 2 | # Microsoft Exchange 2013 HTTPS services with SSL bridging, the 3 | # simple way (1 frontend and 1 backend) 4 | # 5 | # Copyright Exceliance 6 | # v 1.0 - August 19th, 2013 7 | # 8 | # required information: 9 | # : 10 | # : 11 | # : 12 | # : 13 | # : 14 | # : 15 | # : 16 | # end of required information 17 | 18 | ################################################## 19 | # exchange 2013 HTTPS services with SSL bridging # 20 | ################################################## 21 | 22 | ######## Default values for all entries till next defaults section 23 | defaults 24 | option http-server-close # set Connection: close to inspect all HTTP traffic 25 | option dontlognull # Do not log connections with no requests 26 | option redispatch # Try another server in case of connection failure 27 | option contstats # Enable continuous traffic statistics updates 28 | retries 3 # Try to connect up to 3 times in case of failure 29 | timeout connect 5s # 5 seconds max to connect or to stay in queue 30 | timeout http-keep-alive 1s # 1 second max for the client to post next request 31 | timeout http-request 15s # 15 seconds max for the client to send a request 32 | timeout queue 30s # 30 seconds max queued on load balancer 33 | timeout tarpit 1m # tarpit hold tim 34 | backlog 10000 # Size of SYN backlog queue 35 | 36 | frontend ft_exchange_https 37 | bind :80 name http 38 | bind :443 name https crt 39 | mode http 40 | log global 41 | option httplog 42 | capture request header User-Agent len 64 43 | capture request header Host len 32 44 | log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ {%sslv/%sslc/%[ssl_fc_sni]/%[ssl_fc_session_id]}\ %{+Q}r 45 | timeout client 25s 46 | maxconn 1000 47 | http-request redirect scheme https code 302 if !{ ssl_fc } 48 | http-request redirect location /owa/ code 302 if { hdr(Host) } { path / } 49 | default_backend bk_exchange_https 50 | 51 | backend bk_exchange_https 52 | balance roundrobin 53 | mode http 54 | log global 55 | option httplog 56 | option forwardfor 57 | default-server inter 3s rise 2 fall 3 58 | timeout server 25s 59 | server :443 maxconn 1000 weight 10 ssl check 60 | server :443 maxconn 1000 weight 10 ssl check 61 | 62 | 63 | 64 | 65 | 66 | -------------------------------------------------------------------------------- /configuration_templates/genconf.ps1: -------------------------------------------------------------------------------- 1 | # genconf.ps1 2 | 3 | # HAProxy configurator script for Windows Powershell 4 | # Copyright Exceliance 5 | # v 1.1 - August 6th, 2013 6 | 7 | # Check if there is only one argument, otherwise exit and print usage 8 | if($args.Count -ne 1) { 9 | Write-Host -ForegroundColor Yellow ("Usage: "+$MyInvocation.InvocationName+"