├── .gitignore ├── LICENSE ├── README.md ├── defaults └── main.yml ├── files ├── certs.sh └── openssl.conf ├── handlers └── main.yml ├── meta └── main.yml ├── tasks ├── configure.yml ├── install.yml ├── install_linux.yml ├── install_osx.yml └── main.yml └── templates ├── common.json ├── connections.json ├── security.json ├── server.json ├── supervisor.d_consul.conf └── unpack /.gitignore: -------------------------------------------------------------------------------- 1 | files/consul.crt 2 | files/consul.csr 3 | files/consul.key 4 | files/consul_ca.crt 5 | 6 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 sgargan 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Ansible role to install the Consul.io Service discovery and clustering framework (http://consul.io). The role targets linux, and aims to be as generic as possible, avoiding package managers, installing to '/opt' and optionally using the supervisord process manager to control the service. 2 | 3 | ```bash 4 | ansible-galaxy install -p sgargan.consul 5 | ``` 6 | 7 | The role can be used to install either the server or the agent. It installs the agent by default, and the server by setting the consul_server flag to true when calling role 8 | 9 | ```yaml 10 | - hosts: consul_agents 11 | sudo: true 12 | 13 | role: 14 | - sgargan.consul 15 | ``` 16 | 17 | ```yaml 18 | - hosts: consul_servers 19 | sudo: true 20 | 21 | role: 22 | - {role: sgargan.consul, consul_server: true} 23 | ``` 24 | 25 | Bootstrapping 26 | ------------- 27 | 28 | Note that by default when creating a server cluster you need to have the servers to elect a leader until all the servers are ready. This is called bootstrapping and can be read about in detail [here](https://consul.io/docs/guides/bootstrapping.html). 29 | 30 | To bootstrap the servers you pass a group of servers to the role as it is being applied, usually this will be the group that the role is being run against e.g. consul_servers. This consul_join group is passed as follows. 31 | 32 | ```yaml 33 | - hosts: consul_servers 34 | sudo: true 35 | 36 | role: 37 | - {role: sgargan.consul, consul_server: true, consul_join: '{{groups["consul_servers"]}}'} 38 | ``` 39 | 40 | Consul will expect the number of servers in the supplied group to join the cluster and will wait to beging the leader election until at least that many servers have joined. 41 | 42 | Joining 43 | ------- 44 | 45 | For the agent role this consul_join can be used to supply the list of servers to join when the agent starts e.g. 46 | 47 | ```yaml 48 | - hosts: consul_agents 49 | sudo: true 50 | 51 | role: 52 | - {role: sgargan.consul, , consul_join: ['consul-1.dc1.example.io', 'consul-2.dc1.example.io', 'consul-2.dc1.example.io']} 53 | ``` 54 | 55 | Using Supervisor 56 | ---------------- 57 | 58 | I like to use Supervisor to manage the Consul process lifecycle, the consul role does not have a dependency on supervisor role, so you must apply it yourself. A complimentary Supervisor role can be downloaded from galaxy as follows 59 | 60 | ```bash 61 | ansible-galaxy install -p sgargan.supervisor 62 | ``` 63 | 64 | To use this in a playbook and have the consul role generate a supervisor config you need to invoke the role and supply the consul_supervisor_enabled flag. 65 | 66 | ```yaml 67 | - hosts: consul_servers 68 | sudo: true 69 | 70 | role: 71 | - {role: sgargan.supervisor} 72 | - {role: sgargan.consul, consul_server: true, consul_join: '{{groups["consul_servers"]}}', consul_supervisor_enabled: true} 73 | ``` 74 | 75 | For more information on using supervisor see 76 | -------------------------------------------------------------------------------- /defaults/main.yml: -------------------------------------------------------------------------------- 1 | # install related variables 2 | consul_version: 0.4.1 3 | consul_dir: '/opt/consul_{{consul_version}}' 4 | consul_conf_dir: '{{consul_dir}}/conf.d' 5 | consul_data_dir: '{{consul_dir}}/data' 6 | consul_log_dir: '{{consul_dir}}/logs' 7 | consul_ui_dir: '{{consul_dir}}/dist' 8 | consul_url: https://dl.bintray.com/mitchellh/consul/{{consul_version}} 9 | consul_ui_url: https://dl.bintray.com/mitchellh/consul/{{consul_version}}_web_ui.zip 10 | consul_user: consul 11 | consul_group: consul 12 | consul_log_level: info 13 | consul_bootstrap_expect: 2 14 | consul_supervisor_enabled: false 15 | consul_supervisord_config_dir: /etc/supervisor.d 16 | consul_join: [] 17 | 18 | consul_root_cert_installed: false 19 | 20 | # make sure these get interpreted as strings not booleans 21 | json_true: 'true' 22 | json_false: 'false' 23 | 24 | # config values from http://www.consul.io/docs/agent/options.html 25 | 26 | consul_datacenter: dc1 27 | 28 | consul_interface: 0.0.0.0 29 | consul_client_address: '{{consul_interface}}' 30 | consul_advertise_address: '{{consul_interface}}' 31 | consul_bind_address: '{{consul_interface}}' 32 | 33 | consul_acl_enabled: true 34 | consul_acl_datacenter: '{{consul_datacenter}}' 35 | consul_acl_default_policy: 'allow' 36 | consul_acl_down_policy: 'extend-cache' 37 | 38 | consul_acl_master_token: 4791402A-D875-4C18-8316-E652DBA53B18 39 | consul_acl_token: C818728E-5D0F-4E97-8CDA-C4E4F9518744 40 | consul_acl_ttl: 30s 41 | 42 | consul_dns_address: '{{consul_client_address}}' 43 | consul_http_address: '{{consul_client_address}}' 44 | consul_rpc_address: '{{consul_client_address}}' 45 | 46 | consul_dns_port: 8600 47 | consul_http_port: 8500 48 | consul_rpc_port: 8400 49 | consul_serf_lan_port: 8301 50 | consul_serf_wan_port: 8302 51 | consul_server_port: 8300 52 | 53 | consul_connection_verify_enabled: true 54 | consul_ca_file: '{{consul_dir}}/conf.d/consul_ca.crt' 55 | consul_verify_incoming: '{{json_true}}' 56 | consul_verify_outgoing: '{{json_true}}' 57 | 58 | consul_cert_file: '{{consul_dir}}/conf.d/consul.crt' 59 | consul_key_file: '{{consul_dir}}/conf.d/consul.key' 60 | 61 | consul_check_update_interval: 5m 62 | 63 | consul_disable_anonymous_signature: '{{json_true}}' 64 | consul_disable_remote_exec: '{{json_true}}' 65 | consul_disable_update_check: '{{json_false}}' 66 | 67 | consul_dns_enabled: true 68 | consul_dns_allow_stale: '{{json_false}}' 69 | consul_dns_max_stale: 5s 70 | consul_dns_node_ttl: 0s 71 | consul_dns_enable_truncate: '{{json_false}}' 72 | consul_dns_domain: "consul." 73 | 74 | consul_enable_debug: '{{json_false}}' 75 | 76 | consul_enable_syslog: '{{json_true}}' 77 | 78 | consul_encrypt_enabled: '{{json_false}}' 79 | consul_encrypt_key: WemSfNqARydofnJFh5bDEA== 80 | 81 | consul_leave_on_terminate: '{{json_true}}' 82 | 83 | consul_dns_recursor_enabled: false 84 | consul_dns_recursor: somehost 85 | 86 | consul_rejoin_after_leave: '{{json_true}}' 87 | 88 | consul_retry_join: '{{consul_join}}' 89 | 90 | consul_retry_interval: 30s 91 | 92 | consul_retry_max: 10 93 | 94 | consul_server: false 95 | 96 | consul_server_name: '{{inventory_hostname}}' 97 | consul_node_name: '{{inventory_hostname}}' 98 | 99 | consul_skip_leave_on_interrupt: '{{json_false}}' 100 | 101 | consul_statsd_enabled: false 102 | consul_statsd_addr: http://somesite 103 | 104 | consul_statsite_enabled: false 105 | consul_statsite_addr: http://somesite 106 | 107 | consul_syslog_facility: LOCAL0 108 | 109 | consul_verify_incoming: '{{json_true}}' 110 | 111 | consul_verify_outgoing: '{{json_true}}' 112 | -------------------------------------------------------------------------------- /files/certs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [ -e consul_ca.crt ]; then 4 | echo "Certs already exist, aborting creation." 5 | exit 0 6 | fi 7 | 8 | script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 9 | cd $script_dir 10 | 11 | openssl genrsa -out consul.key 4096 12 | 13 | openssl req -new -x509 -days 365 -key consul.key -subj '/C=US/ST=Somestate/L=sometown/O=somecompany/OU=someorg/CN=someguy' -out consul_ca.crt 14 | 15 | openssl req -new -key consul.key -out consul.csr -subj '/C=US/ST=Somestate/L=sometown/O=somecompany/OU=someorg/CN=*' 16 | 17 | openssl x509 -extfile openssl.conf -extensions ssl_client -req -days 365 -in consul.csr -CA consul_ca.crt -CAkey consul.key -out consul.crt -set_serial 01 18 | -------------------------------------------------------------------------------- /files/openssl.conf: -------------------------------------------------------------------------------- 1 | [ ssl_client ] 2 | extendedKeyUsage = serverAuth,clientAuth 3 | -------------------------------------------------------------------------------- /handlers/main.yml: -------------------------------------------------------------------------------- 1 | - name: supervisor started 2 | shell: ps -deaf | grep supervisord | grep -v grep | awk '{print $2};' | xargs kill -1 3 | when: consul_supervisor_enabled and not ansible_os_family == 'Darwin' 4 | notify: restart consul 5 | 6 | - name: restart consul 7 | supervisorctl: name=consul state=restarted 8 | when: consul_supervisor_enabled and not ansible_os_family == 'Darwin' 9 | -------------------------------------------------------------------------------- /meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | galaxy_info: 3 | author: Steve Gargan 4 | description: Roles to install Consul and run consul. 5 | license: MIT 6 | min_ansible_version: 1.7 7 | 8 | platforms: 9 | - name: GenericUNIX 10 | versions: 11 | - any 12 | 13 | categories: 14 | - clustering 15 | - monitoring 16 | -------------------------------------------------------------------------------- /tasks/configure.yml: -------------------------------------------------------------------------------- 1 | - name: copy over certificates and keys for security 2 | copy: src='{{item.name}}' dest='{{item.dest}}' mode=600 3 | with_items: 4 | - {name: 'consul_ca.crt', dest: '{{consul_ca_file}}'} 5 | - {name: 'consul.crt', dest: '{{consul_cert_file}}'} 6 | - {name: 'consul.key', dest: '{{consul_key_file}}'} 7 | 8 | - name: add common configuration 9 | template: src={{item}} dest={{consul_conf_dir}}/{{item}} 10 | with_items: 11 | - common.json 12 | - connections.json 13 | - security.json 14 | notify: 15 | - restart consul 16 | 17 | - name: Add server configuration 18 | template: src=server.json dest={{consul_conf_dir}}/server.json 19 | when: consul_server 20 | notify: 21 | - restart consul 22 | 23 | - name: change ownership of consul files 24 | file: path={{consul_dir}} owner={{consul_user}} group={{consul_group}} recurse=true 25 | when: not ansible_os_family == 'Darwin' 26 | 27 | - name: Install the supervisord config script 28 | template: src=supervisor.d_consul.conf dest={{consul_supervisord_config_dir}}/consul.conf owner=root group=root mode=0644 29 | notify: 30 | - supervisor started 31 | - restart consul 32 | when: consul_supervisor_enabled and not ansible_os_family == 'Darwin' 33 | 34 | - name: suggest supervisor 35 | debug: msg="Supervisor process management for consul is not enabled. Consul will not be started automatically. Consider applying role sgargan.supervisord" 36 | when: not consul_supervisor_enabled 37 | -------------------------------------------------------------------------------- /tasks/install.yml: -------------------------------------------------------------------------------- 1 | - include: install_linux.yml 2 | when: not ansible_os_family == 'Darwin' 3 | 4 | - include: install_osx.yml 5 | when: ansible_os_family == 'Darwin' 6 | -------------------------------------------------------------------------------- /tasks/install_linux.yml: -------------------------------------------------------------------------------- 1 | - get_url: url='{{consul_url}}_linux_amd64.zip' dest=/tmp/consul.zip validate_certs=no 2 | 3 | - get_url: url='{{consul_ui_url}}' dest=/tmp/consul_ui.zip validate_certs=no 4 | when: consul_server == true 5 | 6 | - group: name={{consul_group}} 7 | 8 | - user: name={{consul_user}} group={{consul_group}} 9 | 10 | - name: create consul dir 11 | file: path={{item}} state=directory owner={{consul_user}} group={{consul_group}} 12 | with_items: 13 | - '{{consul_dir}}' 14 | - '{{consul_dir}}/bin' 15 | - '{{consul_conf_dir}}' 16 | - '{{consul_data_dir}}' 17 | - '{{consul_log_dir}}' 18 | 19 | - name: copy over distro agnostic unpack script 20 | template: src=unpack dest=/tmp/unpack mode=755 21 | 22 | - name: unpack the consul archive 23 | command: /tmp/unpack /tmp/consul.zip {{consul_dir}}/bin 24 | 25 | - name: unpack the consul_ui archive 26 | command: /tmp/unpack /tmp/consul_ui.zip {{consul_dir}} 27 | when: consul_server == true 28 | 29 | - name: remove tmp consul zips 30 | file: path=/tmp/{{item}}.zip state=absent 31 | with_items: 32 | - consul_ui 33 | - consul 34 | 35 | - name: make link to consul folder 36 | file: src='{{consul_dir}}' dest=/opt/consul state=link 37 | 38 | - name: make consul executable 39 | file: path="{{consul_dir}}/bin/consul" mode=755 40 | 41 | - name: make consul available on the path 42 | command: update-alternatives --install "/usr/bin/consul" "consul" "{{consul_dir}}/bin/consul" 100 43 | -------------------------------------------------------------------------------- /tasks/install_osx.yml: -------------------------------------------------------------------------------- 1 | - get_url: url='{{consul_url}}_darwin_amd64.zip' dest=/tmp/consul.zip validate_certs=no 2 | 3 | - get_url: url='{{consul_ui_url}}' dest=/tmp/consul_ui.zip validate_certs=no 4 | 5 | - name: create consul dir 6 | file: path={{item}} state=directory 7 | with_items: 8 | - '{{consul_dir}}' 9 | - '{{consul_dir}}/bin' 10 | - '{{consul_conf_dir}}' 11 | - '{{consul_data_dir}}' 12 | - '{{consul_log_dir}}' 13 | 14 | - name: copy over distro agnostic unpack script 15 | template: src=unpack dest=/tmp/unpack mode=755 16 | 17 | - name: unpack the consul archive 18 | command: /tmp/unpack /tmp/consul.zip {{consul_dir}}/bin 19 | 20 | - name: unpack the consul_ui archive 21 | command: /tmp/unpack /tmp/consul_ui.zip {{consul_dir}} 22 | when: consul_server == true 23 | 24 | - name: remove tmp consul zips 25 | file: path=/tmp/{{item}}.zip state=absent 26 | with_items: 27 | - consul_ui 28 | - consul 29 | 30 | - name: make consul executable 31 | file: path="{{consul_dir}}/bin/consul" mode=755 32 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | - include: install.yml 2 | tags: 3 | - install 4 | 5 | - include: configure.yml 6 | tags: 7 | - configure 8 | -------------------------------------------------------------------------------- /templates/common.json: -------------------------------------------------------------------------------- 1 | { 2 | "datacenter": "{{consul_datacenter}}", 3 | 4 | "data_dir": "{{consul_data_dir}}", 5 | 6 | "log_level": "{{consul_log_level}}", 7 | 8 | "node_name": "{{consul_node_name}}", 9 | 10 | "check_update_interval": "{{consul_check_update_interval}}", 11 | 12 | "disable_update_check": {{consul_disable_update_check}}, 13 | 14 | "enable_debug": {{consul_enable_debug}}, 15 | 16 | "enable_syslog": {{consul_enable_syslog}}, 17 | 18 | "leave_on_terminate": {{consul_leave_on_terminate}}, 19 | 20 | "skip_leave_on_interrupt": {{consul_skip_leave_on_interrupt}}, 21 | 22 | {% if consul_statsd_enabled %}"statsd_addr": "{{consul_statsd_addr}}",{% endif %} 23 | 24 | {% if consul_statsite_enabled %}"statsite_addr": "{{consul_statsite_addr}}",{% endif %} 25 | 26 | "syslog_facility": "{{consul_syslog_facility}}" 27 | 28 | } 29 | -------------------------------------------------------------------------------- /templates/connections.json: -------------------------------------------------------------------------------- 1 | { 2 | {%if consul_join|length > 0 %} 3 | "start_join": [{% for server in consul_join %} "{{server}}" {% if not loop.last %}, {% endif %}{% endfor %}], 4 | {% endif %} 5 | {%if consul_retry_join|length > 0 %} 6 | "retry_join": [{% for server in consul_join %} "{{server}}" {% if not loop.last %}, {% endif %}{% endfor %}], 7 | "rejoin_after_leave": {{consul_rejoin_after_leave}}, 8 | "retry_interval": "{{consul_retry_interval}}", 9 | {% endif %} 10 | 11 | "addresses": { 12 | {%if consul_dns_enabled %}"dns": "{{consul_dns_address}}",{% endif %} 13 | "http": "{{consul_http_address}}", 14 | "rpc": "{{consul_rpc_address}}", 15 | "advertise_addr": "{{consul_advertise_address}}" 16 | }, 17 | 18 | "client_addr": "{{consul_client_address}}", 19 | 20 | "bind_addr": "{{consul_bind_address}}", 21 | 22 | "ports": { 23 | {%if consul_dns_enabled %}"dns": {{consul_dns_port}}, 24 | {% endif %} 25 | "http": {{consul_http_port}}, 26 | "rpc": {{consul_rpc_port}}, 27 | "serf_lan": {{consul_serf_lan_port}}, 28 | "serf_wan": {{consul_serf_wan_port}}, 29 | "server": {{consul_server_port}} 30 | }, 31 | 32 | {%if consul_dns_enabled %} 33 | "dns_config": { 34 | "max_stale": "{{consul_dns_max_stale}}", 35 | "node_ttl": "{{consul_dns_node_ttl}}", 36 | "enable_truncate": {{consul_dns_enable_truncate}}, 37 | "domain": "{{consul_dns_domain}}" 38 | }{% endif %} 39 | 40 | {%if consul_dns_recursor_enabled %}"recursor": "{{consul_dns_recursor}}",{% endif %} 41 | } 42 | -------------------------------------------------------------------------------- /templates/security.json: -------------------------------------------------------------------------------- 1 | { 2 | 3 | {% if consul_acl_enabled %} 4 | "acl_datacenter": "{{consul_acl_datacenter}}", 5 | 6 | {%if consul_acl_default_policy %}"acl_default_policy": "{{consul_acl_default_policy}}",{% endif %} 7 | 8 | {% if consul_acl_down_policy %}"acl_down_policy": "{{consul_acl_down_policy}}",{% endif %} 9 | 10 | {% if consul_server %}"acl_master_token": "{{consul_acl_master_token}}",{% endif %} 11 | 12 | {% if not consul_server %}"acl_token": "{{consul_acl_token}}",{% endif %} 13 | 14 | {% if consul_acl_ttl %}"acl_ttl": "{{consul_acl_ttl}}",{% endif %} 15 | 16 | {% endif %} 17 | {% if consul_encrypt_enabled %}"encrypt": "{{consul_encrypt_key}}",{% endif %} 18 | 19 | {% if consul_connection_verify_enabled %} 20 | "ca_file": "{{consul_ca_file}}", 21 | 22 | "verify_incoming": {{consul_verify_incoming}}, 23 | 24 | "verify_outgoing": {{consul_verify_outgoing}}, 25 | {% endif %} 26 | 27 | "cert_file": "{{consul_cert_file}}", 28 | 29 | "key_file": "{{consul_key_file}}", 30 | 31 | "disable_anonymous_signature": {{consul_disable_anonymous_signature}}, 32 | 33 | "disable_remote_exec": {{consul_disable_remote_exec}} 34 | 35 | } 36 | -------------------------------------------------------------------------------- /templates/server.json: -------------------------------------------------------------------------------- 1 | { 2 | "server": true, 3 | 4 | "server_name": "{{consul_server_name}}", 5 | 6 | "ui_dir": "{{consul_ui_dir}}", 7 | 8 | "bootstrap_expect": {{consul_join | length}} 9 | } 10 | -------------------------------------------------------------------------------- /templates/supervisor.d_consul.conf: -------------------------------------------------------------------------------- 1 | [program:consul] 2 | command={{consul_dir}}/bin/consul agent -config-dir={{consul_conf_dir}} 3 | user={{consul_user}} 4 | group={{consul_group}} 5 | autostart=true 6 | autorestart=true 7 | startsecs=10 8 | startretries=999 9 | stdout_logfile={{consul_log_dir}}/consul.log 10 | redirect_stderr=true 11 | stdout_logfile_maxbytes=20MB 12 | stdout_logfile_backups=10 13 | priority=40 14 | -------------------------------------------------------------------------------- /templates/unpack: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import sys, zipfile 4 | 5 | zip = sys.argv[1] 6 | dest = sys.argv[2] 7 | 8 | with zipfile.ZipFile(zip, "r") as z: 9 | z.extractall(dest) 10 | --------------------------------------------------------------------------------