├── README.md ├── install_docker_machine_compose.sh ├── inventory └── hosts ├── main.tf ├── output.tf ├── playbook.retry ├── playbook.yml ├── security-groups.tf └── variable.tf /README.md: -------------------------------------------------------------------------------- 1 | **docker-swarm-using-terraform-ansible** 2 | 3 | A node in a swarm cluster is any machine with docker engine installed and capable of hosting containers/services (When we run docker engine under swarm mode we often call applications as services). This is also referred as Docker node. A Docker node can be a physical machine or one or more virtual machines running on a physical host or cloud server. It is recommended to spread your docker nodes across multiple physical machines to provide availability and reliability for the applications running on the hosts. Docker Swarm environment consists of one or more manager nodes. To deploy an application on Docker Swarm we submit a request in the form of service definition to a manager node. Manager node performs orchestration and cluster management functions required to maintain the desired state of the farm. If there are multiple manager nodes in a swarm, the nodes elect a leader to conduct orchestration which implements leader election strategy. 4 | 5 | Step-1 6 | In this post we can see how to install Terraform and how to setup the AWS account for working ahead. After installing Terraform and setting up AWS account go to the next step. 7 | NOTE: You need to create and download a key-pair using aws management console. Mine is : docker-key.pem 8 | 9 | Step-2 10 | Create a directory named swarm-deploy. create three files named variable.tf, security-groups.tf, main.tf and output.tf. In variable.tf file add the following 11 | 12 | ``` 13 | ### variable.tf 14 | variable "aws_region" { 15 | description = "AWS region on which we will setup the swarm cluster" 16 | default = "eu-west-1" 17 | } 18 | variable "ami" { 19 | description = "Amazon Linux AMI" 20 | default = "ami-04d10c7d" 21 | } 22 | variable "instance_type" { 23 | description = "Instance type" 24 | default = "t2.micro" 25 | } 26 | variable "key_path" { 27 | description = "SSH Public Key path" 28 | default = "/path-to-keyfile/docker-key.pem" 29 | } 30 | variable "key_name" { 31 | description = "Desired name of Keypair..." 32 | default = "docker-key" 33 | } 34 | variable "bootstrap_path" { 35 | description = "Script to install Docker Engine" 36 | default = "install_docker_machine_compose.sh" 37 | } 38 | ``` 39 | 40 | In this file I’m using region eu-west-1 and Ubuntu-16.04 amazon machine image. You can set yours… :) 41 | In security-groups.tf file add the following 42 | 43 | ``` 44 | ### security-groups.tf 45 | resource "aws_security_group" "sgswarm" { 46 | name = "sgswarm" 47 | tags { 48 | Name = "sgswarm" 49 | } 50 | # Allow all inbound 51 | ingress { 52 | from_port = 0 53 | to_port = 65535 54 | protocol = "tcp" 55 | cidr_blocks = ["0.0.0.0/0"] 56 | } 57 | egress { 58 | from_port = 0 59 | to_port = 65535 60 | protocol = "tcp" 61 | cidr_blocks = ["0.0.0.0/0"] 62 | } 63 | # Enable ICMP 64 | ingress { 65 | from_port = -1 66 | to_port = -1 67 | protocol = "icmp" 68 | cidr_blocks = ["0.0.0.0/0"] 69 | } 70 | } 71 | ``` 72 | 73 | In main.tf add the following 74 | 75 | ``` 76 | ### main.tf 77 | # Specify the provider and access details 78 | provider "aws" { 79 | access_key = "your-aws-access-key" 80 | secret_key = "your-aws-secret-access-key" 81 | region = "${var.aws_region}" 82 | } 83 | resource "aws_instance" "master" { 84 | ami = "${var.ami}" 85 | instance_type = "${var.instance_type}" 86 | key_name = "${var.key_name}" 87 | user_data = "${file("${var.bootstrap_path}")}" 88 | vpc_security_group_ids = ["${aws_security_group.sgswarm.id}"] 89 | tags { 90 | Name = "master" 91 | } 92 | } 93 | resource "aws_instance" "worker1" { 94 | ami = "${var.ami}" 95 | instance_type = "${var.instance_type}" 96 | key_name = "${var.key_name}" 97 | user_data = "${file("${var.bootstrap_path}")}" 98 | vpc_security_group_ids = ["${aws_security_group.sgswarm.id}"] 99 | tags { 100 | Name = "worker 1" 101 | } 102 | } 103 | resource "aws_instance" "worker2" { 104 | ami = "${var.ami}" 105 | instance_type = "${var.instance_type}" 106 | key_name = "${var.key_name}" 107 | user_data = "${file("${var.bootstrap_path}")}" 108 | vpc_security_group_ids = ["${aws_security_group.sgswarm.id}"] 109 | tags { 110 | Name = "worker 2" 111 | } 112 | } 113 | ``` 114 | 115 | In output.tf file add the following 116 | 117 | ``` 118 | ### output.tf 119 | output "master_public_ip" { 120 | value = ["${aws_instance.master.public_ip}"] 121 | } 122 | output "worker1_public_ip" { 123 | value = ["${aws_instance.worker1.public_ip}"] 124 | } 125 | output "worker2_public_ip" { 126 | value = ["${aws_instance.worker2.public_ip}"] 127 | } 128 | ``` 129 | 130 | Step-3 131 | Create a shell script named install_docker_machine_compose.sh which will install docker. This script will execute in the provision time of EC2… 132 | 133 | ``` 134 | #!/bin/bash 135 | export LC_ALL=C 136 | sudo apt-get update -y 137 | #sudo apt-get upgrade -y 138 | ### install python-minimal 139 | sudo apt-get install python-minimal -y 140 | # install docker-engine 141 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - 142 | sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" 143 | sudo apt-get update 144 | sudo apt-get install -y docker-ce 145 | echo "Docker installed..." 146 | sudo usermod -aG docker ${whoami} 147 | sudo systemctl enable docker 148 | sudo systemctl start docker 149 | echo "########################################" 150 | echo "########################################" 151 | echo "##################### install docker-compose ########################" 152 | sudo curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose 153 | sudo chmod +x /usr/local/bin/docker-compose 154 | docker-compose --version 155 | echo "docker-compose installed..." 156 | echo "########################################" 157 | echo "########################################" 158 | echo "#################### install docker-machine #########################" 159 | curl -L https://github.com/docker/machine/releases/download/v0.12.2/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine 160 | chmod +x /tmp/docker-machine 161 | sudo cp /tmp/docker-machine /usr/local/bin/docker-machine 162 | echo "docker-machine installed..." 163 | ``` 164 | 165 | Step-4 166 | Install Ansible 167 | 168 | ``` 169 | $ sudo apt-add-repository ppa:ansible/ansible 170 | Press ENTER to accept the PPA addition. 171 | $ sudo apt-get update 172 | $ sudo apt-get install ansible 173 | ``` 174 | 175 | Step-5 176 | There are a couple of ways of setting up a swarm cluster. You can create a cluster using any virtualized environments like Hyper-V, virtual box. The number of hosts running in a swarm cluster will be restricted to the host’s CPU and memory capacity. Traditionally on premise environments are setup using multiple physical nodes. The second way of setting up swarm environment is by using hosted environments like Azure or AWS. 177 | 178 | We’ll create ansible script for creating swarm cluster(a manager node and two worker nodes). Create a file named playbook.yml in the same swarm-deploy directory and add the following 179 | 180 | ``` 181 | ### playbook.yml 182 | --- 183 | - name: Init Swarm Master 184 | hosts: masters 185 | become: true 186 | gather_facts: False 187 | remote_user: ubuntu 188 | tasks: 189 | - name: Swarm Init 190 | command: sudo usermod -aG docker {{remote_user}} 191 | command: docker swarm init --advertise-addr {{ inventory_hostname }} 192 | - name: Get Worker Token 193 | command: docker swarm join-token worker -q 194 | register: worker_token 195 | - name: Show Worker Token 196 | debug: var=worker_token.stdout 197 | - name: Master Token 198 | command: docker swarm join-token manager -q 199 | register: master_token 200 | - name: Show Master Token 201 | debug: var=master_token.stdout 202 | - name: Join Swarm Cluster 203 | hosts: workers 204 | become: true 205 | remote_user: ubuntu 206 | gather_facts: False 207 | vars: 208 | token: "{{ hostvars[groups['masters'][0]]['worker_token']['stdout'] }}" 209 | master: "{{ hostvars[groups['masters'][0]]['inventory_hostname'] }}" 210 | tasks: 211 | - name: Join Swarm Cluster as a Worker 212 | command: sudo usermod -aG docker {{remote_user}} 213 | command: sudo docker swarm join --token {{ token }} {{ master }}:2377 214 | register: worker 215 | - name: Show Results 216 | debug: var=worker.stdout 217 | - name: Show Errors 218 | debug: var=worker.stderr 219 | ``` 220 | 221 | 222 | Create a directory named inventory and a file named hosts under inventory folder. Change the public ip and key-file-path which you’ll get after running the terraform apply command. 223 | 224 | ``` 225 | [masters] 226 | 52.51.138.1 ansible_user=ubuntu ansible_private_key_file=/path-to-your-keyfile/docker-key.pem 227 | [workers] 228 | 34.240.19.111 ansible_user=ubuntu ansible_private_key_file=/path-to-your-keyfile/docker-key.pem 229 | 52.208.83.236 ansible_user=ubuntu ansible_private_key_file=/path-to-your-keyfile/docker-key.pem 230 | ``` 231 | 232 | 233 | Step-6 234 | All the configuration files and scripts are now set. Run the following commands to deploy three instances using terraform and to create swarm cluser using ansible in those intances. 235 | 236 | ``` 237 | $ terraform init 238 | $ terraform plan 239 | $ terraform apply 240 | ``` 241 | ``` 242 | $ ansible-playbook -i inventory/hosts playbook.yml 243 | ``` 244 | 245 | Our swarm cluster is ready. Let’s check the cluster using ssh in manager node. 246 | 247 | ``` 248 | $ ssh -i your_key_file.pem ubuntu@manager_public_ip 249 | 250 | $ sudo docker node ls 251 | ``` 252 | 253 | That’s all. 254 | -------------------------------------------------------------------------------- /install_docker_machine_compose.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | 4 | export LC_ALL=C 5 | sudo apt-get update -y 6 | #sudo apt-get upgrade -y 7 | 8 | ### install python-minimal 9 | sudo apt-get install python-minimal -y 10 | 11 | # install docker-engine 12 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - 13 | sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" 14 | sudo apt-get update 15 | sudo apt-get install -y docker-ce 16 | echo "Docker installed..." 17 | sudo usermod -aG docker ${whoami} 18 | sudo systemctl enable docker 19 | sudo systemctl start docker 20 | 21 | echo "########################################" 22 | echo "########################################" 23 | 24 | echo "##################### install docker-compose ########################" 25 | sudo curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose 26 | sudo chmod +x /usr/local/bin/docker-compose 27 | docker-compose --version 28 | echo "docker-compose installed..." 29 | 30 | echo "########################################" 31 | echo "########################################" 32 | 33 | echo "#################### install docker-machine #########################" 34 | curl -L https://github.com/docker/machine/releases/download/v0.12.2/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine 35 | chmod +x /tmp/docker-machine 36 | sudo cp /tmp/docker-machine /usr/local/bin/docker-machine 37 | echo "docker-machine installed..." 38 | -------------------------------------------------------------------------------- /inventory/hosts: -------------------------------------------------------------------------------- 1 | [masters] 2 | 13.126.90.53 ansible_user=ubuntu ansible_private_key_file=/home/ratul/developments/devops/keyfile/ec2-core-app.pem 3 | 4 | #ansible_private_key_file=/home/ratul/developments/devops/keyfile/docker-key.pem 5 | 6 | [workers] 7 | #ansible_user=ubuntu 8 | 35.154.160.9 ansible_user=ubuntu ansible_private_key_file=/home/ratul/developments/devops/keyfile/ec2-core-app.pem 9 | 13.126.72.193 ansible_user=ubuntu ansible_private_key_file=/home/ratul/developments/devops/keyfile/ec2-core-app.pem 10 | -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | # Specify the provider and access details 2 | provider "aws" { 3 | access_key = "your_access_key" 4 | secret_key = "your_secret_access_key" 5 | region = "${var.aws_region}" 6 | } 7 | 8 | resource "aws_instance" "master" { 9 | ami = "${var.ami}" 10 | instance_type = "${var.instance_type}" 11 | key_name = "${var.key_name}" 12 | user_data = "${file("${var.bootstrap_path}")}" 13 | vpc_security_group_ids = ["${aws_security_group.sgswarm.id}"] 14 | 15 | tags { 16 | Name = "master" 17 | } 18 | } 19 | 20 | resource "aws_instance" "worker1" { 21 | ami = "${var.ami}" 22 | instance_type = "${var.instance_type}" 23 | key_name = "${var.key_name}" 24 | user_data = "${file("${var.bootstrap_path}")}" 25 | vpc_security_group_ids = ["${aws_security_group.sgswarm.id}"] 26 | 27 | tags { 28 | Name = "worker 1" 29 | } 30 | } 31 | 32 | resource "aws_instance" "worker2" { 33 | ami = "${var.ami}" 34 | instance_type = "${var.instance_type}" 35 | key_name = "${var.key_name}" 36 | user_data = "${file("${var.bootstrap_path}")}" 37 | vpc_security_group_ids = ["${aws_security_group.sgswarm.id}"] 38 | 39 | tags { 40 | Name = "worker 2" 41 | } 42 | } 43 | -------------------------------------------------------------------------------- /output.tf: -------------------------------------------------------------------------------- 1 | output "master_public_ip" { 2 | value = ["${aws_instance.master.public_ip}"] 3 | } 4 | 5 | output "worker1_public_ip" { 6 | value = ["${aws_instance.worker1.public_ip}"] 7 | } 8 | 9 | output "worker2_public_ip" { 10 | value = ["${aws_instance.worker2.public_ip}"] 11 | } 12 | -------------------------------------------------------------------------------- /playbook.retry: -------------------------------------------------------------------------------- 1 | 13.126.90.53 2 | -------------------------------------------------------------------------------- /playbook.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Init Swarm Master 3 | hosts: masters 4 | become: true 5 | gather_facts: False 6 | remote_user: ubuntu 7 | tasks: 8 | - name: Swarm Init 9 | command: sudo usermod -aG docker {{remote_user}} 10 | command: docker swarm init --advertise-addr {{ inventory_hostname }} 11 | 12 | - name: Get Worker Token 13 | command: docker swarm join-token worker -q 14 | register: worker_token 15 | 16 | - name: Show Worker Token 17 | debug: var=worker_token.stdout 18 | 19 | - name: Master Token 20 | command: docker swarm join-token manager -q 21 | register: master_token 22 | 23 | - name: Show Master Token 24 | debug: var=master_token.stdout 25 | 26 | 27 | - name: Join Swarm Cluster 28 | hosts: workers 29 | become: true 30 | remote_user: ubuntu 31 | gather_facts: False 32 | vars: 33 | token: "{{ hostvars[groups['masters'][0]]['worker_token']['stdout'] }}" 34 | master: "{{ hostvars[groups['masters'][0]]['inventory_hostname'] }}" 35 | tasks: 36 | - name: Join Swarm Cluster as a Worker 37 | command: sudo usermod -aG docker {{remote_user}} 38 | command: sudo docker swarm join --token {{ token }} {{ master }}:2377 39 | register: worker 40 | 41 | - name: Show Results 42 | debug: var=worker.stdout 43 | 44 | - name: Show Errors 45 | debug: var=worker.stderr 46 | -------------------------------------------------------------------------------- /security-groups.tf: -------------------------------------------------------------------------------- 1 | resource "aws_security_group" "sgswarm" { 2 | name = "sgswarm" 3 | tags { 4 | Name = "sgswarm" 5 | } 6 | 7 | # Allow all inbound 8 | ingress { 9 | from_port = 0 10 | to_port = 65535 11 | protocol = "tcp" 12 | cidr_blocks = ["0.0.0.0/0"] 13 | } 14 | 15 | egress { 16 | from_port = 0 17 | to_port = 65535 18 | protocol = "tcp" 19 | cidr_blocks = ["0.0.0.0/0"] 20 | } 21 | 22 | # Enable ICMP 23 | ingress { 24 | from_port = -1 25 | to_port = -1 26 | protocol = "icmp" 27 | cidr_blocks = ["0.0.0.0/0"] 28 | } 29 | } 30 | -------------------------------------------------------------------------------- /variable.tf: -------------------------------------------------------------------------------- 1 | variable "aws_region" { 2 | description = "AWS region on which we will setup the swarm cluster" 3 | default = "ap-south-1" 4 | } 5 | 6 | variable "ami" { 7 | description = "Amazon Linux AMI" 8 | default = "ami-8da8d2e2" 9 | } 10 | 11 | variable "instance_type" { 12 | description = "Instance type" 13 | default = "t2.micro" 14 | } 15 | 16 | variable "key_path" { 17 | description = "SSH Public Key path" 18 | default = "/home/ratul/developments/devops/keyfile/ec2-core-app.pem" 19 | } 20 | 21 | variable "key_name" { 22 | description = "Desired name of Keypair..." 23 | default = "ec2-core-app" 24 | } 25 | 26 | variable "bootstrap_path" { 27 | description = "Script to install Docker Engine" 28 | default = "install_docker_machine_compose.sh" 29 | } 30 | --------------------------------------------------------------------------------