├── Dockerfile ├── Makefile ├── README.md ├── config └── server.json ├── consul-bootstrap-definition.json ├── consul-server-definition.json └── register-tasks.sh /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM consul:0.8.1 2 | ADD ./config /config/ 3 | RUN apk update && apk add bash 4 | EXPOSE 8300 5 | EXPOSE 8301 6 | EXPOSE 8301 7 | EXPOSE 8302 8 | EXPOSE 8400 9 | EXPOSE 8500 10 | EXPOSE 8600 11 | ENTRYPOINT ["/bin/bash", "-c", "(export JIP=$(wget -qO- 169.254.169.254/latest/meta-data/local-ipv4);export JNODE=$(wget -qO- 169.254.169.254/latest/meta-data/local-hostname);consul agent -client=0.0.0.0 -node=$JNODE -retry-join-ec2-tag-key=$EC2_TAG_KEY -retry-join-ec2-tag-value=$EC2_TAG_VALUE $CONSUL_PARAMS -server -data-dir=/data -advertise=$JIP -config-dir=/config)"] 12 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | 2 | build: 3 | docker build -t majest/docker-consul-ecs:$(VERSION) . 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Consul ECS 2 | ========== 3 | 4 | 5 | This image is based on the [Docker-maintained consul image](https://store.docker.com/images/fec1f6b9-0a26-49b7-990b-d001a3496cc6). It solves the problem of setting up consul servers on ECS which is a bit tricky with the AWS task definition. 6 | 7 | ### Requirements ### 8 | 9 | * You will need AWS CLI installed or alternatively register the definitions manually. 10 | * You will need to set up an ECS Cluster 11 | * You will need to set a tag for the cluster's Auto Scaling Group (ASG) 12 | 13 | ### Task Definition Explanation ### 14 | 15 | The image and task definition are constructed in a way that it could be easy to modify the parameters using env variables. 16 | 17 | There are two task definitions, bootstrap and server which are used to set up the environment. 18 | 19 | The [Bootstrap task definition](consul-bootstrap-definition.json) uses the environmental variable `CONSUL_PARAMS` to set the consul in bootstrap mode (useful for testing). 20 | 21 | The [Server task definition](consul-server-definition.json) uses the environmental variable `CONSUL_PARAMS` to set the expected number of servers to 3 (feel free to change). [Consul recommends](https://www.consul.io/docs/guides/bootstrapping.html) using "3 or 5 total servers per datacenter". 22 | 23 | 24 | There are to extra env variables that are defined when container starts. `JIP` sets the ip of the host machine that will be advertised, `JNODE` is used to set the name of the node. 25 | 26 | Task definitions include dns server ip. Make sure it's correct for your environment. You can check that using ```ifconfig``` and comparing the ip in docker0 section on one of your ECS instances. 27 | 28 | You can check if the nodes are registered using dig. 29 | 30 | `dig @0.0.0.0 -p 53 INSTANCEHOSTNAME.node.consul` 31 | 32 | ### Setup ### 33 | 34 | 1. Create your cluster. 35 | * Make sure Auto-scaling groups are enabled as we need to tweak those later. 36 | 2. (*Optional*) Create Repository and host your own version of this docker image. 37 | 3. If creating your own repository, modify the `image` variable in the appropriate task definition file to point to your repository. 38 | 4. Modify `__TAGKEY__` and `__TAGVALUE__` in [consul-server-definition.json](consul-server-definition.json) to values you want to use to identify these servers. 39 | 5. In the AWS Web Console, EC2 -> Auto Scaling Groups: 40 | 41 | Add a Tag with the key/value that you set in the task definition 42 | 43 | 6. Register Task definitions. By either: 44 | * On your local machine, run `register-tasks.sh` OR 45 | * Register tasks manually via the AWS Web Console 46 | 7. Create a service in your cluster 47 | 48 | ### Potential Problems ### 49 | 50 | ie: Things to check first 51 | 52 | * Do Security groups allow ingress on [the right ports](https://www.consul.io/docs/agent/options.html#ports)? 53 | * Specifically ensure that your servers' Security Groups are allowed, AND 54 | * that the consul servers' Security Group itself is allowed. 55 | * Does your IAM role have the permission `ec2:DescribeInstances`? 56 | * It is recommended to create your own role/policy with only the permissions you need. However, for testing puposes **only** the policy *AmazonEC2ReadOnlyAccess* contains this permission, along with much more Read Access than you likely need. 57 | * More information on the Consul ec2 flags [available here](https://www.consul.io/docs/agent/options.html#_retry_join_ec2_tag_key) 58 | 59 | ### UI ### 60 | 61 | Ui should be available at ```http://awsinstanceip:8500/ui``` 62 | 63 | Alternatively open the tunnel to the instance ```ssh -LnN 8500:localhost:8500 user@awsinstance``` so the ui can be available on your local host ```http://localhost:8500/ui```. It's potentially useful for services development wehere you need the consul cluster available locally. 64 | -------------------------------------------------------------------------------- /config/server.json: -------------------------------------------------------------------------------- 1 | { 2 | "ui": true, 3 | "dns_config": { 4 | "allow_stale": false 5 | } 6 | } 7 | -------------------------------------------------------------------------------- /consul-bootstrap-definition.json: -------------------------------------------------------------------------------- 1 | { 2 | "containerDefinitions": [{ 3 | "essential": true, 4 | "cpu": 20, 5 | "memory": 128, 6 | "image": "majest/docker-consul-ecs:latest", 7 | "name": "consul-bootstrap", 8 | "mountPoints": [{ 9 | "containerPath": "/data", 10 | "sourceVolume": "consuldata", 11 | "readOnly": false 12 | }], 13 | "environment": [{ 14 | "name": "CONSUL_PARAMS", 15 | "value": "-bootstrap" 16 | }], 17 | "portMappings": [{ 18 | "containerPort": 8300, 19 | "hostPort": 8300, 20 | "protocol": "tcp" 21 | }, { 22 | "containerPort": 8301, 23 | "hostPort": 8301, 24 | "protocol": "tcp" 25 | }, { 26 | "containerPort": 8301, 27 | "hostPort": 8301, 28 | "protocol": "udp" 29 | }, { 30 | "containerPort": 8302, 31 | "hostPort": 8302, 32 | "protocol": "tcp" 33 | }, { 34 | "containerPort": 8400, 35 | "hostPort": 8400, 36 | "protocol": "tcp" 37 | }, { 38 | "containerPort": 8500, 39 | "hostPort": 8500, 40 | "protocol": "tcp" 41 | }, { 42 | "containerPort": 8600, 43 | "hostPort": 53, 44 | "protocol": "udp" 45 | }], 46 | "dnsServers": [ 47 | "172.17.0.1", 48 | "8.8.8.8" 49 | ], 50 | "dnsSearchDomains": ["service.consul"] 51 | }], 52 | "family": "consul-bootstrap", 53 | "volumes": [{ 54 | "host": { 55 | "sourcePath": "/opt/rtc/consul" 56 | }, 57 | "name": "consuldata" 58 | }] 59 | } 60 | -------------------------------------------------------------------------------- /consul-server-definition.json: -------------------------------------------------------------------------------- 1 | { 2 | "containerDefinitions": [{ 3 | "essential": true, 4 | "cpu": 20, 5 | "memory": 128, 6 | "image": "majest/docker-consul-ecs:latest", 7 | "name": "consul-server", 8 | "mountPoints": [{ 9 | "containerPath": "/data", 10 | "sourceVolume": "consuldata", 11 | "readOnly": false 12 | }], 13 | "portMappings": [{ 14 | "containerPort": 8300, 15 | "hostPort": 8300, 16 | "protocol": "tcp" 17 | }, { 18 | "containerPort": 8301, 19 | "hostPort": 8301, 20 | "protocol": "tcp" 21 | }, { 22 | "containerPort": 8301, 23 | "hostPort": 8301, 24 | "protocol": "udp" 25 | }, { 26 | "containerPort": 8302, 27 | "hostPort": 8302, 28 | "protocol": "tcp" 29 | }, { 30 | "containerPort": 8400, 31 | "hostPort": 8400, 32 | "protocol": "tcp" 33 | }, { 34 | "containerPort": 8500, 35 | "hostPort": 8500, 36 | "protocol": "tcp" 37 | }, { 38 | "containerPort": 8600, 39 | "hostPort": 53, 40 | "protocol": "udp" 41 | }], 42 | "dnsServers": [ 43 | "172.17.0.1", 44 | "8.8.8.8" 45 | ], 46 | "environment": [{ 47 | "name": "CONSUL_PARAMS", 48 | "value": "-bootstrap-expect 3" 49 | }, { 50 | "name": "EC2_TAG_KEY", 51 | "value": "__TAGKEY__" 52 | }, { 53 | "name": "EC2_TAG_VALUE", 54 | "value": "__TAGVALUE__" 55 | } 56 | ], 57 | "dnsSearchDomains": ["service.consul"] 58 | }], 59 | "family": "consul-server", 60 | "volumes": [{ 61 | "host": { 62 | "sourcePath": "/opt/rtc/consul" 63 | }, 64 | "name": "consuldata" 65 | }] 66 | } 67 | -------------------------------------------------------------------------------- /register-tasks.sh: -------------------------------------------------------------------------------- 1 | aws ecs register-task-definition --cli-input-json file://consul-bootstrap-definition.json 2 | aws ecs register-task-definition --cli-input-json file://consul-server-definition.json 3 | --------------------------------------------------------------------------------