├── .gitignore ├── LICENSE ├── README.md ├── ci └── ecs │ ├── ecs-bluegreen.py │ ├── requirements.txt │ ├── sample.yml │ └── update-ecs.yml ├── ecs-cluster ├── iam.tf ├── main.tf ├── outputs.tf ├── variables.tf └── versions.tf ├── efs ├── cloud-config.yaml.tpl ├── main.tf ├── outputs.tf ├── variables.tf └── versions.tf ├── repository ├── iam.tf ├── main.tf ├── outputs.tf ├── variables.tf └── versions.tf ├── scaling ├── iam.tf ├── main.tf ├── variables.tf └── versions.tf ├── tests ├── ecs-cluster.tf └── outputs.tf └── user-data ├── main.tf ├── outputs.tf ├── templates └── node-exporter.service.tpl ├── variables.tf └── versions.tf /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled files 2 | *.tfstate 3 | *.tfstate.backup 4 | .terraform 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Skyscrapers 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # terraform-ecs 2 | Terraform ECS module 3 | 4 | ## ecs-cluster 5 | Setup a basic ECS cluster with the needed IAM rights. 6 | 7 | ### Available variables: 8 | * [`project`]: String(required): Project name 9 | * [`environment`]: String(required): For what environment is this cluster needed (eg staging, production, ...) 10 | ### Output 11 | * [`cluster_name`]: String: The name of the ECS cluster. 12 | * [`cluster_id`]: String: The ID (ARN) of the ECS cluster. 13 | * [`ecs-instance-profile`]: IAM instance profile that you need to give to your instances. 14 | * [`ecs_instance_profile_arn`]: IAM instance profile arn that you need to give to your instances. 15 | * [`ecs-service-role`]: IAM service role for ECS to manage your loadbalancers 16 | * [`ecs-instance-role`]: IAM instance role name, useful to attach extra policies 17 | 18 | ### Example 19 | ``` 20 | module "ecs_cluster" { 21 | source = "ecs_cluster" 22 | project = "${var.project}" 23 | environment = "${terraform.env}" 24 | } 25 | ``` 26 | 27 | ## Repository 28 | Creates an Elastic Container Registry with associated pull & push policies. 29 | The created pull & push policies can be used as policy attachments on AWS roles. 30 | 31 | ### Available variables: 32 | * [`repository_name`]: String(required): The name of the ECR repository 33 | * [`expire_after`]: Integer(optional): The amount of days after which untagged images expire. Set to 0 if you do not want a lifecycle policy. (default: 30) 34 | 35 | ### Output 36 | * [`repository_url`]: The url to the ECR repository 37 | * [`repository_push_policy`]: The id of the push policy to this ECR repository. 38 | * [`repository_pull_policy`]: The id of the pull policy to this ECR repository. 39 | 40 | ### Example 41 | ``` 42 | module "jenkins-repo" { 43 | source = "github.com/skyscrapers/terraform-ecs//repository" 44 | repository_name = "jenkins" 45 | repository_description = "Jenkins Master" 46 | expire_after = 14 47 | } 48 | ``` 49 | 50 | ## EFS 51 | Creates an Elastic Filesystem, mount points and cloud-config to mount at boot 52 | 53 | ### Available variables: 54 | * [`subnets`]: String(required): The subnets to create the mount point in 55 | * [`subnet_amount`]: String(required): The amount of subnets 56 | * [`project`]: String(optional): The project to tag EFS (default: "") 57 | * [`mount_point`]: String(optional): The mount point of EFS on the system (default: /media/efs) 58 | * [`name`]: String(optional): The name to tag EFS (default: efs) 59 | * [`security_groups`]: List(optional): The security groups to associate with the mount points (default: [] adds default security group) 60 | 61 | ### Output 62 | * [`cloud_config`]: The cloud config to mount efs at boot 63 | * [`dns_name`]: The DNS name of the elastic file system 64 | * [`efs_id`]: ID of the EFS 65 | * [`kms_key_id`]: KMS key used to encrypt EFS 66 | * [`mount_target_ids`]: List of mount target ids 67 | * [`mount_target_interface_ids`]: List of mount target interface ids 68 | 69 | ### Example 70 | ``` 71 | module "efs" { 72 | source = "efs" 73 | project = "myproject" 74 | subnets = "${module.vpc.private_db}" 75 | subnet_amount = 3 76 | security_groups = ["sg-ac66e1d6"] 77 | } 78 | ``` 79 | 80 | ## scaling 81 | Setup a ECS cluster and service scaling. 82 | 83 | ### Available variables: 84 | * [`cluster_name`]: String(required): The name of the ECS cluster. 85 | * [`service_name`]: String(required): The name of the ECS service. 86 | * [`evaluation_periods`]: String(optional): Evaluation period for the cloudwatch alarms (default: 4) 87 | * [`threshold_down`]: String(optional): Threshold when scaling down needs to happen. (default: 25) 88 | * [`threshold_up`]: String(optional): Threshold when scaling up needs to happen. (default: 75) 89 | * [`scale_up_adjustment`]: String(optional): Amount of tasks per check (maximum) that will be added. (default: 1) 90 | * [`scale_down_adjustment`]: String(optional): Amount of tasks per check (maximum) that will be removed. (default: -1) 91 | * [`period_down`]: String(optional): How long the threshold needs to be reached before a downscale happens. (default: 120) 92 | * [`period_up`]: String(optional): How long the threshold needs to be reached before a upscale happens. (default: 60) 93 | * [`statistic`]: String(optional): On what statistic the scaling needs to be based upon. (default: Average) 94 | * [`min_capacity`]: String(optional): Minimum amount of ECS task to run for a service. (default: 1) 95 | * [`max_capacity`]: String(optional): Maximum amount of ECS task to run for a service. (default: 4) 96 | * [`datapoints_to_alarm_up`]: String(optional): ) The number of datapoints that must be breaching to trigger the alarm to scale up (default: 4) 97 | * [`datapoints_to_alarm_down`]: String(optional): The number of datapoints that must be breaching to trigger the alarm to scale down (default: 4) 98 | 99 | 100 | ### Output 101 | 102 | ### Example 103 | ``` 104 | module "service-scaling" { 105 | source = "scaling" 106 | cluster_name = "ecs-production" 107 | service_name = "test-service" 108 | min_capacity = "2" 109 | max_capacity = "10" 110 | ecs_autoscale_group_name = "asg-production" 111 | } 112 | ``` 113 | 114 | ## user-data 115 | Setup a ECS cluster and service user-data. 116 | 117 | ### Available variables: 118 | * [`project`]: String(optional): The project to tag EFS (default: "") 119 | * [`environment`]: String(required): For what environment is this cluster needed (eg staging, production, ...) 120 | * [`cluster_name`]: String(required): The name of the ECS cluster. 121 | * [`teleport_version`]: String(optional): The version of Teleport to be used (default: 4.4.5). 122 | * [`teleport_server`]: String(optional): the name of the teleport server to connect to(default: "") 123 | * [`teleport_auth_token`]: String(optional): Teleport server node token (default: "") 124 | 125 | ### Output 126 | 127 | ### Example 128 | ``` 129 | module "service-user-data" { 130 | source = "user-data" 131 | project = "${var.project}" 132 | environment = "${terraform.workspace}" 133 | teleport_auth_token = "${data.aws_kms_secret.secret.auth_token}" 134 | teleport_version = "${var.teleport_version}" 135 | teleport_server = "${var.teleport_server}" 136 | cluster_name = "${module.ecs_cluster.cluster_name}" 137 | } 138 | ``` 139 | 140 | ## CI/CD 141 | 142 | In the `ci` folder you can find some tasks definition for automating deployment with `concourse` 143 | 144 | ### ECS 145 | This folder contians a script that can be used to automatically update the ecs AMI. 146 | It also contains a concourse task definition that allows to use the script in a concourse pipeline. 147 | 148 | In order to add this task in a pipeline you need to specify the following inputs: 149 | * [`terraform-ecs`] : This git repo 150 | * [`terraform-repo`] : The git repo with the terraform code that contains the ecs cluster 151 | * [`ami`] : The ami resource that we want to deploy 152 | 153 | You can customize your pipeline with the following params: 154 | * [`AWS_ACCESS_KEY_ID`] : The key id to use to authenticate to AWS 155 | * [`AWS_SECRET_ACCESS_KEY`] : The secret key to use to authenticate to AWS 156 | * [`TF_PROJECT_FOLDER`] : The folder that contains the ecs cluster definition 157 | * [`TF_VERSION`] : The terraform version of the project 158 | * [`AWS_DEFAULT_REGION`] : The AWS region where the ecs cluster has been deployed 159 | * [`TIMEOUT`] : The time the script will wait for the container to migrate to the new instances 160 | * [`TF_ENVIRONMENT`] : The terraform workspace to target 161 | 162 | An example of a concourse pipeline can be found in the [example pipeline](ci/ecs/sample.yml) 163 | -------------------------------------------------------------------------------- /ci/ecs/ecs-bluegreen.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2.7 2 | 3 | import boto3 4 | import getopt 5 | import sys 6 | import subprocess 7 | import time 8 | from datetime import datetime 9 | import json 10 | 11 | 12 | def main(argv): 13 | helptext = 'ecs-bluegreen.py -f -a -c -t -e [--role-arn ]' 14 | 15 | try: 16 | opts, args = getopt.getopt(argv, "hsrf:a:c:t:e:", ["folder=", "ami=", "command=", "timeout=", "environment=", "role-arn="]) 17 | except getopt.GetoptError: 18 | print(helptext) 19 | sys.exit(2) 20 | 21 | if opts: 22 | for opt, arg in opts: 23 | if opt == '-h': 24 | print(helptext) 25 | sys.exit(2) 26 | elif opt in ("-f", "--folder"): 27 | projectPath = arg 28 | elif opt in ("-a", "--ami"): 29 | ami = arg 30 | elif opt in ("-c", "--command"): 31 | command = arg 32 | elif opt in ("-t", "--timeout"): 33 | maxTimeout = int(arg) 34 | elif opt in ("-e", "--environment"): 35 | environment = arg 36 | elif opt in ("-s"): 37 | stopScaling = True 38 | elif opt in ("-r","--force-recycle"): 39 | forceRecycle = True 40 | elif opt in ("--role-arn"): 41 | assumeRoleArn = arg 42 | else: 43 | print(helptext) 44 | sys.exit(2) 45 | 46 | if 'command' not in locals(): 47 | command = 'plan' 48 | 49 | if 'maxTimeout' not in locals(): 50 | maxTimeout = 200 51 | 52 | if 'projectPath' not in locals(): 53 | print('Please give your folder path of your Terraform project') 54 | print(helptext) 55 | sys.exit(2) 56 | 57 | if 'ami' not in locals(): 58 | print('Please give an AMI as argument') 59 | print(helptext) 60 | sys.exit(2) 61 | 62 | if 'environment' not in locals(): 63 | environment = None 64 | 65 | if 'stopScaling' not in locals(): 66 | stopScaling = False 67 | 68 | if 'forceRecycle' not in locals(): 69 | forceRecycle = False 70 | 71 | if 'assumeRoleArn' not in locals(): 72 | assumeRoleArn = None 73 | 74 | # Get a boto3 session 75 | global awsSession 76 | awsSession = getBotoSession(assumeRoleArn) 77 | 78 | # Retrieve autoscaling group names and ecs cluster info 79 | tf_output = getTerraformOutput(projectPath) 80 | output_requirements = ['blue_asg_id','green_asg_id','ecs_cluster_name'] 81 | # Verify TF output 82 | if len([x for x in output_requirements if x not in tf_output.keys()]) > 0: 83 | print("Missing required output from terraform stack " + str([x for x in output_requirements if x not in tf_output.keys()])) 84 | sys.exit(2) 85 | agBlue = tf_output['blue_asg_id'] 86 | agGreen = tf_output['green_asg_id'] 87 | ecs_cluster = tf_output['ecs_cluster_name'] 88 | # Retrieve autoscaling groups information 89 | info = getAutoscalingInfo([agBlue, agGreen]) 90 | # Determine the active autoscaling group 91 | active = getActive(info) 92 | desiredInstanceCount = info['AutoScalingGroups'][active]['DesiredCapacity'] * 2 93 | # Bring up the not active autoscaling group with the new AMI 94 | scaleUpAutoscaling(info, active, ami, command, projectPath, environment) 95 | 96 | # Retrieve autoscaling groups information (we need to do this again because the launchconig has changed and we need this in a later phase) 97 | info = getAutoscalingInfo([agBlue, agGreen]) 98 | if 'apply' in command: 99 | print('Waiting for 30 seconds to get autoscaling status') 100 | time.sleep(30) 101 | timeout = 30 102 | ecs_info = getECSInfo(ecs_cluster) 103 | while len(ecs_info['containerInstanceArns']) != desiredInstanceCount: 104 | print("Current active cluster nodes: %s desired: %s" % (str(len(ecs_info['containerInstanceArns'])),str(desiredInstanceCount))) 105 | if timeout > maxTimeout: 106 | print('Rolling back, reached timeout while registering instances to ECS cluster') 107 | rollbackAutoscaling(info, active, ami, command, projectPath, environment) 108 | print('Rollback complete: unable to get the desired amount of nodes registered to the ECS cluster') 109 | sys.exit(2) 110 | 111 | print('Waiting for 10 seconds to get ecs status') 112 | time.sleep(10) 113 | timeout += 10 114 | ecs_info = getECSInfo(ecs_cluster) 115 | removeInstancesFromECS(info['AutoScalingGroups'][active],ecs_cluster,ecs_info,forceRecycle,maxTimeout) 116 | print('Deactivating the autoscaling') 117 | scaleDownAutoscaling(info, active, ami, command, projectPath, environment) 118 | 119 | def getBotoSession(assumeRoleArn): 120 | if assumeRoleArn: 121 | sts_client = boto3.client('sts') 122 | 123 | # Call the assume_role method of the STSConnection object and pass the role 124 | # ARN and a role session name. 125 | assumed_role_object = sts_client.assume_role( 126 | RoleArn = assumeRoleArn, 127 | RoleSessionName = "bluegreen" 128 | ) 129 | 130 | return boto3.Session( 131 | aws_access_key_id = assumed_role_object['Credentials']['AccessKeyId'], 132 | aws_secret_access_key = assumed_role_object['Credentials']['SecretAccessKey'], 133 | aws_session_token = assumed_role_object['Credentials']['SessionToken'], 134 | ) 135 | else: 136 | return boto3.Session() 137 | 138 | def removeInstancesFromECS(old_asg,ecs_cluster,ecs_info,forceRecycle,maxTimeout=500): 139 | client = awsSession.client('ecs') 140 | ecs_instances_list = describeECSInstance(ecs_info['containerInstanceArns'],ecs_cluster) 141 | old_cluster_instance_ids = [ecs_instance['containerInstanceArn'] for ecs_instance in ecs_instances_list['containerInstances'] if ecs_instance['ec2InstanceId'] in [instance['InstanceId'] for instance in old_asg['Instances']]] 142 | response = client.update_container_instances_state( 143 | cluster=ecs_cluster, 144 | containerInstances=old_cluster_instance_ids, 145 | status='DRAINING' 146 | ) 147 | time.sleep(60) 148 | timeout=60 149 | for instance in old_cluster_instance_ids: 150 | instance_details = describeECSInstance([instance],ecs_cluster) 151 | while instance_details['containerInstances'][0]['runningTasksCount'] != 0: 152 | if timeout > maxTimeout: 153 | if forceRecycle: 154 | print("Force recycling of instance: "+instance) 155 | break 156 | else: 157 | print("Unable to cleanup instance %s in %s seconds " % (instance,str(timeout))) 158 | sys.exit(2) 159 | print("still active number of tasks: " + str(instance_details['containerInstances'][0]['runningTasksCount'])) 160 | time.sleep(10) 161 | instance_details = describeECSInstance([instance],ecs_cluster) 162 | timeout += 10 163 | response = client.deregister_container_instance( 164 | cluster=ecs_cluster, 165 | containerInstance=instance, 166 | force=False 167 | ) 168 | timeout=0 169 | 170 | def describeECSInstance(container_instance_ids,ecs_cluster): 171 | client = awsSession.client('ecs') 172 | response = client.describe_container_instances( 173 | cluster=ecs_cluster, 174 | containerInstances=container_instance_ids 175 | ) 176 | return response 177 | 178 | def getECSInfo(ecs_cluster): 179 | client = awsSession.client('ecs') 180 | response = client.list_container_instances( 181 | cluster=ecs_cluster 182 | ) 183 | return response 184 | 185 | def getTerraformOutput(projectPath, output=''): 186 | process = subprocess.Popen('terraform output -json' + output, shell=True, cwd=projectPath, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 187 | std_out, std_err = process.communicate() 188 | if process.returncode != 0: 189 | err_msg = "%s. Code: %s" % (std_err.strip(), process.returncode) 190 | print(err_msg) 191 | sys.exit(process.returncode) 192 | tf_output=json.loads(std_out) 193 | 194 | try: 195 | return {name : tf_output[name]["value"] for name in tf_output.keys()} 196 | except: 197 | print(std_out) 198 | 199 | def getAutoscalingInfo(asgs): 200 | client = awsSession.client('autoscaling') 201 | response = client.describe_auto_scaling_groups( 202 | AutoScalingGroupNames=asgs, 203 | MaxRecords=2 204 | ) 205 | return response 206 | 207 | 208 | def getAmi(launchconfig): 209 | client = awsSession.client('autoscaling') 210 | response = client.describe_launch_configurations( 211 | LaunchConfigurationNames=[ 212 | launchconfig, 213 | ], 214 | MaxRecords=1 215 | ) 216 | return response['LaunchConfigurations'][0]['ImageId'] 217 | 218 | 219 | def getLaunchconfigDate(launchconfig): 220 | client = awsSession.client('autoscaling') 221 | response = client.describe_launch_configurations( 222 | LaunchConfigurationNames=[ 223 | launchconfig, 224 | ], 225 | MaxRecords=1 226 | ) 227 | return response['LaunchConfigurations'][0]['CreatedTime'] 228 | 229 | 230 | def getActive(info): 231 | if info['AutoScalingGroups'][1]['DesiredCapacity'] == 0: 232 | print('Blue is active') 233 | return 0 234 | elif info['AutoScalingGroups'][0]['DesiredCapacity'] == 0: 235 | print('Green is active') 236 | return 1 237 | else: 238 | blueDate = getLaunchconfigDate(info['AutoScalingGroups'][0]['LaunchConfigurationName']) 239 | greenDate = getLaunchconfigDate(info['AutoScalingGroups'][1]['LaunchConfigurationName']) 240 | # use the ASG with the oldest launch config 241 | if blueDate < greenDate: 242 | print('Blue has oldest launchconfig') 243 | return 1 244 | else: 245 | print('Green has oldest launchconfig') 246 | return 0 247 | 248 | 249 | def scaleUpAutoscaling(info, active, ami, command, projectPath, environment): 250 | blueMin = info['AutoScalingGroups'][active]['MinSize'] 251 | blueMax = info['AutoScalingGroups'][active]['MaxSize'] 252 | blueDesired = info['AutoScalingGroups'][active]['DesiredCapacity'] 253 | 254 | greenMin = info['AutoScalingGroups'][active]['MinSize'] 255 | greenMax = info['AutoScalingGroups'][active]['MaxSize'] 256 | greenDesired = info['AutoScalingGroups'][active]['DesiredCapacity'] 257 | 258 | if active == 0: 259 | blueAMI = getAmi(info['AutoScalingGroups'][active]['LaunchConfigurationName']) 260 | greenAMI = ami 261 | elif active == 1: 262 | blueAMI = ami 263 | greenAMI = getAmi(info['AutoScalingGroups'][active]['LaunchConfigurationName']) 264 | else: 265 | print('No acive AMI') 266 | sys.exit(1) 267 | 268 | updateAutoscaling(command, blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, projectPath, environment) 269 | 270 | 271 | def scaleDownAutoscaling(info, active, ami, command, projectPath, environment): 272 | blueAMI = getAmi(info['AutoScalingGroups'][0]['LaunchConfigurationName']) 273 | greenAMI = getAmi(info['AutoScalingGroups'][1]['LaunchConfigurationName']) 274 | if active == 0: 275 | blueMin = 0 276 | blueMax = 0 277 | blueDesired = 0 278 | 279 | greenMin = info['AutoScalingGroups'][active]['MinSize'] 280 | greenMax = info['AutoScalingGroups'][active]['MaxSize'] 281 | greenDesired = info['AutoScalingGroups'][active]['DesiredCapacity'] 282 | elif active == 1: 283 | blueMin = info['AutoScalingGroups'][active]['MinSize'] 284 | blueMax = info['AutoScalingGroups'][active]['MaxSize'] 285 | blueDesired = info['AutoScalingGroups'][active]['DesiredCapacity'] 286 | 287 | greenMin = 0 288 | greenMax = 0 289 | greenDesired = 0 290 | else: 291 | print('No acive AMI') 292 | sys.exit(1) 293 | 294 | updateAutoscaling(command, blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, projectPath, environment) 295 | 296 | 297 | def rollbackAutoscaling(info, active, ami, command, projectPath, environment): 298 | blueAMI = getAmi(info['AutoScalingGroups'][0]['LaunchConfigurationName']) 299 | greenAMI = getAmi(info['AutoScalingGroups'][1]['LaunchConfigurationName']) 300 | 301 | if active == 0: 302 | blueMin = info['AutoScalingGroups'][0]['MinSize'] 303 | blueMax = info['AutoScalingGroups'][0]['MaxSize'] 304 | blueDesired = info['AutoScalingGroups'][0]['DesiredCapacity'] 305 | 306 | greenMin = 0 307 | greenMax = 0 308 | greenDesired = 0 309 | elif active == 1: 310 | greenMin = info['AutoScalingGroups'][1]['MinSize'] 311 | greenMax = info['AutoScalingGroups'][1]['MaxSize'] 312 | greenDesired = info['AutoScalingGroups'][1]['DesiredCapacity'] 313 | 314 | blueMin = 0 315 | blueMax = 0 316 | blueDesired = 0 317 | else: 318 | print('No acive AMI') 319 | sys.exit(1) 320 | 321 | updateAutoscaling(command, blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, projectPath, environment) 322 | 323 | 324 | def stopAutoscaling(info, active, ami, command, projectPath, environment): 325 | blueMin = info['AutoScalingGroups'][active]['MinSize'] 326 | blueMax = info['AutoScalingGroups'][active]['MaxSize'] 327 | blueDesired = 0 328 | 329 | greenMin = info['AutoScalingGroups'][active]['MinSize'] 330 | greenMax = info['AutoScalingGroups'][active]['MaxSize'] 331 | greenDesired = 0 332 | 333 | if active == 0: 334 | blueAMI = getAmi(info['AutoScalingGroups'][active]['LaunchConfigurationName']) 335 | greenAMI = ami 336 | elif active == 1: 337 | blueAMI = ami 338 | greenAMI = getAmi(info['AutoScalingGroups'][active]['LaunchConfigurationName']) 339 | else: 340 | print('No acive AMI') 341 | sys.exit(1) 342 | 343 | updateAutoscaling(command, blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, projectPath, environment) 344 | 345 | 346 | def updateAutoscaling(command, blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, projectPath, environment): 347 | command = 'terraform %s %s' % (command, buildTerraformVars(blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, environment)) 348 | print(command) 349 | process = subprocess.Popen(command, shell=True, cwd=projectPath, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 350 | out, err = process.communicate() 351 | print('stdoutput') 352 | print(out) 353 | if process.returncode != 0: 354 | print('stderror') 355 | print(err) 356 | sys.exit(process.returncode) 357 | 358 | 359 | def buildTerraformVars(blueMax, blueMin, blueDesired, blueAMI, greenMax, greenMin, greenDesired, greenAMI, environment): 360 | variables = { 361 | 'blue_max_size': blueMax, 362 | 'blue_min_size': blueMin, 363 | 'blue_desired_capacity': blueDesired, 364 | 'blue_ami': blueAMI, 365 | 'green_max_size': greenMax, 366 | 'green_min_size': greenMin, 367 | 'green_desired_capacity': greenDesired, 368 | 'green_ami': greenAMI 369 | } 370 | out = [] 371 | 372 | # When using terraform environments, set the environment tfvars file 373 | if environment is not None: 374 | out.append('-var-file=%s' % (environment)) 375 | 376 | for key, value in variables.iteritems(): 377 | out.append('-var \'%s=%s\'' % (key, value)) 378 | 379 | return ' '.join(out) 380 | 381 | 382 | if __name__ == "__main__": 383 | main(sys.argv[1:]) 384 | -------------------------------------------------------------------------------- /ci/ecs/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3 2 | awscli 3 | -------------------------------------------------------------------------------- /ci/ecs/sample.yml: -------------------------------------------------------------------------------- 1 | --- 2 | resource_types: 3 | - name: ami 4 | type: docker-image 5 | source: 6 | repository: jdub/ami-resource 7 | 8 | - name: git-skyscrapers 9 | type: git 10 | source: 11 | uri: git@github.com:skyscrapers/xxxxx.git 12 | branch: master 13 | private_key: ((PRIVATE_KEY)) 14 | - name: git-terraform-ecs 15 | type: git 16 | source: 17 | uri: git@github.com:skyscrapers/terraform-ecs.git 18 | branch: ecs-automation 19 | private_key: ((PRIVATE_KEY)) 20 | - name: ecs-ami 21 | type: ami 22 | check_every: 1h 23 | source: 24 | aws_access_key_id: ((AWS_CONCOURSE.key)) 25 | aws_secret_access_key: ((AWS_CONCOURSE.secret)) 26 | region: eu-west-1 27 | filters: 28 | owner-id: "591542846629" 29 | is-public: true 30 | state: available 31 | name: amzn-ami-*.a-amazon-ecs-optimized 32 | jobs: 33 | - name: update-ecs 34 | plan: 35 | - do: 36 | - aggregate: 37 | - get: git-skyscrapers 38 | - get: ecs-ami 39 | - get: git-terraform-ecs 40 | - task: deploy 41 | file: git-terraform-ecs/ci/ecs/update-ecs.yml 42 | input_mapping: 43 | ami: ecs-ami 44 | terraform-ecs: git-terraform-ecs 45 | terraform-repo: git-skyscrapers 46 | params: 47 | TF_PROJECT_FOLDER: stacks/ecs-cluster 48 | TF_ENVIRONMENT: production 49 | TF_VERSION: 0.11.7 50 | TIMEOUT: 900 51 | AWS_ACCESS_KEY_ID: (( AWS_CONCOURSE.key )) 52 | AWS_SECRET_ACCESS_KEY: (( AWS_CONCOURSE.secret )) 53 | -------------------------------------------------------------------------------- /ci/ecs/update-ecs.yml: -------------------------------------------------------------------------------- 1 | --- 2 | platform: linux 3 | image_resource: 4 | type: registry-image 5 | source: 6 | repository: "python" 7 | tag: "2.7.13-alpine" 8 | 9 | params: 10 | TF_PROJECT_FOLDER: 11 | TF_VERSION: "0.11.7" 12 | AWS_DEFAULT_REGION: eu-west-1 13 | TIMEOUT: 500 14 | TF_ENVIRONMENT: 15 | AWS_ACCESS_KEY_ID: 16 | AWS_SECRET_ACCESS_KEY: 17 | ASSUME_ROLE_ARN: 18 | 19 | inputs: 20 | - name: terraform-repo 21 | - name: terraform-ecs 22 | - name: ami 23 | optional: true 24 | 25 | run: 26 | path: sh 27 | args: 28 | - -exc 29 | - | 30 | # Install dependencies 31 | apk add --update unzip curl jq git 32 | 33 | # Install terraform 34 | curl -s -o "terraform.zip" "https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 35 | unzip terraform.zip 36 | export PATH=$PATH:$PWD 37 | 38 | # Install dependencies of the deployment script 39 | pip install -r terraform-ecs/ci/ecs/requirements.txt 40 | 41 | # Extract AMI id from packer manifest 42 | if [ -d "ami" ]; then 43 | AMI_ID=$(cat ami/id) 44 | fi 45 | 46 | WORKDIR=$PWD 47 | cd terraform-repo/$TF_PROJECT_FOLDER 48 | # Init terraform 49 | terraform init 50 | terraform workspace select $TF_ENVIRONMENT 51 | # Deploy 52 | $WORKDIR/terraform-ecs/ci/ecs/ecs-bluegreen.py -f $WORKDIR/terraform-repo/$TF_PROJECT_FOLDER -a $AMI_ID -c "apply -auto-approve" -t $TIMEOUT -e $WORKDIR/terraform-repo/$TF_PROJECT_FOLDER/$TF_ENVIRONMENT.tfvars --role-arn $ASSUME_ROLE_ARN 53 | -------------------------------------------------------------------------------- /ecs-cluster/iam.tf: -------------------------------------------------------------------------------- 1 | /** 2 | * Instance profile for instances launched by autoscaling 3 | */ 4 | 5 | resource "aws_iam_instance_profile" "ecs-instance-profile" { 6 | name = "ecs-instance-profile-${var.environment}" 7 | role = aws_iam_role.ecs-instance-role.name 8 | } 9 | 10 | resource "aws_iam_role" "ecs-instance-role" { 11 | name = "ecs-instance-role-${var.environment}" 12 | 13 | assume_role_policy = <> /etc/fstab 9 | - mount -a -t nfs4 10 | -------------------------------------------------------------------------------- /efs/main.tf: -------------------------------------------------------------------------------- 1 | resource "aws_efs_file_system" "efs" { 2 | encrypted = true 3 | 4 | tags = { 5 | Name = var.name 6 | Environment = terraform.workspace 7 | Project = var.project 8 | } 9 | } 10 | 11 | resource "aws_efs_mount_target" "efs" { 12 | file_system_id = aws_efs_file_system.efs.id 13 | count = var.subnet_amount 14 | subnet_id = element(var.subnets, count.index) 15 | security_groups = var.security_groups 16 | } 17 | 18 | data "template_file" "efs" { 19 | template = file("${path.module}/cloud-config.yaml.tpl") 20 | 21 | vars = { 22 | mount_point = var.mount_point 23 | dns_name = aws_efs_file_system.efs.dns_name 24 | } 25 | } 26 | 27 | -------------------------------------------------------------------------------- /efs/outputs.tf: -------------------------------------------------------------------------------- 1 | output "cloud_config" { 2 | value = data.template_file.efs.rendered 3 | } 4 | 5 | output "dns_name" { 6 | value = aws_efs_file_system.efs.dns_name 7 | } 8 | 9 | output "kms_key_id" { 10 | value = aws_efs_file_system.efs.kms_key_id 11 | } 12 | 13 | output "efs_id" { 14 | value = aws_efs_file_system.efs.id 15 | } 16 | 17 | output "mount_target_ids" { 18 | value = aws_efs_mount_target.efs.*.id 19 | } 20 | 21 | output "mount_target_interface_ids" { 22 | value = aws_efs_mount_target.efs.*.network_interface_id 23 | } 24 | 25 | -------------------------------------------------------------------------------- /efs/variables.tf: -------------------------------------------------------------------------------- 1 | variable "mount_point" { 2 | default = "/media/efs" 3 | } 4 | 5 | variable "name" { 6 | default = "efs" 7 | } 8 | 9 | variable "project" { 10 | default = "" 11 | } 12 | 13 | variable "security_groups" { 14 | default = [] 15 | type = list(string) 16 | } 17 | 18 | variable "subnets" { 19 | type = list(string) 20 | } 21 | 22 | # value of 'count' cannot be computed 23 | variable "subnet_amount" { 24 | type = string 25 | } 26 | 27 | -------------------------------------------------------------------------------- /efs/versions.tf: -------------------------------------------------------------------------------- 1 | 2 | terraform { 3 | required_version = ">= 0.12" 4 | } 5 | -------------------------------------------------------------------------------- /repository/iam.tf: -------------------------------------------------------------------------------- 1 | data "aws_iam_policy_document" "ecr-repo-push" { 2 | statement { 3 | sid = "1" 4 | 5 | actions = [ 6 | "ecr:DescribeRepositories", 7 | "ecr:ListImages", 8 | "ecr:DescribeRepositories", 9 | "ecr:ListImages", 10 | "ecr:BatchCheckLayerAvailability", 11 | "ecr:BatchGetImage", 12 | "ecr:DescribeImages", 13 | "ecr:GetAuthorizationToken", 14 | "ecr:GetDownloadUrlForLayer", 15 | "ecr:GetRepositoryPolicy", 16 | "ecr:CompleteLayerUpload", 17 | "ecr:InitiateLayerUpload", 18 | "ecr:PutImage", 19 | "ecr:UploadLayerPart", 20 | ] 21 | 22 | resources = [ 23 | aws_ecr_repository.ecr-repo.arn, 24 | ] 25 | } 26 | } 27 | 28 | resource "aws_iam_policy" "ecr-repo-push" { 29 | name = "ecr-${var.repository_name}-push" 30 | path = "/" 31 | description = "Push access to the ${var.repository_name} repository" 32 | 33 | policy = data.aws_iam_policy_document.ecr-repo-push.json 34 | } 35 | 36 | data "aws_iam_policy_document" "ecr-repo-pull" { 37 | statement { 38 | sid = "1" 39 | 40 | actions = [ 41 | "ecr:DescribeRepositories", 42 | "ecr:ListImages", 43 | "ecr:BatchCheckLayerAvailability", 44 | "ecr:BatchGetImage", 45 | "ecr:DescribeImages", 46 | "ecr:GetAuthorizationToken", 47 | "ecr:GetDownloadUrlForLayer", 48 | "ecr:GetRepositoryPolicy", 49 | ] 50 | 51 | resources = [ 52 | aws_ecr_repository.ecr-repo.arn, 53 | ] 54 | } 55 | } 56 | 57 | resource "aws_iam_policy" "ecr-repo-pull" { 58 | name = "ecr-${var.repository_name}-pull" 59 | path = "/" 60 | description = "Pull access to the ${var.repository_name} repository" 61 | 62 | policy = data.aws_iam_policy_document.ecr-repo-pull.json 63 | } 64 | 65 | -------------------------------------------------------------------------------- /repository/main.tf: -------------------------------------------------------------------------------- 1 | resource "aws_ecr_repository" "ecr-repo" { 2 | name = var.repository_name 3 | } 4 | 5 | resource "aws_ecr_lifecycle_policy" "ecr-repo-policy" { 6 | count = var.expire_after == 0 ? 0 : 1 7 | repository = aws_ecr_repository.ecr-repo.name 8 | 9 | policy = <> /etc/ecs/ecs.config 47 | start ecs 48 | 49 | cd /tmp 50 | curl -L "https://get.gravitational.com/teleport-v${var.teleport_version}-linux-amd64-bin.tar.gz" > ./teleport.tar.gz 51 | sudo tar -xzf ./teleport.tar.gz 52 | sudo ./teleport/install 53 | cd /opt 54 | curl -LO "https://github.com/prometheus/node_exporter/releases/download/v${var.node_exporter_version}/node_exporter-${var.node_exporter_version}.linux-amd64.tar.gz" 55 | tar -xzf node_exporter-${var.node_exporter_version}.linux-amd64.tar.gz 56 | mv node_exporter-${var.node_exporter_version}.linux-amd64/node_exporter /usr/local/bin/ 57 | chmod +x /usr/local/bin/node_exporter 58 | systemctl daemon-reload 59 | systemctl enable node_exporter.service 60 | systemctl start node_exporter 61 | EOF 62 | } 63 | 64 | part { 65 | content_type = "text/x-shellscript" 66 | content = module.teleport_bootstrap_script.teleport_bootstrap_script 67 | } 68 | 69 | part { 70 | content_type = "text/cloud-config" 71 | 72 | content = <> /etc/fstab 78 | - mount -a -t nfs4 79 | - chmod -R 777 ${var.efs_mount_point} 80 | EOF 81 | } 82 | } 83 | 84 | module "teleport_bootstrap_script" { 85 | source = "github.com/skyscrapers/terraform-teleport//teleport-bootstrap-script?ref=7.2.1" 86 | auth_server = var.teleport_server 87 | auth_token = var.teleport_auth_token 88 | function = "ecs" 89 | environment = var.environment 90 | project = var.project 91 | } 92 | -------------------------------------------------------------------------------- /user-data/outputs.tf: -------------------------------------------------------------------------------- 1 | output "userdata" { 2 | value = data.template_cloudinit_config.teleport_bootstrap.rendered 3 | } 4 | 5 | -------------------------------------------------------------------------------- /user-data/templates/node-exporter.service.tpl: -------------------------------------------------------------------------------- 1 | - content: | 2 | [Unit] 3 | Description=Node Exporter 4 | 5 | [Service] 6 | Type=simple 7 | Restart=on-failure 8 | ExecStart=/usr/local/bin/node_exporter --collector.textfile.directory /var/lib/node_exporter/textfile_collector 9 | PIDFile=/var/run/node_exporter.pid 10 | ExecReload=/bin/kill -HUP $MAINPID 11 | [Install] 12 | WantedBy=multi-user.target 13 | path: ${service_type_path} 14 | permissions: '${file_permissions}' 15 | -------------------------------------------------------------------------------- /user-data/variables.tf: -------------------------------------------------------------------------------- 1 | variable "environment" { 2 | } 3 | 4 | variable "project" { 5 | } 6 | 7 | variable "cluster_name" { 8 | } 9 | 10 | variable "teleport_version" { 11 | default = "10.1.4" 12 | } 13 | 14 | variable "teleport_server" { 15 | default = "" 16 | } 17 | 18 | variable "teleport_auth_token" { 19 | default = "" 20 | } 21 | 22 | variable "efs_mount_point" { 23 | default = "/efs" 24 | } 25 | 26 | variable "efs_dns_name" { 27 | default = "" 28 | } 29 | 30 | variable "node_exporter_version" { 31 | default = "0.16.0" 32 | } 33 | -------------------------------------------------------------------------------- /user-data/versions.tf: -------------------------------------------------------------------------------- 1 | 2 | terraform { 3 | required_version = ">= 0.12" 4 | } 5 | --------------------------------------------------------------------------------