├── .github └── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md ├── .gitignore ├── README.md ├── automationdocument.tf ├── examples └── amazon-ami-default-example │ ├── README.md │ ├── example.auto.tfvars │ └── main.tf ├── iam.tf ├── labels.tf ├── lambda.tf ├── lambda ├── lambda-trigger-automation.py ├── lambda-update-autoscaling-groups.py └── lambda-update-parameter-store.py ├── linux-user-data.sh ├── main.tf ├── outputs.tf └── variables.tf /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | 5 | --- 6 | 7 | **Describe the bug** 8 | A clear and concise description of what the bug is. 9 | 10 | **To Reproduce** 11 | Steps to reproduce the behavior: 12 | 1. Go to '...' 13 | 2. Click on '....' 14 | 3. Scroll down to '....' 15 | 4. See error 16 | 17 | **Expected behavior** 18 | A clear and concise description of what you expected to happen. 19 | 20 | **Screenshots** 21 | If applicable, add screenshots to help explain your problem. 22 | 23 | **Desktop (please complete the following information):** 24 | - OS: [e.g. iOS] 25 | - Browser [e.g. chrome, safari] 26 | - Version [e.g. 22] 27 | 28 | **Smartphone (please complete the following information):** 29 | - Device: [e.g. iPhone6] 30 | - OS: [e.g. iOS8.1] 31 | - Browser [e.g. stock browser, safari] 32 | - Version [e.g. 22] 33 | 34 | **Additional context** 35 | Add any other context about the problem here. 36 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | 5 | --- 6 | 7 | **Is your feature request related to a problem? Please describe.** 8 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 9 | 10 | **Describe the solution you'd like** 11 | A clear and concise description of what you want to happen. 12 | 13 | **Describe alternatives you've considered** 14 | A clear and concise description of any alternative solutions or features you've considered. 15 | 16 | **Additional context** 17 | Add any other context or screenshots about the feature request here. 18 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.tfstate 2 | *.tfstate.* 3 | **/*.pem 4 | **/.terraform 5 | **/terraform.tfstate 6 | **/terraform.tfstate.backup 7 | 8 | dev.auto.tfvars 9 | *.zip 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # terraform-aws-ssm-ami-bakery 2 | 3 | This Terraform Module creates AWS AMI's that can be easily kept up to date, automatically applying operating system (OS) patches to a Windows or Linux AMI that is already considered to be the most up-to-date or latest AMI. In the example, the default value of the parameter `SourceAmiId` is defined by a Systems Manager Parameter Store parameter called `latestAmi`. The value of `latestAmi` is updated by an AWS Lambda function invoked at the end of the Automation workflow. As a result of this Automation process, the time and effort spent patching AMIs is minimized. 4 | 5 | ## TODO: 6 | - [x] Create Lambda functions 7 | - [x] Create Automation documents 8 | - [x] Test pipeline 9 | - [x] Create a basic example 10 | - [ ] Create a full README.md file 11 | - [ ] [Enable logging](https://docs.aws.amazon.com/systems-manager/latest/userguide/monitoring-ssm-agent.html) of the update processes to cloudwatch 12 | - [ ] Provide inputs for specifying a subnet to launch in 13 | - [ ] Split the single role into 2 roles, one for lambda and one for SSM. 14 | - [ ] Provide inputs for security groups to attach 15 | - [ ] Create output SNS queue, and write the queue arn to Parameter store, so that events can be chained together 16 | - [ ] Import the file `linux-user-data.sh` append the variable `var.additional_userdata` to it, base64 encode it and update the SSM document 17 | - [ ] Create Submodule: Allow setting expiry of old images 18 | - [x] Allow updating of a specified ASG arn to use the new AMI 19 | - [x] Add the ability to wait for approval before updating the ASG 20 | 21 | ## Simple Example 22 | 23 | **This will:** 24 | - Create a SSM automation document for linux 25 | - Create appropriate roles and policies to run the process 26 | - Create the lambda functions needed to trigger the process 27 | - Create the Parameter Store Name `/cp/dev/amazon-linux/LatestAmi` and store the value of `ami` in it 28 | - Provide parameters to the lambda function such as a name template name template "{namespace}-{stage}-{name}-{date}" 29 | - Subscribe the lambda function to an SNS topic for it to be triggered as per [this aws example](http://docs.amazonaws.cn/en_us/AWSEC2/latest/UserGuide/amazon-linux-ami-basics.html#linux-ami-notifications) 30 | 31 | 32 | **When the lambda is triggered it will:** 33 | - Launch the specified AMI as a new instance 34 | - Generate a new ami with the name template "{namespace}-{stage}-{name}-{date}" and in the example below the output template is `cp-dev-amazon-linux-{{global:DATE_TIME}}` 35 | - Install the latest SSM agent on it if it is not installed already. 36 | - Update all of the OS patches (`yum update -y`) 37 | - Shut down the instance 38 | - Terminate the instance 39 | - Trigger the second lambda function to update the Parameter Store Name `/cp/dev/amazon-linux/LatestAmi` with the new AMI id. 40 | 41 | ```hcl 42 | provider "aws" { 43 | region = "eu-west-2" 44 | } 45 | 46 | module "keep_ami_patched" { 47 | source = "git::git@github.com:bitflight-public/terraform-aws-ssm-ami-bakery.git" 48 | namespace = "cp" 49 | stage = "dev" 50 | name = "amazon-linux" 51 | ami = "ami-dc2ecebb" // eu-west-2 amazon ami, but you would normally start with your own customised ami 52 | } 53 | 54 | # For Amazon Linux updates, they are only released in an SNS feed from us-east-1 55 | # Pin a provider to us-east-1 to create a topic subscription to that SNS feed. 56 | # Below we are subscribing the lambda function to this release notification, 57 | # This means that on each release from amazon, we will update our custom image to match. 58 | provider "aws" { 59 | region = "us-east-1" 60 | alias = "us-east-1" 61 | } 62 | 63 | resource "aws_sns_topic_subscription" "trigger_automation" { 64 | provider = "aws.us-east-1" 65 | topic_arn = "arn:aws:sns:us-east-1:137112412989:amazon-linux-ami-updates" 66 | protocol = "lambda" 67 | endpoint = "${module.keep_ami_patched.lambda_endpoint_arn}" 68 | } 69 | ``` 70 | -------------------------------------------------------------------------------- /automationdocument.tf: -------------------------------------------------------------------------------- 1 | resource "aws_ssm_document" "document_linux" { 2 | count = "${var.os_type == "linux" ? 1 : 0 }" 3 | name = "${module.label.id}-linux-document" 4 | document_type = "Automation" 5 | 6 | content = < /tmp/aws-update-linux-instance", 113 | "chmod +x /tmp/aws-update-linux-instance", 114 | "/tmp/aws-update-linux-instance --pre-update-script '{{PreUpdateScript}}' --post-update-script '{{PostUpdateScript}}' --include-packages '{{IncludePackages}}' --exclude-packages '{{ExcludePackages}}' 2>&1 | tee /tmp/aws-update-linux-instance.log" 115 | ] 116 | } 117 | } 118 | }, 119 | { 120 | "name": "stopInstance", 121 | "action": "aws:changeInstanceState", 122 | "maxAttempts": 3, 123 | "timeoutSeconds": 1200, 124 | "onFailure": "Abort", 125 | "inputs": { 126 | "InstanceIds": [ 127 | "{{launchInstance.InstanceIds}}" 128 | ], 129 | "DesiredState": "stopped" 130 | } 131 | }, 132 | { 133 | "name": "createImage", 134 | "action": "aws:createImage", 135 | "maxAttempts": 3, 136 | "onFailure": "Abort", 137 | "inputs": { 138 | "InstanceId": "{{launchInstance.InstanceIds}}", 139 | "ImageName": "{{TargetAmiName}}", 140 | "NoReboot": true, 141 | "ImageDescription": "AMI Generated by EC2 Automation on {{global:DATE_TIME}} from {{SourceAmiId}}" 142 | } 143 | }, 144 | { 145 | "name": "terminateInstance", 146 | "action": "aws:changeInstanceState", 147 | "maxAttempts": 3, 148 | "onFailure": "Continue", 149 | "inputs": { 150 | "InstanceIds": [ 151 | "{{launchInstance.InstanceIds}}" 152 | ], 153 | "DesiredState": "terminated" 154 | } 155 | }, 156 | { 157 | "name":"updateSsmParam", 158 | "action":"aws:invokeLambdaFunction", 159 | "timeoutSeconds":1200, 160 | "maxAttempts":1, 161 | "onFailure":"Abort", 162 | "inputs":{ 163 | "FunctionName":"{{SSMAmiLambdaFunctionName}}", 164 | "Payload":"{\"parameterName\":\"{{SourceAmiParameterName}}\", \"parameterValue\":\"{{createImage.ImageId}}\"}" 165 | } 166 | } 167 | ${local.approval_request}, 168 | { 169 | "name":"updateASG", 170 | "action":"aws:invokeLambdaFunction", 171 | "timeoutSeconds":1200, 172 | "maxAttempts":1, 173 | "onFailure":"Abort", 174 | "inputs": { 175 | "FunctionName": "{{SSMAutomationUpdateAsg}}", 176 | "Payload": "{\"targetASG\":\"{{targetASG}}\", \"newAmiID\":\"{{createImage.ImageId}}\"}" 177 | } 178 | } 179 | ], 180 | "outputs": [ 181 | "createImage.ImageId" 182 | ] 183 | } 184 | DOC 185 | } 186 | 187 | locals { 188 | approval_request_format = { 189 | "NotificationArn" = "{{ApprovalNotificationArn}}" 190 | "Message" = "Please the approve the update of the ASGs." 191 | "timeoutSeconds" = 86000 192 | "onFailure" = "Abort" 193 | "MinRequiredApprovals" = "${min(var.min_num_approvers, length(var.approvers_list))}" 194 | "Approvers" = ["${var.approvers_list}"] 195 | } 196 | 197 | approval_request = "${var.require_approval_to_update_asg == "true" ? ",${jsonencode(local.approval_request_format)}" : ""}" 198 | } 199 | 200 | resource "aws_sns_topic" "approve_asg_updates" { 201 | count = "${var.require_approval_to_update_asg == "true" ? 1 : 0}" 202 | name_prefix = "Automation-${module.label.id}" 203 | } 204 | 205 | resource "aws_ssm_document" "document_windows" { 206 | count = "${var.os_type == "windows" ? 1 : 0 }" 207 | name = "${module.label.id}-windows-document" 208 | document_type = "Automation" 209 | 210 | content = < "$SCRIPT_NAME" 37 | FILE_SIZE=$(du -k /tmp/$SCRIPT_NAME | cut -f1) 38 | echo AWS-UpdateLinuxAmi: Finished downloading script, size: $FILE_SIZE 39 | if [ $FILE_SIZE -gt 0 ]; then 40 | break 41 | else 42 | if [[ $RETRY_COUNT -lt MAX_RETRY_COUNT ]]; then 43 | RETRY_COUNT=$((RETRY_COUNT+1)); 44 | echo AWS-UpdateLinuxAmi: FileSize is 0, retryCount: $RETRY_COUNT 45 | fi 46 | fi 47 | done 48 | 49 | if [ $FILE_SIZE -gt 0 ]; then 50 | chmod +x "$SCRIPT_NAME" 51 | echo AWS-UpdateLinuxAmi: Running UpdateSSMAgent script now .... 52 | ./"$SCRIPT_NAME" --region "$REGION" 53 | else 54 | echo AWS-UpdateLinuxAmi: Unable to download script, quitting .... 55 | fi -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | # SSM AMI Updater 2 | 3 | # Use this data source to validtate that the AMI exists. 4 | data "aws_ami" "info" { 5 | filter { 6 | name = "image-id" 7 | values = ["${var.ami}"] 8 | } 9 | } 10 | 11 | locals { 12 | ami = "${data.aws_ami.info.id}" 13 | } 14 | 15 | module "parameter" { 16 | source = "git::git@github.com:Jamie-BitFlight/terraform-aws-parameter-store.git?ref=bitflight/master" 17 | 18 | parameter_write = [{ 19 | name = "/${var.namespace}/${var.stage}/${var.name}/LatestAmi" 20 | value = "${local.ami}" 21 | type = "String" 22 | overwrite = "true" 23 | }] 24 | } 25 | -------------------------------------------------------------------------------- /outputs.tf: -------------------------------------------------------------------------------- 1 | output "lambda_endpoint_arn" { 2 | value = "${aws_lambda_function.lambda_trigger_automation.arn}" 3 | } 4 | 5 | output "document_name" { 6 | value = "${var.os_type == "linux" ? join("",aws_ssm_document.document_linux.*.name) : "${var.os_type == "windows" ? join("",aws_ssm_document.document_linux.*.name) : "none"}" }" 7 | } 8 | 9 | output "source_ami_id" { 10 | value = "${local.ami}" 11 | } 12 | 13 | output "target_ami_name" { 14 | value = "${local.target_ami_name}" 15 | } 16 | 17 | output "instance_iam_role" { 18 | value = "${aws_iam_instance_profile.ec2_profile.name}" 19 | } 20 | 21 | output "instance_type" { 22 | value = "${var.instance_type}" 23 | } 24 | 25 | output "pre_update_script_url" { 26 | value = "${var.pre_update_script_url}" 27 | } 28 | 29 | output "post_update_script_url" { 30 | value = "${var.post_update_script_url}" 31 | } 32 | 33 | output "include_packages" { 34 | value = "${var.include_packages}" 35 | } 36 | 37 | output "exclude_packages" { 38 | value = "${var.exclude_packages}" 39 | } 40 | 41 | output "source_ami_parameter_name" { 42 | value = "${join("",module.parameter.names)}" 43 | } 44 | 45 | output "source_ami_parameter_value" { 46 | value = "${join("",module.parameter.values)}" 47 | } 48 | -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | # Label variables are stored in the labels.tf file for portibility. 2 | 3 | variable "subnet" { 4 | type = "string" 5 | description = "Which subnet should the AMI be in when building" 6 | default = "" 7 | } 8 | 9 | variable "ami" { 10 | type = "string" 11 | description = "Which base AMI should this module use" 12 | default = "" 13 | } 14 | 15 | variable "os_type" { 16 | type = "string" 17 | default = "linux" 18 | description = "Is the AMI of 'linux' or 'windows'" 19 | } 20 | 21 | variable "additional_role_arns" { 22 | type = "list" 23 | description = "A list of additional role ARNs to attach to the SSM role, that is also attached to the AMI while it builds." 24 | default = [] 25 | } 26 | 27 | variable "target_ami_name_override" { 28 | type = "string" 29 | description = "By default the generated AMI name is based on the format of the terraform-null-label id. This can be used to override that." 30 | default = "" 31 | } 32 | 33 | variable "instance_type" { 34 | type = "string" 35 | description = "Which instance type should be used to build the AMI" 36 | default = "t2.small" 37 | } 38 | 39 | variable "include_packages" { 40 | type = "string" 41 | description = "Only update these named packages. Defaults to 'all' packages." 42 | default = "all" 43 | } 44 | 45 | variable "exclude_packages" { 46 | type = "string" 47 | description = "Exclude these named packages. Defaults to 'none'" 48 | default = "none" 49 | } 50 | 51 | variable "pre_update_script_url" { 52 | type = "string" 53 | description = "URL of a script to run before updates are applied. Default ('none') is to not run a script" 54 | default = "none" 55 | } 56 | 57 | variable "post_update_script_url" { 58 | type = "string" 59 | description = "URL of a script to run after package updates are applied. Default ('none') is to not run a script." 60 | default = "none" 61 | } 62 | 63 | variable "activate" { 64 | type = "string" 65 | description = "If set to true, a build will be run now" 66 | default = "false" 67 | } 68 | 69 | variable "additional_userdata" { 70 | type = "string" 71 | description = "User data for the build" 72 | default = "" 73 | } 74 | 75 | variable "kms_key_arn" { 76 | type = "string" 77 | description = "KMS Key for decrypting/encrypting SSM Parameter store values" 78 | default = "" 79 | } 80 | 81 | variable "target_asg" { 82 | type = "string" 83 | description = "Automatically update this autoscaling group arn to use the new AMI if they were using the old AMI" 84 | default = "" 85 | } 86 | 87 | variable "approvers_list" { 88 | type = "list" 89 | 90 | description = <