├── .gitignore ├── LICENSE ├── README.md ├── cfn-template.yml ├── nested.yml └── pre-commit /.gitignore: -------------------------------------------------------------------------------- 1 | .idea/ 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Will Jordan 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # aws-codepipeline-nested-stack 2 | 3 | This project demonstrates how to set up an "infrastructure continuous delivery" architecture using GitHub, AWS CodePipeline and CloudFormation, with a project containing a nested stack. 4 | 5 | ## Getting Started 6 | 7 | 1. [Fork this repo](fork). 8 | 2. Bootstrap the CloudFormation stack: 9 | 1. [![Launch Stack](https://cdn.rawgit.com/buildkite/cloudformation-launch-stack-button-svg/master/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/new?stackName=aws-codepipeline-nested-stack&templateURL=https://s3.amazonaws.com/wjordan-aws-codepipeline/cfn-template.yml) 10 | 2. Enter the forked repo's owner in the `GitHubOwner` field. 11 | 3. Create a [New personal access token](https://github.com/settings/tokens/new) with `repo` and `admin:repo_hook` scopes, and enter the token in the `GitHubToken` field. 12 | 4. Enter the name of an existing S3 bucket for storing pipeline artifacts in the `ArtifactBucket` field. ([Create a bucket](http://docs.aws.amazon.com/AmazonS3/latest/gsg/CreatingABucket.html) first if necessary.) 13 | 3. Verify the newly-created stack and pipeline. 14 | 1. Check the [CloudFormation Console](https://console.aws.amazon.com/cloudformation) to ensure your stack reaches the `CREATE_COMPLETE` state successfully. 15 | 2. Check the [CodePipeline Console](https://console.aws.amazon.com/codepipeline) to ensure the pipeline's `Source` and `Deploy` stages both completed successfully. 16 | 4. Update the parent CloudFormation stack: 17 | 1. Modify `cfn-template.yml` in the Git repository, and commit/push the change. 18 | 2. For example, try renaming the `Dummy` resource to `dummy2`. 19 | 5. Update the child CloudFormation stack: 20 | 1. Modify `nested.yml` in the Git repository, and commit/push the change. 21 | 2. For example, try renaming the `Dummy` resource to `Dummy2`. 22 | 6. Verify the stack update(s). 23 | a. Check the CodePipeline Console to ensure the pipeline processes the new commit in both stages. 24 | b. Check the CloudFormation Console to ensure your stack reaches the `UPDATE_COMPLETE` state successfully. 25 | c. Verify the created/updated resources in the `Resources` tab of the CloudFormation console match the values in the new template. 26 | 27 | That's it! 28 | 29 | **Note**: The [CloudFormation Service Role](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-servicerole.html) (`CFNRole`) grans full **admin** permissions (`'*'`) to your AWS account. 30 | 31 | For more restricted, fine-grained security, you should move the `CFNRole` and `PipelineRole` resources into a separate CloudFormation stack (or just create them manually), reference them using [`Fn::ImportValue`](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html) (or by a fixed-string name), and ensure that `CFNRole` [grants least privilege](http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege) depending on the Resources in your stack. 32 | 33 | ## References 34 | 35 | Talk from re:Invent 2016, "[Infrastructure Continuous Delivery Using AWS CloudFormation](https://www.youtube.com/watch?v=TDalsML3QqY)" 36 | 37 | -------------------------------------------------------------------------------- /cfn-template.yml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: 2010-09-09 2 | Description: Infrastructure Continuous Delivery with CodePipeline and CloudFormation, with a project containing a nested stack. 3 | Parameters: 4 | ArtifactBucket: 5 | Type: String 6 | Description: Name of existing S3 bucket for storing pipeline artifacts 7 | StackFilename: 8 | Type: String 9 | Default: cfn-template.yml 10 | Description: CloudFormation stack template filename in the Git repo 11 | GitHubOwner: 12 | Type: String 13 | Description: GitHub repository owner 14 | GitHubRepo: 15 | Type: String 16 | Default: aws-codepipeline-nested-stack 17 | Description: GitHub repository name 18 | GitHubBranch: 19 | Type: String 20 | Default: master 21 | Description: GitHub repository branch 22 | GitHubToken: 23 | Type: String 24 | Description: GitHub repository OAuth token 25 | NestedStackFilename: 26 | Type: String 27 | Description: GitHub filename (and S3 Object Key) for nested stack template. 28 | Default: nested.yml 29 | Resources: 30 | Pipeline: 31 | Type: AWS::CodePipeline::Pipeline 32 | Properties: 33 | RoleArn: !GetAtt [PipelineRole, Arn] 34 | ArtifactStore: 35 | Type: S3 36 | Location: !Ref ArtifactBucket 37 | Stages: 38 | - Name: Source 39 | Actions: 40 | - Name: Source 41 | ActionTypeId: 42 | Category: Source 43 | Owner: ThirdParty 44 | Version: 1 45 | Provider: GitHub 46 | Configuration: 47 | Owner: !Ref GitHubOwner 48 | Repo: !Ref GitHubRepo 49 | Branch: !Ref GitHubBranch 50 | OAuthToken: !Ref GitHubToken 51 | OutputArtifacts: [Name: Template] 52 | RunOrder: 1 53 | - Name: Deploy 54 | Actions: 55 | - Name: S3Upload 56 | ActionTypeId: 57 | Category: Invoke 58 | Owner: AWS 59 | Provider: Lambda 60 | Version: 1 61 | InputArtifacts: [Name: Template] 62 | Configuration: 63 | FunctionName: !Ref S3UploadObject 64 | UserParameters: !Ref NestedStackFilename 65 | RunOrder: 1 66 | - Name: Deploy 67 | RunOrder: 2 68 | ActionTypeId: 69 | Category: Deploy 70 | Owner: AWS 71 | Version: 1 72 | Provider: CloudFormation 73 | InputArtifacts: [Name: Template] 74 | Configuration: 75 | ActionMode: REPLACE_ON_FAILURE 76 | RoleArn: !GetAtt [CFNRole, Arn] 77 | StackName: !Ref AWS::StackName 78 | TemplatePath: !Sub "Template::${StackFilename}" 79 | Capabilities: CAPABILITY_IAM 80 | ParameterOverrides: !Sub | 81 | { 82 | "ArtifactBucket": "${ArtifactBucket}", 83 | "StackFilename": "${StackFilename}", 84 | "GitHubOwner": "${GitHubOwner}", 85 | "GitHubRepo": "${GitHubRepo}", 86 | "GitHubBranch": "${GitHubBranch}", 87 | "GitHubToken": "${GitHubToken}", 88 | "NestedStackFilename": "${NestedStackFilename}" 89 | } 90 | CFNRole: 91 | Type: AWS::IAM::Role 92 | Properties: 93 | AssumeRolePolicyDocument: 94 | Statement: 95 | - Action: ['sts:AssumeRole'] 96 | Effect: Allow 97 | Principal: {Service: [cloudformation.amazonaws.com]} 98 | Version: '2012-10-17' 99 | Path: / 100 | ManagedPolicyArns: 101 | # TODO grant least privilege to only allow managing your CloudFormation stack resources 102 | - "arn:aws:iam::aws:policy/AdministratorAccess" 103 | PipelineRole: 104 | Type: AWS::IAM::Role 105 | Properties: 106 | AssumeRolePolicyDocument: 107 | Statement: 108 | - Action: ['sts:AssumeRole'] 109 | Effect: Allow 110 | Principal: {Service: [codepipeline.amazonaws.com]} 111 | Version: '2012-10-17' 112 | Path: / 113 | Policies: 114 | - PolicyName: CodePipelineAccess 115 | PolicyDocument: 116 | Version: '2012-10-17' 117 | Statement: 118 | - Action: 119 | - 's3:*' 120 | - 'cloudformation:*' 121 | - 'iam:PassRole' 122 | - 'lambda:*' 123 | Effect: Allow 124 | Resource: '*' 125 | Dummy: 126 | Type: AWS::CloudFormation::WaitConditionHandle 127 | NestedStack: 128 | Type: AWS::CloudFormation::Stack 129 | Properties: 130 | TemplateURL: !Sub "https://s3.amazonaws.com/${ArtifactBucket}/${NestedStackFilename}" 131 | S3UploadObject: 132 | Type: AWS::Lambda::Function 133 | Properties: 134 | Description: Extracts and uploads the specified InputArtifact file to S3. 135 | Handler: index.handler 136 | Role: !GetAtt LambdaExecutionRole.Arn 137 | Code: 138 | ZipFile: !Sub | 139 | var exec = require('child_process').exec; 140 | var AWS = require('aws-sdk'); 141 | var codePipeline = new AWS.CodePipeline(); 142 | exports.handler = function(event, context, callback) { 143 | var job = event["CodePipeline.job"]; 144 | var s3Download = new AWS.S3({ 145 | credentials: job.data.artifactCredentials, 146 | signatureVersion: 'v4' 147 | }); 148 | var s3Upload = new AWS.S3({ 149 | signatureVersion: 'v4' 150 | }); 151 | var jobId = job.id; 152 | function respond(e) { 153 | var params = {jobId: jobId}; 154 | if (e) { 155 | params['failureDetails'] = { 156 | message: JSON.stringify(e), 157 | type: 'JobFailed', 158 | externalExecutionId: context.invokeid 159 | }; 160 | codePipeline.putJobFailureResult(params, (err, data) => callback(e)); 161 | } else { 162 | codePipeline.putJobSuccessResult(params, (err, data) => callback(e)); 163 | } 164 | } 165 | var filename = job.data.actionConfiguration.configuration.UserParameters; 166 | var location = job.data.inputArtifacts[0].location.s3Location; 167 | var bucket = location.bucketName; 168 | var key = location.objectKey; 169 | var tmpFile = '/tmp/file.zip'; 170 | s3Download.getObject({Bucket: bucket, Key: key}) 171 | .createReadStream() 172 | .pipe(require('fs').createWriteStream(tmpFile)) 173 | .on('finish', ()=>{ 174 | exec(`unzip -p ${!tmpFile} ${!filename}`, (err, stdout)=>{ 175 | if (err) { respond(err); } 176 | s3Upload.putObject({Bucket: bucket, Key: filename, Body: stdout}, (err, data) => respond(err)); 177 | }); 178 | }); 179 | }; 180 | Timeout: 30 181 | Runtime: nodejs4.3 182 | LambdaExecutionRole: 183 | Type: AWS::IAM::Role 184 | Properties: 185 | AssumeRolePolicyDocument: 186 | Version: '2012-10-17' 187 | Statement: 188 | - Effect: Allow 189 | Principal: {Service: [lambda.amazonaws.com]} 190 | Action: ['sts:AssumeRole'] 191 | Path: / 192 | ManagedPolicyArns: 193 | - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 194 | - "arn:aws:iam::aws:policy/AWSCodePipelineCustomActionAccess" 195 | Policies: 196 | - PolicyName: S3Policy 197 | PolicyDocument: 198 | Version: '2012-10-17' 199 | Statement: 200 | - Effect: Allow 201 | Action: 202 | - 's3:PutObject' 203 | - 's3:PutObjectAcl' 204 | Resource: !Sub "arn:aws:s3:::${ArtifactBucket}/${NestedStackFilename}" 205 | -------------------------------------------------------------------------------- /nested.yml: -------------------------------------------------------------------------------- 1 | Resources: 2 | Dummy: 3 | Type: AWS::CloudFormation::WaitConditionHandle 4 | -------------------------------------------------------------------------------- /pre-commit: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # .git/hooks/pre-commit 4 | # 5 | 6 | aws cloudformation package \ 7 | --template-file ./template.yml \ 8 | --s3-bucket cfn-templates 9 | --output-template-file template-package.yml 10 | git add template-package.yml 11 | --------------------------------------------------------------------------------