├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── __init__.py ├── cloudformation ├── codepipeline_ci_cd.yaml └── lambda_deployer.yaml ├── config └── codebuild │ ├── buildspec_deploy_prod_lambda_functions.yaml │ ├── buildspec_deploy_stage_lambda_functions.yaml │ ├── buildspec_e2e_tests.yaml │ ├── buildspec_lint_test.yaml │ ├── buildspec_read_inputs.yaml │ ├── buildspec_template_cfm.yaml │ └── buildspec_unit_tests.yaml ├── images ├── architecture.png ├── cloudformation_pipeline.png └── codecommit_repository.png ├── lambdas ├── __init__.py ├── add │ ├── __init__.py │ ├── add.py │ └── requirements.txt ├── cleaninput │ ├── __init__.py │ ├── cleaninput.py │ └── requirements.txt ├── divide │ ├── __init__.py │ ├── divide.py │ └── requirements.txt ├── multiply │ ├── __init__.py │ ├── multiply.py │ └── requirements.txt ├── requirements.txt └── subtract │ ├── __init__.py │ ├── requirements.txt │ └── subtract.py ├── requirements.txt ├── sm_def.json ├── template_statemachine_cf.py └── tests ├── e2e ├── PipelineExecutionMetadata.py ├── StepFunctionClient.py └── test_e2e.py ├── requirements.txt └── unit └── lambdas ├── test_add.py ├── test_cleaninput.py ├── test_divide.py ├── test_multiply.py └── test_subtract.py /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AWS Step Function CI/CD Pipeline 2 | 3 | > Create a CI/CD Pipeline to test and deploy a Step Function State Machine. 4 | 5 | ## Overview 6 | 7 | This project shows you how to use AWS CodePipeline, AWS CodeBuild, and Serverless Application Model (SAM) to test and deploy an AWS Step Function state machine. This example enables developers to have more confidence that their Step Functions are performing as they would expect before releasing them to production. 8 | 9 | ## Solution Overview 10 | 11 | ![architecture](images/architecture.png) 12 | 13 | This project will create a CodePipeline project that performs the following steps: 14 | 15 | 1. Pull source code from AWS CodeCommit. 16 | 2. Lint AWS Step Function Amazon States Language (ASL) definition. 17 | 3. Run unit tests against AWS Lambda functions that back the state machine. 18 | 4. Use SAM to deploy stage versions of AWS Lambda functions. 19 | 5. Build CloudFormation template artifact for releasing stage state machine. 20 | 6. Deploy stage CloudFormation template to create stage state machine. 21 | 7. Run end-to-end tests against stage Step Function state machine. 22 | 8. Send production release approval email to approvers. 23 | 9. Run CloudFormation delete against stage stack. 24 | 10. Build CloudFormation template artifact for releasing prod state machine. 25 | 11. Use SAM to deploy production version of AWS Lambda functions. 26 | 12. Deploy prod CloudFormation template to create prod state machine. 27 | 28 | ## Pre-requisites 29 | 30 | - Create or use an existing AWS account 31 | - Install Python >= 3.7 32 | - Git Client is installed on your desktop 33 | - Install AWS CLI (if deploying pipeline with CLI) 34 | - OpenSSL is installed or you have another way to Base64 encode a string 35 | 36 | ## Breakdown of Directories 37 | 38 | - [cloudformation](cloudformation) = this directory contains two CloudFormation templates. The first one is called `codepipeline_ci_cd.yaml` and is used to stand up the CodeBuild projects and the CodePipeline project. The second one is called `lambda_deployer.yaml` and this is used for deploying Lambda functions using SAM. 39 | - [config](config) = this directory contains a sub-directory called `codebuild` and contains all of the CodeBuild BuildSpec files used for the various stages in the CI/CD pipeline. 40 | - [lambdas](lambdas) = this directory contains a collection of sub-folders which encompass the lambda functions that will be invoked when the Step Function state machine executes. 41 | - [tests](tests) = this directory contains two sub-folders and is contains all of the needed tests for checkout of the pipeline. The first sub-folder is called [unit](tests/unit) and contains all of the unit tests needed to test the Lambda functions. The second folder is called [e2e](tests/e2e) and contains all of the end-to-end tests that are run against the Step Function state machine. 42 | 43 | ## Deployment Steps 44 | 45 | ### Clone repository locally 46 | 47 | First, clone this repository to your local machine using a command like below: 48 | 49 | ```sh 50 | $ git clone https://github.com/aws-samples/aws-stepfunction-cicd-pipeline-example.git 51 | ``` 52 | 53 | ### Create AWS CodeCommit Repository 54 | 55 | Navigate to the AWS console and search for `CodeCommit`. Once on this page, click on `Create repository`. 56 | 57 | Create a new repository with the name of `CalculationStateMachine`, as seen below: 58 | 59 | ![create repository](images/codecommit_repository.png) 60 | 61 | ### Push Repository Code 62 | 63 | After you have created the CodeCommit repository, you will need to delete the existing git remote and repoint it to CodeCommit using the following command: 64 | 65 | ```sh 66 | $ git remote rm origin 67 | $ git remote add origin https://git-codecommit.us-east-1.amazonaws.com/v1/repos/CalculationStateMachine 68 | $ git add -A 69 | $ git commit -m "Adding all files for state machine pipeline creation" 70 | $ git push -u origin master 71 | ``` 72 | 73 | ### Create Systems Manager Parameter Store Value 74 | 75 | Now that you have pushed this repository into CodeCommit, you will now create a parameter in AWS Systems Manager Parameter Store. This parameter is a Base64 Encoded JSON object that contains the names of the Lambda functions that you will deploy for your Step Function state machine. 76 | 77 | Run the following command to create the parameter: 78 | 79 | ```sh 80 | aws ssm put-parameter \ 81 | --name "StateMachineLambdaFunctions" \ 82 | --value "eyJTdGFnZSI6IFsgeyJDTEVBTklOUFVUX0ZVTkNUSU9OIjogIkNsZWFuSW5wdXQtU3RhZ2UiLAogICJNVUxUSVBMWV9GVU5DVElPTiI6ICJNdWx0aXBseS1TdGFnZSIsCiAgIkFERF9GVU5DVElPTiI6ICJBZGQtU3RhZ2UiLAogICJTVUJUUkFDVF9GVU5DVElPTiI6ICJTdWJ0cmFjdC1TdGFnZSIsCiAgIkRJVklERV9GVU5DVElPTiI6ICJEaXZpZGUtU3RhZ2UifV0sIlByb2QiOiBbeyJDTEVBTklOUFVUX0ZVTkNUSU9OIjogIkNsZWFuSW5wdXQiLAogICJNVUxUSVBMWV9GVU5DVElPTiI6ICJNdWx0aXBseSIsCiAgIkFERF9GVU5DVElPTiI6ICJBZGQiLAogICJTVUJUUkFDVF9GVU5DVElPTiI6ICJTdWJ0cmFjdCIsCiAgIkRJVklERV9GVU5DVElPTiI6ICJEaXZpZGUifV19" \ 83 | --type String 84 | ``` 85 | 86 | The decoded value is: 87 | 88 | ```sh 89 | { 90 | "Stage": [ 91 | { 92 | "CLEANINPUT_FUNCTION": "CleanInput-Stage", 93 | "MULTIPLY_FUNCTION": "Multiply-Stage", 94 | "ADD_FUNCTION": "Add-Stage", 95 | "SUBTRACT_FUNCTION": "Subtract-Stage", 96 | "DIVIDE_FUNCTION": "Divide-Stage" 97 | } 98 | ], 99 | "Prod": [ 100 | { 101 | "CLEANINPUT_FUNCTION": "CleanInput", 102 | "MULTIPLY_FUNCTION": "Multiply", 103 | "ADD_FUNCTION": "Add", 104 | "SUBTRACT_FUNCTION": "Subtract", 105 | "DIVIDE_FUNCTION": "Divide" 106 | } 107 | ] 108 | } 109 | ``` 110 | 111 | This command above will create a new SSM entry with the Base64 Encoded value of the JSON payload above. 112 | 113 | ### Bootstrap Pipeline CloudFormation Template 114 | 115 | The CloudFormation template for bootstrapping the CodePipeline project is located at [codepipeline_ci_cd.yaml](cloudformation/codepipeline_ci_cd.yaml) 116 | 117 | Now that you have created the CodeCommit repository and the Systems Manager Parameter Store value you are ready to create the CloudFormation stack to set up the following resources: 118 | 119 | - AWS CodePipeline Project 120 | - AWS CodeBuild Steps 121 | - S3 Lambda SAM staging Bucket 122 | - S3 CodePipeline artifact Bucket 123 | - IAM CodePipeline and CodeBuild Execution Roles and Policies 124 | - SNS Production Approval Topic and email subscriber 125 | - KMS CodePipeline encryption key 126 | 127 | ![ci_cd_pipeline](images/cloudformation_pipeline.png) 128 | 129 | The CloudFormation stack can be created via the CLI or the AWS console. 130 | 131 | _NOTE:_ you will need to accept the SNS topic subscription via the email that gets generated from AWS. 132 | 133 | ### Create Stack using the AWS CLI (option 1) 134 | 135 | ```sh 136 | $ aws cloudformation deploy --template-file cloudformation/codepipeline_ci_cd.yaml --stack-name step-function-cicd-pipeline --parameter-overrides ProductionApprovalEmail=example@myemail.com StateMachineName=CalculationStateMachine CodeCommitRepository=CalculationStateMachine CodeCommitBranch=master 137 | ``` 138 | 139 | ### Create Stack using the AWS Console (option 2) 140 | 141 | Navigate to the AWS Console and then to CloudFormation. Once here, upload the template `cloudformation/codepipeline_ci_cd.yaml` and then provide the parameters or override the existing default values. 142 | 143 | ### CodePipeline CloudFormation Parameters 144 | 145 | These are the parameters that are specified in the `--parameters-override` section of the CloudFormation Deployment command above. These also correspond to the parameters section of the CloudFormation template itself. 146 | 147 | - StateMachineName = this is the name that the Step Function state machine will be given when it is deployed. 148 | - CodeCommitRepository = this is the name of the AWS CodeCommit repository where this code base will reside. 149 | - CodeCommitBranch = this is the AWS CodeCommit Branch name that we will pull when running the AWS CodePipeline execution. 150 | - ProductionApprovalEmail = this is the email address that will be notified via an SNS Topic Subscription when a Production Approval is requested. 151 | 152 | ## Useful Examples 153 | 154 | ### SSM Base64 Encoded Parameter Example 155 | 156 | This is an example of the value of the JSON object as a Base64 Encoded object: 157 | 158 | ```sh 159 | eyJTdGFnZSI6IFsgeyJDTEVBTklOUFVUX0ZVTkNUSU9OIjogIkNsZWFuSW5wdXQtU3RhZ2UiLAogICJNVUxUSVBMWV9GVU5DVElPTiI6ICJNdWx0aXBseS1TdGFnZSIsCiAgIkFERF9GVU5DVElPTiI6ICJBZGQtU3RhZ2UiLAogICJTVUJUUkFDVF9GVU5DVElPTiI6ICJTdWJ0cmFjdC1TdGFnZSIsCiAgIkRJVklERV9GVU5DVElPTiI6ICJEaXZpZGUtU3RhZ2UifV0sIlByb2QiOiBbeyJDTEVBTklOUFVUX0ZVTkNUSU9OIjogIkNsZWFuSW5wdXQiLAogICJNVUxUSVBMWV9GVU5DVElPTiI6ICJNdWx0aXBseSIsCiAgIkFERF9GVU5DVElPTiI6ICJBZGQiLAogICJTVUJUUkFDVF9GVU5DVElPTiI6ICJTdWJ0cmFjdCIsCiAgIkRJVklERV9GVU5DVElPTiI6ICJEaXZpZGUifV19 160 | ``` 161 | 162 | ### Base64 Encoding JSON object 163 | 164 | In order to base64 encode the JSON object above, run the following command: 165 | 166 | ```sh 167 | $ export VAL='{"Stage": [ {"CLEANINPUT_FUNCTION": "CleanInput-Stage", 168 | "MULTIPLY_FUNCTION": "Multiply-Stage", 169 | "ADD_FUNCTION": "Add-Stage", 170 | "SUBTRACT_FUNCTION": "Subtract-Stage", 171 | "DIVIDE_FUNCTION": "Divide-Stage"}],"Prod": [{"CLEANINPUT_FUNCTION": "CleanInput", 172 | "MULTIPLY_FUNCTION": "Multiply", 173 | "ADD_FUNCTION": "Add", 174 | "SUBTRACT_FUNCTION": "Subtract", 175 | "DIVIDE_FUNCTION": "Divide"}]}' 176 | 177 | $ echo -n $VAL | openssl base64 178 | ``` 179 | 180 | ## Lambda Deployer SAM Template 181 | 182 | There is a [SAM Template](cloudformation/lambda_deployer.yaml) which will get used in one of the CodeBuild steps to deploy all of the Lambda Functions. This will use an All-at-Once deployment approach and will set up all of the necessary Lambda Versions, Lambda Aliases, and appropriate Lambda Functions. 183 | 184 | ## License 185 | 186 | This library is licensed under the MIT-0 License. See the LICENSE file. 187 | 188 | ## Contributors 189 | 190 | - Matt Noyce 191 | - Viyoma Sachdeva 192 | - Srihari Prabaharan 193 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/__init__.py -------------------------------------------------------------------------------- /cloudformation/codepipeline_ci_cd.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | AWSTemplateFormatVersion: 2010-09-09 5 | 6 | Description: Creates a CodePipeline Project with all associated CodeBuild, CloudFormation, and Approvals 7 | 8 | Parameters: 9 | StateMachineName: 10 | Description: The Production name of the State Machine 11 | Type: String 12 | Default: CalculationStateMachine 13 | CodeCommitRepository: 14 | Description: The name of the CodeCommit Repository where configurations are located 15 | Type: String 16 | Default: CalculationStateMachine 17 | CodeCommitBranch: 18 | Description: The CodeCommit Branch to use 19 | Type: String 20 | Default: master 21 | ProductionApprovalEmail: 22 | Description: The email address to send production deployment approval to 23 | Type: String 24 | 25 | Resources: 26 | SNSApprovalTopic: 27 | Type: AWS::SNS::Topic 28 | SNSEmailSubscriber: 29 | Type: AWS::SNS::Subscription 30 | Properties: 31 | Protocol: email 32 | Endpoint: !Ref ProductionApprovalEmail 33 | TopicArn: !Ref SNSApprovalTopic 34 | CodeBuildRole: 35 | Type: AWS::IAM::Role 36 | Properties: 37 | AssumeRolePolicyDocument: 38 | Version: "2012-10-17" 39 | Statement: 40 | - Effect: Allow 41 | Principal: 42 | Service: codebuild.amazonaws.com 43 | Action: sts:AssumeRole 44 | Policies: 45 | - PolicyName: !Sub "CodeCommitPull-${AWS::StackName}" 46 | PolicyDocument: 47 | Version: "2012-10-17" 48 | Statement: 49 | - Effect: Allow 50 | Action: 51 | - s3:* 52 | Resource: "*" 53 | - Effect: Allow 54 | Action: 55 | - codebuild:* 56 | Resource: "*" 57 | - Effect: Allow 58 | Action: 59 | - lambda:* 60 | Resource: "*" 61 | - Effect: Allow 62 | Action: 63 | - cloudformation:* 64 | Resource: "*" 65 | - Effect: Allow 66 | Action: 67 | - iam:* 68 | Resource: "*" 69 | - Effect: Allow 70 | Action: 71 | - codecommit:* 72 | Resource: "*" 73 | - Effect: Allow 74 | Action: 75 | - states:* 76 | Resource: "*" 77 | - Effect: Allow 78 | Action: 79 | - logs:CreateLogStream 80 | - logs:CreateLogGroup 81 | - logs:PutLogEvents 82 | Resource: "*" 83 | CustomResourceLambdaExecutionRole: 84 | Type: AWS::IAM::Role 85 | Properties: 86 | AssumeRolePolicyDocument: 87 | Version: "2012-10-17" 88 | Statement: 89 | - Effect: Allow 90 | Principal: 91 | Service: 92 | - lambda.amazonaws.com 93 | Action: 94 | - sts:AssumeRole 95 | 96 | Path: / 97 | Policies: 98 | - PolicyName: allow-logs 99 | PolicyDocument: 100 | Statement: 101 | - Effect: Allow 102 | Action: 103 | - logs:* 104 | Resource: arn:aws:logs:*:*:* 105 | - Effect: Allow 106 | Action: 107 | - s3:PutObject 108 | - s3:GetObject 109 | - s3:GetObjectVersion 110 | Resource: "*" 111 | - Effect: Allow 112 | Action: 113 | - ssm:PutObject 114 | - ssm:GetObject 115 | - ssm:GetObjectVersion 116 | Resource: "*" 117 | Base64Decode: 118 | Type: AWS::Lambda::Function 119 | Properties: 120 | Runtime: python2.7 121 | Timeout: 60 122 | Handler: index.handler 123 | Role: 124 | Fn::GetAtt: 125 | - "CustomResourceLambdaExecutionRole" 126 | - "Arn" 127 | Code: 128 | ZipFile: | 129 | import base64 130 | import json 131 | import cfnresponse 132 | def handler(event, context): 133 | responseValue = base64.b64decode(event['ResourceProperties'].get('Base64', "")).decode('utf-8') 134 | responseData = {} 135 | responseData['Json'] = responseValue 136 | cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, "CustomResourceBase64") 137 | GetJson: 138 | Type: Custom::Base64Decode 139 | Properties: 140 | ServiceToken: 141 | Fn::GetAtt: 142 | - "Base64Decode" 143 | - "Arn" 144 | Base64: "{{resolve:ssm:StateMachineLambdaFunctions:1}}" 145 | CodePipelineRole: 146 | Type: AWS::IAM::Role 147 | Properties: 148 | AssumeRolePolicyDocument: 149 | Version: "2012-10-17" 150 | Statement: 151 | - Effect: Allow 152 | Principal: 153 | Service: codepipeline.amazonaws.com 154 | Action: sts:AssumeRole 155 | - Effect: Allow 156 | Principal: 157 | Service: cloudformation.amazonaws.com 158 | Action: sts:AssumeRole 159 | Policies: 160 | - PolicyName: !Sub "CodePipeline-${AWS::StackName}" 161 | PolicyDocument: 162 | Version: "2012-10-17" 163 | Statement: 164 | - Effect: Allow 165 | Action: 166 | - codebuild:StartBuild 167 | - codebuild:BatchGetBuilds 168 | - logs:CreateLogStream 169 | - iam:PassRole 170 | Resource: "*" 171 | - Effect: Allow 172 | Action: 173 | - codecommit:* 174 | Resource: "*" 175 | - Effect: Allow 176 | Action: 177 | - s3:* 178 | Resource: "*" 179 | - Effect: Allow 180 | Action: 181 | - cloudformation:* 182 | Resource: "*" 183 | - Effect: Allow 184 | Action: 185 | - SNS:Publish 186 | Resource: !Ref SNSApprovalTopic 187 | 188 | CodePipelineCFMRole: 189 | Type: AWS::IAM::Role 190 | Properties: 191 | AssumeRolePolicyDocument: 192 | Version: "2012-10-17" 193 | Statement: 194 | - Effect: Allow 195 | Principal: 196 | Service: codepipeline.amazonaws.com 197 | Action: sts:AssumeRole 198 | - Effect: Allow 199 | Principal: 200 | Service: cloudformation.amazonaws.com 201 | Action: sts:AssumeRole 202 | Policies: 203 | - PolicyName: !Sub "CodePipeline-${AWS::StackName}-CFM" 204 | PolicyDocument: 205 | Version: "2012-10-17" 206 | Statement: 207 | - Effect: Allow 208 | Action: 209 | - codebuild:StartBuild 210 | - codebuild:BatchGetBuilds 211 | - logs:CreateLogStream 212 | - cloudformation:CreateStack 213 | - cloudformation:CreateChangeSet 214 | - cloudformation:DescribeStacks 215 | - iam:CreateRole 216 | - iam:* 217 | - states:* 218 | Resource: "*" 219 | - Effect: Allow 220 | Action: 221 | - cloudformation:* 222 | Resource: "*" 223 | - Effect: Allow 224 | Action: 225 | - s3:* 226 | Resource: "*" 227 | 228 | CodePipelineArtifactKMSKey: 229 | Type: AWS::KMS::Key 230 | Properties: 231 | Description: !Sub "KMS Key for ${AWS::StackName}" 232 | Enabled: true 233 | EnableKeyRotation: true 234 | KeyPolicy: 235 | Id: key-consolepolicy-3 236 | Version: "2012-10-17" 237 | Statement: 238 | - Sid: Enable IAM User Permissions 239 | Effect: Allow 240 | Principal: 241 | AWS: 242 | - !Sub arn:aws:iam::${AWS::AccountId}:root 243 | Action: kms:* 244 | Resource: "*" 245 | - Sid: Allow access for Key Administrators 246 | Effect: Allow 247 | Principal: 248 | AWS: 249 | - !GetAtt CodePipelineRole.Arn 250 | Action: 251 | - kms:Create* 252 | - kms:Describe* 253 | - kms:Enable* 254 | - kms:List* 255 | - kms:Put* 256 | - kms:Update* 257 | - kms:Revoke* 258 | - kms:Disable* 259 | - kms:Get* 260 | - kms:Delete* 261 | - kms:TagResource 262 | - kms:UntagResource 263 | - kms:ScheduleKeyDeletion 264 | - kms:CancelKeyDeletion 265 | Resource: "*" 266 | - Sid: Allow use of the key 267 | Effect: Allow 268 | Principal: 269 | AWS: 270 | - !GetAtt CodePipelineRole.Arn 271 | - !GetAtt CodeBuildRole.Arn 272 | - !GetAtt CodePipelineCFMRole.Arn 273 | Action: 274 | - kms:Encrypt 275 | - kms:Decrypt 276 | - kms:ReEncrypt* 277 | - kms:GenerateDataKey* 278 | - kms:DescribeKey 279 | Resource: "*" 280 | - Sid: Allow attachment of persistent resources 281 | Effect: Allow 282 | Principal: 283 | AWS: 284 | - !GetAtt CodePipelineRole.Arn 285 | Action: 286 | - kms:CreateGrant 287 | - kms:ListGrants 288 | - kms:RevokeGrant 289 | Resource: "*" 290 | Condition: 291 | Bool: 292 | kms:GrantIsForAWSResource: "true" 293 | 294 | CodePipelineArtifactBucket: 295 | Type: AWS::S3::Bucket 296 | DeletionPolicy: Retain 297 | Properties: 298 | BucketEncryption: 299 | ServerSideEncryptionConfiguration: 300 | - ServerSideEncryptionByDefault: 301 | SSEAlgorithm: aws:kms 302 | KMSMasterKeyID: !Ref CodePipelineArtifactKMSKey 303 | 304 | LambdaSAMCodeBucket: 305 | Type: AWS::S3::Bucket 306 | DeletionPolicy: Retain 307 | 308 | CodeBuildLintProject: 309 | Type: AWS::CodeBuild::Project 310 | Properties: 311 | Artifacts: 312 | Type: CODEPIPELINE 313 | Source: 314 | Type: CODEPIPELINE 315 | BuildSpec: config/codebuild/buildspec_lint_test.yaml 316 | Environment: 317 | ComputeType: BUILD_GENERAL1_SMALL 318 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 319 | Type: LINUX_CONTAINER 320 | Name: !Sub "${AWS::StackName}-Lint" 321 | ServiceRole: !GetAtt CodeBuildRole.Arn 322 | 323 | CodeBuildUnitTestProject: 324 | Type: AWS::CodeBuild::Project 325 | Properties: 326 | Artifacts: 327 | Type: CODEPIPELINE 328 | Source: 329 | Type: CODEPIPELINE 330 | BuildSpec: config/codebuild/buildspec_unit_tests.yaml 331 | Environment: 332 | ComputeType: BUILD_GENERAL1_SMALL 333 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 334 | Type: LINUX_CONTAINER 335 | Name: !Sub "${AWS::StackName}-Unit-Tests" 336 | ServiceRole: !GetAtt CodeBuildRole.Arn 337 | 338 | CodeBuildDeployStageLambdaFunctions: 339 | Type: AWS::CodeBuild::Project 340 | Properties: 341 | Artifacts: 342 | Type: CODEPIPELINE 343 | Source: 344 | Type: CODEPIPELINE 345 | BuildSpec: config/codebuild/buildspec_deploy_stage_lambda_functions.yaml 346 | Environment: 347 | ComputeType: BUILD_GENERAL1_SMALL 348 | PrivilegedMode: true 349 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 350 | Type: LINUX_CONTAINER 351 | EnvironmentVariables: 352 | - Name: SAM_BUILD_BUCKET 353 | Value: !Ref LambdaSAMCodeBucket 354 | - Name: S3_PREFIX 355 | Value: "lambda-stage" 356 | - Name: LAMBDA_DEPLOYER_STACK_NAME 357 | Value: !Sub "${AWS::StackName}-Lambda-Deployer-Stage" 358 | - Name: LAMBDA_FUNCTION_NAMES_JSON 359 | Value: !GetAtt GetJson.Json 360 | Name: !Sub "${AWS::StackName}-Deploy-Lambda-Functions-Stage" 361 | ServiceRole: !GetAtt CodeBuildRole.Arn 362 | 363 | CodeBuildDeployProductionLambdaFunctions: 364 | Type: AWS::CodeBuild::Project 365 | Properties: 366 | Artifacts: 367 | Type: CODEPIPELINE 368 | Source: 369 | Type: CODEPIPELINE 370 | BuildSpec: config/codebuild/buildspec_deploy_prod_lambda_functions.yaml 371 | Environment: 372 | ComputeType: BUILD_GENERAL1_SMALL 373 | PrivilegedMode: true 374 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 375 | Type: LINUX_CONTAINER 376 | EnvironmentVariables: 377 | - Name: SAM_BUILD_BUCKET 378 | Value: !Ref LambdaSAMCodeBucket 379 | - Name: S3_PREFIX 380 | Value: "lambda-production" 381 | - Name: LAMBDA_DEPLOYER_STACK_NAME 382 | Value: !Sub "${AWS::StackName}-Lambda-Deployer-Production" 383 | - Name: LAMBDA_FUNCTION_NAMES_JSON 384 | Value: !GetAtt GetJson.Json 385 | Name: !Sub "${AWS::StackName}-Deploy-Production" 386 | ServiceRole: !GetAtt CodeBuildRole.Arn 387 | 388 | CodeBuildTemplateStateMachineCFMStage: 389 | Type: AWS::CodeBuild::Project 390 | Properties: 391 | Artifacts: 392 | Type: CODEPIPELINE 393 | Source: 394 | Type: CODEPIPELINE 395 | BuildSpec: config/codebuild/buildspec_template_cfm.yaml 396 | Environment: 397 | ComputeType: BUILD_GENERAL1_SMALL 398 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 399 | Type: LINUX_CONTAINER 400 | EnvironmentVariables: 401 | - Name: Lambda_Func 402 | Value: !GetAtt GetJson.Json 403 | - Name: Environment 404 | Value: Stage 405 | Name: !Sub "${AWS::StackName}-Template-Stage-Cloudformation" 406 | ServiceRole: !GetAtt CodeBuildRole.Arn 407 | 408 | CodeBuildTemplateStateMachineCFMProduction: 409 | Type: AWS::CodeBuild::Project 410 | Properties: 411 | Artifacts: 412 | Type: CODEPIPELINE 413 | Source: 414 | Type: CODEPIPELINE 415 | BuildSpec: config/codebuild/buildspec_template_cfm.yaml 416 | Environment: 417 | ComputeType: BUILD_GENERAL1_SMALL 418 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 419 | Type: LINUX_CONTAINER 420 | EnvironmentVariables: 421 | - Name: Lambda_Func 422 | Value: !GetAtt GetJson.Json 423 | - Name: Environment 424 | Value: Prod 425 | Name: !Sub "${AWS::StackName}-Template-Prod-Cloudformation-Template" 426 | ServiceRole: !GetAtt CodeBuildRole.Arn 427 | 428 | CodeBuildE2ETestProject: 429 | Type: AWS::CodeBuild::Project 430 | Properties: 431 | Artifacts: 432 | Type: CODEPIPELINE 433 | Source: 434 | Type: CODEPIPELINE 435 | BuildSpec: config/codebuild/buildspec_e2e_tests.yaml 436 | Environment: 437 | ComputeType: BUILD_GENERAL1_SMALL 438 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 439 | Type: LINUX_CONTAINER 440 | EnvironmentVariables: 441 | - Name: AWS_REGION 442 | Value: !Ref "AWS::Region" 443 | - Name: STATE_MACHINE_NAME 444 | Value: !Sub "${StateMachineName}-Test" 445 | Name: !Sub "${AWS::StackName}-E2E-Tests" 446 | ServiceRole: !GetAtt CodeBuildRole.Arn 447 | 448 | CodeBuildDeployProductionPipeline: 449 | Type: AWS::CodeBuild::Project 450 | Properties: 451 | Artifacts: 452 | Type: CODEPIPELINE 453 | Source: 454 | Type: CODEPIPELINE 455 | Environment: 456 | ComputeType: BUILD_GENERAL1_SMALL 457 | Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0 458 | Type: LINUX_CONTAINER 459 | Name: !Sub "${AWS::StackName}-Deploy-Production-Pipeline" 460 | ServiceRole: !GetAtt CodeBuildRole.Arn 461 | 462 | CodePipelineProject: 463 | Type: AWS::CodePipeline::Pipeline 464 | Properties: 465 | ArtifactStore: 466 | EncryptionKey: 467 | Id: !Ref CodePipelineArtifactKMSKey 468 | Type: KMS 469 | Location: !Ref CodePipelineArtifactBucket 470 | Type: S3 471 | RoleArn: !GetAtt CodePipelineRole.Arn 472 | Stages: 473 | - Name: Source 474 | Actions: 475 | - Name: Source 476 | ActionTypeId: 477 | Category: Source 478 | Owner: AWS 479 | Provider: CodeCommit 480 | Version: "1" 481 | Configuration: 482 | BranchName: !Ref CodeCommitBranch 483 | RepositoryName: !Ref CodeCommitRepository 484 | OutputArtifacts: 485 | - Name: SourceCode 486 | - Name: SourceLint 487 | Actions: 488 | - Name: SourceLint 489 | InputArtifacts: 490 | - Name: SourceCode 491 | ActionTypeId: 492 | Category: Test 493 | Owner: AWS 494 | Version: 1 495 | Provider: CodeBuild 496 | Configuration: 497 | ProjectName: !Ref CodeBuildLintProject 498 | - Name: Unit-Tests 499 | Actions: 500 | - Name: Unit-Tests 501 | InputArtifacts: 502 | - Name: SourceCode 503 | ActionTypeId: 504 | Category: Test 505 | Owner: AWS 506 | Version: 1 507 | Provider: CodeBuild 508 | Configuration: 509 | ProjectName: !Ref CodeBuildUnitTestProject 510 | - Name: Deploy-Lambda-Functions-Stage 511 | Actions: 512 | - Name: Deploy-Lambda-Functions-Stage 513 | InputArtifacts: 514 | - Name: SourceCode 515 | ActionTypeId: 516 | Category: Test 517 | Owner: AWS 518 | Version: 1 519 | Provider: CodeBuild 520 | Configuration: 521 | ProjectName: !Ref CodeBuildDeployStageLambdaFunctions 522 | - Name: Template-Stage-Cloudformation 523 | Actions: 524 | - Name: Template-Stage-Cloudformation 525 | InputArtifacts: 526 | - Name: SourceCode 527 | OutputArtifacts: 528 | - Name: TransformedSourceCodeStage 529 | ActionTypeId: 530 | Category: Test 531 | Owner: AWS 532 | Version: 1 533 | Provider: CodeBuild 534 | Configuration: 535 | ProjectName: !Ref CodeBuildTemplateStateMachineCFMStage 536 | - Name: DeployStagePipeline 537 | Actions: 538 | - Name: DeployStagePipeline 539 | InputArtifacts: 540 | - Name: TransformedSourceCodeStage 541 | ActionTypeId: 542 | Category: Deploy 543 | Owner: AWS 544 | Provider: CloudFormation 545 | Version: "1" 546 | Configuration: 547 | StackName: !Sub "${StateMachineName}-Stage" 548 | TemplatePath: TransformedSourceCodeStage::sm_cfm.json 549 | ActionMode: CREATE_UPDATE 550 | RoleArn: !GetAtt CodePipelineCFMRole.Arn 551 | Capabilities: CAPABILITY_IAM 552 | ParameterOverrides: !Sub '{"StateMachineName": "${StateMachineName}-Test"}' 553 | - Name: E2E-Tests 554 | Actions: 555 | - Name: E2E-Tests 556 | InputArtifacts: 557 | - Name: TransformedSourceCodeStage 558 | ActionTypeId: 559 | Category: Test 560 | Owner: AWS 561 | Version: 1 562 | Provider: CodeBuild 563 | Configuration: 564 | ProjectName: !Ref CodeBuildE2ETestProject 565 | - Name: ProductionApproval 566 | Actions: 567 | - Name: ProductionApproval 568 | ActionTypeId: 569 | Category: Approval 570 | Owner: AWS 571 | Provider: Manual 572 | Version: "1" 573 | Configuration: 574 | NotificationArn: !Ref SNSApprovalTopic 575 | - Name: DestroyStagePipeline 576 | Actions: 577 | - Name: DestroyStagePipeline 578 | InputArtifacts: 579 | - Name: TransformedSourceCodeStage 580 | ActionTypeId: 581 | Category: Deploy 582 | Owner: AWS 583 | Provider: CloudFormation 584 | Version: "1" 585 | Configuration: 586 | StackName: !Sub "${StateMachineName}-Stage" 587 | ActionMode: DELETE_ONLY 588 | RoleArn: !GetAtt CodePipelineCFMRole.Arn 589 | Capabilities: CAPABILITY_IAM 590 | ParameterOverrides: !Sub '{"StateMachineName": "${StateMachineName}"}' 591 | - Name: Template-Prod-Cloudformation-Template 592 | Actions: 593 | - Name: Template-Prod-Cloudformation-Template 594 | InputArtifacts: 595 | - Name: SourceCode 596 | OutputArtifacts: 597 | - Name: TransformedSourceCodeProduction 598 | ActionTypeId: 599 | Category: Test 600 | Owner: AWS 601 | Version: 1 602 | Provider: CodeBuild 603 | Configuration: 604 | ProjectName: !Ref CodeBuildTemplateStateMachineCFMProduction 605 | - Name: Deploy-Production-Lambda-Functions 606 | Actions: 607 | - Name: Deploy-Production-Lambda-Functions 608 | InputArtifacts: 609 | - Name: TransformedSourceCodeProduction 610 | ActionTypeId: 611 | Category: Test 612 | Owner: AWS 613 | Version: 1 614 | Provider: CodeBuild 615 | Configuration: 616 | ProjectName: !Ref CodeBuildDeployProductionLambdaFunctions 617 | - Name: DeployProductionPipeline 618 | Actions: 619 | - Name: DeployProductionPipeline 620 | InputArtifacts: 621 | - Name: TransformedSourceCodeProduction 622 | ActionTypeId: 623 | Category: Deploy 624 | Owner: AWS 625 | Provider: CloudFormation 626 | Version: "1" 627 | Configuration: 628 | StackName: !Sub "${StateMachineName}-Production" 629 | ActionMode: CREATE_UPDATE 630 | RoleArn: !GetAtt CodePipelineCFMRole.Arn 631 | Capabilities: CAPABILITY_IAM 632 | TemplatePath: TransformedSourceCodeProduction::sm_cfm.json 633 | ParameterOverrides: !Sub '{"StateMachineName": "${StateMachineName}"}' 634 | TopicPolicy: 635 | Type: AWS::SNS::TopicPolicy 636 | Properties: 637 | Topics: 638 | - !Ref "SNSApprovalTopic" 639 | PolicyDocument: 640 | Version: "2008-10-17" 641 | Id: __default_policy_ID 642 | Statement: 643 | - Sid: __default_statement_ID 644 | Effect: Allow 645 | Principal: 646 | AWS: "*" 647 | Action: 648 | - SNS:GetTopicAttributes 649 | - SNS:SetTopicAttributes 650 | - SNS:AddPermission 651 | - SNS:RemovePermission 652 | - SNS:DeleteTopic 653 | - SNS:Subscribe 654 | - SNS:ListSubscriptionsByTopic 655 | - SNS:Publish 656 | - SNS:Receive 657 | Resource: 658 | - !Ref "SNSApprovalTopic" 659 | Condition: 660 | StringEquals: 661 | AWS:SourceOwner: 662 | - !Ref "AWS::AccountId" 663 | - Sid: TrustCWEToPublishEventsToMyTopic 664 | Effect: Allow 665 | Principal: 666 | Service: codepipeline.amazonaws.com 667 | Action: sns:Publish 668 | Resource: 669 | - !Ref "SNSApprovalTopic" 670 | -------------------------------------------------------------------------------- /cloudformation/lambda_deployer.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | AWSTemplateFormatVersion: 2010-09-09 5 | Transform: AWS::Serverless-2016-10-31 6 | 7 | Description: Creates all lambda functions needed for the Step Function state machine execution 8 | 9 | Parameters: 10 | CleanInputFunctionName: 11 | Description: The name of the Clean Input Lambda Function 12 | Type: String 13 | MultiplyFunctionName: 14 | Description: The name of the Multiply Lambda Function 15 | Type: String 16 | AddFunctionName: 17 | Description: The name of the Add Lambda Function 18 | Type: String 19 | SubtractFunctionName: 20 | Description: The name of the Subtract Lambda Function 21 | Type: String 22 | DivideFunctionName: 23 | Description: The name of the Divide Lambda Function 24 | Type: String 25 | 26 | Resources: 27 | CleanInputLambdaFunction: 28 | Type: AWS::Serverless::Function 29 | Properties: 30 | Handler: cleaninput.lambda_handler 31 | Runtime: python3.7 32 | CodeUri: lambdas/cleaninput 33 | FunctionName: !Ref CleanInputFunctionName 34 | AutoPublishAlias: live 35 | MemorySize: 128 36 | Timeout: 30 37 | 38 | MultiplyLambdaFunction: 39 | Type: AWS::Serverless::Function 40 | Properties: 41 | Handler: multiply.lambda_handler 42 | Runtime: python3.7 43 | CodeUri: lambdas/multiply 44 | FunctionName: !Ref MultiplyFunctionName 45 | AutoPublishAlias: live 46 | MemorySize: 128 47 | Timeout: 30 48 | 49 | AddLambdaFunction: 50 | Type: AWS::Serverless::Function 51 | Properties: 52 | Handler: add.lambda_handler 53 | Runtime: python3.7 54 | CodeUri: lambdas/add 55 | FunctionName: !Ref AddFunctionName 56 | AutoPublishAlias: live 57 | MemorySize: 128 58 | Timeout: 30 59 | 60 | SubtractLambdaFunction: 61 | Type: AWS::Serverless::Function 62 | Properties: 63 | Handler: subtract.lambda_handler 64 | Runtime: python3.7 65 | CodeUri: lambdas/subtract 66 | FunctionName: !Ref SubtractFunctionName 67 | AutoPublishAlias: live 68 | MemorySize: 128 69 | Timeout: 30 70 | 71 | DivideLambdaFunction: 72 | Type: AWS::Serverless::Function 73 | Properties: 74 | Handler: divide.lambda_handler 75 | Runtime: python3.7 76 | CodeUri: lambdas/divide 77 | FunctionName: !Ref DivideFunctionName 78 | AutoPublishAlias: live 79 | MemorySize: 128 80 | Timeout: 30 81 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_deploy_prod_lambda_functions.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | 6 | phases: 7 | install: 8 | runtime-versions: 9 | python: 3.8 10 | docker: 18 11 | build: 12 | commands: 13 | - echo "Gathering and Setting Lambda Function Environment Variables" 14 | - export CLEANINPUT_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Prod"][0]["CLEANINPUT_FUNCTION"])') 15 | - export MULTIPLY_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Prod"][0]["MULTIPLY_FUNCTION"])') 16 | - export ADD_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Prod"][0]["ADD_FUNCTION"])') 17 | - export SUBTRACT_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Prod"][0]["SUBTRACT_FUNCTION"])') 18 | - export DIVIDE_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Prod"][0]["DIVIDE_FUNCTION"])') 19 | 20 | - echo "Running a SAM Build command" 21 | - sam build -b ./build -s . -t cloudformation/lambda_deployer.yaml -u 22 | 23 | - echo "Running a SAM Package Command" 24 | - > 25 | sam package 26 | --template-file build/template.yaml 27 | --s3-bucket ${SAM_BUILD_BUCKET} 28 | --s3-prefix ${S3_PREFIX} 29 | --output-template-file build/packaged.yaml 30 | 31 | - echo "Running a SAM Deploy Command" 32 | - > 33 | sam deploy 34 | --template-file build/packaged.yaml 35 | --stack-name ${LAMBDA_DEPLOYER_STACK_NAME} 36 | --capabilities CAPABILITY_NAMED_IAM 37 | --s3-prefix ${S3_PREFIX} 38 | --parameter-overrides CleanInputFunctionName=${CLEANINPUT_LAMBDA_NAME} MultiplyFunctionName=${MULTIPLY_LAMBDA_NAME} 39 | AddFunctionName=${ADD_LAMBDA_NAME} SubtractFunctionName=${SUBTRACT_LAMBDA_NAME} DivideFunctionName=${DIVIDE_LAMBDA_NAME} 40 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_deploy_stage_lambda_functions.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | 6 | phases: 7 | install: 8 | runtime-versions: 9 | python: 3.8 10 | docker: 18 11 | build: 12 | commands: 13 | - echo "Gathering and Setting Lambda Function Environment Variables" 14 | - export CLEANINPUT_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Stage"][0]["CLEANINPUT_FUNCTION"])') 15 | - export MULTIPLY_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Stage"][0]["MULTIPLY_FUNCTION"])') 16 | - export ADD_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Stage"][0]["ADD_FUNCTION"])') 17 | - export SUBTRACT_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Stage"][0]["SUBTRACT_FUNCTION"])') 18 | - export DIVIDE_LAMBDA_NAME=$(echo ${LAMBDA_FUNCTION_NAMES_JSON} | python -c 'import json,sys; print(json.load(sys.stdin)["Stage"][0]["DIVIDE_FUNCTION"])') 19 | 20 | - echo "Running a SAM Build command" 21 | - sam build -b ./build -s . -t cloudformation/lambda_deployer.yaml -u 22 | 23 | - echo "Running a SAM Package Command" 24 | - > 25 | sam package 26 | --template-file build/template.yaml 27 | --s3-bucket ${SAM_BUILD_BUCKET} 28 | --s3-prefix ${S3_PREFIX} 29 | --output-template-file build/packaged.yaml 30 | 31 | - echo "Running a SAM Deploy Command" 32 | - > 33 | sam deploy 34 | --template-file build/packaged.yaml 35 | --stack-name ${LAMBDA_DEPLOYER_STACK_NAME} 36 | --capabilities CAPABILITY_NAMED_IAM 37 | --s3-prefix ${S3_PREFIX} 38 | --parameter-overrides CleanInputFunctionName=${CLEANINPUT_LAMBDA_NAME} MultiplyFunctionName=${MULTIPLY_LAMBDA_NAME} 39 | AddFunctionName=${ADD_LAMBDA_NAME} SubtractFunctionName=${SUBTRACT_LAMBDA_NAME} DivideFunctionName=${DIVIDE_LAMBDA_NAME} 40 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_e2e_tests.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | env: 6 | git-credential-helper: yes 7 | phases: 8 | install: 9 | runtime-versions: 10 | python: 3.8 11 | commands: 12 | - pip3 install -r tests/requirements.txt 13 | 14 | build: 15 | commands: 16 | - pytest -vvv -s tests/e2e --junitxml=reports/e2e.xml 17 | 18 | artifacts: 19 | files: 20 | - "**/*" 21 | 22 | reports: 23 | E2ETestReports: 24 | files: 25 | - "**/*" 26 | base-directory: "reports" 27 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_lint_test.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | env: 6 | git-credential-helper: yes 7 | phases: 8 | install: 9 | runtime-versions: 10 | ruby: 2.6 11 | commands: 12 | - yum -y install rubygems 13 | - gem install statelint 14 | 15 | build: 16 | commands: 17 | - statelint sm_def.json 18 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_read_inputs.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | env: 6 | git-credential-helper: yes 7 | phases: 8 | install: 9 | runtime-versions: 10 | ruby: 2.6 11 | 12 | build: 13 | commands: 14 | - ls -alt 15 | 16 | artifacts: 17 | files: 18 | - "**/*" 19 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_template_cfm.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | env: 6 | git-credential-helper: yes 7 | phases: 8 | install: 9 | runtime-versions: 10 | python: 3.8 11 | commands: 12 | - pip3 install -r requirements.txt 13 | 14 | build: 15 | commands: 16 | - python template_statemachine_cf.py 17 | artifacts: 18 | files: 19 | - "**/*" 20 | -------------------------------------------------------------------------------- /config/codebuild/buildspec_unit_tests.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | version: 0.2 5 | env: 6 | git-credential-helper: yes 7 | phases: 8 | install: 9 | runtime-versions: 10 | python: 3.8 11 | commands: 12 | - pip3 install -r tests/requirements.txt 13 | 14 | build: 15 | commands: 16 | - pytest -s -vvv tests/unit/ --junitxml=reports/unit.xml 17 | 18 | reports: 19 | StateMachineReports: 20 | files: 21 | - "**/*" 22 | base-directory: "reports" 23 | -------------------------------------------------------------------------------- /images/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/images/architecture.png -------------------------------------------------------------------------------- /images/cloudformation_pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/images/cloudformation_pipeline.png -------------------------------------------------------------------------------- /images/codecommit_repository.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/images/codecommit_repository.png -------------------------------------------------------------------------------- /lambdas/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/__init__.py -------------------------------------------------------------------------------- /lambdas/add/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/add/__init__.py -------------------------------------------------------------------------------- /lambdas/add/add.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import json 5 | 6 | def lambda_handler(event, context): 7 | print(event) 8 | first = event['input']['first'] 9 | second = event['input']['second'] 10 | third = event['input']['third'] 11 | result = event['input']['result'] 12 | 13 | print(f"FIRST: {first}, SECOND: {second}, THIRD: {third}") 14 | 15 | result = int(result) + int(second) 16 | 17 | print(f"RESULT: {result}") 18 | 19 | response = { 20 | "first": first, 21 | "second": second, 22 | "third": third, 23 | "result": int(result) 24 | } 25 | 26 | event['input'] = response 27 | 28 | return event['input'] 29 | -------------------------------------------------------------------------------- /lambdas/add/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/add/requirements.txt -------------------------------------------------------------------------------- /lambdas/cleaninput/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/cleaninput/__init__.py -------------------------------------------------------------------------------- /lambdas/cleaninput/cleaninput.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import json 5 | 6 | def lambda_handler(event, context): 7 | print(event) 8 | input_vals = event['input'] 9 | 10 | values = input_vals['input'].split(" ") 11 | 12 | input_massaged = { 13 | "first": values[0], 14 | "second": values[1], 15 | "third": values[2] 16 | } 17 | 18 | event['input'] = input_massaged 19 | 20 | return event 21 | -------------------------------------------------------------------------------- /lambdas/cleaninput/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/cleaninput/requirements.txt -------------------------------------------------------------------------------- /lambdas/divide/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/divide/__init__.py -------------------------------------------------------------------------------- /lambdas/divide/divide.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | def lambda_handler(event, context): 4 | print(event) 5 | first = event['input']['first'] 6 | second = event['input']['second'] 7 | third = event['input']['third'] 8 | result = event['input']['result'] 9 | 10 | print(f"FIRST: {first}, SECOND: {second}, THIRD: {third}") 11 | 12 | result = int(result) / int(second) 13 | 14 | print(f"RESULT: {result}") 15 | 16 | response = { 17 | "first": first, 18 | "second": second, 19 | "third": third, 20 | "result": int(result) 21 | } 22 | 23 | event['input'] = response 24 | 25 | return event['input'] -------------------------------------------------------------------------------- /lambdas/divide/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/divide/requirements.txt -------------------------------------------------------------------------------- /lambdas/multiply/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/multiply/__init__.py -------------------------------------------------------------------------------- /lambdas/multiply/multiply.py: -------------------------------------------------------------------------------- 1 | 2 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | # SPDX-License-Identifier: MIT-0 4 | 5 | import json 6 | 7 | def lambda_handler(event, context): 8 | print(event) 9 | first = event['input']['input']['first'] 10 | second = event['input']['input']['second'] 11 | third = event['input']['input']['third'] 12 | 13 | print(f"FIRST: {first}, SECOND: {second}, THIRD: {third}") 14 | 15 | result = int(first) * int(second) * int(third) 16 | 17 | print(f"RESULT: {result}") 18 | 19 | response = { 20 | "first": first, 21 | "second": second, 22 | "third": third, 23 | "result": int(result) 24 | } 25 | 26 | event['input'] = response 27 | 28 | print(event['input']) 29 | 30 | return event['input'] 31 | -------------------------------------------------------------------------------- /lambdas/multiply/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/multiply/requirements.txt -------------------------------------------------------------------------------- /lambdas/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/requirements.txt -------------------------------------------------------------------------------- /lambdas/subtract/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/subtract/__init__.py -------------------------------------------------------------------------------- /lambdas/subtract/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/aws-stepfunction-cicd-pipeline-example/a999062fa3fedd5fd258e389aee6caefd963b71f/lambdas/subtract/requirements.txt -------------------------------------------------------------------------------- /lambdas/subtract/subtract.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import json 5 | 6 | def lambda_handler(event, context): 7 | print(event) 8 | first = event['input']['first'] 9 | second = event['input']['second'] 10 | third = event['input']['third'] 11 | result = event['input']['result'] 12 | 13 | print(f"FIRST: {first}, SECOND: {second}, THIRD: {third}") 14 | 15 | result = int(result) - int(first) 16 | 17 | print(f"RESULT: {result}") 18 | 19 | response = { 20 | "first": first, 21 | "second": second, 22 | "third": third, 23 | "result": int(result) 24 | } 25 | 26 | event['input'] = response 27 | 28 | return event['input'] 29 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | jinja2 -------------------------------------------------------------------------------- /sm_def.json: -------------------------------------------------------------------------------- 1 | { 2 | "Comment": "CalculationStateMachine", 3 | "StartAt": "CleanInput", 4 | "States": { 5 | "CleanInput": { 6 | "Type": "Task", 7 | "Resource": "arn:aws:states:::lambda:invoke", 8 | "Parameters": { 9 | "FunctionName": "{{CleanInput}}", 10 | "Payload": { 11 | "input.$": "$" 12 | } 13 | }, 14 | "Next": "Multiply" 15 | }, 16 | "Multiply": { 17 | "Type": "Task", 18 | "Resource": "arn:aws:states:::lambda:invoke", 19 | "Parameters": { 20 | "FunctionName": "{{Multiply}}", 21 | "Payload": { 22 | "input.$": "$.Payload" 23 | } 24 | }, 25 | "Next": "Choice" 26 | }, 27 | "Choice": { 28 | "Type": "Choice", 29 | "Choices": [ 30 | { 31 | "Variable": "$.Payload.result", 32 | "NumericGreaterThanEquals": 20, 33 | "Next": "Subtract" 34 | } 35 | ], 36 | "Default": "Fail" 37 | }, 38 | "Subtract": { 39 | "Type": "Task", 40 | "Resource": "arn:aws:states:::lambda:invoke", 41 | "Parameters": { 42 | "FunctionName": "{{Subtract}}", 43 | "Payload": { 44 | "input.$": "$.Payload" 45 | } 46 | }, 47 | "Next": "Add" 48 | }, 49 | "Fail": { 50 | "Type": "Fail", 51 | "Cause": "Number Too Small", 52 | "Error": "NumberTooSmallError" 53 | }, 54 | "Add": { 55 | "Type": "Task", 56 | "Resource": "arn:aws:states:::lambda:invoke", 57 | "Parameters": { 58 | "FunctionName": "{{Add}}", 59 | "Payload": { 60 | "input.$": "$.Payload" 61 | } 62 | }, 63 | "Next": "Divide" 64 | }, 65 | "Divide": { 66 | "Type": "Task", 67 | "Resource": "arn:aws:states:::lambda:invoke", 68 | "Parameters": { 69 | "FunctionName": "{{Divide}}", 70 | "Payload": { 71 | "input.$": "$.Payload" 72 | } 73 | }, 74 | "End": true 75 | } 76 | } 77 | } 78 | -------------------------------------------------------------------------------- /template_statemachine_cf.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import sys 6 | import json 7 | from jinja2 import Template 8 | 9 | Lambda_dict = {} 10 | Environment = "" 11 | 12 | def read_sm_def ( 13 | sm_def_file: str 14 | ) -> dict: 15 | """ 16 | Reads state machine definition from a file and returns it as a dictionary. 17 | 18 | Parameters: 19 | sm_def_file (str) = the name of the state machine definition file. 20 | 21 | Returns: 22 | sm_def_dict (dict) = the state machine definition as a dictionary. 23 | """ 24 | 25 | try: 26 | with open(f"{sm_def_file}", "r") as f: 27 | return f.read() 28 | except IOError as e: 29 | print("Path does not exist!") 30 | print(e) 31 | sys.exit(1) 32 | 33 | def template_sm_def ( 34 | sm_def_template: dict 35 | ) -> dict: 36 | Lambda_dict = json.loads(os.environ["Lambda_Func"]) 37 | Environment = os.environ["Environment"] 38 | if Environment == 'Stage': 39 | clean_input_function = Lambda_dict["Stage"][0]["CLEANINPUT_FUNCTION"] 40 | multiply_function = Lambda_dict["Stage"][0]["MULTIPLY_FUNCTION"] 41 | add_function = Lambda_dict["Stage"][0]["ADD_FUNCTION"] 42 | subtract_function = Lambda_dict["Stage"][0]["SUBTRACT_FUNCTION"] 43 | divide_function = Lambda_dict["Stage"][0]["DIVIDE_FUNCTION"] 44 | else: 45 | clean_input_function = Lambda_dict["Prod"][0]["CLEANINPUT_FUNCTION"] 46 | multiply_function = Lambda_dict["Prod"][0]["MULTIPLY_FUNCTION"] 47 | add_function = Lambda_dict["Prod"][0]["ADD_FUNCTION"] 48 | subtract_function = Lambda_dict["Prod"][0]["SUBTRACT_FUNCTION"] 49 | divide_function = Lambda_dict["Prod"][0]["DIVIDE_FUNCTION"] 50 | 51 | tm = Template(sm_def_template) 52 | msg = tm.render( 53 | CleanInput=clean_input_function, 54 | Multiply=multiply_function, 55 | Subtract=subtract_function, 56 | Add=add_function, 57 | Divide=divide_function 58 | ) 59 | 60 | return msg 61 | 62 | def template_state_machine( 63 | sm_def: dict 64 | ) -> dict: 65 | """ 66 | Templates out the CloudFormation for creating a state machine. 67 | 68 | Parameters: 69 | sm_def (dict) = a dictionary definition of the aws states language state machine. 70 | 71 | Returns: 72 | templated_cf (dict) = a dictionary definition of the state machine. 73 | """ 74 | 75 | templated_cf = { 76 | "AWSTemplateFormatVersion": "2010-09-09", 77 | "Description": "Creates the Step Function State Machine and associated IAM roles and policies", 78 | "Parameters": { 79 | "StateMachineName": { 80 | "Description": "The name of the State Machine", 81 | "Type": "String" 82 | } 83 | }, 84 | "Resources": { 85 | "StateMachineLambdaRole": { 86 | "Type": "AWS::IAM::Role", 87 | "Properties": { 88 | "AssumeRolePolicyDocument": { 89 | "Version": "2012-10-17", 90 | "Statement": [ 91 | { 92 | "Effect": "Allow", 93 | "Principal": { 94 | "Service": "states.amazonaws.com" 95 | }, 96 | "Action": "sts:AssumeRole" 97 | } 98 | ] 99 | }, 100 | "Policies": [ 101 | { 102 | "PolicyName": { 103 | "Fn::Sub": "States-Lambda-Execution-${AWS::StackName}-Policy" 104 | }, 105 | "PolicyDocument": { 106 | "Version": "2012-10-17", 107 | "Statement": [ 108 | { 109 | "Effect": "Allow", 110 | "Action": [ 111 | "logs:CreateLogStream", 112 | "logs:CreateLogGroup", 113 | "logs:PutLogEvents", 114 | "sns:*" 115 | ], 116 | "Resource": "*" 117 | }, 118 | { 119 | "Effect": "Allow", 120 | "Action": [ 121 | "lambda:InvokeFunction" 122 | ], 123 | "Resource": "*" 124 | } 125 | ] 126 | } 127 | } 128 | ] 129 | } 130 | }, 131 | "StateMachine": { 132 | "Type": "AWS::StepFunctions::StateMachine", 133 | "Properties": { 134 | "DefinitionString": sm_def, 135 | "RoleArn": { 136 | "Fn::GetAtt": [ 137 | "StateMachineLambdaRole", 138 | "Arn" 139 | ] 140 | }, 141 | "StateMachineName": { 142 | "Ref": "StateMachineName" 143 | } 144 | } 145 | } 146 | } 147 | } 148 | 149 | return templated_cf 150 | 151 | 152 | sm_def_dict = read_sm_def( 153 | sm_def_file='sm_def.json' 154 | ) 155 | 156 | templated_sm_def = template_sm_def( 157 | sm_def_template=sm_def_dict 158 | ) 159 | 160 | print(templated_sm_def) 161 | 162 | cfm_sm_def = template_state_machine( 163 | sm_def=templated_sm_def 164 | ) 165 | 166 | with open("sm_cfm.json", "w") as f: 167 | f.write(json.dumps(cfm_sm_def)) 168 | -------------------------------------------------------------------------------- /tests/e2e/PipelineExecutionMetadata.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import boto3 5 | import json 6 | 7 | """ 8 | This class is a Data Structure that holds metadata about the pipeline execution 9 | In particular it is useful for validating the input and output of the entire pipeline 10 | 11 | """ 12 | class PipelineExecutionMetadata(): 13 | def __init__( 14 | self, 15 | executionarn: str, 16 | status: str, 17 | inputval: str, 18 | output: str, 19 | retries: str 20 | ): 21 | self.executionarn = executionarn 22 | self.status = status 23 | self.inputval = inputval 24 | self.output = output 25 | self.retries = retries 26 | 27 | def __repr__(self): 28 | response = {} 29 | response['executionarn'] = self.executionarn 30 | response['status'] = self.status 31 | response['inputval'] = self.inputval 32 | response['output'] = self.output 33 | response['retries'] = self.retries 34 | return json.dumps(response) 35 | 36 | def get_executionarn(self): 37 | return self.executionarn 38 | 39 | def get_status(self): 40 | return self.status 41 | 42 | def get_inputval(self): 43 | return self.inputval 44 | 45 | def get_output(self): 46 | return self.output 47 | 48 | def get_retries(self): 49 | return self.retries 50 | -------------------------------------------------------------------------------- /tests/e2e/StepFunctionClient.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import boto3 5 | import uuid 6 | import time 7 | import json 8 | from PipelineExecutionMetadata import PipelineExecutionMetadata 9 | 10 | class StepFunctionClient(): 11 | """ 12 | This class chains together necessary API calls to AWS and populates model objects. 13 | 14 | Attributes: 15 | client (boto3) = the boto3 client for step functions. 16 | region (str) = the aws region to create the client in (e.g. 'us-east-1'). 17 | """ 18 | 19 | def __init__(self, aws_region: str): 20 | """ 21 | The constructor for the StepFunctionClient class. 22 | 23 | Parameters: 24 | region (str) = the aws region to create the client in (e.g. 'us-east-1'). 25 | """ 26 | 27 | self.client = boto3.client( 28 | service_name='stepfunctions', 29 | region_name=aws_region 30 | ) 31 | self.region = aws_region 32 | self.accountid = boto3.client("sts").get_caller_identity()["Account"] 33 | 34 | def start_pipeline_synchronous ( 35 | self, 36 | name: str, 37 | input: str 38 | ) -> PipelineExecutionMetadata: 39 | """ 40 | Starts a state machine execution until completion. 41 | 42 | Parameters: 43 | name (str) = the name of the state machine to execute. 44 | input (str) = json string representing input to starting state. 45 | 46 | Returns: 47 | PipelineExecution = an object representing the execution at a high level. 48 | """ 49 | 50 | arn = "arn:aws:states:{}:{}:stateMachine:{}".format(self.region, self.accountid, name) 51 | print(f"Starting Pipeline with ARN of {arn}") 52 | 53 | response = self.client.start_execution( 54 | stateMachineArn=arn, 55 | name=str(uuid.uuid1()), 56 | input=input 57 | ) 58 | 59 | executionarn = response.get('executionArn') 60 | 61 | print(f"Initial Response: {response}") 62 | print(f"Waiting for Execution to Complete for Execution ARN {executionarn}") 63 | 64 | while executionarn in str(self.get_running_execution_history(arn)): 65 | time.sleep(10) 66 | 67 | execution = self.get_pipeline_execution_metadata ( 68 | executionarn=executionarn 69 | ) 70 | 71 | if execution.get_status() == 'RUNNING': 72 | time.sleep(10) 73 | 74 | return self.get_pipeline_execution_metadata ( 75 | executionarn=executionarn 76 | ) 77 | 78 | def get_running_execution_history ( 79 | self, 80 | smarn: str 81 | ) -> list: 82 | """ 83 | Gathers a list of currently running executions by state machine arn. 84 | 85 | Parameters: 86 | smarn (str) = the arn of the state machine. 87 | 88 | Returns: 89 | list = a list of currently-running executions of the state machine. 90 | """ 91 | 92 | return self.client.list_executions( 93 | stateMachineArn=smarn, 94 | statusFilter='RUNNING' 95 | ) 96 | 97 | def get_pipeline_execution_metadata ( 98 | self, 99 | executionarn: str 100 | ) -> PipelineExecutionMetadata: 101 | """ 102 | Gathers high-level metadata about pipeline execution. 103 | 104 | Parameters: 105 | executionarn (str) = the arn of the state machine execution. 106 | 107 | Returns: 108 | PipelineExecutionMetadata = metadata object about pipeline. 109 | 110 | PipelineExecutionMetadata Attributes: 111 | executionarn (str) = the arn of the execution. 112 | status (str) = the status of the execution. 113 | inputval (str) = the starting state input provided to the execution. 114 | output (str) = the output of the execution terminal state. 115 | retries (str) = the number of retries that took place. 116 | """ 117 | 118 | response = self.client.describe_execution ( 119 | executionArn=executionarn 120 | ) 121 | 122 | output = '' 123 | 124 | try: 125 | output = response['output'] 126 | except Exception as e: 127 | output = '' 128 | 129 | return PipelineExecutionMetadata ( 130 | executionarn=response['executionArn'], 131 | status=response['status'], 132 | inputval=response['input'], 133 | output=output, 134 | retries=response['ResponseMetadata']['RetryAttempts'] 135 | ) 136 | 137 | -------------------------------------------------------------------------------- /tests/e2e/test_e2e.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import json 6 | import pytest 7 | from StepFunctionClient import StepFunctionClient 8 | 9 | def test_successful_execution(): 10 | # read environment variables from CodeBuild Project 11 | aws_region = os.environ['AWS_REGION'] 12 | state_machine_name = os.environ['STATE_MACHINE_NAME'] 13 | 14 | client = StepFunctionClient(aws_region=aws_region) 15 | 16 | # start the state machine 17 | pipeline_metadata = client.start_pipeline_synchronous( 18 | name=f"{state_machine_name}", 19 | input='{\"input\": \"1 20 3\"}' 20 | ) 21 | 22 | expected_output = {'first': '1', 'second': '20', 'third': '3', 'result': 3} 23 | 24 | print(pipeline_metadata) 25 | execution_arn = pipeline_metadata.get_executionarn() 26 | execution_metadata = client.get_pipeline_execution_metadata(executionarn=execution_arn) 27 | execution_output = execution_metadata.get_output() 28 | json_output = json.loads(execution_output) 29 | 30 | assert(pipeline_metadata.get_status() == 'SUCCEEDED') 31 | assert(json_output['Payload'] == expected_output) 32 | -------------------------------------------------------------------------------- /tests/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3 2 | pytest -------------------------------------------------------------------------------- /tests/unit/lambdas/test_add.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import sys 6 | 7 | sys.path.append( 8 | os.path.realpath( 9 | os.path.join( 10 | os.path.dirname(__file__), 11 | os.path.pardir, 12 | os.path.pardir, 13 | os.path.pardir, 14 | "lambdas", 15 | "add" 16 | ) 17 | ) 18 | ) 19 | 20 | from add import lambda_handler 21 | 22 | def test_add(): 23 | event = { 24 | "input": { 25 | "first": "1", 26 | "second": "20", 27 | "third": "3", 28 | "result": 59 29 | } 30 | } 31 | 32 | # actual function call 33 | actual_response = lambda_handler( 34 | event=event, 35 | context={} 36 | ) 37 | 38 | # expected response 39 | expected_response = { 40 | 'first': '1', 41 | 'second': '20', 42 | 'third': '3', 43 | 'result': 79 44 | } 45 | 46 | assert(actual_response == expected_response) 47 | -------------------------------------------------------------------------------- /tests/unit/lambdas/test_cleaninput.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import sys 6 | 7 | sys.path.append( 8 | os.path.realpath( 9 | os.path.join( 10 | os.path.dirname(__file__), 11 | os.path.pardir, 12 | os.path.pardir, 13 | os.path.pardir, 14 | "lambdas", 15 | "cleaninput" 16 | ) 17 | ) 18 | ) 19 | 20 | from cleaninput import lambda_handler 21 | 22 | def test_cleaninput(): 23 | event = { 24 | 'input': { 25 | 'input': '1 20 3' 26 | } 27 | } 28 | 29 | # actual function call 30 | actual_response = lambda_handler( 31 | event=event, 32 | context={} 33 | ) 34 | 35 | # expected response 36 | expected_response = { 37 | 'input': { 38 | 'first': '1', 39 | 'second': '20', 40 | 'third': '3' 41 | } 42 | } 43 | 44 | assert(actual_response == expected_response) 45 | -------------------------------------------------------------------------------- /tests/unit/lambdas/test_divide.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import sys 6 | 7 | sys.path.append( 8 | os.path.realpath( 9 | os.path.join( 10 | os.path.dirname(__file__), 11 | os.path.pardir, 12 | os.path.pardir, 13 | os.path.pardir, 14 | "lambdas", 15 | "divide" 16 | ) 17 | ) 18 | ) 19 | 20 | from divide import lambda_handler 21 | 22 | def test_divide(): 23 | event = { 24 | "input": { 25 | "first": "1", 26 | "second": "20", 27 | "third": "3", 28 | "result": 59 29 | } 30 | } 31 | 32 | # actual function call 33 | actual_response = lambda_handler( 34 | event=event, 35 | context={} 36 | ) 37 | 38 | # expected response 39 | expected_response = { 40 | 'first': '1', 41 | 'second': '20', 42 | 'third': '3', 43 | 'result': 2 44 | } 45 | 46 | assert(actual_response == expected_response) 47 | -------------------------------------------------------------------------------- /tests/unit/lambdas/test_multiply.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import sys 6 | 7 | sys.path.append( 8 | os.path.realpath( 9 | os.path.join( 10 | os.path.dirname(__file__), 11 | os.path.pardir, 12 | os.path.pardir, 13 | os.path.pardir, 14 | "lambdas", 15 | "multiply" 16 | ) 17 | ) 18 | ) 19 | 20 | from multiply import lambda_handler 21 | 22 | def test_multiply(): 23 | event = { 24 | "input": { 25 | "input": { 26 | "first": "1", 27 | "second": "20", 28 | "third": "3", 29 | "result": 59 30 | } 31 | } 32 | } 33 | 34 | # actual function call 35 | actual_response = lambda_handler( 36 | event=event, 37 | context={} 38 | ) 39 | 40 | # expected response 41 | expected_response = { 42 | 'first': '1', 43 | 'second': '20', 44 | 'third': '3', 45 | 'result': 60 46 | } 47 | 48 | assert(actual_response == expected_response) 49 | -------------------------------------------------------------------------------- /tests/unit/lambdas/test_subtract.py: -------------------------------------------------------------------------------- 1 | # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # SPDX-License-Identifier: MIT-0 3 | 4 | import os 5 | import sys 6 | 7 | sys.path.append( 8 | os.path.realpath( 9 | os.path.join( 10 | os.path.dirname(__file__), 11 | os.path.pardir, 12 | os.path.pardir, 13 | os.path.pardir, 14 | "lambdas", 15 | "subtract" 16 | ) 17 | ) 18 | ) 19 | 20 | from subtract import lambda_handler 21 | 22 | def test_subtract(): 23 | event = { 24 | "input": { 25 | "first": "1", 26 | "second": "20", 27 | "third": "3", 28 | "result": 59 29 | } 30 | } 31 | 32 | # actual function call 33 | actual_response = lambda_handler( 34 | event=event, 35 | context={} 36 | ) 37 | 38 | # expected response 39 | expected_response = { 40 | 'first': '1', 41 | 'second': '20', 42 | 'third': '3', 43 | 'result': 58 44 | } 45 | 46 | assert(actual_response == expected_response) 47 | --------------------------------------------------------------------------------