├── .github └── PULL_REQUEST_TEMPLATE.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── ddb-backup-cf.json └── ddbbackup.py /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | *Issue #, if available:* 2 | 3 | *Description of changes:* 4 | 5 | 6 | By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. 7 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check [existing open](https://github.com/awslabs/dynamodb-backup-scheduler/issues), or [recently closed](https://github.com/awslabs/dynamodb-backup-scheduler/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/dynamodb-backup-scheduler/labels/help%20wanted) issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Licensing 54 | 55 | See the [LICENSE](https://github.com/awslabs/dynamodb-backup-scheduler/blob/master/LICENSE) file for our project's licensing. We will ask you confirm the licensing of your contribution. 56 | 57 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 58 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Dynamodb Backup Scheduler 2 | 3 | A cloudFormation template - Since DynamoDb backup is ondemand feature - this CF template will help customer to trigger the Lambda code (Python code) in daily basis 4 | 5 | ## License 6 | 7 | This sample code is made available under the MIT-0 license. See the LICENSE file. 8 | -------------------------------------------------------------------------------- /ddb-backup-cf.json: -------------------------------------------------------------------------------- 1 | { 2 | "AWSTemplateFormatVersion": "2010-09-09", 3 | "Description": "Sample template to schedule DynamoDB table backup using Lambda- V0.23", 4 | "Parameters": { 5 | "BucketName":{ 6 | "Type": "String", 7 | "Description": "Enter bucket name" 8 | }, 9 | "FileName":{ 10 | "Type": "String", 11 | "Description": "Enter file name" 12 | }, 13 | "DDBTableName":{ 14 | "Type": "String", 15 | "Description": "Enter DynamoDB Table name to enable backup" 16 | }, 17 | "BackupRetention": { 18 | "Type": "Number", 19 | "Description": "Days keep the backup" 20 | } 21 | }, 22 | "Metadata" : { 23 | "AWS::CloudFormation::Interface" : { 24 | "ParameterGroups" : [ 25 | { 26 | "Label" : { "default" : "Lambda Function" }, 27 | "Parameters" : [ "BucketName", "FileName" ] 28 | }, 29 | { 30 | "Label" : { "default":"Target DynamoDB table to backup" }, 31 | "Parameters" : [ "DDBTableName", "BackupRetention" ] 32 | } 33 | ] 34 | } 35 | }, 36 | "Resources": { 37 | "LambdaServiceExecutionRole": { 38 | "Properties": { 39 | "AssumeRolePolicyDocument": { 40 | "Statement": [ 41 | { 42 | "Action": [ 43 | "sts:AssumeRole" 44 | ], 45 | "Effect": "Allow", 46 | "Principal": { 47 | "Service": [ 48 | "lambda.amazonaws.com" 49 | ] 50 | } 51 | } 52 | ], 53 | "Version": "2012-10-17" 54 | }, 55 | "Path": "/" 56 | }, 57 | "Type": "AWS::IAM::Role" 58 | }, 59 | "LambdaServiceExecutionRolePolicy":{ 60 | "Type": "AWS::IAM::Policy", 61 | "Properties": { 62 | "PolicyName": "DDBBackupLambdaRolePolicy", 63 | "PolicyDocument": { 64 | "Version": "2012-10-17", 65 | "Statement": [{ 66 | "Effect": "Allow", 67 | "Action": ["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"], 68 | "Resource": { 69 | "Fn::Join": [ 70 | "", 71 | [ 72 | "arn:aws:logs:", 73 | {"Ref": "AWS::Region"}, 74 | ":", 75 | {"Ref": "AWS::AccountId"}, 76 | ":", 77 | "log-group:", 78 | "/aws/lambda/", 79 | {"Ref": "DDBBackup"}, 80 | ":*" 81 | ] 82 | ] 83 | } 84 | }, 85 | { 86 | "Effect": "Allow", 87 | "Action": [ "dynamodb:CreateBackup" ], 88 | "Resource": 89 | [{ 90 | "Fn::Join": [ 91 | "", 92 | [ 93 | "arn:aws:dynamodb:", 94 | {"Ref": "AWS::Region"}, 95 | ":", 96 | {"Ref": "AWS::AccountId"}, 97 | ":", 98 | "table/", 99 | {"Ref": "DDBTableName"} 100 | ] 101 | ] 102 | } 103 | ] 104 | }, 105 | { 106 | "Effect": "Allow", 107 | "Action": [ "dynamodb:ListBackups", "dynamodb:DeleteBackup" ], 108 | "Resource": [{ 109 | "Fn::Join": [ 110 | "", 111 | [ 112 | "arn:aws:dynamodb:", 113 | {"Ref": "AWS::Region"}, 114 | ":", 115 | {"Ref": "AWS::AccountId"}, 116 | ":", 117 | "table/", 118 | {"Ref": "DDBTableName"}, 119 | "/backup/*" 120 | ] 121 | ] 122 | } 123 | ] 124 | }] 125 | }, 126 | "Roles": [ { "Ref": "LambdaServiceExecutionRole" } ] 127 | } 128 | }, 129 | "ScheduledEvent": { 130 | "Properties": { 131 | "Description": "CloudWatch event to trigger lambda SG function", 132 | "ScheduleExpression": "rate(1 day)", 133 | "State": "ENABLED", 134 | "Targets": [ 135 | { 136 | "Arn": { "Fn::GetAtt" : ["DDBBackup", "Arn"] }, 137 | "Id": "DDBTarget" 138 | } 139 | ] 140 | }, 141 | "Type": "AWS::Events::Rule" 142 | }, 143 | "LambdaInvokePermission": { 144 | "Type": "AWS::Lambda::Permission", 145 | "Properties": { 146 | "FunctionName" : { "Fn::GetAtt" : ["DDBBackup", "Arn"] }, 147 | "Action": "lambda:InvokeFunction", 148 | "Principal": "events.amazonaws.com", 149 | "SourceArn" : { "Fn::GetAtt" : ["ScheduledEvent", "Arn"] } 150 | } 151 | }, 152 | "DDBBackup": { 153 | "Properties": { 154 | "Code": { 155 | "S3Bucket": {"Ref": "BucketName"}, 156 | "S3Key": {"Ref": "FileName"} 157 | 158 | }, 159 | "Handler": "ddbbackup.lambda_handler", 160 | "MemorySize": 128, 161 | "Role": { 162 | "Fn::GetAtt": [ 163 | "LambdaServiceExecutionRole", 164 | "Arn" 165 | ] 166 | }, 167 | "Runtime": "python3.6", 168 | "Timeout": 300, 169 | "Environment": { 170 | "Variables": { 171 | "DDBTable": { "Ref": "DDBTableName"}, 172 | "BackupRetention": { "Ref": "BackupRetention"} 173 | } 174 | } 175 | }, 176 | "Type": "AWS::Lambda::Function" 177 | } 178 | } 179 | } 180 | -------------------------------------------------------------------------------- /ddbbackup.py: -------------------------------------------------------------------------------- 1 | # Copyright 2017-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 4 | # 5 | # http://aws.amazon.com/apache2.0/ 6 | # 7 | # or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 8 | from __future__ import print_function 9 | from datetime import date, datetime, timedelta 10 | import json 11 | import boto3 12 | import time 13 | from botocore.exceptions import ClientError 14 | import os 15 | ddbRegion = os.environ['AWS_DEFAULT_REGION'] 16 | ddbTable = os.environ['DDBTable'] 17 | backupName = 'Schedule_Backup_V21' 18 | print('Backup started for: ', backupName) 19 | ddb = boto3.client('dynamodb', region_name=ddbRegion) 20 | 21 | # for deleting old backup. It will search for old backup and will escape deleting last backup days you mentioned in the backup retention 22 | #daysToLookBackup=2 23 | daysToLookBackup= int(os.environ['BackupRetention']) 24 | daysToLookBackupL=daysToLookBackup-1 25 | 26 | 27 | def lambda_handler(event, context): 28 | try: 29 | #create backup 30 | ddb.create_backup(TableName=ddbTable,BackupName = backupName) 31 | print('Backup has been taken successfully for table:', ddbTable) 32 | 33 | #check recent backup 34 | lowerDate=datetime.now() - timedelta(days=daysToLookBackupL) 35 | upperDate=datetime.now() 36 | responseLatest = ddb.list_backups(TableName=ddbTable, TimeRangeLowerBound=datetime(lowerDate.year, lowerDate.month, lowerDate.day), TimeRangeUpperBound=datetime(upperDate.year, upperDate.month, upperDate.day)) 37 | latestBackupCount=len(responseLatest['BackupSummaries']) 38 | print('Total backup count in recent days:',latestBackupCount) 39 | 40 | deleteupperDate = datetime.now() - timedelta(days=daysToLookBackup) 41 | print(deleteupperDate) 42 | # TimeRangeLowerBound is the release of Amazon DynamoDB Backup and Restore - Nov 29, 2017 43 | response = ddb.list_backups(TableName=ddbTable, TimeRangeLowerBound=datetime(2017, 11, 29), TimeRangeUpperBound=datetime(deleteupperDate.year, deleteupperDate.month, deleteupperDate.day)) 44 | 45 | #check whether latest backup count is more than two before removing the old backup 46 | if latestBackupCount>=2: 47 | if 'LastEvaluatedBackupArn' in response: 48 | lastEvalBackupArn = response['LastEvaluatedBackupArn'] 49 | else: 50 | lastEvalBackupArn = '' 51 | 52 | while (lastEvalBackupArn != ''): 53 | for record in response['BackupSummaries']: 54 | backupArn = record['BackupArn'] 55 | ddb.delete_backup(BackupArn=backupArn) 56 | print(backupName, 'has deleted this backup:', backupArn) 57 | 58 | response = ddb.list_backups(TableName=ddbTable, TimeRangeLowerBound=datetime(2017, 11, 23), TimeRangeUpperBound=datetime(deleteupperDate.year, deleteupperDate.month, deleteupperDate.day), ExclusiveStartBackupArn=lastEvalBackupArn) 59 | if 'LastEvaluatedBackupArn' in response: 60 | lastEvalBackupArn = response['LastEvaluatedBackupArn'] 61 | else: 62 | lastEvalBackupArn = '' 63 | print ('the end') 64 | else: 65 | print ('Recent backup does not meet the deletion criteria') 66 | 67 | except ClientError as e: 68 | print(e) 69 | 70 | except ValueError as ve: 71 | print('error:',ve) 72 | 73 | except Exception as ex: 74 | print(ex) 75 | 76 | 77 | 78 | --------------------------------------------------------------------------------