├── CHANGELOG.txt ├── LICENSE ├── LICENSE.txt ├── README.md ├── cftemplates ├── snapshots_tool_aurora_dest.json └── snapshots_tool_aurora_source.json └── lambda ├── .gitignore ├── Makefile ├── copy_snapshots_dest_aurora └── lambda_function.py ├── copy_snapshots_no_x_account_aurora └── lambda_function.py ├── delete_old_snapshots_aurora └── lambda_function.py ├── delete_old_snapshots_dest_aurora └── lambda_function.py ├── delete_old_snapshots_no_x_account_aurora └── lambda_function.py ├── share_snapshots_aurora └── lambda_function.py ├── snapshots_tool_utils.py └── take_snapshots_aurora └── lambda_function.py /CHANGELOG.txt: -------------------------------------------------------------------------------- 1 | 0.13 2 | Modified the Cloudformation templates to create one topic for all alarms within the stack. The SNS topic can also be specified at stack creation to help in use cases where the tool is deployed multiple times in the same account and region. 3 | 4 | 0.12 5 | Added support for copying snapshots to a different region without making a cross-account copy 6 | 7 | 0.11 8 | Minor tidying up of code 9 | Added RETENTION_DAYS logic to copy_snapshots_dest.lambda. This allows setting a shorter retention period in the destination account than in the source account 10 | 11 | 0.1 12 | Initial commit -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/aurora-snapshot-tool/37d73aef9f369ab4fe5bba31b3dac791a90adaf7/LICENSE.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Snapshot Tool for Amazon Aurora 2 | 3 | The Snapshot Tool for Amazon Aurora automates the task of creating manual snapshots, copying them into a different account and a different region, and deleting them after a specified number of days. It also allows you to specify the backup schedule (at what times and how often) and a retention period in days. This version will only work with Amazon Aurora MySQL and PostgreSQL instances. For a version that works with other Amazon RDS instances, please visit the [Snapshot Tool for Amazon RDS](https://github.com/awslabs/rds-snapshot-tool). 4 | 5 | ## Getting Started 6 | 7 | ## Building From Source and Deploying 8 | 9 | You will need also to build from source and deploy the code for the Lambda functions to your own bucket in your own account. To build, you need to be on a unix-like system (e.g., macOS or some flavour of Linux) and you need to have `make` and `zip`. 10 | 11 | 1. Create an S3 bucket to hold the Lambda function zip files. The bucket must be in the same region where the Lambda functions will run. And the Lambda functions must run in the same region as the RDS instances. 12 | 13 | 1. Clone the repository using git or downloading from Github 14 | 15 | 1. Edit the `Makefile` file and set `S3DEST` to be the bucket name where you want the functions to go. Set the `AWSARGS`, `AWSCMD` and `ZIPCMD` variables as well. 16 | 17 | 1. Type `make` at the command line. It will call `zip` to make the zip files, and then it will call `aws s3 cp` to copy the zip files to the bucket you named. 18 | 19 | 1. Be sure to use the correct bucket name in the `CodeBucket` parameter when launching the stack in both accounts. 20 | 21 | 22 | To deploy on your accounts, you will need to use the Cloudformation templates provided. 23 | snapshot_tool_aurora_source.json needs to run in the source account (or the account that runs the Aurora clusters) 24 | snapshot_tool_aurora_dest.json needs to run in the destination account (or the account where you'd like to keep your snapshots) 25 | 26 | **IMPORTANT** Run the Cloudformation templates on the same region where your Aurora clusters run (both in the source and destination accounts). If that is not possible because AWS Step Functions is not available, you will need to use the **SourceRegionOverride** parameter explained below. 27 | 28 | ### Source Account 29 | #### Components 30 | The following components will be created in the source account: 31 | * 3 Lambda functions (TakeSnapshotsAurora, ShareSnapshotsAurora, DeleteOldSnapshotsAurora) 32 | * 3 State Machines (Amazon Step Functions) to trigger execution of each Lambda function (stateMachineTakeSnapshotAurora, stateMachineShareSnapshotAurora, stateMachineDeleteOldSnapshotsAurora) 33 | * 3 Cloudwatch Event Rules to trigger the state functions 34 | * 3 Cloudwatch Alarms and 1 associated SNS Topic to alert on State Machines failures 35 | * A Cloudformation stack containing all these resources 36 | 37 | #### Installing in the source account 38 | Run snapshot_tool_aurora_source.json on the Cloudformation console. 39 | You wil need to specify the different parameters. The default values will back up all Aurora clusters in the region at 1AM UTC, once a day. 40 | If your clusters are encrypted, you will need to provide access to the KMS Key to the destination account. You can read more on how to do that here: https://aws.amazon.com/premiumsupport/knowledge-center/share-cmk-account/ 41 | 42 | Here is a break down of each parameter for the source template: 43 | 44 | * **BackupInterval** - how many hours between backup 45 | * **BackupSchedule** - at what times and how often to run backups. Set in accordance with **BackupInterval**. For example, set **BackupInterval** to 8 hours and **BackupSchedule** 0 0,8,16 * * ? * if you want backups to run at 0, 8 and 16 UTC. If your backups run more often than **BackupInterval**, snapshots will only be created when the latest snapshot is older than **BackupInterval** 46 | * **ClusterNamePattern** - set to the names of the clusters you want this tool to back up. You can use a Python regex that will be searched in the cluster identifier. For example, if your clusters are named *prod-01*, *prod-02*, etc, you can set **ClusterNamePattern** to *prod*. The string you specify will be searched anywhere in the name unless you use an anchor such as ^ or $. In most cases, a simple name like "prod" or "dev" will suffice. More information on Python regular expressions here: https://docs.python.org/2/howto/regex.html 47 | * **DestinationAccount** - the account where you want snapshots to be copied to 48 | * **LogLevel** - The log level you want as output to the Lambda functions. ERROR is usually enough. You can increase to INFO or DEBUG. 49 | * **RetentionDays** - the amount of days you want your snapshots to be kept. Snapshots created more than **RetentionDays** ago will be automatically deleted (only if they contain a tag with Key: CreatedBy, Value: Snapshot Tool for Aurora) 50 | * **ShareSnapshots** - Set to TRUE if you are sharing snapshots with a different account. If you set to FALSE, StateMachine, Lambda functions and associated Cloudwatch Alarms related to sharing across accounts will not be created. It is useful if you only want to take backups and manage the retention, but do not need to copy them across accounts or regions. 51 | * **SourceRegionOverride** - if you are running Aurora on a region where Step Functions is not available, this parameter will allow you to override the source region. For example, at the time of this writing, you may be running Aurora in Northern California (us-west-1) and would like to copy your snapshots to Montreal (ca-central-1). Neither region supports Step Functions at the time of this writing so deploying this tool there will not work. The solution is to run this template in a region that supports Step Functions (such as North Virginia or Ohio) and set **SourceRegionOverride** to *us-west-1*. 52 | **IMPORTANT**: deploy to the closest regions for best results. 53 | 54 | * **CodeBucket** - this parameter specifies the bucket where the code for the Lambda functions is located. The Lambda function code is located in the ```lambda``` directory. These files need to be zipped and on the **root* of the bucket or the CloudFormation templates will fail. Please see instructions below on how to build this file hierarchy 55 | * **DeleteOldSnapshots** - Set to TRUE to enable functionanility that will delete snapshots after **RetentionDays**. Set to FALSE if you want to disable this functionality completely. (Associated Lambda and State Machine resources will not be created in the account). **WARNING** If you decide to enable this functionality later on, bear in mind it will delete **all snapshots**, older than **RetentionDays**, created by this tool; not just the ones created after **DeleteOldSnapshots** is set to TRUE. 56 | * **ShareSnapshots** - Set to TRUE to enable functionality that will share snapshots with **DestAccount**. Set to FALSE to completely disable sharing. (Associated Lambda and State Machine resources will not be created in the account.) 57 | * **SnapshotNamePrefix** - Set a name that will be added to the front of the snapshot identifiers when created, so that they are formatted as Addname-ClusterIdentifier-Timestamp. Useful if you need to share snapshots from multiple accounts and need to identify from which account they came from. Use 'NONE' or leave empty if you do not need a prefix (default) 58 | ### Destination Account 59 | #### Components 60 | The following components will be created in the destination account: 61 | * 2 Lambda functions (CopySnapshotsDestAurora, DeleteOldSnapshotsDestAurora) 62 | * 2 State Machines (Amazon Step Functions) to trigger execution of each Lambda function (stateMachineCopySnapshotsDestAurora, stateMachineDeleteOldSnapshotsDestAurora) 63 | * 2 Cloudwatch Event Rules to trigger the state functions 64 | * 2 Cloudwatch Alarms and associated 1 SNS Topic to alert on State Machines failures 65 | * A Cloudformation stack containing all these resources 66 | 67 | On your destination account, you will need to run snapshot_tool_aurora_dest.json on the Cloudformation. As before, you will need to run it in a region where Step Functions is available. 68 | The following parameters are available: 69 | 70 | * **DestinationRegion** - the region where you want your snapshots to be copied. If you set it to the same as the source region, the snapshots will be copied from the source account but will be kept in the source region. This is useful if you would like to keep a copy of your snapshots in a different account but would prefer not to copy them to a different region. 71 | * **CrossAccountCopy** - if you only need to copy snapshots across regions and not to a different account, set this to FALSE. When set to FALSE, any snapshots shared with the account will be ignored. 72 | * **SnapshotPattern** - similar to ClusterNamePattern. See above 73 | * **DeleteOldSnapshots** - Set to TRUE to enable functionanility that will delete snapshots after **RetentionDays**. Set to FALSE if you want to disable this functionality completely. (Associated Lambda and State Machine resources will not be created in the account). **WARNING** If you decide to enable this functionality later on, bear in mind it will delete ALL SNAPSHOTS older than RetentionDays created by this tool, not just the ones created after **DeleteOldSnapshots** is set to TRUE. 74 | * **KmsKeySource** KMS Key to be used for copying encrypted snapshots on the source region. If you are copying to a different region, you will also need to provide a second key in the destination region. 75 | * **KmsKeyDestination** KMS Key to be used for copying encrypted snapshots to the destination region. If you are not copying to a different region, this parameter is not necessary. 76 | * **RetentionDays** - as in the source account, the amount of days you want your snapshots to be kept. **Do not set this parameter to a value lower than the source account.** Snapshots created more than **RetentionDays** ago will be automatically deleted (only if they contain a tag with Key: CopiedBy, Value: Snapshot Tool for Aurora) 77 | 78 | 79 | ## Updating 80 | 81 | This tool is fundamentally stateless. The state is mainly in the tags on the snapshots themselves and the parameters to the CloudFormation stack. If you make changes to the parameters or make changes to the Lambda function code, it is best to delete the stack and then launch the stack again. 82 | 83 | 84 | ## Authors 85 | 86 | * **Marcelo Coronel** - [mrcoronel](https://github.com/mrcoronel) 87 | 88 | ## License 89 | 90 | This project is licensed under the Apache License - see the [LICENSE.txt](LICENSE.txt) file for details 91 | -------------------------------------------------------------------------------- /cftemplates/snapshots_tool_aurora_dest.json: -------------------------------------------------------------------------------- 1 | { 2 | "AWSTemplateFormatVersion": "2010-09-09", 3 | "Parameters": { 4 | "CodeBucket": { 5 | "Type": "String", 6 | "Description": "Name of the bucket that contains the lambda functions to deploy. Leave the default value to download the code from the AWS Managed buckets" 7 | }, 8 | "SnapshotPattern": { 9 | "Type": "String", 10 | "Default": "ALL_SNAPSHOTS", 11 | "Description": "Python regex for matching cluster identifiers to backup. Use \"ALL_SNAPSHOTS\" to back up every Aurora cluster in the region." 12 | }, 13 | "RetentionDays": { 14 | "Type": "Number", 15 | "Default": "7", 16 | "Description": "Number of days to keep snapshots in retention before deleting them" 17 | }, 18 | "DestinationRegion": { 19 | "Type": "String", 20 | "Description": "Destination region for snapshots." 21 | }, 22 | "LogLevel": { 23 | "Type": "String", 24 | "Default": "ERROR", 25 | "Description": "Log level for Lambda functions (DEBUG, INFO, WARN, ERROR, CRITICAL are valid values)." 26 | }, 27 | "SourceRegionOverride": { 28 | "Type": "String", 29 | "Default": "NO", 30 | "Description": "Set to the region where your Aurora clusters run, only if such region does not support Step Functions. Leave as NO otherwise" 31 | }, 32 | "KmsKeyDestination": { 33 | "Type": "String", 34 | "Default": "None", 35 | "Description": "Set to the KMS Key Id in the destination region to re-encrypt encrypted snapshots. Leave None if you are not using encryption" 36 | }, 37 | "KmsKeySource": { 38 | "Type": "String", 39 | "Default": "None", 40 | "Description": "Set to the KMS Key Id in the SOURCE region to re-encrypt encrypted snapshots. Leave None if you are not using encryption" 41 | }, 42 | "DeleteOldSnapshots": { 43 | "Type": "String", 44 | "Default": "TRUE", 45 | "Description": "Set to TRUE to enable deletion of snapshot based on RetentionDays. Set to FALSE to disable", 46 | "AllowedValues": ["TRUE", "FALSE"] 47 | }, 48 | "CrossAccountCopy": { 49 | "Type": "String", 50 | "AllowedValues": ["TRUE", "FALSE"], 51 | "Default": "TRUE", 52 | "Description": "Enable copying snapshots across accounts. Set to FALSE if your source snapshosts are not on a different account" 53 | }, 54 | "SNSTopic": { 55 | "Type": "String", 56 | "Default": "", 57 | "Description": "If you have a topic that you would like subscribed to notifications, enter it here. If empty, the tool will create a new topic" 58 | } 59 | }, 60 | "Conditions": { 61 | "DeleteOld": { 62 | "Fn::Equals": [{ 63 | "Ref": "DeleteOldSnapshots" 64 | }, "TRUE"] 65 | }, 66 | "CrossAccount": { 67 | "Fn::Equals": [{ 68 | "Ref": "CrossAccountCopy" 69 | }, "TRUE"] 70 | }, 71 | "SNSTopicIsEmpty": { 72 | "Fn::Equals": [{ 73 | "Ref": "SNSTopic" 74 | }, ""] 75 | }, 76 | "RegionOverridden": { 77 | "Fn::Not": [{ 78 | "Fn::Equals": [{ 79 | "Ref": "SourceRegionOverride" 80 | }, "NO"] 81 | }] 82 | 83 | 84 | } 85 | }, 86 | "Resources": { 87 | "snsTopicSnapshotsAuroraToolDest": { 88 | "Condition": "SNSTopicIsEmpty", 89 | "Type": "AWS::SNS::Topic", 90 | "Properties": { 91 | "DisplayName": "topic_aurora_tool_dest" 92 | } 93 | }, 94 | "snspolicySnapshotsAuroraDest": { 95 | "Condition": "SNSTopicIsEmpty", 96 | "Type": "AWS::SNS::TopicPolicy", 97 | "Properties": { 98 | "Topics": [{ 99 | "Fn::If": ["SNSTopicIsEmpty", { 100 | "Ref": "snsTopicSnapshotsAuroraToolDest" 101 | }, { 102 | "Ref": "SNSTopic" 103 | }] 104 | }], 105 | "PolicyDocument": { 106 | "Version": "2008-10-17", 107 | "Id": "__default_policy_ID", 108 | "Statement": [{ 109 | "Effect": "Allow", 110 | "Principal": { 111 | "AWS": "*" 112 | }, 113 | "Action": [ 114 | "SNS:GetTopicAttributes", 115 | "SNS:SetTopicAttributes", 116 | "SNS:AddPermission", 117 | "SNS:RemovePermission", 118 | "SNS:DeleteTopic", 119 | "SNS:Subscribe", 120 | "SNS:ListSubscriptionsByTopic", 121 | "SNS:Publish", 122 | "SNS:Receive" 123 | ], 124 | "Resource": "*", 125 | "Condition": { 126 | "StringEquals": { 127 | "AWS:SourceOwner": { 128 | "Ref": "AWS::AccountId" 129 | } 130 | } 131 | } 132 | }] 133 | } 134 | } 135 | }, 136 | "alarmcwCopyFailedDest": { 137 | "Type": "AWS::CloudWatch::Alarm", 138 | "Properties": { 139 | "AlarmDescription": "DB Copy to destination status", 140 | "ActionsEnabled": "true", 141 | "ComparisonOperator": "GreaterThanOrEqualToThreshold", 142 | "EvaluationPeriods": "1", 143 | "MetricName": "ExecutionsFailed", 144 | "Namespace": "AWS/States", 145 | "Period": "300", 146 | "Statistic": "Sum", 147 | "Threshold": "2.0", 148 | "TreatMissingData": "ignore", 149 | "AlarmActions": [{ 150 | "Fn::If": ["SNSTopicIsEmpty", { 151 | "Ref": "snsTopicSnapshotsAuroraToolDest" 152 | }, { 153 | "Ref": "SNSTopic" 154 | }] 155 | }], 156 | "OKActions": [{ 157 | "Fn::If": ["SNSTopicIsEmpty", { 158 | "Ref": "snsTopicSnapshotsAuroraToolDest" 159 | }, { 160 | "Ref": "SNSTopic" 161 | }] 162 | }], 163 | "InsufficientDataActions": [{ 164 | "Fn::If": ["SNSTopicIsEmpty", { 165 | "Ref": "snsTopicSnapshotsAuroraToolDest" 166 | }, { 167 | "Ref": "SNSTopic" 168 | }] 169 | }], 170 | "Dimensions": [{ 171 | "Name": "StateMachineArn", 172 | "Value": { 173 | "Ref": "statemachineCopySnapshotsDestAurora" 174 | } 175 | }] 176 | } 177 | }, 178 | "alarmcwDeleteOldFailedDest": { 179 | "Type": "AWS::CloudWatch::Alarm", 180 | "Condition": "DeleteOld", 181 | "Properties": { 182 | "AlarmDescription": "Failed to delete old snapshots from destination", 183 | "ActionsEnabled": "true", 184 | "ComparisonOperator": "GreaterThanOrEqualToThreshold", 185 | "EvaluationPeriods": "2", 186 | "MetricName": "ExecutionsFailed", 187 | "Namespace": "AWS/States", 188 | "Period": "3600", 189 | "Statistic": "Sum", 190 | "Threshold": "2.0", 191 | "AlarmActions": [{ 192 | "Fn::If": ["SNSTopicIsEmpty", { 193 | "Ref": "snsTopicSnapshotsAuroraToolDest" 194 | }, { 195 | "Ref": "SNSTopic" 196 | }] 197 | }], 198 | "OKActions": [{ 199 | "Fn::If": ["SNSTopicIsEmpty", { 200 | "Ref": "snsTopicSnapshotsAuroraToolDest" 201 | }, { 202 | "Ref": "SNSTopic" 203 | }] 204 | }], 205 | "InsufficientDataActions": [{ 206 | "Fn::If": ["SNSTopicIsEmpty", { 207 | "Ref": "snsTopicSnapshotsAuroraToolDest" 208 | }, { 209 | "Ref": "SNSTopic" 210 | }] 211 | }], 212 | "Dimensions": [{ 213 | "Name": "StateMachineArn", 214 | "Value": { 215 | "Ref": "statemachineDeleteOldSnapshotsDestAurora" 216 | } 217 | }] 218 | } 219 | }, 220 | "iamroleSnapshotsAurora": { 221 | "Type": "AWS::IAM::Role", 222 | "Properties": { 223 | "AssumeRolePolicyDocument": { 224 | "Version": "2012-10-17", 225 | "Statement": [{ 226 | "Effect": "Allow", 227 | "Principal": { 228 | "Service": "lambda.amazonaws.com" 229 | }, 230 | "Action": "sts:AssumeRole" 231 | }] 232 | }, 233 | "ManagedPolicyArns": ["arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"], 234 | "Policies": [{ 235 | "PolicyName": "inline_policy_snapshots_aurora_rds", 236 | "PolicyDocument": { 237 | "Version": "2012-10-17", 238 | "Statement": [{ 239 | "Effect": "Allow", 240 | "Action": [ 241 | "rds:DescribeDBClusters", 242 | "rds:DescribeDBClusterSnapshots" 243 | ], 244 | "Resource": "*" 245 | }, 246 | { 247 | "Effect": "Allow", 248 | "Action": [ 249 | "rds:CreateDBClusterSnapshot", 250 | "rds:DeleteDBClusterSnapshot", 251 | "rds:ModifyDBClusterSnapshotAttribute", 252 | "rds:CopyDBClusterSnapshot", 253 | "rds:DescribeDBClusterSnapshotAttributes", 254 | "rds:ListTagsForResource", 255 | "rds:AddTagsToResource" 256 | ], 257 | "Resource": [ 258 | "arn:aws:rds:*:*:cluster:*", 259 | "arn:aws:rds:*:*:cluster-snapshot:*" 260 | ] 261 | 262 | } 263 | ] 264 | } 265 | 266 | }, 267 | { 268 | "PolicyName": "inline_policy_snapshots_aurora_kms_access", 269 | "PolicyDocument": { 270 | "Version": "2012-10-17", 271 | "Statement": [{ 272 | "Sid": "AllowUseOfTheKey", 273 | "Effect": "Allow", 274 | "Action": [ 275 | "kms:Encrypt", 276 | "kms:Decrypt", 277 | "kms:ReEncrypt*", 278 | "kms:GenerateDataKey*", 279 | "kms:DescribeKey" 280 | ], 281 | "Resource": [ 282 | "arn:aws:kms:*:*:key/*" 283 | ] 284 | }, 285 | { 286 | "Sid": "AllowAttachmentOfPersistentResources", 287 | "Effect": "Allow", 288 | "Action": [ 289 | "kms:CreateGrant", 290 | "kms:ListGrants", 291 | "kms:RevokeGrant" 292 | ], 293 | "Resource": [ 294 | "arn:aws:kms:*:*:key/*" 295 | ], 296 | "Condition": { 297 | "Bool": { 298 | "kms:GrantIsForAWSResource": true 299 | } 300 | } 301 | } 302 | ] 303 | } 304 | } 305 | ] 306 | } 307 | }, 308 | "lambdaCopySnapshotsAurora": { 309 | "Type": "AWS::Lambda::Function", 310 | "Properties": { 311 | "Code": { 312 | "S3Bucket": { 313 | "Ref": "CodeBucket" 314 | }, 315 | "S3Key": { 316 | "Fn::If": ["CrossAccount", "copy_snapshots_dest_aurora.zip", "copy_snapshots_no_x_account_aurora.zip"] 317 | } 318 | }, 319 | "Description": "This functions copies snapshots for Aurora clusters shared with this account. It checks for existing snapshots following the pattern specified in the environment variables with the following format: -YYYY-MM-DD-HH-MM", 320 | "MemorySize": 512, 321 | "Environment": { 322 | "Variables": { 323 | "SNAPSHOT_PATTERN": { 324 | "Ref": "SnapshotPattern" 325 | }, 326 | "DEST_REGION": { 327 | "Ref": "DestinationRegion" 328 | }, 329 | "LOG_LEVEL": { 330 | "Ref": "LogLevel" 331 | }, 332 | "REGION_OVERRIDE": { 333 | "Ref": "SourceRegionOverride" 334 | }, 335 | "KMS_KEY_DEST_REGION": { 336 | "Ref": "KmsKeyDestination" 337 | }, 338 | "KMS_KEY_SOURCE_REGION": { 339 | "Ref": "KmsKeySource" 340 | }, 341 | "RETENTION_DAYS": { 342 | "Ref": "RetentionDays" 343 | } 344 | } 345 | }, 346 | "Role": { 347 | "Fn::GetAtt": ["iamroleSnapshotsAurora", "Arn"] 348 | }, 349 | "Runtime": "python3.7", 350 | "Handler": "lambda_function.lambda_handler", 351 | "Timeout": 300 352 | } 353 | }, 354 | "lambdaDeleteOldDestAurora": { 355 | "Type": "AWS::Lambda::Function", 356 | "Condition": "DeleteOld", 357 | "Properties": { 358 | "Code": { 359 | "S3Bucket": { 360 | "Ref": "CodeBucket" 361 | }, 362 | "S3Key": { 363 | "Fn::If": ["CrossAccount", "delete_old_snapshots_dest_aurora.zip", "delete_old_snapshots_no_x_account_aurora.zip"] 364 | } 365 | }, 366 | "Description": "This function enforces retention on the snapshots shared with the destination account.", 367 | "MemorySize": 512, 368 | "Environment": { 369 | "Variables": { 370 | "SNAPSHOT_PATTERN": { 371 | "Ref": "SnapshotPattern" 372 | }, 373 | "DEST_REGION": { 374 | "Ref": "DestinationRegion" 375 | }, 376 | "RETENTION_DAYS": { 377 | "Ref": "RetentionDays" 378 | }, 379 | "LOG_LEVEL": { 380 | "Ref": "LogLevel" 381 | } 382 | } 383 | }, 384 | "Role": { 385 | "Fn::GetAtt": ["iamroleSnapshotsAurora", "Arn"] 386 | }, 387 | "Runtime": "python3.7", 388 | "Handler": "lambda_function.lambda_handler", 389 | "Timeout": 300 390 | } 391 | }, 392 | "iamroleStateExecution": { 393 | "Type": "AWS::IAM::Role", 394 | "Properties": { 395 | "AssumeRolePolicyDocument": { 396 | "Version": "2012-10-17", 397 | "Statement": [{ 398 | "Effect": "Allow", 399 | "Principal": { 400 | "Service": { 401 | "Fn::Join": ["", ["states.", { 402 | "Ref": "AWS::Region" 403 | }, ".amazonaws.com"]] 404 | } 405 | }, 406 | "Action": "sts:AssumeRole" 407 | }] 408 | }, 409 | "Policies": [{ 410 | "PolicyName": "inline_policy_aurora_snapshot", 411 | "PolicyDocument": { 412 | "Version": "2012-10-17", 413 | "Statement": [{ 414 | "Effect": "Allow", 415 | "Action": [ 416 | "lambda:InvokeFunction" 417 | ], 418 | "Resource": [{ 419 | "Fn::GetAtt": ["lambdaCopySnapshotsAurora", "Arn"] 420 | }, { 421 | "Fn::GetAtt": ["lambdaDeleteOldDestAurora", "Arn"] 422 | }] 423 | 424 | }] 425 | } 426 | }] 427 | } 428 | }, 429 | "statemachineCopySnapshotsDestAurora": { 430 | "Type": "AWS::StepFunctions::StateMachine", 431 | "Properties": { 432 | "DefinitionString": { 433 | "Fn::Join": ["", [{ 434 | "Fn::Join": ["\n", [ 435 | " {\"Comment\":\"Copies snapshots locally and then to DEST_REGION\",", 436 | " \"StartAt\":\"CopySnapshots\",", 437 | " \"States\":{", 438 | " \"CopySnapshots\":{", 439 | " \"Type\":\"Task\",", 440 | " \"Resource\": " 441 | ]] 442 | }, 443 | "\"", 444 | { 445 | "Fn::GetAtt": ["lambdaCopySnapshotsAurora", "Arn"] 446 | }, "\"\n,", 447 | { 448 | "Fn::Join": ["\n", [ 449 | " \"Retry\":[", 450 | " {", 451 | " \"ErrorEquals\":[ ", 452 | " \"SnapshotToolException\"", 453 | " ],", 454 | " \"IntervalSeconds\":300,", 455 | " \"MaxAttempts\":5,", 456 | " \"BackoffRate\":1", 457 | " },", 458 | " {", 459 | " \"ErrorEquals\":[ ", 460 | " \"States.ALL\"], ", 461 | " \"IntervalSeconds\": 30,", 462 | " \"MaxAttempts\": 20,", 463 | " \"BackoffRate\": 1", 464 | " }", 465 | " ],", 466 | " \"End\": true ", 467 | " }", 468 | " }}" 469 | ]] 470 | } 471 | ]] 472 | }, 473 | "RoleArn": { 474 | "Fn::GetAtt": ["iamroleStateExecution", "Arn"] 475 | } 476 | } 477 | }, 478 | "statemachineDeleteOldSnapshotsDestAurora": { 479 | "Type": "AWS::StepFunctions::StateMachine", 480 | "Condition": "DeleteOld", 481 | "Properties": { 482 | "DefinitionString": { 483 | "Fn::Join": ["", [{ 484 | "Fn::Join": ["\n", [ 485 | " {\"Comment\":\"DeleteOld for Aurora snapshots in destination region\",", 486 | " \"StartAt\":\"DeleteOldDestRegion\",", 487 | " \"States\":{", 488 | " \"DeleteOldDestRegion\":{", 489 | " \"Type\":\"Task\",", 490 | " \"Resource\": " 491 | ]] 492 | }, 493 | "\"", 494 | { 495 | "Fn::GetAtt": ["lambdaDeleteOldDestAurora", "Arn"] 496 | }, "\"\n,", 497 | { 498 | "Fn::Join": ["\n", [ 499 | " \"Retry\":[", 500 | " {", 501 | " \"ErrorEquals\":[ ", 502 | " \"SnapshotToolException\"", 503 | " ],", 504 | " \"IntervalSeconds\":600,", 505 | " \"MaxAttempts\":3,", 506 | " \"BackoffRate\":1", 507 | " },", 508 | " {", 509 | " \"ErrorEquals\":[ ", 510 | " \"States.ALL\"], ", 511 | " \"IntervalSeconds\": 30,", 512 | " \"MaxAttempts\": 20,", 513 | " \"BackoffRate\": 1", 514 | " }", 515 | " ],", 516 | " \"End\": true ", 517 | " }", 518 | " }}" 519 | ]] 520 | } 521 | ]] 522 | }, 523 | "RoleArn": { 524 | "Fn::GetAtt": ["iamroleStateExecution", "Arn"] 525 | } 526 | } 527 | }, 528 | "iamroleStepInvocation": { 529 | "Type": "AWS::IAM::Role", 530 | "Properties": { 531 | "AssumeRolePolicyDocument": { 532 | "Version": "2012-10-17", 533 | "Statement": [{ 534 | "Effect": "Allow", 535 | "Principal": { 536 | "Service": "events.amazonaws.com" 537 | }, 538 | "Action": "sts:AssumeRole" 539 | }] 540 | }, 541 | "Policies": [{ 542 | "PolicyName": "inline_policy_state_invocation", 543 | "PolicyDocument": { 544 | "Version": "2012-10-17", 545 | "Statement": [{ 546 | "Effect": "Allow", 547 | "Action": [ 548 | "states:StartExecution" 549 | ], 550 | "Resource": [{ 551 | "Ref": "statemachineCopySnapshotsDestAurora" 552 | }, { 553 | "Ref": "statemachineDeleteOldSnapshotsDestAurora" 554 | }] 555 | }] 556 | } 557 | }] 558 | } 559 | }, 560 | "cwEventCopySnapshotsAurora": { 561 | "Type": "AWS::Events::Rule", 562 | "Properties": { 563 | "Description": "Triggers the Aurora Copy state machine in the destination account", 564 | "ScheduleExpression": { 565 | "Fn::Join": ["", ["cron(", "/30 * * * ? *", ")"]] 566 | }, 567 | "State": "ENABLED", 568 | "Targets": [{ 569 | "Arn": { 570 | "Ref": "statemachineCopySnapshotsDestAurora" 571 | }, 572 | "Id": "Target1", 573 | "RoleArn": { 574 | "Fn::GetAtt": ["iamroleStepInvocation", "Arn"] 575 | } 576 | }] 577 | } 578 | }, 579 | "cwEventDeleteOldSnapshotsAurora": { 580 | "Type": "AWS::Events::Rule", 581 | "Condition": "DeleteOld", 582 | "Properties": { 583 | "Description": "Triggers the Aurora DeleteOld state machine in the destination account", 584 | "ScheduleExpression": { 585 | "Fn::Join": ["", ["cron(", "0 /1 * * ? *", ")"]] 586 | }, 587 | "State": "ENABLED", 588 | "Targets": [{ 589 | "Arn": { 590 | "Ref": "statemachineDeleteOldSnapshotsDestAurora" 591 | }, 592 | "Id": "Target1", 593 | "RoleArn": { 594 | "Fn::GetAtt": ["iamroleStepInvocation", "Arn"] 595 | } 596 | }] 597 | } 598 | } 599 | }, 600 | "Outputs": { 601 | "SNSTopic": { 602 | "Description": "Subscribe to this topic to receive alerts of failures with the snapshots tool", 603 | "Value": { 604 | "Fn::If": ["SNSTopicIsEmpty", { 605 | "Ref": "snsTopicSnapshotsAuroraToolDest" 606 | }, { 607 | "Ref": "SNSTopic" 608 | }] 609 | } 610 | } 611 | }, 612 | "Description": "Snapshots for Aurora cross-region and cross-account (destination account stack)" 613 | } -------------------------------------------------------------------------------- /cftemplates/snapshots_tool_aurora_source.json: -------------------------------------------------------------------------------- 1 | { 2 | "AWSTemplateFormatVersion": "2010-09-09", 3 | "Parameters": { 4 | "CodeBucket": { 5 | "Type": "String", 6 | "Description": "Name of the bucket that contains the lambda functions to deploy. Leave the default value to download the code from the AWS Managed buckets" 7 | }, 8 | "ClusterNamePattern": { 9 | "Type": "String", 10 | "Default": "ALL_CLUSTERS", 11 | "Description": "Python regex for matching cluster identifiers to backup. Use \"ALL_CLUSTERS\" to back up every Aurora cluster in the region." 12 | }, 13 | "BackupInterval": { 14 | "Type": "Number", 15 | "Default": "24", 16 | "Description": "Interval for backups in hours. Default is 24" 17 | }, 18 | "DestinationAccount": { 19 | "Type": "Number", 20 | "Default": "000000000000", 21 | "Description": "Destination account with no dashes." 22 | }, 23 | "ShareSnapshots": { 24 | "Type": "String", 25 | "Default": "TRUE", 26 | "AllowedValues": ["TRUE", "FALSE"] 27 | }, 28 | "BackupSchedule": { 29 | "Type": "String", 30 | "Default": "0 1 * * ? *", 31 | "Description": "Backup schedule in Cloudwatch Event cron format. Needs to run at least once for every Interval. The default value runs once every at 1AM UTC. More information: http://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html" 32 | }, 33 | "RetentionDays": { 34 | "Type": "Number", 35 | "Default": "7", 36 | "Description": "Number of days to keep snapshots in retention before deleting them" 37 | }, 38 | "LogLevel": { 39 | "Type": "String", 40 | "Default": "ERROR", 41 | "Description": "Log level for Lambda functions (DEBUG, INFO, WARN, ERROR, CRITICAL are valid values)." 42 | }, 43 | "SourceRegionOverride": { 44 | "Type": "String", 45 | "Default": "NO", 46 | "Description": "Set to the region where your Aurora clusters run, only if such region does not support Step Functions. Leave as NO otherwise" 47 | }, 48 | "DeleteOldSnapshots": { 49 | "Type": "String", 50 | "Default": "TRUE", 51 | "Description": "Set to TRUE to enable deletion of snapshot based on RetentionDays. Set to FALSE to disable", 52 | "AllowedValues": ["TRUE", "FALSE"] 53 | }, 54 | "SNSTopic": { 55 | "Type": "String", 56 | "Default": "", 57 | "Description": "If you have a topic that you would like subscribed to notifications, enter it here. If empty, the tool will create a new topic" 58 | }, 59 | "SnapshotNamePrefix": { 60 | "Type": "String", 61 | "Default": "", 62 | "Description": "Add a name/tag to the front of the snapshot identifier, so they are formatted like this: ADD_NAME-CLUSTERIDENTIFIER-TIMESTAMP" 63 | } 64 | }, 65 | "Conditions": { 66 | "Share": { 67 | "Fn::Equals": [{ 68 | "Ref": "ShareSnapshots" 69 | }, "TRUE"] 70 | }, 71 | "DeleteOld": { 72 | "Fn::Equals": [{ 73 | "Ref": "DeleteOldSnapshots" 74 | }, "TRUE"] 75 | }, 76 | "SNSTopicIsEmpty": { 77 | "Fn::Equals": [{ 78 | "Ref": "SNSTopic" 79 | }, ""] 80 | }, 81 | "RegionOverridden": { 82 | "Fn::Not": [{ 83 | "Fn::Equals": [{ 84 | "Ref": "SourceRegionOverride" 85 | }, "NO"] 86 | }] 87 | } 88 | 89 | }, 90 | "Resources": { 91 | "snsTopicSnapshotsAuroraTool": { 92 | "Condition": "SNSTopicIsEmpty", 93 | "Type": "AWS::SNS::Topic", 94 | "Properties": { 95 | "DisplayName": "topic_aurora_tool" 96 | } 97 | }, 98 | "snspolicySnapshotsAurora": { 99 | "Condition": "SNSTopicIsEmpty", 100 | "Type": "AWS::SNS::TopicPolicy", 101 | "Properties": { 102 | "Topics": [{ 103 | "Fn::If": ["SNSTopicIsEmpty", { 104 | "Ref": "snsTopicSnapshotsAuroraTool" 105 | }, { 106 | "Ref": "SNSTopic" 107 | }] 108 | }], 109 | "PolicyDocument": { 110 | "Version": "2008-10-17", 111 | "Id": "__default_policy_ID", 112 | "Statement": [{ 113 | "Effect": "Allow", 114 | "Principal": { 115 | "AWS": "*" 116 | }, 117 | "Action": [ 118 | "SNS:GetTopicAttributes", 119 | "SNS:SetTopicAttributes", 120 | "SNS:AddPermission", 121 | "SNS:RemovePermission", 122 | "SNS:DeleteTopic", 123 | "SNS:Subscribe", 124 | "SNS:ListSubscriptionsByTopic", 125 | "SNS:Publish", 126 | "SNS:Receive" 127 | ], 128 | "Resource": "*", 129 | "Condition": { 130 | "StringEquals": { 131 | "AWS:SourceOwner": { 132 | "Ref": "AWS::AccountId" 133 | } 134 | } 135 | } 136 | }] 137 | } 138 | } 139 | }, 140 | "alarmcwBackupsFailed": { 141 | "Type": "AWS::CloudWatch::Alarm", 142 | "Properties": { 143 | "AlarmDescription": "DB backup status", 144 | "ActionsEnabled": "true", 145 | "ComparisonOperator": "GreaterThanOrEqualToThreshold", 146 | "EvaluationPeriods": "1", 147 | "MetricName": "ExecutionsFailed", 148 | "Namespace": "AWS/States", 149 | "Period": "300", 150 | "Statistic": "Sum", 151 | "Threshold": "1.0", 152 | "TreatMissingData": "ignore", 153 | "AlarmActions": [{ 154 | "Fn::If": ["SNSTopicIsEmpty", { 155 | "Ref": "snsTopicSnapshotsAuroraTool" 156 | }, { 157 | "Ref": "SNSTopic" 158 | }] 159 | }], 160 | "OKActions": [{ 161 | "Fn::If": ["SNSTopicIsEmpty", { 162 | "Ref": "snsTopicSnapshotsAuroraTool" 163 | }, { 164 | "Ref": "SNSTopic" 165 | }] 166 | }], 167 | "InsufficientDataActions": [{ 168 | "Fn::If": ["SNSTopicIsEmpty", { 169 | "Ref": "snsTopicSnapshotsAuroraTool" 170 | }, { 171 | "Ref": "SNSTopic" 172 | }] 173 | }], 174 | "Dimensions": [{ 175 | "Name": "StateMachineArn", 176 | "Value": { 177 | "Ref": "stateMachineTakeSnapshotsAurora" 178 | } 179 | }] 180 | } 181 | }, 182 | 183 | "alarmcwShareFailed": { 184 | "Condition": "Share", 185 | "Type": "AWS::CloudWatch::Alarm", 186 | "Properties": { 187 | "AlarmDescription": "DB backup transfer to DR region status", 188 | "ActionsEnabled": "true", 189 | "ComparisonOperator": "GreaterThanOrEqualToThreshold", 190 | "EvaluationPeriods": "2", 191 | "MetricName": "ExecutionsFailed", 192 | "Namespace": "AWS/States", 193 | "Period": "3600", 194 | "Statistic": "Sum", 195 | "Threshold": "6.0", 196 | "AlarmActions": [{ 197 | "Fn::If": ["SNSTopicIsEmpty", { 198 | "Ref": "snsTopicSnapshotsAuroraTool" 199 | }, { 200 | "Ref": "SNSTopic" 201 | }] 202 | }], 203 | "OKActions": [{ 204 | "Fn::If": ["SNSTopicIsEmpty", { 205 | "Ref": "snsTopicSnapshotsAuroraTool" 206 | }, { 207 | "Ref": "SNSTopic" 208 | }] 209 | }], 210 | "InsufficientDataActions": [{ 211 | "Fn::If": ["SNSTopicIsEmpty", { 212 | "Ref": "snsTopicSnapshotsAuroraTool" 213 | }, { 214 | "Ref": "SNSTopic" 215 | }] 216 | }], 217 | "Dimensions": [{ 218 | "Name": "StateMachineArn", 219 | "Value": { 220 | "Ref": "statemachineShareSnapshotsAurora" 221 | } 222 | }] 223 | } 224 | }, 225 | "alarmcwDeleteOldFailed": { 226 | "Condition": "DeleteOld", 227 | "Type": "AWS::CloudWatch::Alarm", 228 | "Properties": { 229 | "ActionsEnabled": "true", 230 | "ComparisonOperator": "GreaterThanOrEqualToThreshold", 231 | "EvaluationPeriods": "2", 232 | "MetricName": "ExecutionsFailed", 233 | "Namespace": "AWS/States", 234 | "Period": "3600", 235 | "Statistic": "Sum", 236 | "Threshold": "2.0", 237 | "AlarmActions": [{ 238 | "Fn::If": ["SNSTopicIsEmpty", { 239 | "Ref": "snsTopicSnapshotsAuroraTool" 240 | }, { 241 | "Ref": "SNSTopic" 242 | }] 243 | }], 244 | "OKActions": [{ 245 | "Fn::If": ["SNSTopicIsEmpty", { 246 | "Ref": "snsTopicSnapshotsAuroraTool" 247 | }, { 248 | "Ref": "SNSTopic" 249 | }] 250 | }], 251 | "InsufficientDataActions": [{ 252 | "Fn::If": ["SNSTopicIsEmpty", { 253 | "Ref": "snsTopicSnapshotsAuroraTool" 254 | }, { 255 | "Ref": "SNSTopic" 256 | }] 257 | }], 258 | "Dimensions": [{ 259 | "Name": "StateMachineArn", 260 | "Value": { 261 | "Ref": "statemachineDeleteOldSnapshotsAurora" 262 | } 263 | }] 264 | } 265 | }, 266 | "iamroleSnapshotsAurora": { 267 | "Type": "AWS::IAM::Role", 268 | "Properties": { 269 | "AssumeRolePolicyDocument": { 270 | "Version": "2012-10-17", 271 | "Statement": [{ 272 | "Effect": "Allow", 273 | "Principal": { 274 | "Service": "lambda.amazonaws.com" 275 | }, 276 | "Action": "sts:AssumeRole" 277 | }] 278 | }, 279 | "ManagedPolicyArns": ["arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"], 280 | "Policies": [{ 281 | "PolicyName": "inline_policy_snapshots_aurora_rds", 282 | "PolicyDocument": { 283 | "Version": "2012-10-17", 284 | "Statement": [{ 285 | "Effect": "Allow", 286 | "Action": [ 287 | "rds:DescribeDBClusters", 288 | "rds:DescribeDBClusterSnapshots" 289 | ], 290 | "Resource": "*" 291 | }, 292 | { 293 | "Effect": "Allow", 294 | "Action": [ 295 | "rds:CreateDBClusterSnapshot", 296 | "rds:DeleteDBClusterSnapshot", 297 | "rds:ModifyDBClusterSnapshotAttribute", 298 | "rds:DescribeDBClusterSnapshotAttributes", 299 | "rds:ListTagsForResource", 300 | "rds:AddTagsToResource" 301 | ], 302 | "Resource": [{ 303 | "Fn::Join": [":", ["arn:aws:rds", { 304 | "Fn::If": ["RegionOverridden", { 305 | "Ref": "SourceRegionOverride" 306 | }, { 307 | "Ref": "AWS::Region" 308 | }] 309 | }, "*:cluster:*"]] 310 | }, 311 | { 312 | "Fn::Join": [":", ["arn:aws:rds", { 313 | "Fn::If": ["RegionOverridden", { 314 | "Ref": "SourceRegionOverride" 315 | }, { 316 | "Ref": "AWS::Region" 317 | }] 318 | }, "*:cluster-snapshot:*"]] 319 | } 320 | ] 321 | } 322 | ] 323 | } 324 | 325 | }] 326 | } 327 | }, 328 | "lambdaTakeSnapshotsAurora": { 329 | "Type": "AWS::Lambda::Function", 330 | "Properties": { 331 | "Code": { 332 | "S3Bucket": { 333 | "Ref": "CodeBucket" 334 | }, 335 | "S3Key": "take_snapshots_aurora.zip" 336 | }, 337 | "Description": "This functions triggers snapshots creation for Aurora clusters. It checks for existing snapshots following the pattern and interval specified in the environment variables with the following format: -YYYY-MM-DD-HH-MM", 338 | "MemorySize": 512, 339 | "Environment": { 340 | "Variables": { 341 | "INTERVAL": { 342 | "Ref": "BackupInterval" 343 | }, 344 | "PATTERN": { 345 | "Ref": "ClusterNamePattern" 346 | }, 347 | "LOG_LEVEL": { 348 | "Ref": "LogLevel" 349 | }, 350 | "REGION_OVERRIDE": { 351 | "Ref": "SourceRegionOverride" 352 | }, 353 | "SNAPSHOT_NAME_PREFIX": { 354 | "Ref": "SnapshotNamePrefix" 355 | } 356 | } 357 | }, 358 | "Role": { 359 | "Fn::GetAtt": [ 360 | "iamroleSnapshotsAurora", 361 | "Arn" 362 | ] 363 | }, 364 | "Runtime": "python3.7", 365 | "Handler": "lambda_function.lambda_handler", 366 | "Timeout": 300 367 | } 368 | }, 369 | "lambdaShareSnapshotsAurora": { 370 | "Type": "AWS::Lambda::Function", 371 | "Condition": "Share", 372 | "Properties": { 373 | "Code": { 374 | "S3Bucket": { 375 | "Ref": "CodeBucket" 376 | }, 377 | "S3Key": "share_snapshots_aurora.zip" 378 | }, 379 | "Description": "This function shares snapshots created by the aurora_take_snapshots function with DEST_ACCOUNT specified in the environment variables. ", 380 | "MemorySize": 512, 381 | "Environment": { 382 | "Variables": { 383 | "DEST_ACCOUNT": { 384 | "Ref": "DestinationAccount" 385 | }, 386 | "LOG_LEVEL": { 387 | "Ref": "LogLevel" 388 | }, 389 | "PATTERN": { 390 | "Ref": "ClusterNamePattern" 391 | }, 392 | "REGION_OVERRIDE": { 393 | "Ref": "SourceRegionOverride" 394 | } 395 | } 396 | }, 397 | "Role": { 398 | "Fn::GetAtt": ["iamroleSnapshotsAurora", "Arn"] 399 | }, 400 | "Runtime": "python3.7", 401 | "Handler": "lambda_function.lambda_handler", 402 | "Timeout": 300 403 | } 404 | }, 405 | "lambdaDeleteOldSnapshotsAurora": { 406 | "Type": "AWS::Lambda::Function", 407 | "Condition": "DeleteOld", 408 | "Properties": { 409 | "Code": { 410 | "S3Bucket": { 411 | "Ref": "CodeBucket" 412 | }, 413 | "S3Key": "delete_old_snapshots_aurora.zip" 414 | }, 415 | "Description": "This function shares snapshots created by the aurora_take_snapshots function with DEST_ACCOUNT specified in the environment variables. ", 416 | "MemorySize": 512, 417 | "Environment": { 418 | "Variables": { 419 | "RETENTION_DAYS": { 420 | "Ref": "RetentionDays" 421 | }, 422 | "PATTERN": { 423 | "Ref": "ClusterNamePattern" 424 | }, 425 | "LOG_LEVEL": { 426 | "Ref": "LogLevel" 427 | }, 428 | "REGION_OVERRIDE": { 429 | "Ref": "SourceRegionOverride" 430 | } 431 | } 432 | }, 433 | "Role": { 434 | "Fn::GetAtt": ["iamroleSnapshotsAurora", "Arn"] 435 | }, 436 | "Runtime": "python3.7", 437 | "Handler": "lambda_function.lambda_handler", 438 | "Timeout": 300 439 | } 440 | }, 441 | "iamroleStateExecution": { 442 | "Type": "AWS::IAM::Role", 443 | "Properties": { 444 | "AssumeRolePolicyDocument": { 445 | "Version": "2012-10-17", 446 | "Statement": [{ 447 | "Effect": "Allow", 448 | "Principal": { 449 | "Service": { 450 | "Fn::Join": ["", ["states.", { 451 | "Ref": "AWS::Region" 452 | }, ".amazonaws.com"]] 453 | } 454 | }, 455 | "Action": "sts:AssumeRole" 456 | }] 457 | }, 458 | "Policies": [{ 459 | "PolicyName": "inline_policy_snapshots_aurora", 460 | "PolicyDocument": { 461 | "Version": "2012-10-17", 462 | "Statement": [{ 463 | "Effect": "Allow", 464 | "Action": [ 465 | "lambda:InvokeFunction" 466 | ], 467 | "Resource": { 468 | "Fn::If": ["Share", [{ 469 | "Fn::GetAtt": ["lambdaTakeSnapshotsAurora", "Arn"] 470 | }, { 471 | "Fn::GetAtt": ["lambdaShareSnapshotsAurora", "Arn"] 472 | }, { 473 | "Fn::GetAtt": ["lambdaDeleteOldSnapshotsAurora", "Arn"] 474 | }], 475 | [{ 476 | "Fn::GetAtt": ["lambdaTakeSnapshotsAurora", "Arn"] 477 | }, { 478 | "Fn::GetAtt": ["lambdaDeleteOldSnapshotsAurora", "Arn"] 479 | }] 480 | ] 481 | } 482 | }] 483 | } 484 | }] 485 | } 486 | }, 487 | "stateMachineTakeSnapshotsAurora": { 488 | "Type": "AWS::StepFunctions::StateMachine", 489 | "Properties": { 490 | "DefinitionString": { 491 | "Fn::Join": ["", [{ 492 | "Fn::Join": ["\n", [ 493 | " {\"Comment\":\"Triggers snapshot backup for Aurora clusters\",", 494 | " \"StartAt\":\"TakeSnapshots\",", 495 | " \"States\":{", 496 | " \"TakeSnapshots\":{", 497 | " \"Type\":\"Task\",", 498 | " \"Resource\": " 499 | ]] 500 | }, 501 | "\"", 502 | { 503 | "Fn::GetAtt": ["lambdaTakeSnapshotsAurora", "Arn"] 504 | }, "\"\n,", 505 | { 506 | "Fn::Join": ["\n", [ 507 | " \"Retry\":[", 508 | " {", 509 | " \"ErrorEquals\":[ ", 510 | " \"SnapshotToolException\"", 511 | " ],", 512 | " \"IntervalSeconds\":300,", 513 | " \"MaxAttempts\":20,", 514 | " \"BackoffRate\":1", 515 | " },", 516 | " {", 517 | " \"ErrorEquals\":[ ", 518 | " \"States.ALL\"], ", 519 | " \"IntervalSeconds\": 30,", 520 | " \"MaxAttempts\": 20,", 521 | " \"BackoffRate\": 1", 522 | " }", 523 | " ],", 524 | " \"End\": true ", 525 | " }", 526 | " }}" 527 | ]] 528 | } 529 | ]] 530 | }, 531 | "RoleArn": { 532 | "Fn::GetAtt": ["iamroleStateExecution", "Arn"] 533 | } 534 | } 535 | }, 536 | "statemachineShareSnapshotsAurora": { 537 | "Type": "AWS::StepFunctions::StateMachine", 538 | "Condition": "Share", 539 | "Properties": { 540 | "DefinitionString": { 541 | "Fn::Join": ["", [{ 542 | "Fn::Join": ["\n", [ 543 | " {\"Comment\":\"Shares snapshots with DEST_ACCOUNT\",", 544 | " \"StartAt\":\"ShareSnapshots\",", 545 | " \"States\":{", 546 | " \"ShareSnapshots\":{", 547 | " \"Type\":\"Task\",", 548 | " \"Resource\": " 549 | ]] 550 | }, 551 | "\"", 552 | { 553 | "Fn::GetAtt": ["lambdaShareSnapshotsAurora", "Arn"] 554 | }, "\"\n,", 555 | { 556 | "Fn::Join": ["\n", [ 557 | " \"Retry\":[", 558 | " {", 559 | " \"ErrorEquals\":[ ", 560 | " \"SnapshotToolException\"", 561 | " ],", 562 | " \"IntervalSeconds\":300,", 563 | " \"MaxAttempts\":3,", 564 | " \"BackoffRate\":1", 565 | " },", 566 | " {", 567 | " \"ErrorEquals\":[ ", 568 | " \"States.ALL\"], ", 569 | " \"IntervalSeconds\": 30,", 570 | " \"MaxAttempts\": 10,", 571 | " \"BackoffRate\": 1", 572 | " }", 573 | " ],", 574 | " \"End\": true ", 575 | " }", 576 | " }}" 577 | ]] 578 | } 579 | ]] 580 | }, 581 | "RoleArn": { 582 | "Fn::GetAtt": ["iamroleStateExecution", "Arn"] 583 | } 584 | } 585 | }, 586 | "statemachineDeleteOldSnapshotsAurora": { 587 | "Type": "AWS::StepFunctions::StateMachine", 588 | "Condition": "DeleteOld", 589 | "Properties": { 590 | "DefinitionString": { 591 | "Fn::Join": ["", [{ 592 | "Fn::Join": ["\n", [ 593 | " {\"Comment\":\"DeleteOld management for Aurora snapshots\",", 594 | " \"StartAt\":\"DeleteOld\",", 595 | " \"States\":{", 596 | " \"DeleteOld\":{", 597 | " \"Type\":\"Task\",", 598 | " \"Resource\": " 599 | ]] 600 | }, 601 | "\"", 602 | { 603 | "Fn::GetAtt": ["lambdaDeleteOldSnapshotsAurora", "Arn"] 604 | }, "\"\n,", 605 | { 606 | "Fn::Join": ["\n", [ 607 | " \"Retry\":[", 608 | " {", 609 | " \"ErrorEquals\":[ ", 610 | " \"SnapshotToolException\"", 611 | " ],", 612 | " \"IntervalSeconds\":300,", 613 | " \"MaxAttempts\":7,", 614 | " \"BackoffRate\":1", 615 | " },", 616 | " {", 617 | " \"ErrorEquals\":[ ", 618 | " \"States.ALL\"], ", 619 | " \"IntervalSeconds\": 30,", 620 | " \"MaxAttempts\": 10,", 621 | " \"BackoffRate\": 1", 622 | " }", 623 | " ],", 624 | " \"End\": true ", 625 | " }", 626 | " }}" 627 | ]] 628 | } 629 | ]] 630 | }, 631 | "RoleArn": { 632 | "Fn::GetAtt": ["iamroleStateExecution", "Arn"] 633 | } 634 | } 635 | }, 636 | "iamroleStepInvocation": { 637 | "Type": "AWS::IAM::Role", 638 | "Properties": { 639 | "AssumeRolePolicyDocument": { 640 | "Version": "2012-10-17", 641 | "Statement": [{ 642 | "Effect": "Allow", 643 | "Principal": { 644 | "Service": "events.amazonaws.com" 645 | }, 646 | "Action": "sts:AssumeRole" 647 | }] 648 | }, 649 | "Policies": [{ 650 | "PolicyName": "inline_policy_state_invocation", 651 | "PolicyDocument": { 652 | "Version": "2012-10-17", 653 | "Statement": [{ 654 | "Effect": "Allow", 655 | "Action": [ 656 | "states:StartExecution" 657 | ], 658 | "Resource": { 659 | "Fn::If": ["Share", [{ 660 | "Ref": "stateMachineTakeSnapshotsAurora" 661 | }, { 662 | "Ref": "statemachineShareSnapshotsAurora" 663 | }, { 664 | "Ref": "statemachineDeleteOldSnapshotsAurora" 665 | }], 666 | [{ 667 | "Ref": "stateMachineTakeSnapshotsAurora" 668 | }, { 669 | "Ref": "statemachineDeleteOldSnapshotsAurora" 670 | }] 671 | ] 672 | } 673 | }] 674 | } 675 | }] 676 | } 677 | }, 678 | "cwEventBackupAurora": { 679 | "Type": "AWS::Events::Rule", 680 | "Properties": { 681 | "Description": "Triggers the BackupAurora state machine", 682 | "ScheduleExpression": { 683 | "Fn::Join": ["", ["cron(", { 684 | "Ref": "BackupSchedule" 685 | }, ")"]] 686 | }, 687 | "State": "ENABLED", 688 | "Targets": [{ 689 | "Arn": { 690 | "Ref": "stateMachineTakeSnapshotsAurora" 691 | }, 692 | "Id": "Target1", 693 | "RoleArn": { 694 | "Fn::GetAtt": ["iamroleStepInvocation", "Arn"] 695 | } 696 | }] 697 | } 698 | }, 699 | "cwEventShareSnapshotsAurora": { 700 | "Type": "AWS::Events::Rule", 701 | "Condition": "Share", 702 | "Properties": { 703 | "Description": "Triggers the ShareSnapshotsAurora state machine", 704 | "ScheduleExpression": { 705 | "Fn::Join": ["", ["cron(", "/10 * * * ? *", ")"]] 706 | }, 707 | "State": "ENABLED", 708 | "Targets": [{ 709 | "Arn": { 710 | "Ref": "statemachineShareSnapshotsAurora" 711 | }, 712 | "Id": "Target1", 713 | "RoleArn": { 714 | "Fn::GetAtt": ["iamroleStepInvocation", "Arn"] 715 | } 716 | }] 717 | } 718 | }, 719 | "cwEventAuroraDeleteOldSnapshotsAurora": { 720 | "Type": "AWS::Events::Rule", 721 | "Condition": "DeleteOld", 722 | "Properties": { 723 | "Description": "Triggers the DeleteOldSnapshotsAurora state machine", 724 | "ScheduleExpression": { 725 | "Fn::Join": ["", ["cron(", "0 /1 * * ? *", ")"]] 726 | }, 727 | "State": "ENABLED", 728 | "Targets": [{ 729 | "Arn": { 730 | "Ref": "statemachineDeleteOldSnapshotsAurora" 731 | }, 732 | "Id": "Target1", 733 | "RoleArn": { 734 | "Fn::GetAtt": ["iamroleStepInvocation", "Arn"] 735 | } 736 | }] 737 | } 738 | } 739 | }, 740 | "Outputs": { 741 | "SNSTopic": { 742 | "Description": "Subscribe to this topic to receive alerts of failures with the snapshots tool", 743 | "Value": { 744 | "Fn::If": ["SNSTopicIsEmpty", { 745 | "Ref": "snsTopicSnapshotsAuroraTool" 746 | }, { 747 | "Ref": "SNSTopic" 748 | }] 749 | } 750 | } 751 | }, 752 | "Description": "Snapshots Tool for Aurora cross-region and cross-account (source account stack)" 753 | } -------------------------------------------------------------------------------- /lambda/.gitignore: -------------------------------------------------------------------------------- 1 | ._* 2 | *.zip 3 | -------------------------------------------------------------------------------- /lambda/Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | # Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 4 | # http://aws.amazon.com/apache2.0/ 5 | # or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 6 | 7 | # Makefile for generating zip files for lambda functions and then copying them 8 | # to S3 for deployment. This Makefile will NOT WORK unless you fill in the S3DEST 9 | # and AWSARGS variables below. Once those parameters are established, simply type 10 | # 'make' or 'gmake' (depending on your UNIX-like OS) and it will build. 11 | # 12 | # Behaviour: 13 | # Creates a file named ._foo.whatever based on foo.whatever.Uploads foo.whatever to 14 | # the S3 bucket. The ._ file is a hack to figure out whether the file has 15 | # been modified since the last time we uploaded to s3. 16 | 17 | # Override S3 destination by changing this variable or setting it in the 18 | # environment 19 | S3DEST?=[YOUR BUCKET HERE] 20 | 21 | # Set these if, for example, you use profiles on the AWS command line 22 | # or if your 'aws' executable is in a weird place. 23 | AWSARGS=--region [YOUR REGION] --profile [YOUR PROFILE, or 'default', or remove this] 24 | AWSCMD=aws 25 | ZIPCMD=zip 26 | 27 | # disable all implicit make rules 28 | .SUFFIXES: 29 | 30 | # if you define "._foo" as a file on this line, then it will zip up a 31 | # folder called foo, adding a standard file into it to make foo.zip. 32 | all: ._copy_snapshots_dest_aurora \ 33 | ._copy_snapshots_no_x_account_aurora \ 34 | ._delete_old_snapshots_dest_aurora \ 35 | ._delete_old_snapshots_no_x_account_aurora \ 36 | ._delete_old_snapshots_aurora \ 37 | ._share_snapshots_aurora \ 38 | ._take_snapshots_aurora 39 | 40 | clean: 41 | rm -f ._* 42 | 43 | ._%: %.zip 44 | "$(AWSCMD)" $(AWSARGS) s3 cp "$<" "s3://$(S3DEST)" \ 45 | --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers 46 | cp "$<" "$@" 47 | 48 | # This rule is a BSD make style rule that says "to make foo.zip, call 49 | # 'zip -jqr foo snapshot_tool_utils.py'" 50 | %.zip: % 51 | $(ZIPCMD) -jqr "$@" "$<" snapshots_tool_utils.py -------------------------------------------------------------------------------- /lambda/copy_snapshots_dest_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | # copy_snapshots_dest_aurora 12 | # This lambda function will copy shared Aurora snapshots that match the regex specified in the environment variable PATTERN, into the account where it runs. If the snapshot is shared and exists in the local region, it will copy it to the region specified in the environment variable DEST_REGION. If it finds that the snapshots are shared, exist in the local and destination regions, it will delete them from the local region. Copying snapshots cross-account and cross-region need to be separate operations. This function will need to run as many times necessary for the workflow to complete. 13 | # Set PATTERN to a regex that matches your Aurora cluster identifiers (by default: -cluster) 14 | # Set DEST_REGION to the destination AWS region 15 | import boto3 16 | from datetime import datetime 17 | import time 18 | import os 19 | import logging 20 | import re 21 | from snapshots_tool_utils import * 22 | 23 | # Initialize everything 24 | LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 25 | PATTERN = os.getenv('SNAPSHOT_PATTERN', 'ALL_SNAPSHOTS') 26 | DESTINATION_REGION = os.getenv('DEST_REGION').strip() 27 | KMS_KEY_DEST_REGION = os.getenv('KMS_KEY_DEST_REGION', 'None').strip() 28 | KMS_KEY_SOURCE_REGION = os.getenv('KMS_KEY_SOURCE_REGION', 'None').strip() 29 | RETENTION_DAYS = int(os.getenv('RETENTION_DAYS')) 30 | TIMESTAMP_FORMAT = '%Y-%m-%d-%H-%M' 31 | 32 | if os.getenv('REGION_OVERRIDE', 'NO') != 'NO': 33 | REGION = os.getenv('REGION_OVERRIDE').strip() 34 | else: 35 | REGION = os.getenv('AWS_DEFAULT_REGION') 36 | 37 | 38 | logger = logging.getLogger() 39 | logger.setLevel(LOGLEVEL.upper()) 40 | 41 | 42 | 43 | def lambda_handler(event, context): 44 | # Describe all snapshots 45 | pending_copies = 0 46 | client = boto3.client('rds', region_name=REGION) 47 | response = paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots', IncludeShared=True) 48 | 49 | shared_snapshots = get_shared_snapshots(PATTERN, response) 50 | own_snapshots = get_own_snapshots_dest(PATTERN, response) 51 | 52 | # Get list of snapshots in DEST_REGION 53 | client_dest = boto3.client('rds', region_name=DESTINATION_REGION) 54 | response_dest = paginate_api_call(client_dest, 'describe_db_cluster_snapshots', 'DBClusterSnapshots') 55 | own_dest_snapshots = get_own_snapshots_dest(PATTERN, response_dest) 56 | 57 | for shared_identifier, shared_attributes in shared_snapshots.items(): 58 | 59 | if shared_identifier not in own_snapshots.keys() and shared_identifier not in own_dest_snapshots.keys(): 60 | # Check date 61 | creation_date = get_timestamp(shared_identifier, shared_snapshots) 62 | if creation_date: 63 | time_difference = datetime.now() - creation_date 64 | days_difference = time_difference.total_seconds() / 3600 / 24 65 | 66 | # Only copy if it's newer than RETENTION_DAYS 67 | if days_difference < RETENTION_DAYS: 68 | 69 | # Copy to own account 70 | try: 71 | copy_local(shared_identifier, shared_attributes) 72 | 73 | except Exception as e: 74 | pending_copies += 1 75 | logger.error(e) 76 | logger.error('Local copy pending: %s' % shared_identifier) 77 | 78 | else: 79 | if REGION != DESTINATION_REGION: 80 | pending_copies += 1 81 | logger.error('Remote copy pending: %s' % shared_identifier) 82 | 83 | else: 84 | logger.info('Not copying %s locally. Older than %s days' % (shared_identifier, RETENTION_DAYS)) 85 | 86 | else: 87 | logger.info('Not copying %s locally. No valid timestamp' % shared_identifier) 88 | 89 | 90 | # Copy to DESTINATION_REGION 91 | elif shared_identifier not in own_dest_snapshots.keys() and shared_identifier in own_snapshots.keys() and REGION != DESTINATION_REGION: 92 | if own_snapshots[shared_identifier]['Status'] == 'available': 93 | try: 94 | copy_remote(shared_identifier, own_snapshots[shared_identifier]) 95 | 96 | except Exception as e: 97 | pending_copies += 1 98 | logger.error(e) 99 | logger.error('Remote copy pending: %s: %s' % ( 100 | shared_identifier, own_snapshots[shared_identifier]['Arn'])) 101 | else: 102 | pending_copies += 1 103 | logger.error('Remote copy pending: %s: %s' % ( 104 | shared_identifier, own_snapshots[shared_identifier]['Arn'])) 105 | 106 | # Delete local snapshots 107 | elif shared_identifier in own_dest_snapshots.keys() and shared_identifier in own_snapshots.keys() and own_dest_snapshots[shared_identifier]['Status'] == 'available' and REGION != DESTINATION_REGION: 108 | 109 | response = client.delete_db_cluster_snapshot( 110 | DBClusterSnapshotIdentifier=shared_identifier 111 | ) 112 | 113 | logger.info('Deleting local snapshot: %s' % shared_identifier) 114 | 115 | if pending_copies > 0: 116 | log_message = 'Copies pending: %s. Needs retrying' % pending_copies 117 | logger.error(log_message) 118 | raise SnapshotToolException(log_message) 119 | 120 | 121 | if __name__ == '__main__': 122 | lambda_handler(None, None) 123 | -------------------------------------------------------------------------------- /lambda/copy_snapshots_no_x_account_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | # copy_snapshots_no_x_account_aurora 12 | # This lambda function will copy source Aurora snapshots that match the regex specified in the environment variable PATTERN into DEST_REGION. This function will need to run as many times necessary for the workflow to complete. 13 | # Set PATTERN to a regex that matches your Aurora cluster identifiers (by default: -cluster) 14 | # Set DEST_REGION to the destination AWS region 15 | import boto3 16 | from datetime import datetime 17 | import time 18 | import os 19 | import logging 20 | import re 21 | from snapshots_tool_utils import * 22 | 23 | # Initialize everything 24 | LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 25 | PATTERN = os.getenv('SNAPSHOT_PATTERN', 'ALL_SNAPSHOTS') 26 | DESTINATION_REGION = os.getenv('DEST_REGION').strip() 27 | KMS_KEY_DEST_REGION = os.getenv('KMS_KEY_DEST_REGION', 'None').strip() 28 | KMS_KEY_SOURCE_REGION = os.getenv('KMS_KEY_SOURCE_REGION', 'None').strip() 29 | RETENTION_DAYS = int(os.getenv('RETENTION_DAYS')) 30 | TIMESTAMP_FORMAT = '%Y-%m-%d-%H-%M' 31 | 32 | if os.getenv('REGION_OVERRIDE', 'NO') != 'NO': 33 | REGION = os.getenv('REGION_OVERRIDE').strip() 34 | else: 35 | REGION = os.getenv('AWS_DEFAULT_REGION') 36 | 37 | 38 | logger = logging.getLogger() 39 | logger.setLevel(LOGLEVEL.upper()) 40 | 41 | 42 | 43 | def lambda_handler(event, context): 44 | # Describe all snapshots 45 | pending_copies = 0 46 | client = boto3.client('rds', region_name=REGION) 47 | response = paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots') 48 | 49 | source_snapshots = get_own_snapshots_source(PATTERN, response) 50 | own_snapshots_encryption = get_own_snapshots_dest(PATTERN, response) 51 | 52 | # Get list of snapshots in DEST_REGION 53 | client_dest = boto3.client('rds', region_name=DESTINATION_REGION) 54 | response_dest = paginate_api_call(client_dest, 'describe_db_cluster_snapshots', 'DBClusterSnapshots') 55 | dest_snapshots = get_own_snapshots_dest(PATTERN, response_dest) 56 | 57 | 58 | for source_identifier, source_attributes in source_snapshots.items(): 59 | creation_date = get_timestamp(source_identifier, source_snapshots) 60 | if creation_date: 61 | time_difference = datetime.now() - creation_date 62 | days_difference = time_difference.total_seconds() / 3600 / 24 63 | 64 | # Only copy if it's newer than RETENTION_DAYS 65 | if days_difference < RETENTION_DAYS: 66 | # Copy to DESTINATION_REGION 67 | if source_identifier not in dest_snapshots.keys() and REGION != DESTINATION_REGION: 68 | if source_snapshots[source_identifier]['Status'] == 'available': 69 | try: 70 | copy_remote(source_identifier, own_snapshots_encryption[source_identifier]) 71 | 72 | except Exception as e: 73 | pending_copies += 1 74 | logger.error(e) 75 | logger.error('Remote copy pending: %s: %s' % ( 76 | source_identifier, source_snapshots[source_identifier]['Arn'])) 77 | else: 78 | pending_copies += 1 79 | logger.error('Remote copy pending: %s: %s' % ( 80 | source_identifier, source_snapshots[source_identifier]['Arn'])) 81 | else: 82 | logger.info('Not copying %s locally. Older than %s days' % (source_identifier, RETENTION_DAYS)) 83 | 84 | else: 85 | logger.info('Not copying %s locally. No valid timestamp' % source_identifier) 86 | 87 | if pending_copies > 0: 88 | log_message = 'Copies pending: %s. Needs retrying' % pending_copies 89 | logger.error(log_message) 90 | raise SnapshotToolException(log_message) 91 | 92 | 93 | if __name__ == '__main__': 94 | lambda_handler(None, None) 95 | -------------------------------------------------------------------------------- /lambda/delete_old_snapshots_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | # delete_old_snapshots_aurora 12 | # This Lambda function will delete snapshots that have expired and match the regex set in the PATTERN environment variable. It will also look for a matching timestamp in the following format: YYYY-MM-DD-HH-mm 13 | # Set PATTERN to a regex that matches your Aurora cluster identifiers (by default: -cluster) 14 | import boto3 15 | from datetime import datetime 16 | import time 17 | import os 18 | import logging 19 | import re 20 | from snapshots_tool_utils import * 21 | 22 | LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 23 | PATTERN = os.getenv('PATTERN', 'ALL_CLUSTERS') 24 | RETENTION_DAYS = int(os.getenv('RETENTION_DAYS', '7')) 25 | TIMESTAMP_FORMAT = '%Y-%m-%d-%H-%M' 26 | 27 | if os.getenv('REGION_OVERRIDE', 'NO') != 'NO': 28 | REGION = os.getenv('REGION_OVERRIDE').strip() 29 | else: 30 | REGION = os.getenv('AWS_DEFAULT_REGION') 31 | 32 | 33 | logger = logging.getLogger() 34 | logger.setLevel(LOGLEVEL.upper()) 35 | 36 | 37 | 38 | 39 | def lambda_handler(event, context): 40 | pending_delete = 0 41 | client = boto3.client('rds', region_name=REGION) 42 | response = paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots') 43 | 44 | filtered_list = get_own_snapshots_source(PATTERN, response) 45 | 46 | for snapshot in filtered_list.keys(): 47 | 48 | creation_date = get_timestamp(snapshot, filtered_list) 49 | 50 | if creation_date: 51 | 52 | difference = datetime.now() - creation_date 53 | 54 | days_difference = difference.total_seconds() / 3600 / 24 55 | 56 | logger.debug('%s created %s days ago' % 57 | (snapshot, days_difference)) 58 | 59 | # if we are past RETENTION_DAYS 60 | if days_difference > RETENTION_DAYS: 61 | 62 | # delete it 63 | logger.info('Deleting %s' % snapshot) 64 | 65 | try: 66 | client.delete_db_cluster_snapshot( 67 | DBClusterSnapshotIdentifier=snapshot) 68 | 69 | except Exception as e: 70 | pending_delete += 1 71 | logger.info(e) 72 | logger.info('Could not delete %s ' % snapshot) 73 | 74 | else: 75 | # Not older than RETENTION_DAYS 76 | logger.debug('%s created less than %s days. Not deleting' % (snapshot, RETENTION_DAYS)) 77 | 78 | else: 79 | # Did not have a timestamp 80 | logger.debug('Not deleting %s. Could not find a timestamp in the name' % snapshot) 81 | 82 | 83 | if pending_delete > 0: 84 | message = 'Snapshots pending delete: %s' % pending_delete 85 | logger.error(message) 86 | raise SnapshotToolException(message) 87 | 88 | 89 | if __name__ == '__main__': 90 | lambda_handler(None, None) 91 | -------------------------------------------------------------------------------- /lambda/delete_old_snapshots_dest_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | # delete_old_snapshots_dest_aurora 12 | # This lambda function will delete manual snapshots that have expired in the region specified in the environment variable DEST_REGION, and according to the environment variables PATTERN and RETENTION_DAYS. 13 | # Set PATTERN to a regex that matches your Aurora cluster identifiers (by default: -cluster) 14 | # Set DEST_REGION to the destination AWS region 15 | # Set RETENTION_DAYS to the amount of days snapshots need to be kept before deleting 16 | import boto3 17 | import time 18 | import os 19 | import logging 20 | from datetime import datetime 21 | import re 22 | from snapshots_tool_utils import * 23 | 24 | # Initialize everything 25 | DEST_REGION = os.getenv('DEST_REGION', os.getenv('AWS_DEFAULT_REGION')).strip() 26 | LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 27 | PATTERN = os.getenv('PATTERN', 'ALL_SNAPSHOTS') 28 | RETENTION_DAYS = int(os.getenv('RETENTION_DAYS')) 29 | TIMESTAMP_FORMAT = '%Y-%m-%d-%H-%M' 30 | 31 | 32 | logger = logging.getLogger() 33 | logger.setLevel(LOGLEVEL.upper()) 34 | 35 | 36 | 37 | def lambda_handler(event, context): 38 | delete_pending = 0 39 | 40 | # Search for all snapshots 41 | client = boto3.client('rds', region_name=DEST_REGION) 42 | response = paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots') 43 | 44 | # Filter out the ones not created automatically or with other methods 45 | filtered_list = get_own_snapshots_dest(PATTERN, response) 46 | 47 | 48 | for snapshot in filtered_list.keys(): 49 | creation_date = get_timestamp(snapshot, filtered_list) 50 | 51 | if creation_date: 52 | 53 | snapshot_arn = filtered_list[snapshot]['Arn'] 54 | response_tags = client.list_tags_for_resource( 55 | ResourceName=snapshot_arn) 56 | 57 | if search_tag_copied(response_tags): 58 | 59 | difference = datetime.now() - creation_date 60 | days_difference = difference.total_seconds() / 3600 / 24 61 | # if we are past RETENTION_DAYS 62 | 63 | if days_difference > RETENTION_DAYS: 64 | 65 | # delete it 66 | logger.info('Deleting %s. %s days old' % 67 | (snapshot, days_difference)) 68 | 69 | try: 70 | client.delete_db_cluster_snapshot( 71 | DBClusterSnapshotIdentifier=snapshot) 72 | 73 | except Exception as e: 74 | delete_pending += 1 75 | logger.error(e) 76 | logger.error('Could not delete %s' % snapshot) 77 | 78 | else: 79 | logger.info('Not deleting %s. Only %s days old' % 80 | (snapshot, days_difference)) 81 | 82 | else: 83 | logger.info( 84 | 'Not deleting %s. Did not find correct tag' % snapshot) 85 | 86 | else: 87 | logger.debug( 88 | 'Not deleting %s. Did not find a timestamp' % snapshot) 89 | 90 | 91 | if delete_pending > 0: 92 | 93 | log_message = 'Snapshots pending delete: %s' % delete_pending 94 | logger.error(log_message) 95 | raise SnapshotToolException(log_message) 96 | 97 | 98 | if __name__ == '__main__': 99 | lambda_handler(None, None) 100 | -------------------------------------------------------------------------------- /lambda/delete_old_snapshots_no_x_account_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | # delete_old_snapshots_dest_aurora 12 | # This lambda function will delete manual snapshots that have expired in the region specified in the environment variable DEST_REGION, and according to the environment variables PATTERN and RETENTION_DAYS. 13 | # Set PATTERN to a regex that matches your Aurora cluster identifiers (by default: -cluster) 14 | # Set DEST_REGION to the destination AWS region 15 | # Set RETENTION_DAYS to the amount of days snapshots need to be kept before deleting 16 | import boto3 17 | import time 18 | import os 19 | import logging 20 | from datetime import datetime 21 | import re 22 | from snapshots_tool_utils import * 23 | 24 | # Initialize everything 25 | DEST_REGION = os.getenv('DEST_REGION', os.getenv('AWS_DEFAULT_REGION')).strip() 26 | LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 27 | PATTERN = os.getenv('PATTERN', 'ALL_SNAPSHOTS') 28 | RETENTION_DAYS = int(os.getenv('RETENTION_DAYS')) 29 | TIMESTAMP_FORMAT = '%Y-%m-%d-%H-%M' 30 | 31 | 32 | logger = logging.getLogger() 33 | logger.setLevel(LOGLEVEL.upper()) 34 | 35 | 36 | 37 | def lambda_handler(event, context): 38 | delete_pending = 0 39 | 40 | # Search for all snapshots 41 | client = boto3.client('rds', region_name=DEST_REGION) 42 | response = paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots') 43 | 44 | # Filter out the ones not created automatically or with other methods 45 | filtered_list = get_own_snapshots_no_x_account(PATTERN, response, DEST_REGION) 46 | 47 | 48 | for snapshot in filtered_list.keys(): 49 | creation_date = get_timestamp(snapshot, filtered_list) 50 | 51 | if creation_date: 52 | 53 | snapshot_arn = filtered_list[snapshot]['Arn'] 54 | response_tags = client.list_tags_for_resource( 55 | ResourceName=snapshot_arn) 56 | 57 | if search_tag_created(response_tags): 58 | 59 | difference = datetime.now() - creation_date 60 | days_difference = difference.total_seconds() / 3600 / 24 61 | # if we are past RETENTION_DAYS 62 | 63 | if days_difference > RETENTION_DAYS: 64 | 65 | # delete it 66 | logger.info('Deleting %s. %s days old' % 67 | (snapshot, days_difference)) 68 | 69 | try: 70 | client.delete_db_cluster_snapshot( 71 | DBClusterSnapshotIdentifier=snapshot) 72 | 73 | except Exception: 74 | delete_pending += 1 75 | logger.info('Could not delete %s' % snapshot) 76 | 77 | else: 78 | logger.info('Not deleting %s. Only %s days old' % 79 | (snapshot, days_difference)) 80 | 81 | else: 82 | logger.info( 83 | 'Not deleting %s. Did not find correct tag' % snapshot) 84 | 85 | else: 86 | logger.debug( 87 | 'Not deleting %s. Did not find a timestamp' % snapshot) 88 | 89 | 90 | if delete_pending > 0: 91 | 92 | log_message = 'Snapshots pending delete: %s' % delete_pending 93 | logger.error(log_message) 94 | raise SnapshotToolException(log_message) 95 | 96 | 97 | if __name__ == '__main__': 98 | lambda_handler(None, None) 99 | -------------------------------------------------------------------------------- /lambda/share_snapshots_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | # share_snapshots_aurora 12 | # This Lambda function shares snapshots created by aurora_take_snapshot with the account set in the environment variable DEST_ACCOUNT 13 | # It will only share snapshots tagged with shareAndCopy and a value of YES 14 | import boto3 15 | from datetime import datetime 16 | import time 17 | import os 18 | import logging 19 | import re 20 | from snapshots_tool_utils import * 21 | 22 | 23 | # Initialize from environment variable 24 | LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 25 | DEST_ACCOUNTID = str(os.getenv('DEST_ACCOUNT', '000000000000')).strip() 26 | PATTERN = os.getenv('PATTERN', 'ALL_CLUSTERS') 27 | 28 | if os.getenv('REGION_OVERRIDE', 'NO') != 'NO': 29 | REGION = os.getenv('REGION_OVERRIDE').strip() 30 | else: 31 | REGION = os.getenv('AWS_DEFAULT_REGION') 32 | 33 | 34 | logger = logging.getLogger() 35 | logger.setLevel(LOGLEVEL.upper()) 36 | 37 | 38 | 39 | def lambda_handler(event, context): 40 | pending_snapshots = 0 41 | client = boto3.client('rds', region_name=REGION) 42 | response = paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots', SnapshotType='manual') 43 | filtered = get_own_snapshots_share(PATTERN, response) 44 | 45 | # Search all snapshots for the correct tag 46 | for snapshot_identifier,snapshot_object in filtered.items(): 47 | snapshot_arn = snapshot_object['Arn'] 48 | response_tags = client.list_tags_for_resource( 49 | ResourceName=snapshot_arn) 50 | 51 | if snapshot_object['Status'].lower() == 'available' and search_tag_share(response_tags): 52 | try: 53 | # Share snapshot with dest_account 54 | response_modify = client.modify_db_cluster_snapshot_attribute( 55 | DBClusterSnapshotIdentifier=snapshot_identifier, 56 | AttributeName='restore', 57 | ValuesToAdd=[ 58 | DEST_ACCOUNTID 59 | ] 60 | ) 61 | except Exception as e: 62 | logger.error('Exception sharing {}: {}'.format(snapshot_identifier, e)) 63 | pending_snapshots += 1 64 | 65 | if pending_snapshots > 0: 66 | log_message = 'Could not share all snapshots. Pending: %s' % pending_snapshots 67 | logger.error(log_message) 68 | raise SnapshotToolException(log_message) 69 | 70 | 71 | if __name__ == '__main__': 72 | lambda_handler(None, None) 73 | -------------------------------------------------------------------------------- /lambda/snapshots_tool_utils.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | 12 | # snapshots_tool_utils 13 | # Support module for the Snapshots Tool for Aurora 14 | 15 | import boto3 16 | from datetime import datetime 17 | import time 18 | import os 19 | import logging 20 | import re 21 | 22 | 23 | # Initialize everything 24 | _LOGLEVEL = os.getenv('LOG_LEVEL', 'ERROR').strip() 25 | 26 | _DEST_ACCOUNTID = str(os.getenv('DEST_ACCOUNT', '000000000000')).strip() 27 | 28 | _DESTINATION_REGION = os.getenv( 29 | 'DEST_REGION', os.getenv('AWS_DEFAULT_REGION')).strip() 30 | 31 | _KMS_KEY_DEST_REGION = os.getenv('KMS_KEY_DEST_REGION', 'None').strip() 32 | 33 | _KMS_KEY_SOURCE_REGION = os.getenv('KMS_KEY_SOURCE_REGION', 'None').strip() 34 | 35 | _TIMESTAMP_FORMAT = '%Y-%m-%d-%H-%M' 36 | 37 | if os.getenv('REGION_OVERRIDE', 'NO') != 'NO': 38 | _REGION = os.getenv('REGION_OVERRIDE').strip() 39 | else: 40 | _REGION = os.getenv('AWS_DEFAULT_REGION') 41 | 42 | _SUPPORTED_ENGINES = [ 'aurora', 'aurora-mysql', 'aurora-postgresql', 'neptune'] 43 | 44 | logger = logging.getLogger() 45 | logger.setLevel(_LOGLEVEL.upper()) 46 | 47 | 48 | class SnapshotToolException(Exception): 49 | pass 50 | 51 | 52 | def search_tag_created(response): 53 | # Takes a describe_db_cluster_snapshots response and searches for our shareAndCopy tag 54 | try: 55 | 56 | for tag in response['TagList']: 57 | if tag['Key'] == 'CreatedBy' and tag['Value'] == 'Snapshot Tool for Aurora': 58 | return True 59 | 60 | except Exception: 61 | return False 62 | 63 | else: 64 | return False 65 | 66 | 67 | def filter_clusters(pattern, cluster_list): 68 | # Takes the response from describe-db-clusters and filters according to pattern in DBClusterIdentifier 69 | filtered_list = [] 70 | 71 | for cluster in cluster_list['DBClusters']: 72 | 73 | if pattern == 'ALL_CLUSTERS' and cluster['Engine'] in _SUPPORTED_ENGINES: 74 | filtered_list.append(cluster) 75 | 76 | else: 77 | match = re.search(pattern, cluster['DBClusterIdentifier']) 78 | 79 | if match and cluster['Engine'] in _SUPPORTED_ENGINES: 80 | filtered_list.append(cluster) 81 | 82 | return filtered_list 83 | 84 | 85 | def get_snapshot_identifier(snapshot): 86 | # Function that will return the Snapshot identifier given an ARN 87 | match = re.match('arn:aws:rds:.*:.*:cluster-snapshot:(.+)', 88 | snapshot['DBClusterSnapshotArn']) 89 | return match.group(1) 90 | 91 | 92 | def get_own_snapshots_source(pattern, response): 93 | # Filters our own snapshots 94 | filtered = {} 95 | for snapshot in response['DBClusterSnapshots']: 96 | 97 | if snapshot['SnapshotType'] == 'manual' and re.search(pattern, snapshot['DBClusterIdentifier']) and snapshot['Engine'] in _SUPPORTED_ENGINES: 98 | client = boto3.client('rds', region_name=_REGION) 99 | response_tags = client.list_tags_for_resource( 100 | ResourceName=snapshot['DBClusterSnapshotArn']) 101 | 102 | if search_tag_created(response_tags): 103 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 104 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 105 | #Changed the next line to search for ALL_CLUSTERS or ALL_SNAPSHOTS so it will work with no-x-account 106 | elif snapshot['SnapshotType'] == 'manual' and (pattern == 'ALL_CLUSTERS' or pattern == 'ALL_SNAPSHOTS') and snapshot['Engine'] in _SUPPORTED_ENGINES: 107 | client = boto3.client('rds', region_name=_REGION) 108 | response_tags = client.list_tags_for_resource( 109 | ResourceName=snapshot['DBClusterSnapshotArn']) 110 | 111 | if search_tag_created(response_tags): 112 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 113 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 114 | 115 | return filtered 116 | 117 | def get_own_snapshots_no_x_account(pattern, response, REGION): 118 | # Filters our own snapshots 119 | filtered = {} 120 | for snapshot in response['DBClusterSnapshots']: 121 | 122 | if snapshot['SnapshotType'] == 'manual' and re.search(pattern, snapshot['DBClusterIdentifier']) and snapshot['Engine'] in _SUPPORTED_ENGINES: 123 | client = boto3.client('rds', region_name=REGION) 124 | response_tags = client.list_tags_for_resource( 125 | ResourceName=snapshot['DBClusterSnapshotArn']) 126 | 127 | if search_tag_created(response_tags): 128 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 129 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 130 | #Changed the next line to search for ALL_CLUSTERS or ALL_SNAPSHOTS so it will work with no-x-account 131 | elif snapshot['SnapshotType'] == 'manual' and pattern == 'ALL_SNAPSHOTS' and snapshot['Engine'] in _SUPPORTED_ENGINES: 132 | client = boto3.client('rds', region_name=REGION) 133 | response_tags = client.list_tags_for_resource( 134 | ResourceName=snapshot['DBClusterSnapshotArn']) 135 | 136 | if search_tag_created(response_tags): 137 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 138 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 139 | 140 | return filtered 141 | 142 | def get_own_snapshots_share(pattern, response): 143 | # Filter manual snapshots by pattern. Returns a dict of snapshots with DBClusterSnapshotIdentifier as key and Status, DBClusterIdentifier as attributes 144 | filtered = {} 145 | for snapshot in response['DBClusterSnapshots']: 146 | if snapshot['SnapshotType'] == 'manual' and re.search(pattern, snapshot['DBClusterIdentifier']) and snapshot['Engine'] in _SUPPORTED_ENGINES: 147 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 148 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 149 | elif snapshot['SnapshotType'] == 'manual' and pattern == 'ALL_CLUSTERS' and snapshot['Engine'] in _SUPPORTED_ENGINES: 150 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 151 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 152 | return filtered 153 | 154 | 155 | def get_shared_snapshots(pattern, response): 156 | # Returns a dict with only shared snapshots filtered by pattern, with DBSnapshotIdentifier as key and the response as attribute 157 | filtered = {} 158 | for snapshot in response['DBClusterSnapshots']: 159 | if snapshot['SnapshotType'] == 'shared' and re.search(pattern, snapshot['DBClusterIdentifier']) and snapshot['Engine'] in _SUPPORTED_ENGINES: 160 | filtered[get_snapshot_identifier(snapshot)] = { 161 | 'Arn': snapshot['DBClusterSnapshotIdentifier'], 'StorageEncrypted': snapshot['StorageEncrypted'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 162 | if snapshot['StorageEncrypted'] is True: 163 | filtered[get_snapshot_identifier( 164 | snapshot)]['KmsKeyId'] = snapshot['KmsKeyId'] 165 | 166 | elif snapshot['SnapshotType'] == 'shared' and pattern == 'ALL_SNAPSHOTS' and snapshot['Engine'] in _SUPPORTED_ENGINES: 167 | filtered[get_snapshot_identifier(snapshot)] = { 168 | 'Arn': snapshot['DBClusterSnapshotIdentifier'], 'StorageEncrypted': snapshot['StorageEncrypted'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 169 | if snapshot['StorageEncrypted'] is True: 170 | filtered[get_snapshot_identifier( 171 | snapshot)]['KmsKeyId'] = snapshot['KmsKeyId'] 172 | return filtered 173 | 174 | 175 | def get_own_snapshots_dest(pattern, response): 176 | # Returns a dict with local snapshots, filtered by pattern, with DBClusterSnapshotIdentifier as key and Arn, Status as attributes 177 | filtered = {} 178 | for snapshot in response['DBClusterSnapshots']: 179 | 180 | if snapshot['SnapshotType'] == 'manual' and re.search(pattern, snapshot['DBClusterIdentifier']) and snapshot['Engine'] in _SUPPORTED_ENGINES: 181 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 182 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'StorageEncrypted': snapshot['StorageEncrypted'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 183 | 184 | if snapshot['StorageEncrypted'] is True: 185 | filtered[snapshot['DBClusterSnapshotIdentifier'] 186 | ]['KmsKeyId'] = snapshot['KmsKeyId'] 187 | 188 | elif snapshot['SnapshotType'] == 'manual' and pattern == 'ALL_SNAPSHOTS' and snapshot['Engine'] in _SUPPORTED_ENGINES: 189 | filtered[snapshot['DBClusterSnapshotIdentifier']] = { 190 | 'Arn': snapshot['DBClusterSnapshotArn'], 'Status': snapshot['Status'], 'StorageEncrypted': snapshot['StorageEncrypted'], 'DBClusterIdentifier': snapshot['DBClusterIdentifier']} 191 | 192 | if snapshot['StorageEncrypted'] is True: 193 | filtered[snapshot['DBClusterSnapshotIdentifier'] 194 | ]['KmsKeyId'] = snapshot['KmsKeyId'] 195 | 196 | return filtered 197 | 198 | 199 | def copy_local(snapshot_identifier, snapshot_object): 200 | client = boto3.client('rds', region_name=_REGION) 201 | 202 | tags = [{ 203 | 'Key': 'CopiedBy', 204 | 'Value': 'Snapshot Tool for Aurora' 205 | }] 206 | 207 | if snapshot_object['StorageEncrypted']: 208 | logger.info('Copying encrypted snapshot %s locally' % 209 | snapshot_identifier) 210 | 211 | response = client.copy_db_cluster_snapshot( 212 | SourceDBClusterSnapshotIdentifier=snapshot_object['Arn'], 213 | TargetDBClusterSnapshotIdentifier=snapshot_identifier, 214 | KmsKeyId=_KMS_KEY_SOURCE_REGION, 215 | Tags=tags) 216 | 217 | else: 218 | logger.info('Copying snapshot %s locally' % snapshot_identifier) 219 | 220 | response = client.copy_db_cluster_snapshot( 221 | SourceDBClusterSnapshotIdentifier=snapshot_object['Arn'], 222 | TargetDBClusterSnapshotIdentifier=snapshot_identifier, 223 | Tags=tags) 224 | 225 | return response 226 | 227 | 228 | def copy_remote(snapshot_identifier, snapshot_object): 229 | client = boto3.client('rds', region_name=_DESTINATION_REGION) 230 | 231 | if snapshot_object['StorageEncrypted']: 232 | logger.info('Copying encrypted snapshot %s to remote region %s' % 233 | (snapshot_object['Arn'], _DESTINATION_REGION)) 234 | 235 | response = client.copy_db_cluster_snapshot( 236 | SourceDBClusterSnapshotIdentifier=snapshot_object['Arn'], 237 | TargetDBClusterSnapshotIdentifier=snapshot_identifier, 238 | KmsKeyId=_KMS_KEY_DEST_REGION, 239 | SourceRegion=_REGION, 240 | CopyTags=True) 241 | 242 | else: 243 | logger.info('Copying snapshot %s to remote region %s' % 244 | (snapshot_object['Arn'], _DESTINATION_REGION)) 245 | 246 | response = client.copy_db_cluster_snapshot( 247 | SourceDBClusterSnapshotIdentifier=snapshot_object['Arn'], 248 | TargetDBClusterSnapshotIdentifier=snapshot_identifier, 249 | SourceRegion=_REGION, 250 | CopyTags=True) 251 | 252 | return response 253 | 254 | 255 | def get_timestamp(snapshot_identifier, snapshot_list): 256 | 257 | # Searches for a timestamp on a snapshot name 258 | pattern = '%s-(.+)' % snapshot_list[snapshot_identifier]['DBClusterIdentifier'] 259 | 260 | date_time = re.search(pattern, snapshot_identifier) 261 | 262 | if date_time is not None: 263 | try: 264 | return datetime.strptime(date_time.group(1), _TIMESTAMP_FORMAT) 265 | 266 | except Exception: 267 | return None 268 | 269 | return None 270 | 271 | 272 | def get_timestamp_no_minute(snapshot_identifier, snapshot_list): 273 | 274 | # Get a timestamp from the name of a snapshot and strip out the minutes 275 | pattern = '%s-(.+)-\d{2}' % snapshot_list[snapshot_identifier]['DBClusterIdentifier'] 276 | 277 | date_time = re.search(pattern, snapshot_identifier) 278 | 279 | if date_time is not None: 280 | return datetime.strptime(date_time.group(1), '%Y-%m-%d-%H') 281 | 282 | 283 | def get_latest_snapshot_ts(cluster_identifier, filtered_snapshots): 284 | 285 | # Get latest snapshot for a specific DBClusterIdentifier 286 | timestamps = [] 287 | 288 | for snapshot, snapshot_object in filtered_snapshots.items(): 289 | 290 | if snapshot_object['DBClusterIdentifier'] == cluster_identifier: 291 | 292 | timestamp = get_timestamp_no_minute(snapshot, filtered_snapshots) 293 | 294 | if timestamp is not None: 295 | 296 | timestamps.append(timestamp) 297 | 298 | if len(timestamps) > 0: 299 | 300 | return max(timestamps) 301 | 302 | else: 303 | return None 304 | 305 | 306 | def requires_backup(backup_interval, cluster, filtered_snapshots): 307 | 308 | # Returns True if latest snapshot is older than INTERVAL 309 | latest = get_latest_snapshot_ts( 310 | cluster['DBClusterIdentifier'], filtered_snapshots) 311 | 312 | if latest is not None: 313 | 314 | backup_age = datetime.now() - latest 315 | 316 | if backup_age.total_seconds() >= (backup_interval * 60 * 60): 317 | return True 318 | 319 | else: 320 | return False 321 | 322 | elif latest is None: 323 | return True 324 | 325 | 326 | def paginate_api_call(client, api_call, objecttype, *args, **kwargs): 327 | #Takes an RDS boto client and paginates through api_call calls and returns a list of objects of objecttype 328 | response = {} 329 | response[objecttype] = [] 330 | 331 | # Create a paginator 332 | paginator = client.get_paginator(api_call) 333 | 334 | # Create a PageIterator from the Paginator 335 | page_iterator = paginator.paginate(**kwargs) 336 | for page in page_iterator: 337 | for item in page[objecttype]: 338 | response[objecttype].append(item) 339 | 340 | return response 341 | 342 | 343 | def search_tag_share(response): 344 | # Takes a describe_db_cluster_snapshots response and searches for our shareAndCopy tag 345 | try: 346 | 347 | for tag in response['TagList']: 348 | 349 | if tag['Key'] == 'shareAndCopy' and tag['Value'] == 'YES': 350 | 351 | for tag2 in response['TagList']: 352 | 353 | if tag2['Key'] == 'CreatedBy' and tag2['Value'] == 'Snapshot Tool for Aurora': 354 | 355 | return True 356 | 357 | except Exception: 358 | return False 359 | 360 | return False 361 | 362 | 363 | def search_tag_copied(response): 364 | try: 365 | 366 | for tag in response['TagList']: 367 | 368 | if tag['Key'] == 'CopiedBy' and tag['Value'] == 'Snapshot Tool for Aurora': 369 | return True 370 | 371 | except Exception: 372 | return False 373 | 374 | return False 375 | 376 | -------------------------------------------------------------------------------- /lambda/take_snapshots_aurora/lambda_function.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. 3 | 4 | Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at 5 | 6 | http://aws.amazon.com/apache2.0/ 7 | 8 | or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 9 | ''' 10 | 11 | 12 | # take_snapshots_aurora 13 | # This lambda function takes a snapshot of Aurora clusters according to the environment variable PATTERN and INTERVAL 14 | # Set PATTERN to a regex that matches your Aurora cluster identifiers (by default: -cluster) 15 | # Set INTERVAL to the amount of hours between backups. This function will list available manual snapshots and only trigger a new one if the latest is older than INTERVAL hours 16 | import boto3 17 | from datetime import datetime 18 | import os 19 | import logging 20 | from snapshots_tool_utils import * 21 | 22 | # Initialize everything 23 | LOGLEVEL = os.getenv('LOG_LEVEL').strip() 24 | BACKUP_INTERVAL = int(os.getenv('INTERVAL', '24')) 25 | PATTERN = os.getenv('PATTERN', 'ALL_CLUSTERS') 26 | SNAPSHOT_NAME_PREFIX = os.getenv('SNAPSHOT_NAME_PREFIX', 'NONE') 27 | 28 | if os.getenv('REGION_OVERRIDE', 'NO') != 'NO': 29 | REGION = os.getenv('REGION_OVERRIDE').strip() 30 | else: 31 | REGION = os.getenv('AWS_DEFAULT_REGION') 32 | 33 | 34 | logger = logging.getLogger() 35 | logger.setLevel(LOGLEVEL.upper()) 36 | 37 | 38 | 39 | def lambda_handler(event, context): 40 | 41 | client = boto3.client('rds', region_name=REGION) 42 | response = paginate_api_call(client, 'describe_db_clusters', 'DBClusters') 43 | now = datetime.now() 44 | pending_backups = 0 45 | filtered_clusters = filter_clusters(PATTERN, response) 46 | filtered_snapshots = get_own_snapshots_source(PATTERN, paginate_api_call(client, 'describe_db_cluster_snapshots', 'DBClusterSnapshots')) 47 | 48 | for db_cluster in filtered_clusters: 49 | 50 | timestamp_format = now.strftime('%Y-%m-%d-%H-%M') 51 | 52 | if requires_backup(BACKUP_INTERVAL, db_cluster, filtered_snapshots): 53 | 54 | backup_age = get_latest_snapshot_ts( 55 | db_cluster['DBClusterIdentifier'], 56 | filtered_snapshots) 57 | 58 | if backup_age is not None: 59 | logger.info('Backing up %s. Backed up %s minutes ago' % ( 60 | db_cluster['DBClusterIdentifier'], ((now - backup_age).total_seconds() / 60))) 61 | 62 | else: 63 | logger.info('Backing up %s. No previous backup found' % 64 | db_cluster['DBClusterIdentifier']) 65 | 66 | if SNAPSHOT_NAME_PREFIX != 'NONE' and SNAPSHOT_NAME_PREFIX != '': 67 | snapshot_identifier = '%s-%s-%s' % ( 68 | SNAPSHOT_NAME_PREFIX, db_cluster['DBClusterIdentifier'], timestamp_format 69 | ) 70 | else: 71 | snapshot_identifier = '%s-%s' % ( 72 | db_cluster['DBClusterIdentifier'], timestamp_format) 73 | 74 | try: 75 | response = client.create_db_cluster_snapshot( 76 | DBClusterSnapshotIdentifier=snapshot_identifier, 77 | DBClusterIdentifier=db_cluster['DBClusterIdentifier'], 78 | Tags=[{'Key': 'CreatedBy', 'Value': 'Snapshot Tool for Aurora'}, { 79 | 'Key': 'CreatedOn', 'Value': timestamp_format}, {'Key': 'shareAndCopy', 'Value': 'YES'}] 80 | ) 81 | except Exception as e: 82 | logger.error(e) 83 | pending_backups += 1 84 | else: 85 | 86 | backup_age = get_latest_snapshot_ts( 87 | db_cluster['DBClusterIdentifier'], 88 | filtered_snapshots) 89 | 90 | logger.info('Skipped %s. Does not require backup. Backed up %s minutes ago' % ( 91 | db_cluster['DBClusterIdentifier'], (now - backup_age).total_seconds() / 60)) 92 | 93 | if pending_backups > 0: 94 | log_message = 'Could not back up every cluster. Backups pending: %s' % pending_backups 95 | logger.error(log_message) 96 | raise SnapshotToolException(log_message) 97 | 98 | 99 | if __name__ == '__main__': 100 | lambda_handler(None, None) 101 | 102 | 103 | --------------------------------------------------------------------------------