├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── Dockerfile ├── LICENSE ├── README.md ├── deploy_cf.sh ├── deploy_lambda.sh ├── docker-compose.yml ├── dynamodb_alarms_cf ├── deploy.sh ├── dynamodb_alarms_cf.yaml └── undeploy.sh ├── dynamodb_metrics_lambda ├── deploy.sh ├── dynamodb_cloudwatch.py ├── requirements.txt ├── run_shell.sh ├── serverless.yml └── undeploy.sh ├── run_shell.sh ├── undeploy_cf.sh └── undeploy_lambda.sh /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.7.3-stretch 2 | ENV SHELL /bin/bash 3 | 4 | USER root 5 | 6 | COPY dynamodb_metrics_lambda /tmp/src/dynamodb_metrics_lambda 7 | COPY dynamodb_alarms_cf /tmp/src/dynamodb_alarms_cf 8 | 9 | RUN apt-get -y update && \ 10 | apt-get install -y lsb-release iproute2 sudo vim curl build-essential && \ 11 | curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash - && \ 12 | apt-get install -y nodejs awscli && \ 13 | npm install -g serverless@1.43.0 && \ 14 | apt-get install -y git && \ 15 | chmod 755 /tmp/src/dynamodb_metrics_lambda/run_shell.sh && \ 16 | chmod 755 /tmp/src/dynamodb_metrics_lambda/deploy.sh && \ 17 | chmod 755 /tmp/src/dynamodb_metrics_lambda/undeploy.sh && \ 18 | chmod 755 /tmp/src/dynamodb_alarms_cf/deploy.sh && \ 19 | chmod 755 /tmp/src/dynamodb_alarms_cf/undeploy.sh 20 | 21 | CMD ["/bin/bash"] 22 | ENTRYPOINT ["/bin/bash", "-c"] 23 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Project Title 2 | 3 | Amazon DynamoDB Monitoring Tools including a sample cloudformation template for monitoring various types of DynamoDB tables, indices, global tables, streams, and related lambda functions. 4 | 5 | ## Getting Started 6 | 7 | These instructions will get you a copy of the project up and running in your AWS account. See deployment for notes on how to deploy the project on a live system. 8 | 9 | The cloudformation template will deploy a DynamoDB on-demand table, a DynamoDB provisioned throughput table, a GSI on each table, a set of reference cloudwatch alarms for monitoring the resources as well as an SNS topic for receiving these cloudwatch alarm notifications. 10 | 11 | There also is a lambda function which is responsible for gathering and publishing custom cloudwatch metrics. 12 | 13 | ### Prerequisites 14 | 15 | In practice you should modify these alarms to match your table and GSI configurations in your own CF templates and deploy the lambda function using serverless or some other deployment technology. 16 | 17 | A docker image file is included which allows you to easily deploy the samples into your own account without having to install any prerequisites besides the docker toolchain. 18 | 19 | ### Lambda Function Environment Variables 20 | 21 | The lambda function accepts a number of environment variables for overriding default settings. These are: 22 | 23 | * CLOUDWATCH_CUSTOM_NAMESPACE - By default the lambda function will publish metrics to the "Custom_DynamoDB" namespace. If you'd like to change it, set the CLOUDWATCH_CUSTOM_NAMESPACE environment variable 24 | * DYNAMODB_ACCOUNT_TABLE_LIMIT - By default the lambda function will assume your DynamoDB Account Table Limit is 256. There is no API call to determine your account table limit, so if you've asked AWS to increase this limit for your account you must set the DYNAMODB_ACCOUNT_TABLE_LIMIT to that value for the lambda function to calculate the AccountTableLimitPct custom metric properly. 25 | 26 | ### Installing 27 | 28 | All the following commands assume you've installed docker on your system and have exported the AWS environment variables. 29 | 30 | To deploy the lambda function: 31 | 32 | ``` 33 | bash deploy_lambda.sh 34 | ``` 35 | 36 | You can also remove the lambda function: 37 | 38 | ``` 39 | bash undeploy_lambda.sh 40 | ``` 41 | 42 | To deploy the CF template you need to specify an email to associate with the SNS Topic: 43 | 44 | ``` 45 | DYNAMODB_SNS_EMAIL=bob@example.com bash deploy_cf.sh 46 | ``` 47 | 48 | You can also remove the CF template: 49 | 50 | ``` 51 | bash undeploy_cf.sh 52 | ``` 53 | 54 | If you want to test out the deployment by running a shell inside the docker container, use the run_shell.sh script 55 | 56 | ``` 57 | bash run_shell.sh 58 | ``` 59 | 60 | In another window you can drop into a shell inside the docker container: 61 | 62 | ``` 63 | docker exec -it run_shell_dynamodb_metrics_lambda /bin/bash 64 | cd /tmp/src/dynamodb_metrics_lambda 65 | python dynamodb_cloudwatch.py 66 | bash deploy.sh 67 | bash undeploy.sh 68 | ``` 69 | 70 | ## License 71 | 72 | This project is licensed under the MIT-0 License - see the [LICENSE](LICENSE) file for details 73 | -------------------------------------------------------------------------------- /deploy_cf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | : "${DYNAMODB_SNS_EMAIL:?Need to set DYNAMODB_SNS_EMAIL to valid email address}" 7 | 8 | EMAIL_REGEX="^[a-z0-9!#\$%&'*+/=?^_\`{|}~-]+(\.[a-z0-9!#$%&'*+/=?^_\`{|}~-]+)*@([a-z0-9]([a-z0-9-]*[a-z0-9])?\.)+[a-z0-9]([a-z0-9-]*[a-z0-9])?\$" 9 | 10 | if [[ ! $DYNAMODB_SNS_EMAIL =~ $EMAIL_REGEX ]] ; then 11 | echo "Need to set DYNAMODB_SNS_EMAIL to valid email address" 12 | exit 1 13 | fi 14 | 15 | trap "docker-compose -f docker-compose.yml rm --force deploy_dynamodb_alarms_cf" SIGINT SIGTERM 16 | docker-compose -f docker-compose.yml build --no-cache deploy_dynamodb_alarms_cf 17 | docker-compose -f docker-compose.yml up deploy_dynamodb_alarms_cf 18 | docker-compose -f docker-compose.yml rm --force deploy_dynamodb_alarms_cf 19 | -------------------------------------------------------------------------------- /deploy_lambda.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | 7 | trap "docker-compose -f docker-compose.yml rm --force deploy_dynamodb_metrics_lambda" SIGINT SIGTERM 8 | docker-compose -f docker-compose.yml build --no-cache deploy_dynamodb_metrics_lambda && \ 9 | docker-compose -f docker-compose.yml up deploy_dynamodb_metrics_lambda 10 | docker-compose -f docker-compose.yml rm --force deploy_dynamodb_metrics_lambda 11 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '2' 2 | services: 3 | run_shell_dynamodb_metrics_lambda: 4 | build: 5 | context: . 6 | image: run-shell-dynamodb-metrics-lambda 7 | container_name: run_shell_dynamodb_metrics_lambda 8 | command: /tmp/src/dynamodb_metrics_lambda/run_shell.sh 9 | environment: 10 | - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} 11 | - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} 12 | - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION} 13 | deploy_dynamodb_metrics_lambda: 14 | build: 15 | context: . 16 | image: deploy-dynamodb-metrics-lambda 17 | container_name: deploy_dynamodb_metrics_lambda 18 | command: /tmp/src/dynamodb_metrics_lambda/deploy.sh 19 | environment: 20 | - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} 21 | - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} 22 | - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION} 23 | undeploy_dynamodb_metrics_lambda: 24 | build: 25 | context: . 26 | image: undeploy-dynamodb-metrics-lambda 27 | container_name: undeploy_dynamodb_metrics_lambda 28 | command: /tmp/src/dynamodb_metrics_lambda/undeploy.sh 29 | environment: 30 | - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} 31 | - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} 32 | - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION} 33 | deploy_dynamodb_alarms_cf: 34 | build: 35 | context: . 36 | image: deploy-dynamodb-alarms-cf 37 | container_name: deploy_dynamodb_alarms_cf 38 | command: /tmp/src/dynamodb_alarms_cf/deploy.sh 39 | environment: 40 | - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} 41 | - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} 42 | - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION} 43 | - DYNAMODB_SNS_EMAIL=${DYNAMODB_SNS_EMAIL} 44 | undeploy_dynamodb_alarms_cf: 45 | build: 46 | context: . 47 | image: undeploy-dynamodb-alarms-cf 48 | container_name: undeploy_dynamodb_alarms_cf 49 | command: /tmp/src/dynamodb_alarms_cf/undeploy.sh 50 | environment: 51 | - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} 52 | - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} 53 | - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION} 54 | -------------------------------------------------------------------------------- /dynamodb_alarms_cf/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | : "${DYNAMODB_SNS_EMAIL:?Need to set DYNAMODB_SNS_EMAIL to valid email address}" 7 | 8 | EMAIL_REGEX="^[a-z0-9!#\$%&'*+/=?^_\`{|}~-]+(\.[a-z0-9!#$%&'*+/=?^_\`{|}~-]+)*@([a-z0-9]([a-z0-9-]*[a-z0-9])?\.)+[a-z0-9]([a-z0-9-]*[a-z0-9])?\$" 9 | STACK_NAME=dynamodb-monitoring 10 | 11 | if [[ ! $DYNAMODB_SNS_EMAIL =~ $EMAIL_REGEX ]] ; then 12 | echo "Need to set DYNAMODB_SNS_EMAIL to valid email address" 13 | exit 1 14 | fi 15 | 16 | # I coudln't figure out a way to put most of the variables in a params file and still pass in the email parameter from the environment 17 | # If anybody knows a way, it would be nice to know. 18 | if ! aws cloudformation describe-stacks --stack-name $STACK_NAME > /dev/null 2>&1; then 19 | aws cloudformation create-stack --stack-name $STACK_NAME --template-body file:///tmp/src/dynamodb_alarms_cf/dynamodb_alarms_cf.yaml --parameters \ 20 | ParameterKey=DynamoDBProvisionedTableName,ParameterValue=dynamodb-provisioned-monitoring \ 21 | ParameterKey=DynamoDBOnDemandTableName,ParameterValue=dynamodb-ondemand-monitoring \ 22 | ParameterKey=DynamoDBGlobalTableName,ParameterValue=dynamodb-gt-monitoring \ 23 | ParameterKey=DynamoDBGlobalTableReceivingRegion,ParameterValue=us-west-2 \ 24 | ParameterKey=DynamoDBStreamLambdaFunctionName,ParameterValue=FooDataForwardToKinesis \ 25 | ParameterKey=DynamoDBCustomNamespace,ParameterValue=Custom_DynamoDB \ 26 | ParameterKey=DynamoDBSNSEmail,ParameterValue=$DYNAMODB_SNS_EMAIL \ 27 | --capabilities CAPABILITY_IAM 28 | else 29 | aws cloudformation update-stack --stack-name $STACK_NAME --template-body file:///tmp/src/dynamodb_alarms_cf/dynamodb_alarms_cf.yaml --parameters \ 30 | ParameterKey=DynamoDBProvisionedTableName,ParameterValue=dynamodb-provisioned-monitoring \ 31 | ParameterKey=DynamoDBOnDemandTableName,ParameterValue=dynamodb-ondemand-monitoring \ 32 | ParameterKey=DynamoDBGlobalTableName,ParameterValue=dynamodb-gt-monitoring \ 33 | ParameterKey=DynamoDBGlobalTableReceivingRegion,ParameterValue=us-west-2 \ 34 | ParameterKey=DynamoDBStreamLambdaFunctionName,ParameterValue=FooDataForwardToKinesis \ 35 | ParameterKey=DynamoDBCustomNamespace,ParameterValue=Custom_DynamoDB \ 36 | ParameterKey=DynamoDBSNSEmail,ParameterValue=$DYNAMODB_SNS_EMAIL \ 37 | --capabilities CAPABILITY_IAM 38 | fi 39 | -------------------------------------------------------------------------------- /dynamodb_alarms_cf/dynamodb_alarms_cf.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: "2010-09-09" 2 | Description: Sample template for monitoring DynamoDB 3 | Parameters: 4 | DynamoDBProvisionedTableName: 5 | Description: Name of DynamoDB Provisioned Table to create 6 | Type: String 7 | MinLength: 3 8 | MaxLength: 255 9 | ConstraintDescription : https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-naming-rules 10 | DynamoDBOnDemandTableName: 11 | Description: Name of DynamoDB On-Demand Table to create 12 | Type: String 13 | MinLength: 3 14 | MaxLength: 255 15 | ConstraintDescription : https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-naming-rules 16 | DynamoDBGlobalTableName: 17 | Description: Name of pre-existing DynamoDB Global Table 18 | Type: String 19 | MinLength: 3 20 | MaxLength: 255 21 | ConstraintDescription : https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-naming-rules 22 | DynamoDBGlobalTableReceivingRegion: 23 | Description: Replica Region for Global Table referred to by DynamoDBGlobalTableName 24 | Type: String 25 | MinLength: 3 26 | MaxLength: 255 27 | ConstraintDescription : https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html 28 | DynamoDBStreamLambdaFunctionName: 29 | Description: DynamoDB Stream Lambda Function Name 30 | Type: String 31 | MinLength: 1 32 | MaxLength: 140 33 | ConstraintDescription : https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html 34 | DynamoDBCustomNamespace: 35 | Description : Custom namespace used by DynamoDB Monitoring Lambda 36 | Type: String 37 | MinLength: 1 38 | MaxLength: 255 39 | ConstraintDescription : https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html 40 | DynamoDBSNSEmail: 41 | Description : Email Address subscribed to newly created SNS Topic 42 | Type: String 43 | AllowedPattern: "^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\\.[a-zA-Z0-9-.]+$" 44 | MinLength: 1 45 | MaxLength: 255 46 | 47 | Resources: 48 | 49 | DynamoDBMonitoringSNSTopic: 50 | Type: AWS::SNS::Topic 51 | Properties: 52 | DisplayName: DynamoDB Monitoring SNS Topic 53 | Subscription: 54 | - Endpoint: !Ref DynamoDBSNSEmail 55 | Protocol: email 56 | TopicName: dynamodb-monitoring 57 | DynamoDBScalingRole: 58 | Type: "AWS::IAM::Role" 59 | Properties: 60 | AssumeRolePolicyDocument: 61 | Version: "2012-10-17" 62 | Statement: 63 | - 64 | Effect: Allow 65 | Principal: 66 | Service: 67 | - 68 | "application-autoscaling.amazonaws.com" 69 | Action: 70 | - 71 | "sts:AssumeRole" 72 | Path: "/" 73 | Policies: 74 | - 75 | PolicyName: root 76 | PolicyDocument: 77 | Version: "2012-10-17" 78 | Statement: 79 | - 80 | Effect: Allow 81 | Action: 82 | - "dynamodb:DescribeTable" 83 | - "dynamodb:UpdateTable" 84 | - "cloudwatch:PutMetricAlarm" 85 | - "cloudwatch:DescribeAlarms" 86 | - "cloudwatch:GetMetricStatistics" 87 | - "cloudwatch:SetAlarmState" 88 | - "cloudwatch:DeleteAlarms" 89 | Resource: "*" 90 | 91 | DynamoDBProvisionedTable: 92 | Type: "AWS::DynamoDB::Table" 93 | Properties: 94 | TableName: !Ref DynamoDBProvisionedTableName 95 | AttributeDefinitions: 96 | - 97 | AttributeName: userId 98 | AttributeType: S 99 | - 100 | AttributeName: city 101 | AttributeType: S 102 | - 103 | AttributeName: signupDate 104 | AttributeType: S 105 | KeySchema: 106 | - 107 | AttributeName: userId 108 | KeyType: HASH 109 | ProvisionedThroughput: 110 | ReadCapacityUnits: 5 111 | WriteCapacityUnits: 5 112 | GlobalSecondaryIndexes: 113 | - IndexName: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 114 | KeySchema: 115 | - AttributeName: city 116 | KeyType: HASH 117 | - AttributeName: signupDate 118 | KeyType: RANGE 119 | Projection: 120 | ProjectionType: ALL 121 | ProvisionedThroughput: 122 | ReadCapacityUnits: 5 123 | WriteCapacityUnits: 5 124 | ProvisionedTableReadCapacityScalableTarget: 125 | Type: "AWS::ApplicationAutoScaling::ScalableTarget" 126 | Properties: 127 | MaxCapacity: 50 128 | MinCapacity: 10 129 | ResourceId: !Sub table/${DynamoDBProvisionedTableName} 130 | RoleARN: !GetAtt DynamoDBScalingRole.Arn 131 | ScalableDimension: "dynamodb:table:ReadCapacityUnits" 132 | ServiceNamespace: dynamodb 133 | ProvisionedTableReadScalingPolicy: 134 | Type: "AWS::ApplicationAutoScaling::ScalingPolicy" 135 | Properties: 136 | PolicyName: ReadAutoScalingPolicy 137 | PolicyType: TargetTrackingScaling 138 | ScalingTargetId: !Ref ProvisionedTableReadCapacityScalableTarget 139 | TargetTrackingScalingPolicyConfiguration: 140 | TargetValue: 70 141 | ScaleInCooldown: 60 142 | ScaleOutCooldown: 60 143 | PredefinedMetricSpecification: 144 | PredefinedMetricType: DynamoDBReadCapacityUtilization 145 | ProvisionedTableWriteCapacityScalableTarget: 146 | Type: "AWS::ApplicationAutoScaling::ScalableTarget" 147 | Properties: 148 | MaxCapacity: 100 149 | MinCapacity: 5 150 | ResourceId: !Sub table/${DynamoDBProvisionedTableName} 151 | RoleARN: !GetAtt DynamoDBScalingRole.Arn 152 | ScalableDimension: "dynamodb:table:WriteCapacityUnits" 153 | ServiceNamespace: dynamodb 154 | ProvisionedTableWriteScalingPolicy: 155 | Type: "AWS::ApplicationAutoScaling::ScalingPolicy" 156 | Properties: 157 | PolicyName: WriteAutoScalingPolicy 158 | PolicyType: TargetTrackingScaling 159 | ScalingTargetId: !Ref ProvisionedTableWriteCapacityScalableTarget 160 | TargetTrackingScalingPolicyConfiguration: 161 | TargetValue: 70 162 | ScaleInCooldown: 60 163 | ScaleOutCooldown: 60 164 | PredefinedMetricSpecification: 165 | PredefinedMetricType: DynamoDBWriteCapacityUtilization 166 | 167 | DynamoDBOnDemandTable: 168 | Type: "AWS::DynamoDB::Table" 169 | Properties: 170 | TableName: !Ref DynamoDBOnDemandTableName 171 | AttributeDefinitions: 172 | - 173 | AttributeName: userId 174 | AttributeType: S 175 | - 176 | AttributeName: city 177 | AttributeType: S 178 | - 179 | AttributeName: signupDate 180 | AttributeType: S 181 | KeySchema: 182 | - 183 | AttributeName: userId 184 | KeyType: HASH 185 | BillingMode: PAY_PER_REQUEST 186 | GlobalSecondaryIndexes: 187 | - IndexName: !Join [ '-', [!Ref DynamoDBOnDemandTableName, 'gsi1'] ] 188 | KeySchema: 189 | - AttributeName: city 190 | KeyType: HASH 191 | - AttributeName: signupDate 192 | KeyType: RANGE 193 | Projection: 194 | ProjectionType: ALL 195 | 196 | DynamoDBAccountReadCapAlarm: 197 | Type: 'AWS::CloudWatch::Alarm' 198 | Properties: 199 | AlarmName: 'DynamoDBAccountReadCapAlarm' 200 | AlarmDescription: 'Alarm when account approaches maximum read capacity limit' 201 | AlarmActions: 202 | - !Ref DynamoDBMonitoringSNSTopic 203 | Namespace: 'AWS/DynamoDB' 204 | MetricName: 'AccountProvisionedReadCapacityUtilization' 205 | Statistic: 'Maximum' 206 | Unit: 'Percent' 207 | Threshold: 80 208 | ComparisonOperator: 'GreaterThanThreshold' 209 | Period: 300 210 | EvaluationPeriods: 1 211 | DynamoDBAccountWriteCapAlarm: 212 | Type: 'AWS::CloudWatch::Alarm' 213 | Properties: 214 | AlarmName: 'DynamoDBAccountWriteCapAlarm' 215 | AlarmDescription: 'Alarm when account approaches maximum write capacity limit' 216 | AlarmActions: 217 | - !Ref DynamoDBMonitoringSNSTopic 218 | Namespace: 'AWS/DynamoDB' 219 | MetricName: 'AccountProvisionedWriteCapacityUtilization' 220 | Statistic: 'Maximum' 221 | Unit: 'Percent' 222 | Threshold: 80 223 | ComparisonOperator: 'GreaterThanThreshold' 224 | Period: 300 225 | EvaluationPeriods: 1 226 | DynamoDBTableReadCapAlarm: 227 | Type: 'AWS::CloudWatch::Alarm' 228 | Properties: 229 | AlarmName: 'DynamoDBTableReadCapAlarm' 230 | AlarmDescription: 'Alarm when table capacity approaches maximum account read capacity limit' 231 | AlarmActions: 232 | - !Ref DynamoDBMonitoringSNSTopic 233 | Namespace: 'AWS/DynamoDB' 234 | MetricName: 'MaxProvisionedTableReadCapacityUtilization' 235 | Statistic: 'Maximum' 236 | Unit: 'Percent' 237 | Threshold: 80 238 | ComparisonOperator: 'GreaterThanThreshold' 239 | Period: 300 240 | EvaluationPeriods: 1 241 | DynamoDBTableWriteCapAlarm: 242 | Type: 'AWS::CloudWatch::Alarm' 243 | Properties: 244 | AlarmName: 'DynamoDBTableWriteCapAlarm' 245 | AlarmDescription: 'Alarm when table capacity approaches maximum account write capacity limit' 246 | AlarmActions: 247 | - !Ref DynamoDBMonitoringSNSTopic 248 | Namespace: 'AWS/DynamoDB' 249 | MetricName: 'MaxProvisionedTableWriteCapacityUtilization' 250 | Statistic: 'Maximum' 251 | Unit: 'Percent' 252 | Threshold: 80 253 | ComparisonOperator: 'GreaterThanThreshold' 254 | Period: 300 255 | EvaluationPeriods: 1 256 | DynamoDBAccountTableLimitAlarm: 257 | Type: 'AWS::CloudWatch::Alarm' 258 | Properties: 259 | AlarmName: 'DynamoDBAccountTableLimitAlarm' 260 | AlarmDescription: 'Alarm when account approaches total limit of number of DynamoDB Tables' 261 | AlarmActions: 262 | - !Ref DynamoDBMonitoringSNSTopic 263 | Namespace: !Ref DynamoDBCustomNamespace 264 | MetricName: 'AccountTableLimitPct' 265 | Statistic: 'Maximum' 266 | Unit: 'Percent' 267 | Threshold: 80 268 | ComparisonOperator: 'GreaterThanThreshold' 269 | Period: 60 270 | EvaluationPeriods: 2 271 | DynamoDBTableReadThrottlingAlarm: 272 | Type: 'AWS::CloudWatch::Alarm' 273 | Properties: 274 | AlarmName: 'DynamoDBTableReadThrottlingAlarm' 275 | AlarmDescription: 'Alarm when table read throttle requests exceed 2% of total number of read requests' 276 | AlarmActions: 277 | - !Ref DynamoDBMonitoringSNSTopic 278 | Metrics: 279 | - Id: 'e1' 280 | Expression: '(m1/m2)*100' 281 | Label: TableReadThrottlesOverTotalReads 282 | - Id: 'm1' 283 | MetricStat: 284 | Metric: 285 | Namespace: 'AWS/DynamoDB' 286 | MetricName: 'ReadThrottleEvents' 287 | Dimensions: 288 | - Name: 'TableName' 289 | Value: !Ref DynamoDBProvisionedTableName 290 | Period: 60 291 | Stat: 'SampleCount' 292 | Unit: 'Count' 293 | ReturnData: False 294 | - Id: 'm2' 295 | MetricStat: 296 | Metric: 297 | Namespace: 'AWS/DynamoDB' 298 | MetricName: 'ConsumedReadCapacityUnits' 299 | Dimensions: 300 | - Name: 'TableName' 301 | Value: !Ref DynamoDBProvisionedTableName 302 | Period: 60 303 | Stat: 'SampleCount' 304 | Unit: 'Count' 305 | ReturnData: False 306 | EvaluationPeriods: 2 307 | Threshold: 2.0 308 | ComparisonOperator: 'GreaterThanThreshold' 309 | DynamoDBGSIReadThrottlingAlarm: 310 | Type: 'AWS::CloudWatch::Alarm' 311 | Properties: 312 | AlarmName: 'DynamoDBGSIReadThrottlingAlarm' 313 | AlarmDescription: 'Alarm when GSI read throttle requests exceed 2% of total number of read requests' 314 | AlarmActions: 315 | - !Ref DynamoDBMonitoringSNSTopic 316 | Metrics: 317 | - Id: 'e1' 318 | Expression: '(m1/m2)*100' 319 | Label: GSIReadThrottlesOverTotalReads 320 | - Id: 'm1' 321 | MetricStat: 322 | Metric: 323 | Namespace: 'AWS/DynamoDB' 324 | MetricName: 'ReadThrottleEvents' 325 | Dimensions: 326 | - Name: 'TableName' 327 | Value: !Ref DynamoDBProvisionedTableName 328 | - Name: 'GlobalSecondaryIndexName' 329 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 330 | Period: 60 331 | Stat: 'SampleCount' 332 | Unit: 'Count' 333 | ReturnData: False 334 | - Id: 'm2' 335 | MetricStat: 336 | Metric: 337 | Namespace: 'AWS/DynamoDB' 338 | MetricName: 'ConsumedReadCapacityUnits' 339 | Dimensions: 340 | - Name: 'TableName' 341 | Value: !Ref DynamoDBProvisionedTableName 342 | - Name: 'GlobalSecondaryIndexName' 343 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 344 | Period: 60 345 | Stat: 'SampleCount' 346 | Unit: 'Count' 347 | ReturnData: False 348 | EvaluationPeriods: 2 349 | Threshold: 2.0 350 | ComparisonOperator: 'GreaterThanThreshold' 351 | DynamoDBTableWriteThrottlingAlarm: 352 | Type: 'AWS::CloudWatch::Alarm' 353 | Properties: 354 | AlarmName: 'DynamoDBTableWriteThrottlingAlarm' 355 | AlarmDescription: 'Alarm when table write throttle requests exceed 2% of total number of write requests' 356 | AlarmActions: 357 | - !Ref DynamoDBMonitoringSNSTopic 358 | Metrics: 359 | - Id: 'e1' 360 | Expression: '(m1/m2)*100' 361 | Label: TableWriteThrottlesOverTotalWrites 362 | - Id: 'm1' 363 | MetricStat: 364 | Metric: 365 | Namespace: 'AWS/DynamoDB' 366 | MetricName: 'WriteThrottleEvents' 367 | Dimensions: 368 | - Name: 'TableName' 369 | Value: !Ref DynamoDBProvisionedTableName 370 | Period: 60 371 | Stat: 'SampleCount' 372 | Unit: 'Count' 373 | ReturnData: False 374 | - Id: 'm2' 375 | MetricStat: 376 | Metric: 377 | Namespace: 'AWS/DynamoDB' 378 | MetricName: 'ConsumedWriteCapacityUnits' 379 | Dimensions: 380 | - Name: 'TableName' 381 | Value: !Ref DynamoDBProvisionedTableName 382 | Period: 60 383 | Stat: 'SampleCount' 384 | Unit: 'Count' 385 | ReturnData: False 386 | EvaluationPeriods: 2 387 | Threshold: 2.0 388 | ComparisonOperator: 'GreaterThanThreshold' 389 | DynamoDBGSIWriteThrottlingAlarm: 390 | Type: 'AWS::CloudWatch::Alarm' 391 | Properties: 392 | AlarmName: 'DynamoDBGSIWriteThrottlingAlarm' 393 | AlarmDescription: 'Alarm when GSI write throttle requests exceed 2% of total number of write requests' 394 | AlarmActions: 395 | - !Ref DynamoDBMonitoringSNSTopic 396 | Metrics: 397 | - Id: 'e1' 398 | Expression: '(m1/m2)*100' 399 | Label: GSIWriteThrottlesOverTotalWrites 400 | - Id: 'm1' 401 | MetricStat: 402 | Metric: 403 | Namespace: 'AWS/DynamoDB' 404 | MetricName: 'WriteThrottleEvents' 405 | Dimensions: 406 | - Name: 'TableName' 407 | Value: !Ref DynamoDBProvisionedTableName 408 | - Name: 'GlobalSecondaryIndexName' 409 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 410 | Period: 60 411 | Stat: 'SampleCount' 412 | Unit: 'Count' 413 | ReturnData: False 414 | - Id: 'm2' 415 | MetricStat: 416 | Metric: 417 | Namespace: 'AWS/DynamoDB' 418 | MetricName: 'ConsumedWriteCapacityUnits' 419 | Dimensions: 420 | - Name: 'TableName' 421 | Value: !Ref DynamoDBProvisionedTableName 422 | - Name: 'GlobalSecondaryIndexName' 423 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 424 | Period: 60 425 | Stat: 'SampleCount' 426 | Unit: 'Count' 427 | ReturnData: False 428 | EvaluationPeriods: 2 429 | Threshold: 2.0 430 | ComparisonOperator: 'GreaterThanThreshold' 431 | DynamoDBTableSystemErrorAlarm: 432 | Type: 'AWS::CloudWatch::Alarm' 433 | Properties: 434 | AlarmName: 'DynamoDBTableSystemErrorAlarm' 435 | AlarmDescription: 'Alarm when system errors exceed 2% of total number of requests' 436 | AlarmActions: 437 | - !Ref DynamoDBMonitoringSNSTopic 438 | Metrics: 439 | - Id: 'e1' 440 | Expression: 'm1/(m2+m3)*100' 441 | Label: SystemErrorsOverTotalRequests 442 | - Id: 'm1' 443 | MetricStat: 444 | Metric: 445 | Namespace: 'AWS/DynamoDB' 446 | MetricName: 'SystemErrors' 447 | Dimensions: 448 | - Name: 'TableName' 449 | Value: !Ref DynamoDBProvisionedTableName 450 | Period: 60 451 | Stat: 'SampleCount' 452 | Unit: 'Count' 453 | ReturnData: False 454 | - Id: 'm2' 455 | MetricStat: 456 | Metric: 457 | Namespace: 'AWS/DynamoDB' 458 | MetricName: 'ConsumedReadCapacityUnits' 459 | Dimensions: 460 | - Name: 'TableName' 461 | Value: !Ref DynamoDBProvisionedTableName 462 | Period: 60 463 | Stat: 'SampleCount' 464 | Unit: 'Count' 465 | ReturnData: False 466 | - Id: 'm3' 467 | MetricStat: 468 | Metric: 469 | Namespace: 'AWS/DynamoDB' 470 | MetricName: 'ConsumedWriteCapacityUnits' 471 | Dimensions: 472 | - Name: 'TableName' 473 | Value: !Ref DynamoDBProvisionedTableName 474 | Period: 60 475 | Stat: 'SampleCount' 476 | Unit: 'Count' 477 | ReturnData: False 478 | EvaluationPeriods: 20 479 | Threshold: 2.0 480 | ComparisonOperator: 'GreaterThanThreshold' 481 | DynamoDBGSISystemErrorAlarm: 482 | Type: 'AWS::CloudWatch::Alarm' 483 | Properties: 484 | AlarmName: 'DynamoDBGSISystemErrorAlarm' 485 | AlarmDescription: 'Alarm when GSI system errors exceed 2% of total number of requests' 486 | AlarmActions: 487 | - !Ref DynamoDBMonitoringSNSTopic 488 | Metrics: 489 | - Id: 'e1' 490 | Expression: 'm1/(m2+m3)*100' 491 | Label: GSISystemErrorsOverTotalRequests 492 | - Id: 'm1' 493 | MetricStat: 494 | Metric: 495 | Namespace: 'AWS/DynamoDB' 496 | MetricName: 'SystemErrors' 497 | Dimensions: 498 | - Name: 'TableName' 499 | Value: !Ref DynamoDBProvisionedTableName 500 | - Name: 'GlobalSecondaryIndexName' 501 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 502 | Period: 60 503 | Stat: 'SampleCount' 504 | Unit: 'Count' 505 | ReturnData: False 506 | - Id: 'm2' 507 | MetricStat: 508 | Metric: 509 | Namespace: 'AWS/DynamoDB' 510 | MetricName: 'ConsumedReadCapacityUnits' 511 | Dimensions: 512 | - Name: 'TableName' 513 | Value: !Ref DynamoDBProvisionedTableName 514 | - Name: 'GlobalSecondaryIndexName' 515 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 516 | Period: 60 517 | Stat: 'SampleCount' 518 | Unit: 'Count' 519 | ReturnData: False 520 | - Id: 'm3' 521 | MetricStat: 522 | Metric: 523 | Namespace: 'AWS/DynamoDB' 524 | MetricName: 'ConsumedWriteCapacityUnits' 525 | Dimensions: 526 | - Name: 'TableName' 527 | Value: !Ref DynamoDBProvisionedTableName 528 | - Name: 'GlobalSecondaryIndexName' 529 | Value: !Join [ '-', [!Ref DynamoDBProvisionedTableName, 'gsi1'] ] 530 | Period: 60 531 | Stat: 'SampleCount' 532 | Unit: 'Count' 533 | ReturnData: False 534 | EvaluationPeriods: 20 535 | Threshold: 2.0 536 | ComparisonOperator: 'GreaterThanThreshold' 537 | DynamoDBTableUserErrorAlarm: 538 | Type: 'AWS::CloudWatch::Alarm' 539 | Properties: 540 | AlarmName: 'DynamoDBTableUserErrorAlarm' 541 | AlarmDescription: 'Alarm when user errors exceed 2% of total number of requests' 542 | AlarmActions: 543 | - !Ref DynamoDBMonitoringSNSTopic 544 | Metrics: 545 | - Id: 'e1' 546 | Expression: 'm1/(q1+m1)*100' 547 | Label: UserErrorsOverTotalRequests 548 | - Id: 'm1' 549 | MetricStat: 550 | Metric: 551 | Namespace: 'AWS/DynamoDB' 552 | MetricName: 'UserErrors' 553 | Period: 60 554 | Stat: 'SampleCount' 555 | Unit: 'Count' 556 | ReturnData: False 557 | - Id: 'q1' 558 | Expression: 'SELECT COUNT(SuccessfulRequestLatency) FROM SCHEMA("AWS/DynamoDB", TableName, Operation)' 559 | Period: 60 560 | ReturnData: False 561 | EvaluationPeriods: 2 562 | Threshold: 2.0 563 | ComparisonOperator: 'GreaterThanThreshold' 564 | DynamoDBTableConditionCheckAlarm: 565 | Type: 'AWS::CloudWatch::Alarm' 566 | Properties: 567 | AlarmName: 'DynamoDBTableConditionCheckWritelarm' 568 | AlarmDescription: 'Alarm when condition check errors are too high' 569 | AlarmActions: 570 | - !Ref DynamoDBMonitoringSNSTopic 571 | Namespace: 'AWS/DynamoDB' 572 | MetricName: 'ConditionalCheckFailedRequests' 573 | Statistic: 'Sum' 574 | Unit: 'Count' 575 | Threshold: 100 576 | ComparisonOperator: 'GreaterThanThreshold' 577 | Period: 60 578 | EvaluationPeriods: 2 579 | DynamoDBTableTransactionConflictAlarm: 580 | Type: 'AWS::CloudWatch::Alarm' 581 | Properties: 582 | AlarmName: 'DynamoDBTableTransactionConflictAlarm' 583 | AlarmDescription: 'Alarm when transaction conflict errors are too high' 584 | AlarmActions: 585 | - !Ref DynamoDBMonitoringSNSTopic 586 | Namespace: 'AWS/DynamoDB' 587 | MetricName: 'TransactionConflict' 588 | Statistic: 'Sum' 589 | Unit: 'Count' 590 | Threshold: 100 591 | ComparisonOperator: 'GreaterThanThreshold' 592 | Period: 60 593 | EvaluationPeriods: 2 594 | DynamoDBTableASReadAlarm: 595 | Type: 'AWS::CloudWatch::Alarm' 596 | Properties: 597 | AlarmName: 'DynamoDBTableASReadAlarm' 598 | AlarmDescription: 'Alarm when table auto scaling read setting approaches table AS maximum' 599 | AlarmActions: 600 | - !Ref DynamoDBMonitoringSNSTopic 601 | Namespace: !Ref DynamoDBCustomNamespace 602 | MetricName: 'ProvisionedReadCapacityAutoScalingPct' 603 | Dimensions: 604 | - Name: 'TableName' 605 | Value: !Ref DynamoDBProvisionedTableName 606 | Statistic: 'Maximum' 607 | Unit: 'Percent' 608 | Threshold: 90 609 | ComparisonOperator: 'GreaterThanThreshold' 610 | Period: 60 611 | EvaluationPeriods: 2 612 | DynamoDBTableASWriteAlarm: 613 | Type: 'AWS::CloudWatch::Alarm' 614 | Properties: 615 | AlarmName: 'DynamoDBTableASWriteAlarm' 616 | AlarmDescription: 'Alarm when table auto scaling write setting approaches table AS maximum' 617 | AlarmActions: 618 | - !Ref DynamoDBMonitoringSNSTopic 619 | Namespace: !Ref DynamoDBCustomNamespace 620 | MetricName: 'ProvisionedWriteCapacityAutoScalingPct' 621 | Dimensions: 622 | - Name: 'TableName' 623 | Value: !Ref DynamoDBProvisionedTableName 624 | Statistic: 'Maximum' 625 | Unit: 'Percent' 626 | Threshold: 90 627 | ComparisonOperator: 'GreaterThanThreshold' 628 | Period: 60 629 | EvaluationPeriods: 2 630 | 631 | # Alarms for OnDemand Tables 632 | DynamoDBOnDemandTableReadLimitAlarm: 633 | Type: 'AWS::CloudWatch::Alarm' 634 | Properties: 635 | AlarmName: 'DynamoDBOnDemandTableReadLimitAlarm' 636 | AlarmDescription: 'Alarm when consumed table reads approach the account limit' 637 | AlarmActions: 638 | - !Ref DynamoDBMonitoringSNSTopic 639 | Metrics: 640 | - Id: 'e1' 641 | Expression: '(((m1 / 300) / m2) * 100)' 642 | Label: TableReadsOverMaxReadLimit 643 | - Id: 'm1' 644 | MetricStat: 645 | Metric: 646 | Namespace: 'AWS/DynamoDB' 647 | MetricName: 'ConsumedReadCapacityUnits' 648 | Dimensions: 649 | - Name: 'TableName' 650 | Value: !Ref DynamoDBOnDemandTableName 651 | Period: 300 652 | Stat: 'SampleCount' 653 | Unit: 'Count' 654 | ReturnData: False 655 | - Id: 'm2' 656 | MetricStat: 657 | Metric: 658 | Namespace: 'AWS/DynamoDB' 659 | MetricName: 'AccountMaxTableLevelReads' 660 | Period: 300 661 | Stat: 'Maximum' 662 | ReturnData: False 663 | EvaluationPeriods: 2 664 | Threshold: 90 665 | ComparisonOperator: 'GreaterThanThreshold' 666 | 667 | DynamoDBOnDemandTableWriteLimitAlarm: 668 | Type: 'AWS::CloudWatch::Alarm' 669 | Properties: 670 | AlarmName: 'DynamoDBOnDemandTableWriteLimitAlarm' 671 | AlarmDescription: 'Alarm when consumed table reads approach the account limit' 672 | AlarmActions: 673 | - !Ref DynamoDBMonitoringSNSTopic 674 | Metrics: 675 | - Id: 'e1' 676 | Expression: '(((m1 / 300) / m2) * 100)' 677 | Label: TableWritesOverMaxWriteLimit 678 | - Id: 'm1' 679 | MetricStat: 680 | Metric: 681 | Namespace: 'AWS/DynamoDB' 682 | MetricName: 'ConsumedWriteCapacityUnits' 683 | Dimensions: 684 | - Name: 'TableName' 685 | Value: !Ref DynamoDBOnDemandTableName 686 | Period: 300 687 | Stat: 'SampleCount' 688 | Unit: 'Count' 689 | ReturnData: False 690 | - Id: 'm2' 691 | MetricStat: 692 | Metric: 693 | Namespace: 'AWS/DynamoDB' 694 | MetricName: 'AccountMaxTableLevelWrites' 695 | Period: 300 696 | Stat: 'Maximum' 697 | ReturnData: False 698 | EvaluationPeriods: 2 699 | Threshold: 90 700 | ComparisonOperator: 'GreaterThanThreshold' 701 | 702 | DynamoDBOnDemandGSIReadLimitAlarm: 703 | Type: 'AWS::CloudWatch::Alarm' 704 | Properties: 705 | AlarmName: 'DynamoDBOnDemandGSIReadLimitAlarm' 706 | AlarmDescription: 'Alarm when consumed GSI reads approach the account limit' 707 | AlarmActions: 708 | - !Ref DynamoDBMonitoringSNSTopic 709 | Metrics: 710 | - Id: 'e1' 711 | Expression: '(((m1 / 300) / m2) * 100)' 712 | Label: GSIReadsOverMaxReadLimit 713 | - Id: 'm1' 714 | MetricStat: 715 | Metric: 716 | Namespace: 'AWS/DynamoDB' 717 | MetricName: 'ConsumedReadCapacityUnits' 718 | Dimensions: 719 | - Name: 'TableName' 720 | Value: !Ref DynamoDBOnDemandTableName 721 | - Name: 'GlobalSecondaryIndexName' 722 | Value: !Join [ '-', [!Ref DynamoDBOnDemandTableName, 'gsi1'] ] 723 | Period: 300 724 | Stat: 'SampleCount' 725 | Unit: 'Count' 726 | ReturnData: False 727 | - Id: 'm2' 728 | MetricStat: 729 | Metric: 730 | Namespace: 'AWS/DynamoDB' 731 | MetricName: 'AccountMaxTableLevelReads' 732 | Period: 300 733 | Stat: 'Maximum' 734 | ReturnData: False 735 | EvaluationPeriods: 2 736 | Threshold: 90 737 | ComparisonOperator: 'GreaterThanThreshold' 738 | 739 | DynamoDBOnDemandGSIWriteLimitAlarm: 740 | Type: 'AWS::CloudWatch::Alarm' 741 | Properties: 742 | AlarmName: 'DynamoDBOnDemandGSIWriteLimitAlarm' 743 | AlarmDescription: 'Alarm when consumed GSI reads approach the account limit' 744 | AlarmActions: 745 | - !Ref DynamoDBMonitoringSNSTopic 746 | Metrics: 747 | - Id: 'e1' 748 | Expression: '(((m1 / 300) / m2) * 100)' 749 | Label: GSIWritesOverMaxWriteLimit 750 | - Id: 'm1' 751 | MetricStat: 752 | Metric: 753 | Namespace: 'AWS/DynamoDB' 754 | MetricName: 'ConsumedWriteCapacityUnits' 755 | Dimensions: 756 | - Name: 'TableName' 757 | Value: !Ref DynamoDBOnDemandTableName 758 | - Name: 'GlobalSecondaryIndexName' 759 | Value: !Join [ '-', [!Ref DynamoDBOnDemandTableName, 'gsi1'] ] 760 | Period: 300 761 | Stat: 'SampleCount' 762 | Unit: 'Count' 763 | ReturnData: False 764 | - Id: 'm2' 765 | MetricStat: 766 | Metric: 767 | Namespace: 'AWS/DynamoDB' 768 | MetricName: 'AccountMaxTableLevelWrites' 769 | Period: 300 770 | Stat: 'Maximum' 771 | ReturnData: False 772 | EvaluationPeriods: 2 773 | Threshold: 90 774 | ComparisonOperator: 'GreaterThanThreshold' 775 | 776 | # Alarms for Global Tables 777 | DynamoDBGTReplLatencyAlarm: 778 | Type: 'AWS::CloudWatch::Alarm' 779 | Properties: 780 | AlarmName: 'DynamoDBGTReplLatencyAlarm' 781 | AlarmDescription: 'Alarm when global table replication latency exceeds 3 minutes (180k ms)' 782 | AlarmActions: 783 | - !Ref DynamoDBMonitoringSNSTopic 784 | Namespace: 'AWS/DynamoDB' 785 | MetricName: 'ReplicationLatency' 786 | Dimensions: 787 | - Name: 'TableName' 788 | Value: !Ref DynamoDBGlobalTableName 789 | - Name: 'ReceivingRegion' 790 | Value: !Ref DynamoDBGlobalTableReceivingRegion 791 | Statistic: 'Average' 792 | Threshold: 180000 793 | ComparisonOperator: 'GreaterThanThreshold' 794 | Period: 60 795 | EvaluationPeriods: 15 796 | 797 | # Alarms for Lambda Functions 798 | DynamoStreamLambdaIteratorAgeAlarm: 799 | Type: 'AWS::CloudWatch::Alarm' 800 | Properties: 801 | AlarmName: 'DynamoStreamLambdaIteratorAgeAlarm' 802 | AlarmDescription: 'Alarm when lambda iterator age exceeds 30 seconds (30k ms)' 803 | AlarmActions: 804 | - !Ref DynamoDBMonitoringSNSTopic 805 | Namespace: 'AWS/Lambda' 806 | MetricName: 'IteratorAge' 807 | Dimensions: 808 | - Name: 'Function' 809 | Value: !Ref DynamoDBStreamLambdaFunctionName 810 | - Name: 'Resource' 811 | Value: !Ref DynamoDBStreamLambdaFunctionName 812 | Statistic: 'Average' 813 | Threshold: 30000 814 | ComparisonOperator: 'GreaterThanThreshold' 815 | Period: 60 816 | EvaluationPeriods: 2 817 | -------------------------------------------------------------------------------- /dynamodb_alarms_cf/undeploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | 4 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 5 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 6 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 7 | 8 | STACK_NAME=dynamodb-monitoring 9 | 10 | aws cloudformation delete-stack --stack-name $STACK_NAME 11 | -------------------------------------------------------------------------------- /dynamodb_metrics_lambda/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | 7 | pip install -t /tmp/src/dynamodb_metrics_lambda/vendored/ -r /tmp/src/dynamodb_metrics_lambda/requirements.txt 8 | 9 | cd /tmp/src/dynamodb_metrics_lambda; serverless deploy --region $AWS_DEFAULT_REGION || exit 1 10 | -------------------------------------------------------------------------------- /dynamodb_metrics_lambda/dynamodb_cloudwatch.py: -------------------------------------------------------------------------------- 1 | import sys, os 2 | here = os.path.dirname(os.path.realpath(__file__)) 3 | vendored_dir = os.path.join(here, 'vendored') 4 | sys.path.append(vendored_dir) 5 | import boto3 6 | import json 7 | import datetime 8 | 9 | # Create CloudWatch client 10 | cloudwatch = boto3.client('cloudwatch') 11 | ddb = boto3.client('dynamodb') 12 | aas = boto3.client('application-autoscaling') 13 | 14 | # Constants 15 | DEFAULT_DYNAMODB_TABLE_LIMIT = 256 16 | FIVE_MINS_SECS = 300 17 | 18 | # We can't use AWS/DynamoDB since its reserved 19 | # We'll let people override it by changing CLOUDWATCH_CUSTOM_NAMESPACE env 20 | DEFAULT_CLOUDWATCH_CUSTOM_NAMESPACE = "Custom_DynamoDB" 21 | CLOUDWATCH_CUSTOM_NAMESPACE = DEFAULT_CLOUDWATCH_CUSTOM_NAMESPACE 22 | if 'CLOUDWATCH_CUSTOM_NAMESPACE' in os.environ: 23 | CLOUDWATCH_CUSTOM_NAMESPACE = os.environ['CLOUDWATCH_CUSTOM_NAMESPACE'] 24 | 25 | AAS_MAX_RESOURCE_ID_LENGTH = 1600 26 | 27 | # Globals 28 | ddb_account_limits = None 29 | ddb_tables = {} 30 | ddb_total_provisioned_rcu = 0 31 | ddb_total_provisioned_wcu = 0 32 | ddb_total_consumed_rcu = 0 33 | ddb_total_consumed_wcu = 0 34 | 35 | def success_response(event, context): 36 | body = { 37 | "message": "Executed successfully", 38 | "input": event 39 | } 40 | 41 | response = { 42 | "statusCode": 200, 43 | "body": json.dumps(body) 44 | } 45 | 46 | return response 47 | 48 | def load_dynamodb_limits(event, context): 49 | global ddb_account_limits 50 | ddb_account_limits = ddb.describe_limits() 51 | ddb_account_limits.pop('ResponseMetadata', None) 52 | 53 | # Since there's no way to query the max table limit we will allow them to override 54 | # this with an environment variable for the lambda function 55 | if 'DYNAMODB_ACCOUNT_TABLE_LIMIT' in os.environ: 56 | ddb_account_limits['AccountMaxTables'] = os.environ['DYNAMODB_ACCOUNT_TABLE_LIMIT'] 57 | else: 58 | ddb_account_limits['AccountMaxTables'] = DEFAULT_DYNAMODB_TABLE_LIMIT 59 | #print(ddb_account_limits) 60 | 61 | def load_dynamodb_tables(event, context): 62 | global ddb_tables 63 | global ddb_total_provisioned_rcu 64 | global ddb_total_provisioned_wcu 65 | 66 | paginator = ddb.get_paginator('list_tables') 67 | for response in paginator.paginate(): 68 | for table_name in response['TableNames']: 69 | #if table_name == 'dynamodb-speed-test-blog': 70 | #if table_name == 'bank': 71 | ddb_tables[table_name] = {} 72 | # print(ddb_tables) 73 | 74 | for table in ddb_tables.keys(): 75 | response = ddb.describe_table(TableName=table) 76 | #print(response) 77 | if response['Table']['TableStatus'] != 'ACTIVE': 78 | return 79 | 80 | # Older tables that existed before the on demand feature shipped might not have this field 81 | if 'BillingModeSummary' in response['Table'] and 'BillingMode' in response['Table']['BillingModeSummary']: 82 | ddb_tables[table]['BillingMode'] = response['Table']['BillingModeSummary']['BillingMode'] 83 | else: 84 | ddb_tables[table]['BillingMode'] = "PROVISIONED" 85 | 86 | ddb_tables[table]['ProvisionedThroughput'] = response['Table']['ProvisionedThroughput'] 87 | # We don't need this field and it messes up our object->json dump 88 | if 'LastIncreaseDateTime' in ddb_tables[table]['ProvisionedThroughput']: 89 | ddb_tables[table]['ProvisionedThroughput'].pop('LastIncreaseDateTime') 90 | if 'LastDecreaseDateTime' in ddb_tables[table]['ProvisionedThroughput']: 91 | ddb_tables[table]['ProvisionedThroughput'].pop('LastDecreaseDateTime') 92 | ddb_total_provisioned_rcu += ddb_tables[table]['ProvisionedThroughput']['ReadCapacityUnits'] 93 | ddb_total_provisioned_wcu += ddb_tables[table]['ProvisionedThroughput']['WriteCapacityUnits'] 94 | ddb_tables[table]['autoscaling'] = {'ReadCapacityUnits' : None, 'WriteCapacityUnits' : None} 95 | ddb_tables[table]['gsis'] = {} 96 | if 'GlobalSecondaryIndexes' in response['Table']: 97 | for gsi in response['Table']['GlobalSecondaryIndexes']: 98 | ddb_tables[table]['gsis'][gsi['IndexName']] = {} 99 | ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput'] = gsi['ProvisionedThroughput'] 100 | # We don't need this field and it messes up our object->json dump 101 | if 'LastIncreaseDateTime' in ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput']: 102 | ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput'].pop('LastIncreaseDateTime') 103 | if 'LastDecreaseDateTime' in ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput']: 104 | ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput'].pop('LastDecreaseDateTime') 105 | ddb_total_provisioned_rcu += ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput']['ReadCapacityUnits'] 106 | ddb_total_provisioned_wcu += ddb_tables[table]['gsis'][gsi['IndexName']]['ProvisionedThroughput']['WriteCapacityUnits'] 107 | ddb_tables[table]['gsis'][gsi['IndexName']]['autoscaling'] = {'ReadCapacityUnits' : None, 'WriteCapacityUnits' : None} 108 | 109 | def gather_dynamodb_consumption(event, context): 110 | global ddb_tables 111 | 112 | #gather_table_config(event, context) 113 | 114 | # We need a resource ID array with one entry for every table (table/) and one entry for every GSI (table//index/) 115 | # We can put up to 1600 ResourceIds in the array both for describe_scalable_targets 116 | # https://docs.aws.amazon.com/autoscaling/application/APIReference/API_DescribeScalableTargets.html 117 | # To avoid throttling from the AAS service we will build arrays that have up to 1600 ResourceIds and store an array of those 118 | # arrays which we can loop through afterwards to minimize the number of calls we make to AAS service 119 | # Its possible a customer has had their number of tables account limit increased so there could be thousands of tables and thousands of GSIs 120 | # As we go through the results from the DescribeScalableTargets we will build a map of resource_ids to use in calling DescribeScalingPolicies 121 | dst_resource_id_arrays = [] 122 | tmp_dst_resource_ids = [] 123 | 124 | dsp_resource_ids = {} 125 | 126 | for table in ddb_tables.keys(): 127 | tmp_dst_resource_ids.append('table/' + table) 128 | if AAS_MAX_RESOURCE_ID_LENGTH == len(tmp_dst_resource_ids): 129 | dst_resource_id_arrays.append(tmp_dst_resource_ids) 130 | tmp_dst_resource_ids = [] 131 | for gsi in ddb_tables[table]['gsis'].keys(): 132 | tmp_dst_resource_ids.append('table/' + table + '/index/' + gsi) 133 | if AAS_MAX_RESOURCE_ID_LENGTH == len(tmp_dst_resource_ids): 134 | dst_resource_id_arrays.append(tmp_dst_resource_ids) 135 | tmp_dst_resource_ids = [] 136 | if len(tmp_dst_resource_ids) > 0: 137 | dst_resource_id_arrays.append(tmp_dst_resource_ids) 138 | 139 | for dst_resource_id_array in dst_resource_id_arrays: 140 | aas_paginator = aas.get_paginator('describe_scalable_targets') 141 | for aas_response in aas_paginator.paginate(ServiceNamespace='dynamodb', ResourceIds=dst_resource_id_array): 142 | #print(aas_response) 143 | for target in aas_response['ScalableTargets']: 144 | # The responses will be a mix of tables and indexes so we need to figure out which this is 145 | if target['ScalableDimension'].startswith('dynamodb:table:'): 146 | # ResourceId = "table/ 147 | aas_table_name = target['ResourceId'].split('/')[1] 148 | # Slice off the leading "dynamodb:table:" from the Scalable Dimension 149 | aas_scalable_dimension = target['ScalableDimension'][len("dynamodb:table:"):] 150 | ddb_tables[aas_table_name]['autoscaling'][aas_scalable_dimension] = {} 151 | ddb_tables[aas_table_name]['autoscaling'][aas_scalable_dimension]['min'] = target['MinCapacity'] 152 | ddb_tables[aas_table_name]['autoscaling'][aas_scalable_dimension]['max'] = target['MaxCapacity'] 153 | #tmp_dsp_resources.append({'ResourceId': target['ResourceId'], 'ScalableDimension': target['ScalableDimension'], 'type': 'table'}) 154 | dsp_resource_ids[target['ResourceId']] = {'type' : 'table', 'table_name' : aas_table_name} 155 | elif target['ScalableDimension'].startswith('dynamodb:index:'): 156 | # Slice off the leading "table/
/index/" from the ResourceId 157 | # ResourceId = "table/
/index/" 158 | aas_table_name = target['ResourceId'].split('/')[1] 159 | aas_index_name = target['ResourceId'].split('/')[3] 160 | #aas_index_name = target['ResourceId'][len("table/" + table + "/index/"):] 161 | # Slice off the leading "dynamodb:index:" from the Scalable Dimension 162 | aas_scalable_dimension = target['ScalableDimension'][len("dynamodb:index:"):] 163 | ddb_tables[aas_table_name]['gsis'][aas_index_name]['autoscaling'][aas_scalable_dimension] = {} 164 | ddb_tables[aas_table_name]['gsis'][aas_index_name]['autoscaling'][aas_scalable_dimension]['min'] = target['MinCapacity'] 165 | ddb_tables[aas_table_name]['gsis'][aas_index_name]['autoscaling'][aas_scalable_dimension]['max'] = target['MaxCapacity'] 166 | #tmp_dsp_resources.append({'ResourceId': target['ResourceId'], 'ScalableDimension': target['ScalableDimension'], 'type': 'index'}) 167 | dsp_resource_ids[target['ResourceId']] = {'type' : 'index', 'table_name' : aas_table_name, 'index_name' : aas_index_name} 168 | else: 169 | raise Exception(f"unknown ScalableDimension {target['ScalableDimension']}") 170 | 171 | for dsp_resource_id in dsp_resource_ids.keys(): 172 | aas_dsp_paginator = aas.get_paginator('describe_scaling_policies') 173 | for aas_policy_response in aas_dsp_paginator.paginate(ServiceNamespace='dynamodb', ResourceId=dsp_resource_id): 174 | for policy in aas_policy_response['ScalingPolicies']: 175 | if 'table' == dsp_resource_ids[dsp_resource_id]['type']: 176 | # Slice off the leading "dynamodb:table:" from the Scalable Dimension 177 | aas_scalable_dimension = policy['ScalableDimension'][len("dynamodb:table:"):] 178 | ddb_tables[dsp_resource_ids[dsp_resource_id]['table_name']]['autoscaling'][aas_scalable_dimension]['target'] = policy['TargetTrackingScalingPolicyConfiguration']['TargetValue'] 179 | elif 'index' == dsp_resource_ids[dsp_resource_id]['type']: 180 | # Slice off the leading "dynamodb:index:" from the Scalable Dimension 181 | aas_scalable_dimension = policy['ScalableDimension'][len("dynamodb:index:"):] 182 | ddb_tables[dsp_resource_ids[dsp_resource_id]['table_name']]['gsis'][dsp_resource_ids[dsp_resource_id]['index_name']]['autoscaling'][aas_scalable_dimension]['target'] = aas_policy_response['ScalingPolicies'][0]['TargetTrackingScalingPolicyConfiguration']['TargetValue'] 183 | else: 184 | raise Exception(f"unknown resource type {dsp_resource_id['type']}") 185 | 186 | def gather_dynamodb_metrics(event, context): 187 | global ddb_tables 188 | global ddb_total_consumed_rcu 189 | global ddb_total_consumed_wcu 190 | 191 | for table in ddb_tables.keys(): 192 | ddb_tables[table]['metrics'] = {} 193 | # paginator = cloudwatch.get_paginator('list_metrics') 194 | # for response in paginator.paginate(Dimensions=[{'Name': 'TableName','Value': table}], 195 | # Namespace='AWS/DynamoDB'): 196 | # print(response['Metrics']) 197 | response = cloudwatch.get_metric_data( 198 | MetricDataQueries=[ 199 | { 200 | 'Id' : 'consumed_rcu', 201 | 'MetricStat': { 202 | 'Metric': { 203 | 'Namespace': 'AWS/DynamoDB', 204 | 'MetricName': 'ConsumedReadCapacityUnits', 205 | 'Dimensions': [{'Name': 'TableName', 'Value': table}] 206 | }, 207 | 'Period': FIVE_MINS_SECS, 208 | 'Stat': 'Average', 209 | 'Unit': 'Count' 210 | }, 211 | }, 212 | { 213 | 'Id' : 'consumed_wcu', 214 | 'MetricStat': { 215 | 'Metric': { 216 | 'Namespace': 'AWS/DynamoDB', 217 | 'MetricName': 'ConsumedWriteCapacityUnits', 218 | 'Dimensions': [{'Name': 'TableName', 'Value': table}] 219 | }, 220 | 'Period': FIVE_MINS_SECS, 221 | 'Stat': 'Average', 222 | 'Unit': 'Count' 223 | } 224 | } 225 | ], 226 | StartTime=datetime.datetime.now() - datetime.timedelta(minutes=15), 227 | EndTime=datetime.datetime.now(), 228 | MaxDatapoints=5 229 | ) 230 | for result in response['MetricDataResults']: 231 | ddb_tables[table]['metrics'][result['Id']] = 0.0 232 | if len(result['Values']) > 0: 233 | ddb_tables[table]['metrics'][result['Id']] = result['Values'][0] 234 | #print(response) 235 | 236 | def publish_dynamodb_account_metrics(event, context): 237 | global ddb_tables 238 | global ddb_account_limits 239 | 240 | cloudwatch.put_metric_data( 241 | MetricData=[ 242 | { 243 | 'MetricName': 'AccountTableLimitPct', 244 | 'Unit': 'Percent', 245 | 'Value': len(ddb_tables.keys()) / ddb_account_limits['AccountMaxTables'] 246 | } 247 | ], 248 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 249 | ) 250 | 251 | def publish_dynamodb_provisioned_table_metrics(table, event, context): 252 | global ddb_tables 253 | global ddb_account_limits 254 | 255 | if ddb_tables[table]['autoscaling']['ReadCapacityUnits'] is not None: 256 | cloudwatch.put_metric_data( 257 | MetricData=[ 258 | { 259 | 'MetricName': 'ProvisionedReadCapacityAutoScalingPct', 260 | 'Dimensions': [{'Name': 'TableName', 'Value': table}], 261 | 'Unit': 'Percent', 262 | 'Value': ddb_tables[table]['ProvisionedThroughput']['ReadCapacityUnits'] / ddb_tables[table]['autoscaling']['ReadCapacityUnits']['max'] 263 | } 264 | ], 265 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 266 | ) 267 | 268 | if ddb_tables[table]['autoscaling']['WriteCapacityUnits'] is not None: 269 | cloudwatch.put_metric_data( 270 | MetricData=[ 271 | { 272 | 'MetricName': 'ProvisionedWriteCapacityAutoScalingPct', 273 | 'Dimensions': [{'Name': 'TableName', 'Value': table}], 274 | 'Unit': 'Percent', 275 | 'Value': ddb_tables[table]['ProvisionedThroughput']['WriteCapacityUnits'] / ddb_tables[table]['autoscaling']['WriteCapacityUnits']['max'] 276 | } 277 | ], 278 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 279 | ) 280 | 281 | for gsi in ddb_tables[table]['gsis'].keys(): 282 | if ddb_tables[table]['gsis'][gsi]['autoscaling']['ReadCapacityUnits'] is not None: 283 | cloudwatch.put_metric_data( 284 | MetricData=[ 285 | { 286 | 'MetricName': 'ProvisionedReadCapacityAutoScalingPct', 287 | 'Dimensions': [{'Name': 'GlobalSecondaryIndexName', 'Value': gsi}, {'Name': 'TableName', 'Value': table}], 288 | 'Unit': 'Percent', 289 | 'Value': ddb_tables[table]['gsis'][gsi]['ProvisionedThroughput']['ReadCapacityUnits'] / ddb_tables[table]['autoscaling']['ReadCapacityUnits']['max'] 290 | } 291 | ], 292 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 293 | ) 294 | 295 | if ddb_tables[table]['gsis'][gsi]['autoscaling']['WriteCapacityUnits'] is not None: 296 | cloudwatch.put_metric_data( 297 | MetricData=[ 298 | { 299 | 'MetricName': 'ProvisionedWriteCapacityAutoScalingPct', 300 | 'Dimensions': [{'Name': 'GlobalSecondaryIndexName', 'Value': gsi}, {'Name': 'TableName', 'Value': table}], 301 | 'Unit': 'Percent', 302 | 'Value': ddb_tables[table]['gsis'][gsi]['ProvisionedThroughput']['WriteCapacityUnits'] / ddb_tables[table]['autoscaling']['WriteCapacityUnits']['max'] 303 | } 304 | ], 305 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 306 | ) 307 | 308 | def publish_dynamodb_ondemand_table_metrics(table, event, context): 309 | global ddb_tables 310 | global ddb_account_limits 311 | 312 | cloudwatch.put_metric_data( 313 | MetricData=[ 314 | { 315 | 'MetricName': 'ConsumedReadCapacityTableLimitPct', 316 | 'Dimensions': [{'Name': 'TableName', 'Value': table}], 317 | 'Unit': 'Percent', 318 | 'Value': ddb_tables[table]['ProvisionedThroughput']['ReadCapacityUnits'] / ddb_account_limits['TableMaxReadCapacityUnits'] 319 | } 320 | ], 321 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 322 | ) 323 | 324 | cloudwatch.put_metric_data( 325 | MetricData=[ 326 | { 327 | 'MetricName': 'ConsumedWriteCapacityTableLimitPct', 328 | 'Dimensions': [{'Name': 'TableName', 'Value': table}], 329 | 'Unit': 'Percent', 330 | 'Value': ddb_tables[table]['ProvisionedThroughput']['WriteCapacityUnits'] / ddb_account_limits['TableMaxWriteCapacityUnits'] 331 | } 332 | ], 333 | Namespace=CLOUDWATCH_CUSTOM_NAMESPACE 334 | ) 335 | 336 | def publish_dynamodb_table_metrics(event, context): 337 | global ddb_tables 338 | global ddb_account_limits 339 | 340 | for table in ddb_tables.keys(): 341 | if ddb_tables[table]['BillingMode'] == 'PROVISIONED': 342 | publish_dynamodb_provisioned_table_metrics(table, event, context) 343 | elif ddb_tables[table]['BillingMode'] == 'PAY_PER_REQUEST': 344 | publish_dynamodb_ondemand_table_metrics(table, event, context) 345 | else: 346 | raise Exception(f"Unknown billing mode {ddb_tables[table]['BillingMode']} for table {table}") 347 | 348 | def publish_dynamodb_metrics(event, context): 349 | global ddb_tables 350 | global ddb_account_limits 351 | global ddb_total_provisioned_rcu 352 | global ddb_total_provisioned_wcu 353 | load_dynamodb_limits(event, context) 354 | load_dynamodb_tables(event, context) 355 | gather_dynamodb_consumption(event, context) 356 | gather_dynamodb_metrics(event, context) 357 | 358 | # can't use this because sometimes timestamps show up under ProvisionedThroughput.LastIncreaseDateTime 359 | #print(json.dumps(ddb_tables, sort_keys=True, indent=4, separators=(',', ': '))) 360 | #print(ddb_tables) 361 | print(f"Using {len(ddb_tables.keys())} of max {ddb_account_limits['AccountMaxTables']} tables") 362 | print(f"DynamoDB AccountMaxReadCapacityUnits: {ddb_account_limits['AccountMaxReadCapacityUnits']}") 363 | print(f"DynamoDB AccountMaxWriteCapacityUnits: {ddb_account_limits['AccountMaxWriteCapacityUnits']}") 364 | print(f"DynamoDB TableMaxReadCapacityUnits: {ddb_account_limits['TableMaxReadCapacityUnits']}") 365 | print(f"DynamoDB TableMaxWriteCapacityUnits: {ddb_account_limits['TableMaxWriteCapacityUnits']}") 366 | print(f"DynamoDB Total Provisioned RCU: {ddb_total_provisioned_rcu}") 367 | print(f"DynamoDB Total Provisioned WCU: {ddb_total_provisioned_wcu}") 368 | 369 | publish_dynamodb_account_metrics(event, context) 370 | publish_dynamodb_table_metrics(event, context) 371 | 372 | return success_response(event, context) 373 | 374 | if __name__ == "__main__": 375 | response = publish_dynamodb_metrics({}, {}) 376 | print(f'{response}') 377 | -------------------------------------------------------------------------------- /dynamodb_metrics_lambda/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3 2 | retrying 3 | -------------------------------------------------------------------------------- /dynamodb_metrics_lambda/run_shell.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | 7 | pip install -t /tmp/src/dynamodb_metrics_lambda/vendored/ -r /tmp/src/dynamodb_metrics_lambda/requirements.txt 8 | 9 | echo "Run 'docker exec -it run_shell_dynamodb_metrics_lambda /bin/bash'" 10 | echo "Press [CTRL+C] to stop.." 11 | while true 12 | do 13 | sleep 1 14 | done 15 | -------------------------------------------------------------------------------- /dynamodb_metrics_lambda/serverless.yml: -------------------------------------------------------------------------------- 1 | service: dynamodb-monitoring-test 2 | 3 | frameworkVersion: ">=1.1.0 <2.0.0" 4 | 5 | provider: 6 | name: aws 7 | region: ${opt:region} 8 | runtime: python3.7 9 | timeout: 60 # 1 minute 10 | iamRoleStatements: 11 | - Effect: Allow 12 | Action: 13 | - dynamodb:DescribeLimits 14 | - dynamodb:DescribeTable 15 | - dynamodb:ListTables 16 | - autoscaling:Describe* 17 | - application-autoscaling:Describe* 18 | - cloudwatch:GetMetricData 19 | - cloudwatch:PutMetricData 20 | Resource: "*" 21 | 22 | resources: 23 | Resources: 24 | MonitoringDynamoDbTable: 25 | Type: 'AWS::DynamoDB::Table' 26 | #DeletionPolicy: Retain 27 | Properties: 28 | AttributeDefinitions: 29 | - 30 | AttributeName: id 31 | AttributeType: S 32 | KeySchema: 33 | - 34 | AttributeName: id 35 | KeyType: HASH 36 | BillingMode: PROVISIONED 37 | ProvisionedThroughput: 38 | ReadCapacityUnits: 5 39 | WriteCapacityUnits: 5 40 | TableName: dynamodb-monitoring-test 41 | StreamSpecification: 42 | StreamViewType: NEW_AND_OLD_IMAGES 43 | MonitoringDynamoDbTableUEAlarm: 44 | Type: 'AWS::CloudWatch::Alarm' 45 | Properties: 46 | AlarmName: !Join ['', [!Ref MonitoringDynamoDbTable, '-ue-alarm']] 47 | AlarmDescription: 'Alarm when DynamoDB user errors occur' 48 | Namespace: 'AWS/DynamoDB' 49 | MetricName: 'UserErrors' 50 | Dimensions: 51 | - 52 | Name: 'TableName' 53 | Value: !Ref MonitoringDynamoDbTable 54 | Statistic: 'Sum' 55 | Threshold: 0 56 | ComparisonOperator: 'GreaterThanThreshold' 57 | Period: 60 58 | Unit: 'Count' 59 | EvaluationPeriods: 1 60 | MonitoringDynamoDbTableSEAlarm: 61 | Type: 'AWS::CloudWatch::Alarm' 62 | Properties: 63 | AlarmName: !Join ['', [!Ref MonitoringDynamoDbTable, '-se-alarm']] 64 | AlarmDescription: 'Alarm when DynamoDB system errors occur' 65 | Namespace: 'AWS/DynamoDB' 66 | MetricName: 'SystemErrors' 67 | Dimensions: 68 | - 69 | Name: 'TableName' 70 | Value: !Ref MonitoringDynamoDbTable 71 | Statistic: 'Sum' 72 | Threshold: 0 73 | ComparisonOperator: 'GreaterThanThreshold' 74 | Period: 60 75 | Unit: 'Count' 76 | EvaluationPeriods: 1 77 | 78 | functions: 79 | dynamodb-cloudwatch-publish-metrics: 80 | handler: dynamodb_cloudwatch.publish_dynamodb_metrics 81 | environment: 82 | TARGET_DDB_TABLE: !Ref MonitoringDynamoDbTable 83 | events: 84 | - schedule: rate(1 minute) 85 | -------------------------------------------------------------------------------- /dynamodb_metrics_lambda/undeploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | 4 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 5 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 6 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 7 | 8 | cd /tmp/src/dynamodb_metrics_lambda; serverless remove --region $AWS_DEFAULT_REGION || exit 1 9 | -------------------------------------------------------------------------------- /run_shell.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | 7 | trap "docker-compose -f docker-compose.yml rm --force run_shell_dynamodb_metrics_lambda" SIGINT SIGTERM 8 | docker-compose -f docker-compose.yml build --no-cache run_shell_dynamodb_metrics_lambda && \ 9 | docker-compose -f docker-compose.yml up run_shell_dynamodb_metrics_lambda 10 | docker-compose -f docker-compose.yml rm --force run_shell_dynamodb_metrics_lambda 11 | -------------------------------------------------------------------------------- /undeploy_cf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | 7 | trap "docker-compose -f docker-compose.yml rm --force undeploy_dynamodb_alarms_cf" SIGINT SIGTERM 8 | docker-compose -f docker-compose.yml build --no-cache undeploy_dynamodb_alarms_cf 9 | docker-compose -f docker-compose.yml up undeploy_dynamodb_alarms_cf 10 | docker-compose -f docker-compose.yml rm --force undeploy_dynamodb_alarms_cf 11 | -------------------------------------------------------------------------------- /undeploy_lambda.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | : "${AWS_ACCESS_KEY_ID:?Need to set AWS_ACCESS_KEY_ID non-empty}" 4 | : "${AWS_SECRET_ACCESS_KEY:?Need to set AWS_SECRET_ACCESS_KEY non-empty}" 5 | : "${AWS_DEFAULT_REGION:?Need to set AWS_DEFAULT_REGION non-empty}" 6 | 7 | trap "docker-compose -f docker-compose.yml rm --force undeploy_dynamodb_metrics_lambda" SIGINT SIGTERM 8 | docker-compose -f docker-compose.yml build --no-cache undeploy_dynamodb_metrics_lambda && \ 9 | docker-compose -f docker-compose.yml up undeploy_dynamodb_metrics_lambda 10 | docker-compose -f docker-compose.yml rm --force undeploy_dynamodb_metrics_lambda 11 | --------------------------------------------------------------------------------