├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── CloudWatchAlarmFormattedEmail-ArchitectureDiagram.png ├── CloudWatchAlarmFormattedEmail-SampleOutput.png ├── LICENSE ├── README.md ├── cloudformation └── cwalarm-formatted-email.cfn.json └── src └── cwalarm-formatted-email-lambda.py /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /CloudWatchAlarmFormattedEmail-ArchitectureDiagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-alarm-formatted-email/ee32d78f6e39e544d6da4d4e1aafdf06523a8f00/CloudWatchAlarmFormattedEmail-ArchitectureDiagram.png -------------------------------------------------------------------------------- /CloudWatchAlarmFormattedEmail-SampleOutput.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-alarm-formatted-email/ee32d78f6e39e544d6da4d4e1aafdf06523a8f00/CloudWatchAlarmFormattedEmail-SampleOutput.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CloudWatchAlarmFormattedEmail - Formatting CloudWatch alarms email notifications using SES 2 | 3 | This solution enables you to customize CloudWatch alarm email alerts using Amazon Simple Email Service (Amazon SES) and AWS Lambda. You will be able to extract the relevant bits of information that you need from the alarm message and display them in an easy-to-read HTML format. 4 | 5 |   6 | 7 | # Architecture 8 | 9 | ![Architecture Diagram](./CloudWatchAlarmFormattedEmail-ArchitectureDiagram.png) 10 | 11 | The diagram shows the following workflow: 12 | 13 | 1. Amazon CloudWatch alarm state changes and sends notification alert to the associated SNS topic. 14 | 2. Amazon SNS will invoke the subscribed Lambda function. 15 | 3. Lambda function will parse the alarm message and call the Amazon SES API. 16 | 4. Amazon SES will apply the HTML formatting based on a specified template and send the formatted email. 17 | 18 |   19 | 20 | # Requirements 21 | 22 | 1. AWS account & IAM User or Role 23 | 2. [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed. 24 | 25 | 3. A verified [Amazon SES identity](https://docs.aws.amazon.com/ses/latest/dg/verify-addresses-and-domains.html) which will be used as the sender email. 26 | 27 |   28 | # Deployment 29 | 1. Clone the **amazon-cloudwatch-alarm-formatted-email** github repository to your computer using the following command: 30 | ``` 31 | git clone https://github.com/aws-samples/amazon-cloudwatch-alarm-formatted-email 32 | ``` 33 | 34 | 2. Configure the [AWS credential keys](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) to authenticate and authorize the AWS CLI commands on your account. 35 | 36 | 3. Verify an email identity in Amazon SES. This is the email address that you will use as sender: 37 | ``` 38 | aws ses verify-email-identity --email-address 39 | ``` 40 | After issuing this command, you should recive an email for the verification request. You will need to click on the URL link to confirm that you are authorized to use this email address. 41 | 42 | 4. [Create an S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) that will be used to store and access the CWAlarmFormattedEmail Lambda function deployment package if you don't already have one. 43 | 44 | 5. Create a zip file containing the CWAlarmFormattedEmail Lambda function code located in the [src](./src/) directory. This is the deployment package of the AWS Lambda function. 45 | 46 | For MAC: 47 | ``` 48 | zip -j amazon-cloudwatch-alarm-formatted-email.zip src/* 49 | ``` 50 | 51 | For Windows (Powershell): 52 | ``` 53 | Compress-Archive ./src/* amazon-cloudwatch-alarm-formatted-email.zip 54 | ``` 55 | 56 | 57 | 6. Copy the **amazon-cloudwatch-alarm-formatted-email.zip** file to your S3 bucket: 58 | ``` 59 | aws s3 cp amazon-cloudwatch-alarm-formatted-email.zip s3:// 60 | ``` 61 | 62 | 7. Deploy the CloudFormation stack that will create the: 63 | - AWS Lambda function along with its IAM execution role 64 | - Amazon SES HTML template email 65 | - Amazon SNS topic along with its topic policy 66 | 67 |   68 | 69 | ``` 70 | aws cloudformation create-stack --stack-name amazon-cloudwatch-alarm-formatted-email \ 71 | --template-body file://cloudformation/cwalarm-formatted-email.cfn.json \ 72 | --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND \ 73 | --parameters ParameterKey=pS3DeploymentBucket,ParameterValue= \ 74 | ParameterKey=pS3DeploymentKey,ParameterValue= \ 75 | ParameterKey=pEmailSource,ParameterValue= \ 76 | ParameterKey=pEmailToAddress,ParameterValue= \ 77 | ParameterKey=pEmailCcAddress,ParameterValue= \ 78 | ParameterKey=pEmailReplyToAddress,ParameterValue= \ 79 | --region 80 | ``` 81 | # Usage 82 | After the deployment, you should be able to test out the soltuion by creating a [CloudWatch Alarm based on EC2 metric](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-createalarm.html) and associating it to the SNS topic deployed by the CloudFormation stack. 83 | 84 | Once the alarm is triggered, you should receive an HTML formatted email notification to the address specified in the ```pEmailToAddress``` CloudFormation paramter. 85 | 86 | **Note**: Although most of the time each message will be delivered to your application exactly once, the distributed nature of Amazon SNS and transient network conditions could result in occasional, duplicate messages at the subscriber end. 87 | 88 | 89 |   90 | 91 | # Sample Output 92 | 93 | ![Samples Output](./CloudWatchAlarmFormattedEmail-SampleOutput.png) 94 |   95 | 96 | # Customization 97 | 98 | To change how the email looks, you need to update the HTML code in the ```HtmlPart``` property found in the Cloudformation stack [cwalarm-formatted-email.cfn.json](./cloudformation/cwalarm-formatted-email.cfn.json) 99 | 100 | The CWAlarmFormattedEmail Lambda function extracts some common values from the alarm event message (account-id, region etc.) and an EC2 specific value (instance-id). 101 | To use the solution with non-EC2 metrics, make sure to update the Lambda function code. 102 | 103 |   104 | 105 | # Security 106 | 107 | Amazon SNS provides in-transit encryption by default. It is recommended to also enable server side encryption using AWS KMS [Customer managed CMK](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html). 108 | 109 | 110 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 111 | 112 |   113 | 114 | # License 115 | 116 | This library is licensed under the MIT-0 License. See the LICENSE file. 117 | 118 | -------------------------------------------------------------------------------- /cloudformation/cwalarm-formatted-email.cfn.json: -------------------------------------------------------------------------------- 1 | { 2 | "AWSTemplateFormatVersion": "2010-09-09", 3 | "Description": "AWS CloudFormation Template for formatted CloudWatch alarm email notification", 4 | "Transform": "AWS::Serverless-2016-10-31", 5 | "Parameters": { 6 | "pS3DeploymentBucket": { 7 | "Type": "String", 8 | "Description": "S3 bucket that contains Lambda zip package" 9 | }, 10 | "pS3DeploymentKey": { 11 | "Type": "String", 12 | "Description": "S3 key of the Lambda zip package" 13 | }, 14 | "pEmailSource": { 15 | "Type": "String", 16 | "Description": "Sender email address. Should be a verified identity in SES" 17 | }, 18 | "pEmailToAddress": { 19 | "Type": "String", 20 | "Description": "Receiver (To:) email address" 21 | }, 22 | "pEmailCcAddress": { 23 | "Type": "String", 24 | "Description": "Receiver (Cc:) email address" 25 | }, 26 | "pEmailReplyToAddress": { 27 | "Type": "String", 28 | "Description": "Reply To email address" 29 | } 30 | }, 31 | "Metadata": { 32 | "AWS::CloudFormation::Interface": { 33 | "ParameterGroups": [ 34 | { 35 | "Label": { 36 | "default": "AWS Lambda Configuration" 37 | }, 38 | "Parameters": [ 39 | "pS3DeploymentBucket", 40 | "pS3DeploymentKey" 41 | ] 42 | }, 43 | { 44 | "Label": { 45 | "default": "Amazon SES Configuration" 46 | }, 47 | "Parameters": [ 48 | "pEmailSource", 49 | "pEmailToAddress", 50 | "pEmailCcAddress", 51 | "pEmailReplyToAddress" 52 | ] 53 | } 54 | ], 55 | "ParameterLabels": {} 56 | } 57 | }, 58 | "Resources": { 59 | "rFormattedEmailLambdaFunction": { 60 | "Type": "AWS::Serverless::Function", 61 | "Properties": { 62 | "Handler": "cwalarm-formatted-email-lambda.lambda_handler", 63 | "Runtime": "python3.8", 64 | "Description": "CloudWatch alarms email formatter", 65 | "Environment": { 66 | "Variables": { 67 | "EMAIL_SOURCE": { 68 | "Ref": "pEmailSource" 69 | }, 70 | "EMAIL_TO_ADDRESSES": { 71 | "Ref": "pEmailToAddress" 72 | }, 73 | "EMAIL_CC_ADDRESSES": { 74 | "Ref": "pEmailCcAddress" 75 | }, 76 | "EMAIL_REPLY_TO_ADRESSES": { 77 | "Ref": "pEmailReplyToAddress" 78 | }, 79 | "SES_TEMPLATE_CRITICAL": "TemplateCritical" 80 | } 81 | }, 82 | "CodeUri": { 83 | "Bucket": { 84 | "Ref": "pS3DeploymentBucket" 85 | }, 86 | "Key": { 87 | "Ref": "pS3DeploymentKey" 88 | } 89 | }, 90 | "Timeout": 60, 91 | "Role": { 92 | "Fn::GetAtt": [ 93 | "rFunctionRole", 94 | "Arn" 95 | ] 96 | } 97 | } 98 | }, 99 | "rFunctionRole": { 100 | "Type": "AWS::IAM::Role", 101 | "Properties": { 102 | "Path": "/", 103 | "AssumeRolePolicyDocument": { 104 | "Version": "2012-10-17", 105 | "Statement": [ 106 | { 107 | "Effect": "Allow", 108 | "Principal": { 109 | "Service": "lambda.amazonaws.com" 110 | }, 111 | "Action": [ 112 | "sts:AssumeRole" 113 | ] 114 | } 115 | ] 116 | }, 117 | "Policies": [ 118 | { 119 | "PolicyName": "Lambda_Permissions_SES", 120 | "PolicyDocument": { 121 | "Version": "2012-10-17", 122 | "Statement": [ 123 | { 124 | "Effect": "Allow", 125 | "Action": [ 126 | "ses:SendEmail", 127 | "ses:SendTemplatedEmail", 128 | "ses:SendRawEmail" 129 | ], 130 | "Resource":{ 131 | "Fn::Sub": "arn:${AWS::Partition}:ses:${AWS::Region}:${AWS::AccountId}:identity/*" 132 | 133 | } 134 | } 135 | ] 136 | } 137 | }, 138 | { 139 | "PolicyName": "Lambda_Permissions_CWLogs", 140 | "PolicyDocument": { 141 | "Version": "2012-10-17", 142 | "Statement": [ 143 | { 144 | "Effect": "Allow", 145 | "Action": [ 146 | "logs:CreateLogGroup", 147 | "logs:CreateLogStream", 148 | "logs:PutLogEvents", 149 | "logs:DescribeLogGroups" 150 | ], 151 | "Resource": { 152 | "Fn::Sub": "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:*" 153 | } } 154 | ] 155 | } 156 | } 157 | ] 158 | } 159 | }, 160 | "rLambdaFunctionPermission": { 161 | "Type": "AWS::Lambda::Permission", 162 | "Properties": { 163 | "FunctionName": { 164 | "Fn::GetAtt": [ 165 | "rFormattedEmailLambdaFunction", 166 | "Arn" 167 | ] 168 | }, 169 | "Action": "lambda:InvokeFunction", 170 | "Principal": "sns.amazonaws.com", 171 | "SourceArn": { 172 | "Ref": "rCwAlarmSns" 173 | } 174 | } 175 | }, 176 | "rCwAlarmSns": { 177 | "Type": "AWS::SNS::Topic", 178 | "Properties": { 179 | "Subscription": [ 180 | { 181 | "Protocol": "lambda", 182 | "Endpoint": { 183 | "Fn::GetAtt": [ 184 | "rFormattedEmailLambdaFunction", 185 | "Arn" 186 | ] 187 | } 188 | } 189 | ] 190 | } 191 | }, 192 | "rCwAlarmSnsPolicy": { 193 | "Type": "AWS::SNS::TopicPolicy", 194 | "Properties": { 195 | "PolicyDocument": { 196 | "Version": "2012-10-17", 197 | "Statement": [ 198 | { 199 | "Effect": "Allow", 200 | "Principal": { 201 | "Service": [ 202 | "cloudwatch.amazonaws.com" 203 | ] 204 | }, 205 | "Action": "SNS:Publish", 206 | "Resource": [ 207 | { 208 | "Ref": "rCwAlarmSns" 209 | } 210 | ] 211 | } 212 | ] 213 | }, 214 | "Topics": [ 215 | { 216 | "Ref": "rCwAlarmSns" 217 | } 218 | ] 219 | } 220 | }, 221 | "rTemplateCritical": { 222 | "Type": "AWS::SES::Template", 223 | "Properties": { 224 | "Template": { 225 | "TemplateName": "TemplateCritical", 226 | "SubjectPart": "CRITICAL Alarm on {{alarm}}", 227 | "HtmlPart": "

⚠Your Amazon CloudWatch alarm was triggered

ImpactCritical
Alarm Name{{alarm}}
Account

{{account}} {{region}})

Instance-id{{instanceId}}
Date-Time{{datetime}}
ReasonCurrent value {{value}} is {{comparisonoperator}} {{threshold}}
" 228 | } 229 | } 230 | } 231 | } 232 | } -------------------------------------------------------------------------------- /src/cwalarm-formatted-email-lambda.py: -------------------------------------------------------------------------------- 1 | import json 2 | import boto3 3 | import os 4 | import logging 5 | 6 | logger = logging.getLogger() 7 | logger.setLevel(logging.INFO) 8 | 9 | 10 | def lambda_handler(event, context): 11 | 12 | logger.info(f'event received: {event}') 13 | 14 | # Parse cloudwatch alarm JSON message 15 | message = json.loads(event['Records'][0]['Sns']['Message']) 16 | alarm_name = message['AlarmName'] 17 | aws_account_id = message['AWSAccountId'] 18 | region = message['Region'] 19 | threshold = message['Trigger']['Threshold'] 20 | new_state_reason = message['NewStateReason'] 21 | state_change_time_full = message['StateChangeTime'] 22 | state_change_time = state_change_time_full.split(".", 1)[0] 23 | dimensions = message['Trigger']['Dimensions'] 24 | comparison_operator = message['Trigger']['ComparisonOperator'] 25 | metric_current_value = new_state_reason[new_state_reason.find("[")+1:new_state_reason.find("[")+5] 26 | 27 | # Get instance-id from metric dimension 28 | try: 29 | dimension = next(item for item in dimensions if item['name'] == 'InstanceId') 30 | instance_id = dimension['value'] 31 | except Exception as e: 32 | logger.error(f'Could not find EC2 instance-id in alarm event. Make sure Alarm is based on EC2 metric') 33 | raise e 34 | 35 | # Get SES configuration from lambda env variables 36 | email_template = os.environ['SES_TEMPLATE_CRITICAL'] 37 | source_email = os.environ['EMAIL_SOURCE'] 38 | destination_to_email = os.environ['EMAIL_TO_ADDRESSES'] 39 | destination_cc_email = os.environ['EMAIL_CC_ADDRESSES'] 40 | reply_to_email = os.environ['EMAIL_REPLY_TO_ADRESSES'] 41 | 42 | # Read email addresses and send SES templated email 43 | ses = boto3.client('ses') 44 | try: 45 | response = ses.send_templated_email( 46 | Source=source_email, 47 | Destination={ 48 | 'ToAddresses': [ 49 | destination_to_email, 50 | ], 51 | 'CcAddresses': [ 52 | destination_cc_email, 53 | ] 54 | }, 55 | ReplyToAddresses=[ 56 | reply_to_email, 57 | ], 58 | Template=email_template, 59 | TemplateData='{ \"alarm\":\"'+alarm_name+'\", \"reason\": \"'+new_state_reason+'\", \"account\": \"'+aws_account_id+'\", \"region\": \"'+region+'\", \"datetime\": \"'+state_change_time+'\", \"instanceId\": \"'+instance_id+'\", \"value\": \"'+str(metric_current_value)+'\", \"comparisonoperator\": \"'+comparison_operator+'\", \"threshold\": \"'+str(threshold)+'\" }' 60 | ) 61 | except Exception as e: 62 | logger.error(f'Could not send SES templated email: {e}') 63 | raise 64 | else: 65 | logger.info(f'Alarm email notification successfully sent to {destination_to_email}') 66 | --------------------------------------------------------------------------------