├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── automating-kanban-workflows ├── .gitignore ├── README.md ├── app.py ├── architecture.png ├── automating_kanban_workflows │ ├── __init__.py │ ├── automating_kanban_workflows_stack.py │ └── functions │ │ ├── jira_split_into_subtasks │ │ ├── __init__.py │ │ ├── index.py │ │ └── requirements.txt │ │ ├── jira_task_description_review │ │ ├── __init__.py │ │ ├── index.py │ │ └── requirements.txt │ │ └── trigger_kanban_automation_workflow │ │ ├── __init__.py │ │ └── index.py ├── cdk.json └── requirements.txt ├── improving-code-quality-reviews ├── .gitignore ├── README.md ├── app.py ├── architecture.png ├── cdk.json ├── improving_code_quality_reviews │ ├── __init__.py │ ├── functions │ │ └── run_github_code_review │ │ │ ├── __init__.py │ │ │ ├── index.py │ │ │ └── requirements.txt │ └── improving_code_quality_reviews_stack.py └── requirements.txt └── streamline-incident-response ├── .gitignore ├── README.md ├── app.py ├── architecture.png ├── cdk.json ├── requirements.txt └── streamline_incident_response ├── __init__.py ├── functions ├── chatbot_trigger_generate_report │ ├── __init__.py │ ├── events │ │ └── example_event.json │ └── index.py ├── chatbot_trigger_search_previous_incidents │ ├── __init__.py │ ├── events │ │ └── example_event.json │ └── index.py ├── create_markdown_report │ ├── __init__.py │ └── index.py ├── lookup_cloudtrail_events │ ├── __init__.py │ └── index.py ├── lookup_slack_events │ ├── __init__.py │ ├── index.py │ └── requirements.txt ├── start_ingestion_job │ ├── __init__.py │ └── index.py └── upload_markdown_report │ ├── __init__.py │ └── index.py └── streamline_incident_response_stack.py /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .venv 3 | 4 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # GenAI for DevOps 2 | 3 | This repository showcases three separate solutions demonstrating how DevOps can leverage Generative AI (GenAI) to solve common challenges. Each solution is designed to work independently and has its own detailed README. 4 | 5 | The three solutions in this repository are: 6 | 7 | 1. Automating Kanban Workflows 8 | 2. Improving Code Quality Reviews 9 | 3. Streamlining Incident Response 10 | 11 | Each solution demonstrates a unique application of GenAI in DevOps practices, enhancing efficiency and effectiveness in software development and operations. 12 | 13 | ## Usage Instructions 14 | 15 | ### Installation 16 | 17 | Prerequisites: 18 | - Python 3.12 or later 19 | - AWS CDK CLI v2.x 20 | - AWS CLI configured with appropriate credentials 21 | 22 | ### Configuration 23 | 24 | Each solution requires specific configuration. Please refer to the README in each solution's directory for detailed configuration instructions. 25 | 26 | ### Integration 27 | 28 | For integration details specific to each solution, please consult the individual README files in the respective solution directories. 29 | 30 | ## Security 31 | 32 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 33 | 34 | ## License 35 | 36 | This library is licensed under the MIT-0 License. See the LICENSE file. 37 | 38 | -------------------------------------------------------------------------------- /automating-kanban-workflows/.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | package-lock.json 3 | __pycache__ 4 | .pytest_cache 5 | .venv 6 | *.egg-info 7 | 8 | # CDK asset staging directory 9 | .cdk.staging 10 | cdk.out 11 | -------------------------------------------------------------------------------- /automating-kanban-workflows/README.md: -------------------------------------------------------------------------------- 1 | # Automating Kanban Workflows with AWS and Jira 2 | 3 | This project implements an automated Kanban workflow system that integrates AWS services with Jira for enhanced task management and productivity. 4 | 5 | The Automating Kanban Workflows project leverages AWS CDK to create a serverless architecture that automates key aspects of Kanban board management in Jira. It uses AWS Lambda functions, Step Functions, and Amazon SNS to review task descriptions, split tasks into subtasks, and manage the workflow between different stages of the Kanban process. 6 | 7 | ![Architecture Diagram](architecture.png) 8 | 9 | ## Repository Structure 10 | 11 | - `app.py`: The entry point for the CDK application. 12 | - `automating_kanban_workflows/`: Main package containing the CDK stack and Lambda functions. 13 | - `automating_kanban_workflows_stack.py`: Defines the AWS resources using CDK. 14 | - `functions/`: Contains the Lambda function code for different workflow steps. 15 | - `cdk.json`: Configuration file for the CDK application. 16 | 17 | ## Usage Instructions 18 | 19 | ### Prerequisites 20 | 21 | - Python 3.12 22 | - AWS CDK CLI 23 | - AWS account and configured credentials 24 | - Jira account with API access 25 | 26 | ### Amazon Bedrock Setup 27 | 28 | 1. Ensure you deploy this architecture in the following region 29 | - US East 1 (N. Virginia) 30 | 31 | 2. Enable the following foundation models in the Amazon Bedrock console: 32 | - Claude 3.5 Sonnet v2 (Anthropic) 33 | 34 | Note: You must explicitly enable each model in your AWS account before you can use them. To enable the models: 35 | - Navigate to the Amazon Bedrock console 36 | - Select "Model access" in the left navigation pane 37 | - Choose "Manage model access" 38 | - Select the required models and choose "Request model access" 39 | 40 | ### Installation 41 | 42 | 1. Clone the repository: 43 | ``` 44 | git clone https://github.com/aws-samples/genai-for-devops.git 45 | cd automating-kanban-workflows 46 | ``` 47 | 48 | 2. Install the required dependencies: 49 | ``` 50 | pip install -r requirements.txt 51 | ``` 52 | 53 | 3. Configure your AWS credentials: 54 | ``` 55 | aws configure 56 | ``` 57 | 58 | 4. Set up Jira credentials as environment variables: 59 | ``` 60 | export JIRA_URL= 61 | export JIRA_USERNAME= 62 | export JIRA_API_TOKEN= 63 | ``` 64 | 65 | ### Deployment 66 | 67 | 1. Synthesize the CloudFormation template: 68 | ``` 69 | cdk synth 70 | ``` 71 | 72 | 2. Deploy the stack: 73 | ``` 74 | cdk deploy --context jira_api_token=$JIRA_API_TOKEN --context jira_url=$JIRA_URL --context jira_username=$JIRA_USERNAME 75 | ``` 76 | 77 | 3. Note the output value for `JiraKanbanTopicArn`, which you'll need to configure Jira automation. 78 | 79 | ### Configuration 80 | 81 | 1. In Jira, [set up an automation rule to publish to the SNS topic](https://support.atlassian.com/cloud-automation/docs/configure-aws-sns-for-jira-automation/) when a task is created or edited. 82 | 2. Use the `JiraKanbanTopicArn` from the CDK output as the SNS topic ARN in your Jira automation settings. 83 | 84 | ## Data Flow 85 | 86 | 1. A new task is created or edited in Jira. 87 | 2. Jira automation publishes an event to the SNS topic. 88 | 3. The `trigger_kanban_automation_workflow` Lambda function is triggered by the SNS event. 89 | 4. The Lambda function starts the Step Functions state machine. 90 | 5. The state machine executes the following steps: 91 | a. `jira_task_description_review` Lambda function reviews the task description. 92 | b. If the description passes review, `jira_split_into_subtasks` Lambda function is called. 93 | c. If subtasks are created, they are added to the Jira board. 94 | 6. The workflow completes, and the task is ready for development. 95 | 96 | ## Infrastructure 97 | 98 | The project uses AWS CDK to define the following resources: 99 | 100 | - Lambda Functions: 101 | - `JiraTaskDescriptionReviewLambda`: Reviews Jira task descriptions. 102 | - `JiraSplitIntoSubtasksLambda`: Splits tasks into subtasks. 103 | - `TriggerKanbanAutomationWorkflowLambda`: Triggers the workflow when an SNS message is received. 104 | 105 | - Step Functions: 106 | - `JiraKanbanWorkflow`: Orchestrates the task review and subtask creation process. 107 | 108 | - SNS: 109 | - `JiraKanbanTopic`: Receives events from Jira to trigger the workflow. 110 | 111 | - IAM: 112 | - Roles and policies for Lambda functions to access Bedrock and other AWS services. 113 | 114 | ## Troubleshooting 115 | 116 | - If the workflow is not triggering: 117 | 1. Check the SNS topic configuration in Jira. 118 | 2. Verify the Lambda function logs in CloudWatch. 119 | 3. Ensure the Step Functions state machine has the correct permissions. 120 | 121 | - If task descriptions are not being reviewed: 122 | 1. Check the Jira API token and URL in the Lambda environment variables. 123 | 2. Verify the Bedrock model ARN is correct and accessible. 124 | 125 | - For issues with subtask creation: 126 | 1. Review the Jira permissions for the API user. 127 | 2. Check the Lambda function logs for any Jira API errors. 128 | -------------------------------------------------------------------------------- /automating-kanban-workflows/app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import os 3 | 4 | import aws_cdk as cdk 5 | 6 | from automating_kanban_workflows.automating_kanban_workflows_stack import AutomatingKanbanWorkflowsStack 7 | from cdk_nag import AwsSolutionsChecks 8 | 9 | app = cdk.App() 10 | AutomatingKanbanWorkflowsStack(app, "AutomatingKanbanWorkflowsStack",) 11 | 12 | cdk.Aspects.of(app).add(AwsSolutionsChecks(verbose=True)) 13 | app.synth() 14 | -------------------------------------------------------------------------------- /automating-kanban-workflows/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/automating-kanban-workflows/architecture.png -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/automating-kanban-workflows/automating_kanban_workflows/__init__.py -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/automating_kanban_workflows_stack.py: -------------------------------------------------------------------------------- 1 | from aws_cdk import ( 2 | Duration, 3 | Stack, 4 | CfnOutput, 5 | RemovalPolicy, 6 | aws_iam as iam, 7 | aws_lambda as _lambda, 8 | aws_logs as logs, 9 | aws_sns as sns, 10 | aws_sns_subscriptions as subs, 11 | aws_stepfunctions as sfn, 12 | aws_stepfunctions_tasks as task 13 | ) 14 | from constructs import Construct 15 | from cdk_nag import NagSuppressions 16 | 17 | class AutomatingKanbanWorkflowsStack(Stack): 18 | """ 19 | CDK Stack that creates an automated Kanban workflow integration between Jira and AWS. 20 | 21 | This stack creates Lambda functions, Step Functions state machine, and SNS topic 22 | to automate Jira task management with AI-powered review and subtask creation. 23 | """ 24 | 25 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 26 | """ 27 | Initialize the Kanban automation stack. 28 | 29 | Args: 30 | scope (Construct): The scope in which to define this construct 31 | construct_id (str): The scoped construct ID 32 | **kwargs: Additional arguments to pass to the Stack constructor 33 | """ 34 | super().__init__(scope, construct_id, **kwargs) 35 | 36 | # Create Lambda function for reviewing Jira task descriptions 37 | jira_task_description_review = create_lambda_function( 38 | self, 39 | "JiraTaskDescriptionReview", 40 | "jira_task_description_review", 41 | { 42 | "JIRA_API_TOKEN": self.node.try_get_context("jira_api_token"), 43 | "JIRA_USERNAME": self.node.try_get_context("jira_username"), 44 | "JIRA_URL": self.node.try_get_context("jira_url"), 45 | "MODEL_ID": "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account) 46 | }, 47 | include_dependencies=True 48 | ) 49 | 50 | # Add Bedrock permissions to the task review Lambda 51 | jira_task_description_review.add_to_role_policy( 52 | statement=iam.PolicyStatement( 53 | actions=[ 54 | "bedrock:InvokeModel" 55 | ], 56 | resources=[ 57 | "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account), 58 | "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 59 | "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 60 | "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0" 61 | ] 62 | ) 63 | ) 64 | 65 | # Create Lambda function for splitting tasks into subtasks 66 | jira_split_into_subtasks_function = create_lambda_function( 67 | self, 68 | "JiraSplitIntoSubtasks", 69 | "jira_split_into_subtasks", 70 | { 71 | "JIRA_API_TOKEN": self.node.try_get_context("jira_api_token"), 72 | "JIRA_USERNAME": self.node.try_get_context("jira_username"), 73 | "JIRA_URL": self.node.try_get_context("jira_url"), 74 | "MODEL_ID": "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account) 75 | }, 76 | include_dependencies=True 77 | ) 78 | 79 | # Add Bedrock permissions to the subtask creation Lambda 80 | jira_split_into_subtasks_function.add_to_role_policy( 81 | statement=iam.PolicyStatement( 82 | actions=[ 83 | "bedrock:InvokeModel" 84 | ], 85 | resources=[ 86 | "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account), 87 | "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 88 | "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 89 | "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0" 90 | ] 91 | ) 92 | ) 93 | 94 | # Create Step Functions tasks 95 | jira_task_description_review_task = task.LambdaInvoke(self, "JiraTaskDescriptionReviewTask", lambda_function=jira_task_description_review) 96 | jira_split_into_subtasks_task = task.LambdaInvoke(self, "JiraSplitIntoSubtasksTask", lambda_function=jira_split_into_subtasks_function, input_path="$.Payload") 97 | 98 | # Create Step Functions choice and success states 99 | jira_success_choice = sfn.Choice(self, "JiraSuccessChoice") 100 | jira_success = sfn.Succeed(self, "JiraSuccess") 101 | 102 | # Define the Step Functions workflow 103 | definition = jira_task_description_review_task.next(jira_success_choice) 104 | 105 | # Add conditional logic to the workflow 106 | jira_success_choice.when( 107 | sfn.Condition.boolean_equals("$.Payload.proceed", True), 108 | jira_split_into_subtasks_task 109 | ).otherwise(jira_success) 110 | 111 | # Create the Step Functions state machine 112 | step_function = sfn.StateMachine( 113 | self, 114 | "JiraKanbanWorkflow", 115 | definition_body=sfn.DefinitionBody.from_chainable(definition), 116 | tracing_enabled=True, 117 | logs=sfn.LogOptions( 118 | destination=logs.LogGroup( 119 | self, 120 | "JiraKanbanWorkflowLogGroup", 121 | retention=logs.RetentionDays.ONE_DAY, # Adjust retention period as needed 122 | removal_policy=RemovalPolicy.DESTROY 123 | ), 124 | level=sfn.LogLevel.ALL, # Log all events 125 | include_execution_data=True 126 | ) 127 | ) 128 | 129 | # Add suppression for the Step Functions role 130 | NagSuppressions.add_resource_suppressions( 131 | step_function.role, 132 | suppressions=[ 133 | { 134 | "id": "AwsSolutions-IAM5", 135 | "reason": "Step Functions requires these permissions for X-Ray tracing, and Lambda invocations as per AWS documentation: https://docs.aws.amazon.com/step-functions/latest/dg/procedure-create-iam-role.html" 136 | } 137 | ], 138 | apply_to_children=True 139 | ) 140 | 141 | # Create SNS topic for Jira notifications 142 | topic = sns.Topic(self, "JiraKanbanTopic", enforce_ssl=True) 143 | 144 | # Add policy allowing Jira's AWS account to publish to the topic 145 | topic_policy = iam.PolicyStatement( 146 | effect=iam.Effect.ALLOW, 147 | principals=[iam.AccountPrincipal("815843069303")], #Using account ID as documented here: https://support.atlassian.com/cloud-automation/docs/configure-aws-sns-for-jira-automation/ 148 | actions=["sns:Publish"], 149 | resources=[topic.topic_arn] 150 | ) 151 | topic.add_to_resource_policy(topic_policy) 152 | 153 | # Create trigger Lambda and add necessary permissions 154 | trigger_jira_kanban_function = create_lambda_function(self, "TriggerKanbanAutomationWorkflow", "trigger_kanban_automation_Workflow", {"STEP_FUNCTIONS_ARN": step_function.state_machine_arn}) 155 | topic.add_subscription(subs.LambdaSubscription(trigger_jira_kanban_function)) 156 | step_function.grant_start_execution(trigger_jira_kanban_function) 157 | 158 | # Output the SNS topic ARN for reference 159 | CfnOutput(self, "JiraKanbanTopicArn", value=topic.topic_arn) 160 | 161 | 162 | def create_lambda_function(self, purpose: str, folder_name: str, environment={}, include_dependencies=False): 163 | """ 164 | Helper function to create Lambda functions with consistent configuration. 165 | 166 | Args: 167 | purpose (str): Identifier for the Lambda function 168 | folder_name (str): Name of the folder containing the function code 169 | environment (dict): Environment variables for the function 170 | include_dependencies (bool): Whether to include dependencies from requirements.txt 171 | 172 | Returns: 173 | _lambda.Function: The created Lambda function 174 | """ 175 | command = ["bash", "-c", "cp -au . /asset-output"] 176 | 177 | if include_dependencies: 178 | command = ["bash", "-c", "pip install -r requirements.txt -t /asset-output && cp -au . /asset-output"] 179 | 180 | lambda_function = _lambda.Function(self, 181 | "{}Lambda".format(purpose), 182 | handler="index.lambda_handler", 183 | runtime=_lambda.Runtime.PYTHON_3_13, 184 | code=_lambda.Code.from_asset( 185 | "./automating_kanban_workflows/functions/{}".format(folder_name), 186 | bundling=dict( 187 | image=_lambda.Runtime.PYTHON_3_13.bundling_image, 188 | command=command 189 | ) 190 | ), 191 | timeout=Duration.minutes(5), 192 | architecture=_lambda.Architecture.X86_64, 193 | environment=environment 194 | ) 195 | 196 | NagSuppressions.add_resource_suppressions( 197 | lambda_function.role, 198 | suppressions=[ 199 | { 200 | "id": "AwsSolutions-IAM4", 201 | "reason": "Lambda function requires wildcard permissions for logs as name is dynamically generated.", 202 | } 203 | ], 204 | apply_to_children=True 205 | ) 206 | 207 | return lambda_function 208 | -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/jira_split_into_subtasks/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/automating-kanban-workflows/automating_kanban_workflows/functions/jira_split_into_subtasks/__init__.py -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/jira_split_into_subtasks/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | from atlassian import Jira 4 | import os 5 | 6 | # Fetch Jira credentials from environment variables 7 | jira_url = os.environ.get('JIRA_URL') 8 | jira_username = os.environ.get('JIRA_USERNAME') 9 | jira_api_token = os.environ.get('JIRA_API_TOKEN') 10 | 11 | # Initialize Jira client with authentication credentials 12 | jira = Jira( 13 | url=jira_url, 14 | username=jira_username, 15 | password=jira_api_token 16 | ) 17 | 18 | # Initialize Amazon Bedrock client for AI model interactions 19 | bedrock_client = boto3.client('bedrock-runtime') 20 | 21 | def lambda_handler(event, context): 22 | """ 23 | AWS Lambda handler that processes Jira tasks and splits them into subtasks using AI. 24 | 25 | Args: 26 | event (dict): Contains the Jira task key in format {'taskKey': 'PROJECT-123'} 27 | context (LambdaContext): AWS Lambda context object 28 | 29 | Returns: 30 | dict: Response containing success status and message 31 | """ 32 | 33 | # Extract the Jira issue key from the event 34 | issue_key = event['taskKey'] 35 | 36 | # Fetch the full issue details from Jira 37 | issue = jira.issue(issue_key) 38 | issue_description = issue['fields']['description'] 39 | 40 | # Skip processing for Bug and Subtask issue types 41 | if issue['fields']['issuetype']['name'] in ['Bug', 'Subtask']: 42 | return {'success': True} 43 | 44 | # Construct the prompt for the AI model 45 | user_message = """You are a technical project manager for Jira Tickets, who breaks down tickets that into subtasks where there may be multiple individuals involved or the time expected to complete is longer than 2 hours. 46 | You will return a result in a JSON format with one attribute key being subtasks. This is a list. If no subtasks are needed this will be empty. 47 | Each would be an object in the list with a key of title and a key of description. Split by logical divisions and provide as much guidance as possible. Make sure the ticket description is high quality. 48 | The parent task description to review is: {} 49 | Only generate subtasks where it is completely neccessary. These are tasks completed by software development engineers, frontend developers and/or DevOps Engineers. Do not include tasks to do testing (including unit and integration) or deployment as this is part of the SDLC. 50 | Investigation and analysis should not have separate subtasks. 51 | Not tasks for analyzing, no tasks for regression testing. 52 | Each task must be able to be deployed separately (increasing deployment frequency). Do not make any assumptions, only use the existing knowledge you have. 53 | Only return JSON, no text. JSON should be a single line 54 | """.format(issue_description) 55 | 56 | # Prepare the conversation format for Bedrock 57 | conversation = [ 58 | { 59 | "role": "user", 60 | "content": [{"text": user_message}], 61 | } 62 | ] 63 | 64 | # Send the request to Bedrock model with specified inference parameters 65 | response = bedrock_client.converse( 66 | modelId=os.environ['MODEL_ID'], 67 | messages=conversation, 68 | inferenceConfig={"maxTokens": 2048, "temperature": 0.5, "topP": 0.9}, 69 | ) 70 | 71 | # Extract and parse the AI model's response 72 | response_text = response["output"]["message"]["content"][0]["text"] 73 | response_json = json.loads(response_text) 74 | 75 | # Create subtasks in Jira based on the AI model's suggestions 76 | for subtask in response_json['subtasks']: 77 | create_subtask(issue_key, subtask['title'], subtask['description']) 78 | 79 | return { 80 | 'statusCode': 200, 81 | 'body': json.dumps('Hello from Lambda!') 82 | } 83 | 84 | def create_subtask(parent_issue_key, summary, description): 85 | """ 86 | Creates a subtask in Jira under a parent issue. 87 | 88 | Args: 89 | parent_issue_key (str): The key of the parent Jira issue (e.g., 'PROJECT-123') 90 | summary (str): The title/summary of the subtask 91 | description (str): Detailed description of the subtask 92 | 93 | Returns: 94 | str or None: The key of the created subtask if successful, None if failed 95 | 96 | Raises: 97 | Exception: If there's an error creating the subtask in Jira 98 | """ 99 | try: 100 | # Prepare the subtask data structure 101 | subtask = { 102 | 'project': {'key': parent_issue_key.split('-')[0]}, 103 | 'summary': summary, 104 | 'description': description, 105 | 'issuetype': {'name': 'Subtask'}, 106 | 'parent': {'key': parent_issue_key} 107 | } 108 | 109 | # Create the subtask in Jira 110 | new_subtask = jira.create_issue(fields=subtask) 111 | 112 | print(f"Subtask created: {new_subtask['key']}") 113 | return new_subtask['key'] 114 | except Exception as e: 115 | print(f"Error creating subtask: {str(e)}") 116 | return None -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/jira_split_into_subtasks/requirements.txt: -------------------------------------------------------------------------------- 1 | atlassian-python-api -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/jira_task_description_review/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/automating-kanban-workflows/automating_kanban_workflows/functions/jira_task_description_review/__init__.py -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/jira_task_description_review/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | from atlassian import Jira 4 | import os 5 | 6 | # Initialize Jira authentication credentials from environment variables for security 7 | jira_url = os.environ.get('JIRA_URL') 8 | jira_username = os.environ.get('JIRA_USERNAME') 9 | jira_api_token = os.environ.get('JIRA_API_TOKEN') 10 | 11 | # Initialize Jira client with authentication details 12 | jira = Jira( 13 | url=jira_url, 14 | username=jira_username, 15 | password=jira_api_token 16 | ) 17 | 18 | # Initialize Amazon Bedrock client for AI-powered task review 19 | bedrock_client = boto3.client('bedrock-runtime') 20 | 21 | def lambda_handler(event, context): 22 | """ 23 | AWS Lambda handler that reviews Jira task descriptions for quality and completeness. 24 | 25 | This function: 26 | 1. Retrieves a Jira task 27 | 2. Checks if the reporter is the automation user 28 | 3. Uses Amazon Bedrock to analyze the task description 29 | 4. Provides feedback and reassigns if necessary 30 | 31 | Args: 32 | event (dict): Contains the Jira task key in format {'taskKey': 'PROJECT-123'} 33 | context (LambdaContext): AWS Lambda context object 34 | 35 | Returns: 36 | dict: Contains: 37 | - proceed (bool): Whether the task passed quality review 38 | - taskKey (str): The original task key 39 | """ 40 | 41 | # Extract the Jira issue key from the event 42 | issue_key = event['taskKey'] 43 | 44 | # Fetch the full issue details from Jira 45 | issue = jira.issue(issue_key) 46 | 47 | # Skip review if the reporter is the automation user 48 | if 'emailAddress' in issue['fields']['reporter'] and issue['fields']['reporter']['emailAddress'] == jira_username: 49 | return {'proceed': True, 'taskKey': event['taskKey']} 50 | 51 | # Get the reporter's account ID for potential reassignment 52 | reporter_account_id = issue['fields']['reporter']['accountId'] 53 | 54 | # Get the issue description for review 55 | issue_description = issue['fields']['description'] 56 | 57 | # Construct the prompt for the AI model 58 | user_message = """You are a reviewer of Jira Tickets, designed to highlight when a ticket is not clear enough for a developer to work on 59 | You will return a result in a JSON format where one attribute key is pass being either true or false. It is false if it does not meet the quality bar. 60 | A second optional JSON attribute key will be called comment where you are providing guidance and provide an example of how the ticket would meet the pass requirements. 61 | Focus on whether a developer would understand without being perdantic. 62 | Ensure there is a general overview, user story, acceptance criteria, implementation details, testing criteria and any additional considerations. 63 | The task description to review is: {} 64 | Only return JSON, no text. JSON should be a single line 65 | """.format(issue_description) 66 | 67 | # Prepare the conversation format for Bedrock 68 | conversation = [ 69 | { 70 | "role": "user", 71 | "content": [{"text": user_message}], 72 | } 73 | ] 74 | 75 | # Send the request to Bedrock model with specified inference parameters 76 | response = bedrock_client.converse( 77 | modelId=os.environ['MODEL_ID'], 78 | messages=conversation, 79 | inferenceConfig={"maxTokens": 512, "temperature": 0.5, "topP": 0.9}, 80 | ) 81 | 82 | # Extract and parse the AI model's response 83 | response_text = response["output"]["message"]["content"][0]["text"] 84 | response_json = json.loads(response_text) 85 | 86 | # If the task doesn't meet quality standards: 87 | # 1. Log the feedback 88 | # 2. Add the feedback as a comment 89 | # 3. Reassign to the original reporter 90 | if response_json['pass'] == False: 91 | print(response_json['comment']) 92 | jira.issue_add_comment(issue_key, response_json['comment']) 93 | jira.update_issue_field( 94 | issue_key, 95 | fields={'assignee': {'accountId': reporter_account_id}} 96 | ) 97 | 98 | return {'proceed': response_json['pass'], 'taskKey': event['taskKey']} 99 | -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/jira_task_description_review/requirements.txt: -------------------------------------------------------------------------------- 1 | atlassian-python-api -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/trigger_kanban_automation_workflow/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/automating-kanban-workflows/automating_kanban_workflows/functions/trigger_kanban_automation_workflow/__init__.py -------------------------------------------------------------------------------- /automating-kanban-workflows/automating_kanban_workflows/functions/trigger_kanban_automation_workflow/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | import os 4 | 5 | # Initialize AWS Step Functions client for workflow orchestration 6 | sfn_client = boto3.client('stepfunctions') 7 | 8 | def lambda_handler(event, context): 9 | """ 10 | AWS Lambda handler that triggers the Kanban automation workflow in Step Functions 11 | when a Jira event is received via SNS. 12 | 13 | This function: 14 | 1. Processes the incoming SNS event 15 | 2. Extracts the Jira issue key 16 | 3. Starts the Step Functions state machine for task review and processing 17 | 18 | Args: 19 | event (dict): SNS event containing Jira webhook payload 20 | context (LambdaContext): AWS Lambda context object 21 | 22 | Returns: 23 | dict: Response containing execution status and details 24 | """ 25 | # Extract the message from the SNS event 26 | message = json.loads(event['Records'][0]['Sns']['Message']) 27 | 28 | # Start the Step Functions workflow with the issue 29 | sfn_client.start_execution( 30 | stateMachineArn=os.environ['STEP_FUNCTIONS_ARN'], 31 | input=json.dumps(message['automationData']) 32 | ) 33 | 34 | return { 35 | 'statusCode': 200, 36 | 'body': json.dumps({ 37 | 'message': 'Successfully started workflow' 38 | }) 39 | } -------------------------------------------------------------------------------- /automating-kanban-workflows/cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "python3 app.py", 3 | "watch": { 4 | "include": [ 5 | "**" 6 | ], 7 | "exclude": [ 8 | "README.md", 9 | "cdk*.json", 10 | "requirements*.txt", 11 | "source.bat", 12 | "**/__init__.py", 13 | "**/__pycache__", 14 | "tests" 15 | ] 16 | }, 17 | "context": { 18 | "@aws-cdk/aws-lambda:recognizeLayerVersion": true, 19 | "@aws-cdk/core:checkSecretUsage": true, 20 | "@aws-cdk/core:target-partitions": [ 21 | "aws", 22 | "aws-cn" 23 | ], 24 | "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, 25 | "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, 26 | "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, 27 | "@aws-cdk/aws-iam:minimizePolicies": true, 28 | "@aws-cdk/core:validateSnapshotRemovalPolicy": true, 29 | "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, 30 | "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, 31 | "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, 32 | "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, 33 | "@aws-cdk/core:enablePartitionLiterals": true, 34 | "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, 35 | "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, 36 | "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, 37 | "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, 38 | "@aws-cdk/aws-route53-patters:useCertificate": true, 39 | "@aws-cdk/customresources:installLatestAwsSdkDefault": false, 40 | "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, 41 | "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, 42 | "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, 43 | "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, 44 | "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, 45 | "@aws-cdk/aws-redshift:columnId": true, 46 | "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true, 47 | "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, 48 | "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, 49 | "@aws-cdk/aws-kms:aliasNameRef": true, 50 | "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, 51 | "@aws-cdk/core:includePrefixInUniqueNameGeneration": true, 52 | "@aws-cdk/aws-efs:denyAnonymousAccess": true, 53 | "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true, 54 | "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true, 55 | "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true, 56 | "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true, 57 | "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true, 58 | "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true, 59 | "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true, 60 | "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true, 61 | "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, 62 | "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true, 63 | "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true, 64 | "@aws-cdk/aws-eks:nodegroupNameAttribute": true, 65 | "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true, 66 | "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true, 67 | "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false, 68 | "@aws-cdk/aws-s3:keepNotificationInImportedBucket": false, 69 | "@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true, 70 | "@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true, 71 | "@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true, 72 | "@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true, 73 | "@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true, 74 | "@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true, 75 | "@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true, 76 | "@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true, 77 | "@aws-cdk/aws-ec2:bastionHostUseAmazonLinux2023ByDefault": true 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /automating-kanban-workflows/requirements.txt: -------------------------------------------------------------------------------- 1 | aws-cdk-lib==2.173.2 2 | constructs>=10.0.0,<11.0.0 3 | cdk-nag==2.34.23 -------------------------------------------------------------------------------- /improving-code-quality-reviews/.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | package-lock.json 3 | __pycache__ 4 | .pytest_cache 5 | .venv 6 | *.egg-info 7 | 8 | # CDK asset staging directory 9 | .cdk.staging 10 | cdk.out 11 | -------------------------------------------------------------------------------- /improving-code-quality-reviews/README.md: -------------------------------------------------------------------------------- 1 | # GitHub Code Review Automation with AWS CDK and Bedrock 2 | 3 | This project implements an automated code review system for GitHub pull requests using AWS CDK, Lambda, API Gateway, and Amazon Bedrock. 4 | 5 | The system is designed to enhance code quality by automatically reviewing pull requests against a predefined set of best practices and providing feedback to developers. It leverages the power of AI through Amazon Bedrock to generate insightful code review comments. 6 | 7 | ![Architecture Diragram](architecture.png) 8 | 9 | ## Repository Structure 10 | 11 | - `app.py`: The main entry point for the CDK application. 12 | - `cdk.json`: Configuration file for the CDK application. 13 | - `improving_code_quality_reviews/functions/run_github_code_review/index.py`: Lambda function that performs the code review. 14 | - `improving_code_quality_reviews/improving_code_quality_reviews_stack.py`: Defines the AWS infrastructure stack. 15 | 16 | ## Usage Instructions 17 | 18 | ### Prerequisites 19 | 20 | - Python 3.12 21 | - AWS CDK CLI 22 | - AWS CLI configured with appropriate credentials 23 | - [GitHub Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with repo scope 24 | - GitHub Webhook with [Secret setup](https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries#creating-a-secret-token) 25 | 26 | ### Amazon Bedrock Setup 27 | 28 | 1. Ensure you deploy this architecture in the following region 29 | - US East 1 (N. Virginia) 30 | 31 | 2. Enable the following foundation models in the Amazon Bedrock console: 32 | - Claude 3.5 Sonnet v2 (Anthropic) 33 | 34 | Note: You must explicitly enable each model in your AWS account before you can use them. To enable the models: 35 | - Navigate to the Amazon Bedrock console 36 | - Select "Model access" in the left navigation pane 37 | - Choose "Manage model access" 38 | - Select the required models and choose "Request model access" 39 | 40 | ### Installation 41 | 42 | 1. Clone the repository: 43 | ``` 44 | git clone https://github.com/aws-samples/genai-for-devops.git 45 | cd improving-code-quality-reviews 46 | ``` 47 | 48 | 2. Install the required dependencies: 49 | ``` 50 | pip install -r requirements.txt 51 | ``` 52 | 53 | 3. Configure your AWS credentials: 54 | ``` 55 | aws configure 56 | ``` 57 | 58 | 4. Set up your GitHub token and secret as an environment variable: 59 | ``` 60 | export GITHUB_TOKEN=your_github_token_here 61 | export GITHUB_SECRET=your_github_secret_here 62 | ``` 63 | 64 | ### Deployment 65 | 66 | 1. Bootstrap your AWS environment (if not already done): 67 | ``` 68 | cdk bootstrap 69 | ``` 70 | 71 | 2. Deploy the stack: 72 | ``` 73 | cdk deploy --context github_token=$GITHUB_TOKEN --context github_secret=$GITHUB_SECRET 74 | ``` 75 | 76 | 3. Note the API Gateway URL output after deployment. 77 | 78 | ### Configuration 79 | 80 | The `cdk.json` file contains various configuration options for the CDK application. You can modify these settings as needed. 81 | 82 | ### Usage 83 | 84 | To use the code review system: 85 | 86 | 1. Set up a GitHub webhook for your repository, pointing to the API Gateway URL with the `/review` endpoint. 87 | 2. Configure the webhook to trigger on pull request events. 88 | 3. When a new pull request is opened or updated, the system will automatically review the changes and post comments on the pull request. 89 | 90 | ## Data Flow 91 | 92 | 1. GitHub sends a webhook payload to the API Gateway when a pull request event occurs. 93 | 2. API Gateway triggers the Lambda function with the event payload. 94 | 3. The Lambda function: 95 | a. Extracts pull request details from the event. 96 | b. Fetches the diff content from GitHub API. 97 | c. Sends the diff to Amazon Bedrock for analysis. 98 | d. Receives the generated review comment from Bedrock. 99 | e. Posts the comment back to the GitHub pull request. 100 | 101 | ## Infrastructure 102 | 103 | The project uses AWS CDK to define and deploy the following infrastructure: 104 | 105 | ### Lambda 106 | - `GitHubReviewLambda`: Python 3.12 function that handles the code review process. 107 | - Environment variables: 108 | - `GITHUB_TOKEN`: GitHub Personal Access Token 109 | - `MODEL_ID`: ARN of the Bedrock model 110 | 111 | ### API Gateway 112 | - `GitHubReviewAPI`: REST API with a single POST endpoint at `/review` 113 | 114 | ### IAM 115 | - Lambda execution role with permissions to invoke Bedrock models 116 | 117 | ## Troubleshooting 118 | 119 | ### Common Issues 120 | 121 | 1. **Lambda function timing out** 122 | - Problem: The Lambda function exceeds the 5-minute timeout. 123 | - Solution: Increase the timeout in the `ImprovingCodeQualityReviewsStack` class: 124 | ```python 125 | timeout=Duration.minutes(10) 126 | ``` 127 | 128 | 2. **Bedrock model invocation failing** 129 | - Problem: "AccessDeniedException" when invoking Bedrock model. 130 | - Diagnostic steps: 131 | 1. Check the Lambda function logs in CloudWatch. 132 | 2. Verify the IAM permissions for Bedrock in the Lambda execution role. 133 | - Solution: Ensure the correct Bedrock model ARN is specified and the Lambda has the necessary permissions. -------------------------------------------------------------------------------- /improving-code-quality-reviews/app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import os 3 | 4 | import aws_cdk as cdk 5 | 6 | from improving_code_quality_reviews.improving_code_quality_reviews_stack import ImprovingCodeQualityReviewsStack 7 | from cdk_nag import AwsSolutionsChecks 8 | 9 | app = cdk.App() 10 | ImprovingCodeQualityReviewsStack(app, "ImprovingCodeQualityReviewsStack") 11 | 12 | cdk.Aspects.of(app).add(AwsSolutionsChecks(verbose=True)) 13 | app.synth() 14 | -------------------------------------------------------------------------------- /improving-code-quality-reviews/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/improving-code-quality-reviews/architecture.png -------------------------------------------------------------------------------- /improving-code-quality-reviews/cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "python3 app.py", 3 | "watch": { 4 | "include": [ 5 | "**" 6 | ], 7 | "exclude": [ 8 | "README.md", 9 | "cdk*.json", 10 | "requirements*.txt", 11 | "source.bat", 12 | "**/__init__.py", 13 | "**/__pycache__", 14 | "tests" 15 | ] 16 | }, 17 | "context": { 18 | "@aws-cdk/aws-lambda:recognizeLayerVersion": true, 19 | "@aws-cdk/core:checkSecretUsage": true, 20 | "@aws-cdk/core:target-partitions": [ 21 | "aws", 22 | "aws-cn" 23 | ], 24 | "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, 25 | "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, 26 | "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, 27 | "@aws-cdk/aws-iam:minimizePolicies": true, 28 | "@aws-cdk/core:validateSnapshotRemovalPolicy": true, 29 | "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, 30 | "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, 31 | "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, 32 | "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, 33 | "@aws-cdk/core:enablePartitionLiterals": true, 34 | "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, 35 | "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, 36 | "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, 37 | "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, 38 | "@aws-cdk/aws-route53-patters:useCertificate": true, 39 | "@aws-cdk/customresources:installLatestAwsSdkDefault": false, 40 | "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, 41 | "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, 42 | "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, 43 | "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, 44 | "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, 45 | "@aws-cdk/aws-redshift:columnId": true, 46 | "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true, 47 | "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, 48 | "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, 49 | "@aws-cdk/aws-kms:aliasNameRef": true, 50 | "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, 51 | "@aws-cdk/core:includePrefixInUniqueNameGeneration": true, 52 | "@aws-cdk/aws-efs:denyAnonymousAccess": true, 53 | "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true, 54 | "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true, 55 | "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true, 56 | "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true, 57 | "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true, 58 | "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true, 59 | "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true, 60 | "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true, 61 | "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, 62 | "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true, 63 | "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true, 64 | "@aws-cdk/aws-eks:nodegroupNameAttribute": true, 65 | "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true, 66 | "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true, 67 | "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false, 68 | "@aws-cdk/aws-s3:keepNotificationInImportedBucket": false, 69 | "@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true, 70 | "@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true, 71 | "@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true, 72 | "@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true, 73 | "@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true, 74 | "@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true, 75 | "@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true, 76 | "@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true, 77 | "@aws-cdk/aws-ec2:bastionHostUseAmazonLinux2023ByDefault": true 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /improving-code-quality-reviews/improving_code_quality_reviews/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/improving-code-quality-reviews/improving_code_quality_reviews/__init__.py -------------------------------------------------------------------------------- /improving-code-quality-reviews/improving_code_quality_reviews/functions/run_github_code_review/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/improving-code-quality-reviews/improving_code_quality_reviews/functions/run_github_code_review/__init__.py -------------------------------------------------------------------------------- /improving-code-quality-reviews/improving_code_quality_reviews/functions/run_github_code_review/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import hashlib 3 | import hmac 4 | import json 5 | import os 6 | import requests 7 | 8 | # Create API Gateway with a single POST endpoint 9 | api_base = 'https://api.github.com' 10 | 11 | # GitHub authentication token retrieved from environment variables for security 12 | github_token = os.environ['GITHUB_TOKEN'] 13 | 14 | # Initialise the Bedrock client to interact with AI models 15 | bedrock_client = boto3.client('bedrock-runtime') 16 | 17 | def lambda_handler(event, context): 18 | """ 19 | Main handler for the Lambda function that processes GitHub pull request events. 20 | 21 | Args: 22 | event (dict): The event data from the API Gateway trigger 23 | context: The Lambda context object 24 | 25 | Returns: 26 | dict: Response object with status code and message 27 | """ 28 | 29 | # Check if the signature header exists 30 | if 'X-Hub-Signature-256' not in event['headers']: 31 | return { 32 | 'statusCode': 403, 33 | 'body': json.dumps('Missing signature') 34 | } 35 | 36 | # Calculate expected signature using HMAC SHA256 37 | hash_object = hmac.new(os.environ['GITHUB_SECRET'].encode('utf-8'), msg=event['body'].encode('utf-8'), digestmod=hashlib.sha256) 38 | expected_signature = "sha256=" + hash_object.hexdigest() 39 | 40 | # Validate that signatures match before proceeding 41 | # See: https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries#validating-webhook-deliveries 42 | if not hmac.compare_digest(expected_signature, event['headers']['X-Hub-Signature-256']): 43 | return { 44 | 'statusCode': 403, 45 | 'body': json.dumps('Signature mismatch') 46 | } 47 | 48 | body = json.loads(event['body']) 49 | 50 | # Skip processing if the event is not a pull request 51 | if 'pull_request' not in body: 52 | return { 53 | 'statusCode': 200, 54 | 'body': json.dumps('Hello there') 55 | } 56 | 57 | # Extract relevant pull request information 58 | pr_number = body['pull_request']['number'] 59 | repo_name = body['repository']['full_name'] 60 | 61 | # Process the pull request in three steps 62 | diff_content = get_diff(pr_number, repo_name) 63 | comment = generate_comment(diff_content) 64 | post_comment(pr_number, repo_name, comment) 65 | 66 | return { 67 | 'statusCode': 200, 68 | 'body': json.dumps('Successfully Completed') 69 | } 70 | 71 | def get_diff(pr_number, repo_name): 72 | """ 73 | Retrieves the diff content from a GitHub pull request. 74 | 75 | Args: 76 | pr_number (int): The pull request number 77 | repo_name (str): The full repository name (owner/repo) 78 | 79 | Returns: 80 | str: The diff content of the pull request 81 | """ 82 | # Configure headers for GitHub API authentication and diff format 83 | headers = { 84 | 'Authorization': f'Bearer {github_token}', 85 | 'Accept': 'application/vnd.github.v3.diff', 86 | 'X-GitHub-Api-Version': '2022-11-28' 87 | } 88 | 89 | # Fetch the pull request diff from GitHub 90 | diff_url = f'{api_base}/repos/{repo_name}/pulls/{pr_number}' 91 | diff_response = requests.get(diff_url, headers=headers, timeout=30) 92 | diff_response.raise_for_status() 93 | diff_content = diff_response.text 94 | 95 | return diff_content 96 | 97 | def generate_comment(diff_content): 98 | """ 99 | Generates a code review comment using Amazon Bedrock's AI model. 100 | 101 | Args: 102 | diff_content (str): The diff content from the pull request 103 | 104 | Returns: 105 | str: AI-generated code review comment 106 | """ 107 | # Get the model ID from environment variables 108 | model_id = os.environ['MODEL_ID'] 109 | 110 | # Construct the prompt for the AI model 111 | user_message = """You are a reviewer of a git pull request, You are looking to identify if the code follows the companys developer checklist before being reviewed. 112 | - New functionality is covered by unit tests 113 | - Code is clean, readable, and follows the project's coding standards and best practices 114 | - Code is well-documented, including inline comments and updated documentation if necessary 115 | - Performance considerations have been taken into account 116 | - Error handling and logging are implemented appropriately 117 | - Security best practices are followed, and potential vulnerabilities are addressed 118 | - Code is free of any sensitive information (e.g., API keys, passwords) 119 | - Backward compatibility 120 | - Infrastructure as code includes monitoring and logging for new components 121 | - Configuration changes are properly validated and tested 122 | - Database migrations are properly managed and tested 123 | - Continuous Integration and Continuous Deployment (CI/CD) pipelines are updated if necessary 124 | - Potential risks and mitigation strategies have been identified and documented 125 | Callout specific examples of the code, where you do reference the file names and wrap the code snippets in ``. Where possible also provide next steps or examples on how to implement the suggestions. 126 | The PR content is below: 127 | {} 128 | """.format(diff_content) 129 | 130 | # Prepare the conversation structure for the AI model 131 | conversation = [ 132 | { 133 | "role": "user", 134 | "content": [{"text": user_message}], 135 | } 136 | ] 137 | 138 | # Send request to Bedrock model with specific inference parameters 139 | response = bedrock_client.converse( 140 | modelId=model_id, 141 | messages=conversation, 142 | inferenceConfig={ 143 | "maxTokens": 4096, # Maximum length of the response 144 | "temperature": 0.5, # Controls randomness (0.5 for balanced output) 145 | "topP": 0.9 # Controls diversity of the response 146 | }, 147 | ) 148 | 149 | # Extract and return the AI model's response 150 | response_text = response["output"]["message"]["content"][0]["text"] 151 | return response_text 152 | 153 | def post_comment(pr_number, repo_name, comment): 154 | """ 155 | Posts the generated review comment to the GitHub pull request. 156 | 157 | Args: 158 | pr_number (int): The pull request number 159 | repo_name (str): The full repository name (owner/repo) 160 | comment (str): The comment text to post 161 | 162 | Returns: 163 | Response: The GitHub API response object 164 | """ 165 | # Prepare the comment URL and data 166 | comment_url = f'{api_base}/repos/{repo_name}/issues/{pr_number}/comments' 167 | comment_data = { 168 | 'body': "Feedback generated by DevOpsBot\n\n{}".format(comment) 169 | } 170 | 171 | # Configure headers for GitHub API authentication 172 | comment_headers = { 173 | 'Authorization': f'token {github_token}', 174 | 'Accept': 'application/vnd.github.v3+json' 175 | } 176 | 177 | # Post the comment to GitHub 178 | comment_response = requests.post(comment_url, headers=comment_headers, json=comment_data, timeout=30) 179 | comment_response.raise_for_status() 180 | return comment_response -------------------------------------------------------------------------------- /improving-code-quality-reviews/improving_code_quality_reviews/functions/run_github_code_review/requirements.txt: -------------------------------------------------------------------------------- 1 | requests -------------------------------------------------------------------------------- /improving-code-quality-reviews/improving_code_quality_reviews/improving_code_quality_reviews_stack.py: -------------------------------------------------------------------------------- 1 | from aws_cdk import ( 2 | aws_apigateway as apigw, 3 | aws_iam as iam, 4 | aws_lambda as _lambda, 5 | aws_logs as logs, 6 | CfnOutput, 7 | Duration, 8 | Stack, 9 | ) 10 | from constructs import Construct 11 | from cdk_nag import NagSuppressions 12 | 13 | class ImprovingCodeQualityReviewsStack(Stack): 14 | """ 15 | A CDK Stack that creates an automated code review system using GitHub and AWS services. 16 | This stack deploys a Lambda function behind an API Gateway to process code review requests. 17 | """ 18 | 19 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 20 | """ 21 | Initialize the stack with required AWS resources. 22 | 23 | Args: 24 | scope: The scope in which to define this construct 25 | construct_id: The scoped construct ID 26 | **kwargs: Additional arguments passed to the Stack class 27 | """ 28 | super().__init__(scope, construct_id, **kwargs) 29 | 30 | # Retrieve GitHub token and secret from context 31 | github_token = self.node.try_get_context("github_token") 32 | github_secret = self.node.try_get_context("github_secret") 33 | 34 | # Create Lambda function for processing GitHub code reviews 35 | github_review_lambda = _lambda.Function(self, 36 | "GitHubReviewLambda", 37 | handler="index.lambda_handler", 38 | runtime=_lambda.Runtime.PYTHON_3_13, 39 | code=_lambda.Code.from_asset( 40 | "./improving_code_quality_reviews/functions/run_github_code_review", 41 | bundling=dict( 42 | image=_lambda.Runtime.PYTHON_3_13.bundling_image, 43 | command=["bash", "-c", "pip install -r requirements.txt -t /asset-output && cp -au . /asset-output"] 44 | ) 45 | ), 46 | timeout=Duration.minutes(5), 47 | architecture=_lambda.Architecture.X86_64, 48 | environment={ 49 | "GITHUB_TOKEN": github_token, 50 | "MODEL_ID": "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account), 51 | "GITHUB_SECRET": github_secret 52 | } 53 | ) 54 | 55 | NagSuppressions.add_resource_suppressions( 56 | github_review_lambda.role, 57 | suppressions=[ 58 | { 59 | "id": "AwsSolutions-IAM4", 60 | "reason": "Lambda function requires wildcard permissions for logs as name is dynamically generated.", 61 | } 62 | ], 63 | apply_to_children=True 64 | ) 65 | 66 | # Add IAM permissions for the Lambda function to invoke Bedrock models 67 | github_review_lambda.add_to_role_policy( 68 | statement=iam.PolicyStatement( 69 | actions=[ 70 | "bedrock:InvokeModel" 71 | ], 72 | resources=[ 73 | "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account), 74 | "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 75 | "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 76 | "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0" 77 | ] 78 | ) 79 | ) 80 | 81 | # Create CloudWatch role for API Gateway 82 | cloudwatch_role = iam.Role( 83 | self, 84 | "ApiGatewayCloudWatchRole", 85 | assumed_by=iam.ServicePrincipal("apigateway.amazonaws.com"), 86 | managed_policies=[ 87 | iam.ManagedPolicy.from_aws_managed_policy_name( 88 | "service-role/AmazonAPIGatewayPushToCloudWatchLogs" 89 | ) 90 | ] 91 | ) 92 | 93 | NagSuppressions.add_resource_suppressions( 94 | cloudwatch_role, 95 | [ 96 | { 97 | "id": "AwsSolutions-IAM4", 98 | "reason": "This role is for an account/region rather than an individual resource." 99 | } 100 | ] 101 | ) 102 | 103 | # Set up API Gateway Account 104 | apigw.CfnAccount( 105 | self, 106 | "ApiGatewayAccount", 107 | cloud_watch_role_arn=cloudwatch_role.role_arn 108 | ) 109 | 110 | # Create a log group for API Gateway 111 | api_log_group = logs.LogGroup(self, "GitHubReviewAPIAccessLogs") 112 | 113 | # Create API Gateway with a single POST endpoint 114 | api = apigw.LambdaRestApi( 115 | self, 116 | "GitHubReviewAPI", 117 | handler=github_review_lambda, 118 | proxy=False, 119 | deploy_options=apigw.StageOptions( 120 | stage_name="prod", # or your desired stage name 121 | logging_level=apigw.MethodLoggingLevel.INFO, # Set logging level 122 | access_log_destination=apigw.LogGroupLogDestination(api_log_group), 123 | access_log_format=apigw.AccessLogFormat.clf() # Common Log Format 124 | ) 125 | ) 126 | 127 | review_method = api.root.add_resource("review").add_method( 128 | "POST" # Only allow POST method 129 | ) 130 | 131 | # API Gateway CDK Nag suppressions 132 | NagSuppressions.add_resource_suppressions( 133 | api, 134 | [ 135 | { 136 | "id": "AwsSolutions-APIG2", 137 | "reason": "Validation is performed by backend Lambda function." 138 | } 139 | ] 140 | ) 141 | NagSuppressions.add_resource_suppressions_by_path( 142 | self, 143 | f"/{self.stack_name}/GitHubReviewAPI/DeploymentStage.prod/Resource", 144 | [ 145 | { 146 | "id": "AwsSolutions-APIG3", 147 | "reason": "This solution is not intended for production. WAF not required for this solution." 148 | } 149 | ] 150 | ) 151 | NagSuppressions.add_resource_suppressions( 152 | review_method, 153 | [ 154 | { 155 | "id": "AwsSolutions-APIG4", 156 | "reason": "This is a webhook endpoint that uses GitHub's webhook secret for authentication" 157 | }, 158 | { 159 | "id": "AwsSolutions-COG4", 160 | "reason": "This is a webhook endpoint that doesn't require Cognito authentication" 161 | } 162 | ] 163 | ) 164 | 165 | CfnOutput(self, "GitHubReviewAPIUrl", value=api.url) -------------------------------------------------------------------------------- /improving-code-quality-reviews/requirements.txt: -------------------------------------------------------------------------------- 1 | aws-cdk-lib==2.173.2 2 | constructs>=10.0.0,<11.0.0 3 | cdk-nag==2.34.23 4 | -------------------------------------------------------------------------------- /streamline-incident-response/.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | package-lock.json 3 | __pycache__ 4 | .pytest_cache 5 | .venv 6 | *.egg-info 7 | 8 | # CDK asset staging directory 9 | .cdk.staging 10 | cdk.out 11 | -------------------------------------------------------------------------------- /streamline-incident-response/README.md: -------------------------------------------------------------------------------- 1 | # Streamline Incident Response with AWS Services 2 | 3 | This project implements an automated incident response system using AWS services to streamline the process of generating incident reports and searching previous incidents. 4 | 5 | The Streamline Incident Response system is designed to automate the process of gathering relevant information when an incident occurs, generate comprehensive incident reports, and provide quick access to historical incident data for faster resolution. 6 | 7 | ![Architecture Diagram](architecture.png) 8 | 9 | ## Repository Structure 10 | 11 | The repository is structured as an AWS CDK application with Lambda functions organized in the `functions` directory. The main CDK stack is defined in `streamline_incident_response_stack.py`. 12 | 13 | Key Files: 14 | - `app.py`: The entry point for the CDK application. 15 | - `cdk.json`: Configuration file for the CDK application. 16 | - `streamline_incident_response_stack.py`: Defines the main infrastructure stack. 17 | 18 | Integration points: 19 | - CloudTrail: Used to retrieve relevant events during an incident. 20 | - Slack: Integration for retrieving Slack messages related to an incident. 21 | - Amazon Bedrock: Used for generating incident reports and searching previous incidents. 22 | - AWS Step Functions: Orchestrates the incident response workflow. 23 | 24 | ## Usage Instructions 25 | 26 | ### Prerequisites 27 | 28 | - Python 3.12 29 | - AWS CDK CLI 30 | - AWS account and configured credentials 31 | - Jira account with API access 32 | 33 | ### Amazon Bedrock Setup 34 | 35 | 1. Ensure you deploy this architecture in the following region 36 | - US East 1 (N. Virginia) 37 | 38 | 2. Enable the following foundation models in the Amazon Bedrock console: 39 | - Claude 3.5 Sonnet v2 (Anthropic) 40 | - Claude 3 Haiku (Anthropic) 41 | 42 | Note: You must explicitly enable each model in your AWS account before you can use them. To enable the models: 43 | - Navigate to the Amazon Bedrock console 44 | - Select "Model access" in the left navigation pane 45 | - Choose "Manage model access" 46 | - Select the required models and choose "Request model access" 47 | 48 | ### Configuration 49 | 50 | Before deploying, you need to set up the following environment variables: 51 | 52 | - `SLACK_API_TOKEN`: Your Slack API token for retrieving messages. 53 | - `SLACK_CHANNEL`: The Slack channel ID to monitor for incident-related messages. 54 | 55 | You can set these in the CDK context or as environment variables. 56 | 57 | ### Installation 58 | 59 | Prerequisites: 60 | - Python 3.12 or later 61 | - AWS CDK CLI v2.x 62 | - AWS CLI configured with appropriate credentials 63 | 64 | To install the project: 65 | 66 | 1. Clone the repository: 67 | ``` 68 | git clone https://github.com/aws-samples/genai-for-devops.git 69 | cd streamline-incident-response 70 | ``` 71 | 72 | 2. Install the required dependencies: 73 | ``` 74 | pip install -r requirements.txt 75 | ``` 76 | 77 | 3. Configure your AWS credentials: 78 | ``` 79 | aws configure 80 | ``` 81 | 82 | 4. Deploy the CDK stack: 83 | ``` 84 | cdk deploy --context slack_api_token=$SLACK_API_TOKEN --context slack_channel=$SLACK_CHANNEL_ID 85 | ``` 86 | 87 | ### AWS Chatbot Setup 88 | 89 | To integrate [AWS Chatbot](https://aws.amazon.com/chatbot/) with this project, follow these steps: 90 | 91 | 1. Set up AWS Chatbot: 92 | - Go to the AWS Management Console and navigate to AWS Chatbot. 93 | - Click on "Configure new client" and select Slack as the chat client. 94 | - Follow the prompts to authorize AWS Chatbot in your Slack workspace. 95 | 96 | 2. Create a Slack channel for incident alerts: 97 | - In Slack, create a new channel (e.g., #incident-alerts) or use an existing one. 98 | 99 | 3. Configure AWS Chatbot for your Slack channel: 100 | - In AWS Chatbot, create a new configuration for the Slack channel you created. 101 | - Select the IAM role that has permissions to invoke Lambda functions. 102 | 103 | 4. Set up custom actions for CloudWatch alarms: 104 | - In AWS Chatbot, go to the "Configured clients" section and select your Slack configuration. 105 | - Under "Notifications", enable CloudWatch alarms. 106 | - In the "Custom actions" section, add two new custom actions: 107 | a. For generating reports: 108 | - Name: Generate Incident Report 109 | - Lambda function: Select the `chatbot_trigger_generate_report` function 110 | b. For searching previous incidents: 111 | - Name: Search Previous Incidents 112 | - Lambda function: Select the `chatbot_trigger_search_previous_incidents` function 113 | 114 | 5. Configure CloudWatch alarms: 115 | - Go to CloudWatch in the AWS Management Console. 116 | - For each alarm you want to integrate: 117 | - Edit the alarm and go to the "Actions" tab. 118 | - Add an action for the "In alarm" state and select "Send notification to AWS Chatbot". 119 | - Choose the Slack channel configuration you created earlier. 120 | 121 | Now, when a CloudWatch alarm is triggered, it will send a notification to your Slack channel. You can use the custom actions to generate an incident report or search for previous incidents directly from Slack. 122 | 123 | ### Getting Started 124 | 125 | Once deployed and AWS Chatbot is set up, the system will automatically respond to CloudWatch alarms: 126 | 127 | 1. When an alarm is triggered, a notification is sent to the configured Slack channel. 128 | 2. Use the "Generate Incident Report" custom action in Slack to start the incident response process. 129 | 3. The `TriggerGenerateReport` Lambda function is invoked, which starts the Step Functions workflow: 130 | - Retrieves relevant CloudTrail events and Slack messages. 131 | - Generates a markdown incident report using Amazon Bedrock. 132 | - Uploads the report to an S3 bucket. 133 | 4. The uploaded report triggers an ingestion job to add the incident data to the Bedrock Knowledge Base. 134 | 5. When the alarm resolves, use the "Search Previous Incidents" custom action in Slack to find similar past incidents. 135 | 136 | ### Common Use Cases 137 | 138 | 1. Generating an Incident Report: 139 | This happens automatically when you use the "Generate Incident Report" custom action in Slack after a CloudWatch alarm is triggered. 140 | 141 | 2. Searching Previous Incidents: 142 | Use the "Search Previous Incidents" custom action in Slack when an alarm resolves. This invokes the `ChatbotTriggerSearchPreviousIncidents` Lambda function. 143 | 144 | ### Troubleshooting 145 | 146 | Common issues and solutions: 147 | 148 | 1. Problem: Lambda function fails due to missing permissions 149 | - Error message: "AccessDeniedException: User is not authorized to perform: [action] on resource: [resource ARN]" 150 | - Diagnostic process: 151 | a. Check the CloudWatch logs for the specific Lambda function 152 | b. Identify the exact permission missing from the error message 153 | - Solution: Add the required permission to the Lambda function's IAM role in the CDK stack 154 | 155 | 2. Problem: Step Functions execution fails 156 | - Error message: "States.TaskFailed: Execution failed due to an error in Lambda function" 157 | - Diagnostic process: 158 | a. Open the Step Functions console and locate the failed execution 159 | b. Identify which state failed and check its input/output 160 | c. Review the CloudWatch logs for the corresponding Lambda function 161 | - Solution: Debug the specific Lambda function that failed within the Step Functions workflow 162 | 163 | ## Data Flow 164 | 165 | The Streamline Incident Response system processes incident data through several stages: 166 | 167 | 1. Incident Detection: CloudWatch alarm triggers a notification in Slack. 168 | 2. Report Generation: User initiates the process using the "Generate Incident Report" custom action in Slack. 169 | 3. Data Gathering: Parallel execution of `LookupCloudTrailEvents` and `LookupSlackEvents` Lambda functions. 170 | 4. Report Generation: `CreateMarkdownReport` Lambda function uses gathered data and Amazon Bedrock to create an incident report. 171 | 5. Storage: `UploadMarkdownReport` Lambda function stores the report in an S3 bucket. 172 | 6. Indexing: `StartIngestionJob` Lambda function is triggered by the S3 upload, ingesting the report into the Bedrock Knowledge Base. 173 | 7. Retrieval: User initiates the "Search Previous Incidents" custom action in Slack, which invokes the `ChatbotTriggerSearchPreviousIncidents` Lambda function to search past incidents using the Knowledge Base. 174 | 175 | Note: The Step Functions workflow orchestrates the core incident response process, ensuring each step is executed in the correct order and handling any potential failures. 176 | 177 | ## Infrastructure 178 | 179 | The Streamline Incident Response system is built using AWS CDK and consists of the following key resources: 180 | 181 | Lambda Functions: 182 | - `LookupCloudTrailEventsLambda`: Retrieves relevant CloudTrail events during an incident. 183 | - `LookupSlackEventsLambda`: Fetches Slack messages related to an incident. 184 | - `CreateMarkdownReportLambda`: Generates an incident report using Amazon Bedrock. 185 | - `UploadMarkdownReportLambda`: Uploads the generated report to S3. 186 | - `TriggerGenerateReportLambda`: Initiates the incident response workflow. 187 | - `StartIngestionJobLambda`: Triggers the ingestion of new incident reports into the Knowledge Base. 188 | - `ChatbotTriggerSearchPreviousIncidentsLambda`: Searches for previous similar incidents. 189 | 190 | S3: 191 | - `IncidentBucket`: Stores generated incident reports. 192 | 193 | Step Functions: 194 | - `GenerateReportWorkflow`: Orchestrates the incident response process. 195 | 196 | Bedrock: 197 | - `IncidentReportsKnowledgeBase`: Stores and indexes incident reports for quick retrieval. 198 | - `IncidentReportDataSource`: Connects the S3 bucket to the Knowledge Base. 199 | 200 | IAM: 201 | - Various IAM roles and policies are created to grant necessary permissions to Lambda functions and other resources. 202 | 203 | The infrastructure is defined in the `StreamlineIncidentResponseStack` class within the `streamline_incident_response_stack.py` file. This stack creates all the necessary AWS resources and sets up the relationships between them to enable the automated incident response workflow. -------------------------------------------------------------------------------- /streamline-incident-response/app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import os 3 | 4 | import aws_cdk as cdk 5 | 6 | from streamline_incident_response.streamline_incident_response_stack import StreamlineIncidentResponseStack 7 | from cdk_nag import AwsSolutionsChecks 8 | 9 | app = cdk.App() 10 | StreamlineIncidentResponseStack(app, "StreamlineIncidentResponseStack",) 11 | 12 | cdk.Aspects.of(app).add(AwsSolutionsChecks(verbose=True)) 13 | app.synth() 14 | -------------------------------------------------------------------------------- /streamline-incident-response/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/architecture.png -------------------------------------------------------------------------------- /streamline-incident-response/cdk.json: -------------------------------------------------------------------------------- 1 | { 2 | "app": "python3 app.py", 3 | "watch": { 4 | "include": [ 5 | "**" 6 | ], 7 | "exclude": [ 8 | "README.md", 9 | "cdk*.json", 10 | "requirements*.txt", 11 | "source.bat", 12 | "**/__init__.py", 13 | "**/__pycache__", 14 | "tests" 15 | ] 16 | }, 17 | "context": { 18 | "@aws-cdk/aws-lambda:recognizeLayerVersion": true, 19 | "@aws-cdk/core:checkSecretUsage": true, 20 | "@aws-cdk/core:target-partitions": [ 21 | "aws", 22 | "aws-cn" 23 | ], 24 | "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, 25 | "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, 26 | "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, 27 | "@aws-cdk/aws-iam:minimizePolicies": true, 28 | "@aws-cdk/core:validateSnapshotRemovalPolicy": true, 29 | "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, 30 | "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, 31 | "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, 32 | "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, 33 | "@aws-cdk/core:enablePartitionLiterals": true, 34 | "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, 35 | "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, 36 | "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, 37 | "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, 38 | "@aws-cdk/aws-route53-patters:useCertificate": true, 39 | "@aws-cdk/customresources:installLatestAwsSdkDefault": false, 40 | "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, 41 | "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, 42 | "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, 43 | "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, 44 | "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, 45 | "@aws-cdk/aws-redshift:columnId": true, 46 | "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true, 47 | "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, 48 | "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, 49 | "@aws-cdk/aws-kms:aliasNameRef": true, 50 | "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, 51 | "@aws-cdk/core:includePrefixInUniqueNameGeneration": true, 52 | "@aws-cdk/aws-efs:denyAnonymousAccess": true, 53 | "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true, 54 | "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true, 55 | "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true, 56 | "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true, 57 | "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true, 58 | "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true, 59 | "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true, 60 | "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true, 61 | "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true, 62 | "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true, 63 | "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true, 64 | "@aws-cdk/aws-eks:nodegroupNameAttribute": true, 65 | "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true, 66 | "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true, 67 | "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false, 68 | "@aws-cdk/aws-s3:keepNotificationInImportedBucket": false, 69 | "@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true, 70 | "@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true, 71 | "@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true, 72 | "@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true, 73 | "@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true, 74 | "@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true, 75 | "@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true, 76 | "@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true, 77 | "@aws-cdk/aws-ec2:bastionHostUseAmazonLinux2023ByDefault": true 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /streamline-incident-response/requirements.txt: -------------------------------------------------------------------------------- 1 | aws-cdk-lib==2.173.2 2 | constructs>=10.0.0,<11.0.0 3 | cdklabs.generative-ai-cdk-constructs==0.1.288 4 | cdk-nag==2.34.23 -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_generate_report/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_generate_report/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_generate_report/events/example_event.json: -------------------------------------------------------------------------------- 1 | { 2 | "metricAlarmName": "ListCategoriesErrors" 3 | } -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_generate_report/index.py: -------------------------------------------------------------------------------- 1 | 2 | import ast 3 | import boto3 4 | import json 5 | import os 6 | 7 | # Initialize AWS service clients 8 | cloudwatch_client = boto3.client('cloudwatch') 9 | sfn_client = boto3.client('stepfunctions') 10 | 11 | def lambda_handler(event, context): 12 | """ 13 | Lambda handler that triggers a Step Functions workflow to generate incident reports 14 | based on CloudWatch alarm history. 15 | 16 | Args: 17 | event: The Lambda event object containing the incident details 18 | context: The Lambda context object 19 | 20 | Returns: 21 | dict: Response containing the execution ARN and status code 22 | """ 23 | 24 | # Extract alarm name from the event 25 | alarm_name = event['metricAlarmName'] 26 | 27 | # Get the alarm history from CloudWatch 28 | response = cloudwatch_client.describe_alarm_history( 29 | AlarmName=alarm_name, 30 | HistoryItemType='StateUpdate', 31 | MaxRecords=100 # Adjust as needed 32 | ) 33 | 34 | # Find the last OK and ALARM state transitions 35 | last_ok_time = None 36 | last_alarm_time = None 37 | for item in response['AlarmHistoryItems']: 38 | if item['HistoryItemType'] == 'StateUpdate': 39 | # Parse the history data 40 | history_data = item['HistoryData'] 41 | if history_data: 42 | # Replace boolean strings and use ast.literal_eval 43 | history_data = history_data.replace('true', 'True').replace('false', 'False') 44 | history_data = ast.literal_eval(history_data) 45 | new_state = history_data['newState']['stateValue'] 46 | timestamp = item['Timestamp'] 47 | 48 | # Store the first occurrence of OK and ALARM states 49 | if new_state == 'OK' and last_ok_time == None: 50 | last_ok_time = timestamp 51 | elif new_state == 'ALARM' and last_alarm_time == None: 52 | last_alarm_time = timestamp 53 | 54 | # Return error if we couldn't find both state transitions 55 | if last_ok_time is None or last_alarm_time is None: 56 | print(f'Unable to determine state transitions for alarm "{alarm_name}"') 57 | return { 58 | 'statusCode': 400, 59 | 'body': 'Report generation failed' 60 | } 61 | 62 | # Format timestamps for the event 63 | event['lastAlarmTime'] = last_alarm_time.strftime('%Y-%m-%dT%H:%M:%S.%fZ') 64 | event['lastOkTime'] = last_ok_time.strftime('%Y-%m-%dT%H:%M:%S.%fZ') 65 | 66 | # Start the Step Functions workflow with the enhanced event data 67 | sfn_client.start_execution( 68 | stateMachineArn=os.environ['STATE_MACHINE_ARN'], 69 | input=json.dumps(event) 70 | ) 71 | 72 | # Return success response 73 | return { 74 | 'statusCode': 200, 75 | 'body': 'Report Generating' 76 | } -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_search_previous_incidents/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_search_previous_incidents/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_search_previous_incidents/events/example_event.json: -------------------------------------------------------------------------------- 1 | { 2 | "metricAlarmName": "ListCategoriesErrors", 3 | "namespace": "AWS/Lambda", 4 | "metric": "Errors" 5 | } -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/chatbot_trigger_search_previous_incidents/index.py: -------------------------------------------------------------------------------- 1 | import json 2 | import boto3 3 | import os 4 | 5 | # Initialize the Bedrock client for AI operations 6 | bedrock_client = boto3.client('bedrock-agent-runtime') 7 | 8 | def lambda_handler(event, context): 9 | """ 10 | Lambda handler that generates a playbook based on previous incidents using Amazon Bedrock. 11 | 12 | Args: 13 | event (dict): Contains alarm information including: 14 | - metricAlarmName: Name of the CloudWatch alarm 15 | - namespace: The CloudWatch namespace 16 | - metric: The specific metric that triggered the alarm 17 | context: Lambda context object 18 | 19 | Returns: 20 | dict: Response containing the generated playbook in markdown format 21 | """ 22 | 23 | # Format the prompt for Bedrock to generate a playbook based on alarm details 24 | user_message = """Create a playbook based on previous incidents of {} helping the resolver to identify and resolve the issue. 25 | The CloudWatch alarm is for {} namespace, for {} metric. 26 | Use other knowledge of this alarm type to further influence the output where there might be gaps. 27 | Format in markdown only 28 | """.format(event['metricAlarmName'], event['namespace'], event['metric']) 29 | 30 | # Query Bedrock knowledge base to generate response based on previous incidents 31 | response = bedrock_client.retrieve_and_generate( 32 | input={ 33 | 'text': user_message 34 | }, 35 | retrieveAndGenerateConfiguration={ 36 | 'type': 'KNOWLEDGE_BASE', 37 | 'knowledgeBaseConfiguration': { 38 | 'knowledgeBaseId': os.environ['KNOWLEDGE_BASE_ID'], 39 | 'modelArn': os.environ['MODEL_ID'] 40 | } 41 | } 42 | ) 43 | 44 | # Extract the generated playbook from the response 45 | response_text = response['output']['text'] 46 | 47 | # Return the formatted playbook with DevOpsBot attribution 48 | return { 49 | 'statusCode': 200, 50 | 'body': "*Guidance provided by DevOpsBot*\n```{}```".format(response_text) 51 | } 52 | -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/create_markdown_report/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/create_markdown_report/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/create_markdown_report/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | from datetime import datetime 3 | import json 4 | import os 5 | 6 | # Initialize Bedrock client for AI operations 7 | bedrock = boto3.client('bedrock-runtime') 8 | 9 | def lambda_handler(event, context): 10 | """ 11 | Lambda handler that creates a markdown incident report using Amazon Bedrock. 12 | 13 | Args: 14 | event (dict): Contains incident information including: 15 | - lastAlarmTime: Timestamp when alarm was triggered 16 | - lastOkTime: Timestamp when system was last OK 17 | - metricAlarmName: Name of the CloudWatch alarm 18 | - parallelResults: Results from parallel executions (CloudTrail and Slack events) 19 | 20 | Returns: 21 | dict: Response containing the generated markdown report 22 | """ 23 | 24 | # Parse timestamps from the event 25 | last_alarm_time = datetime.fromisoformat(event['lastAlarmTime']) 26 | last_ok_time = datetime.fromisoformat(event['lastOkTime']) 27 | alarm_name = event['metricAlarmName'] 28 | 29 | # Initialize empty strings for events data 30 | events = "" 31 | slack_messages = "" 32 | 33 | # Extract CloudTrail and Slack events from parallel execution results 34 | for parallel_result in event["parallelResults"]: 35 | if "cloudtrail_events" in parallel_result["Payload"]: 36 | events = parallel_result["Payload"]["cloudtrail_events"] 37 | 38 | if "slack_events" in parallel_result["Payload"]: 39 | slack_messages = parallel_result["Payload"]["slack_events"] 40 | 41 | # Generate the incident report using collected data 42 | incident_report = generate_incident_report(last_alarm_time, last_ok_time, alarm_name, events, slack_messages) 43 | 44 | # Return the generated markdown report 45 | body = { 46 | 'markdown': incident_report 47 | } 48 | 49 | return body 50 | 51 | def generate_incident_report(from_time, to_time, alarm_name, cloudtrail_events, slack_messages): 52 | """ 53 | Generates a detailed incident report using Amazon Bedrock. 54 | 55 | Args: 56 | from_time (datetime): Start time of the incident 57 | to_time (datetime): End time of the incident 58 | alarm_name (str): Name of the CloudWatch alarm 59 | cloudtrail_events (str): Related CloudTrail events 60 | slack_messages (str): Related Slack messages 61 | 62 | Returns: 63 | str: Generated markdown report 64 | """ 65 | 66 | # Create prompt template for the AI model 67 | user_message = """You are a incident manager responsible for writing up an incident report after it has happened. 68 | The following sections should be in the incident report. 69 | 1. Incident Summary 70 | 2. Timeline of Events 71 | 3. Root Cause Analysis 72 | 4. Impact Assessment 73 | 5. Resolution and Recovery 74 | 6. Lessons Learned 75 | 7. Action Items and Recommendations 76 | You must use only the following information provided to fill out and omit any headings where you do not have enough detail. Focus on actions that are directly related to this. 77 | From time: {} 78 | To time: {} 79 | Alarm name: {} 80 | CloudTrail events: {} 81 | Slack messages: {} 82 | Create the output in markdown format. 83 | """.format(from_time.isoformat(), to_time.isoformat(), alarm_name, cloudtrail_events, slack_messages) 84 | 85 | # Format conversation for Bedrock 86 | conversation = [ 87 | { 88 | "role": "user", 89 | "content": [{"text": user_message}], 90 | } 91 | ] 92 | 93 | # Send request to Bedrock model with specified configuration 94 | response = bedrock.converse( 95 | modelId=os.environ["MODEL_ID"], 96 | messages=conversation, 97 | inferenceConfig={"maxTokens": 4096, "temperature": 0.5, "topP": 0.9}, 98 | ) 99 | 100 | # Extract and return the generated report 101 | response_text = response["output"]["message"]["content"][0]["text"] 102 | return response_text 103 | -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/lookup_cloudtrail_events/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/lookup_cloudtrail_events/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/lookup_cloudtrail_events/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | from datetime import datetime 3 | import json 4 | 5 | # Initialize CloudTrail client for event lookups 6 | cloudtrail = boto3.client('cloudtrail') 7 | 8 | def lambda_handler(event, context): 9 | """ 10 | Lambda handler that looks up CloudTrail events within a specified time window. 11 | 12 | Args: 13 | event (dict): Contains time window information including: 14 | - lastAlarmTime: ISO formatted timestamp when alarm was triggered 15 | - lastOkTime: ISO formatted timestamp when system was last OK 16 | 17 | Returns: 18 | dict: Response containing CloudTrail events found within the time window 19 | """ 20 | 21 | # Parse the time window from ISO formatted strings to datetime objects 22 | last_alarm_time = datetime.fromisoformat(event['lastAlarmTime']) 23 | last_ok_time = datetime.fromisoformat(event['lastOkTime']) 24 | 25 | # Query CloudTrail for events between start_time and end_time 26 | response = cloudtrail.lookup_events( 27 | StartTime=last_alarm_time, 28 | EndTime=last_ok_time, 29 | MaxResults=100 30 | ) 31 | 32 | # Format response with CloudTrail events 33 | body = { 34 | 'cloudtrail_events': json.dumps(response['Events'], cls=CustomJSONEncoder) 35 | } 36 | 37 | return body 38 | 39 | class CustomJSONEncoder(json.JSONEncoder): 40 | """ 41 | Custom JSON encoder to handle datetime objects in CloudTrail events. 42 | 43 | Extends: 44 | json.JSONEncoder: Standard JSON encoder 45 | """ 46 | def default(self, obj): 47 | """ 48 | Convert datetime objects to ISO format string. 49 | 50 | Args: 51 | obj: Object to serialize 52 | 53 | Returns: 54 | str: ISO formatted datetime string if obj is datetime, 55 | otherwise delegates to parent encoder 56 | """ 57 | if isinstance(obj, datetime): 58 | return obj.isoformat() 59 | return super().default(obj) -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/lookup_slack_events/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/lookup_slack_events/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/lookup_slack_events/index.py: -------------------------------------------------------------------------------- 1 | # Import required libraries for datetime handling, environment variables, and Slack API 2 | from datetime import datetime 3 | import os 4 | import json 5 | from slack_sdk import WebClient 6 | from slack_sdk.errors import SlackApiError 7 | 8 | # Initialize Slack client with token from environment variables 9 | slack_client = WebClient(token=os.environ["SLACK_TOKEN"]) 10 | 11 | def lambda_handler(event, context): 12 | """ 13 | Lambda handler that retrieves Slack messages from a specified channel within a time window. 14 | 15 | Args: 16 | event (dict): Contains time window information including: 17 | - lastAlarmTime: ISO formatted timestamp when alarm was triggered 18 | - lastOkTime: ISO formatted timestamp when system was last OK 19 | 20 | Returns: 21 | dict: Response containing Slack messages found within the time window 22 | """ 23 | 24 | # Parse the time window from ISO formatted strings to datetime objects 25 | last_alarm_time = datetime.fromisoformat(event['lastAlarmTime']) 26 | last_ok_time = datetime.fromisoformat(event['lastOkTime']) 27 | 28 | # Retrieve Slack messages within the time window 29 | slack_messages = get_slack_messages(last_alarm_time, last_ok_time) 30 | 31 | # Format response with Slack messages 32 | body = { 33 | 'slack_events': json.dumps(slack_messages) 34 | } 35 | 36 | return body 37 | 38 | def get_slack_messages(from_time, to_time): 39 | """ 40 | Retrieves all Slack messages from the configured channel within the specified time window. 41 | Handles pagination to get all messages if there are more than the default limit. 42 | 43 | Args: 44 | from_time (datetime): Start time to fetch messages from 45 | to_time (datetime): End time to fetch messages until 46 | 47 | Returns: 48 | list: Collection of Slack messages within the time window 49 | """ 50 | 51 | messages = [] 52 | oldest = from_time.timestamp() 53 | latest = to_time.timestamp() 54 | cursor = None 55 | 56 | while True: 57 | # Query Slack API for messages, handling pagination with cursor 58 | response = slack_client.conversations_history( 59 | channel=os.environ['SLACK_CHANNEL'], 60 | oldest=oldest, 61 | latest=latest, 62 | cursor=cursor 63 | ) 64 | 65 | # Add retrieved messages to our collection 66 | messages.extend(response.data['messages']) 67 | 68 | if response.data['has_more']: 69 | cursor = response.data['response_metadata']['next_cursor'] 70 | else: 71 | break 72 | 73 | return messages 74 | -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/lookup_slack_events/requirements.txt: -------------------------------------------------------------------------------- 1 | slack_sdk -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/start_ingestion_job/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/start_ingestion_job/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/start_ingestion_job/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import os 3 | import uuid 4 | 5 | # Initialize Bedrock Agent client for knowledge base operations 6 | bedrock_agent = boto3.client('bedrock-agent') 7 | 8 | def lambda_handler(event, context): 9 | """ 10 | Lambda handler that starts an ingestion job for new documents in the knowledge base. 11 | This function is typically triggered by S3 events when new incident reports are uploaded. 12 | 13 | Args: 14 | event: The Lambda event object (typically S3 event notification) 15 | context: The Lambda context object 16 | 17 | Returns: 18 | dict: Response indicating successful job initiation 19 | """ 20 | 21 | # Start ingestion job with unique client token for idempotency 22 | bedrock_agent.start_ingestion_job( 23 | clientToken=str(uuid.uuid4()), 24 | dataSourceId=os.environ['DATA_SOURCE_ID'], 25 | knowledgeBaseId=os.environ['KNOWLEDGE_BASE_ID'] 26 | ) 27 | 28 | # Return success response 29 | return { 30 | "status": "Success" 31 | } 32 | -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/upload_markdown_report/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/genai-for-devops/2d643721d1f1f9f06de78d9f81afb87ed3632163/streamline-incident-response/streamline_incident_response/functions/upload_markdown_report/__init__.py -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/functions/upload_markdown_report/index.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | from datetime import datetime 3 | import json 4 | import os 5 | 6 | #S3 client 7 | s3 = boto3.client('s3') 8 | 9 | def lambda_handler(event, context): 10 | """ 11 | Lambda handler that uploads a generated markdown incident report to S3. 12 | 13 | Args: 14 | event (dict): Contains incident report information including: 15 | - lastAlarmTime: ISO formatted timestamp of the alarm 16 | - metricAlarmName: Name of the CloudWatch alarm 17 | - reportResult: Contains the generated markdown report 18 | 19 | Returns: 20 | dict: Response indicating successful upload 21 | """ 22 | 23 | # Parse alarm time and name from the event 24 | last_alarm_time = datetime.fromisoformat(event['lastAlarmTime']) 25 | alarm_name = event['metricAlarmName'] 26 | 27 | # Extract the markdown report from the event 28 | incident_report = event['reportResult']['Payload']['markdown'] 29 | 30 | # Upload the report to S3 with timestamp and alarm name in the key 31 | s3.put_object( 32 | Body=incident_report.encode('utf-8'), 33 | Bucket=os.environ['S3_BUCKET_NAME'], 34 | Key="{}_{}.md".format(last_alarm_time.isoformat(), alarm_name) 35 | ) 36 | 37 | # Return success response 38 | return { 39 | "result": "Success" 40 | } 41 | -------------------------------------------------------------------------------- /streamline-incident-response/streamline_incident_response/streamline_incident_response_stack.py: -------------------------------------------------------------------------------- 1 | from aws_cdk import ( 2 | Duration, 3 | Stack, 4 | aws_iam as iam, 5 | aws_lambda as _lambda, 6 | aws_logs as logs, 7 | aws_stepfunctions as sfn, 8 | aws_stepfunctions_tasks as task, 9 | aws_s3 as s3, 10 | aws_s3_notifications as s3n 11 | ) 12 | from constructs import Construct 13 | from cdk_nag import NagSuppressions 14 | 15 | # Import Bedrock constructs for AI integration 16 | from cdklabs.generative_ai_cdk_constructs import ( 17 | bedrock 18 | ) 19 | 20 | class StreamlineIncidentResponseStack(Stack): 21 | """Stack that implements an automated incident response workflow.""" 22 | 23 | def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: 24 | super().__init__(scope, construct_id, **kwargs) 25 | 26 | # Create S3 bucket for storing access logs 27 | logging_bucket = s3.Bucket( 28 | self, 29 | "IncidentBucketLogs", 30 | encryption=s3.BucketEncryption.S3_MANAGED, 31 | enforce_ssl=True, 32 | block_public_access=s3.BlockPublicAccess.BLOCK_ALL 33 | ) 34 | 35 | # Create S3 bucket for storing incident reports with public access blocked 36 | incident_bucket = s3.Bucket( 37 | self, 38 | "IncidentBucket", 39 | block_public_access=s3.BlockPublicAccess.BLOCK_ALL, 40 | enforce_ssl=True, 41 | server_access_logs_bucket=logging_bucket, 42 | server_access_logs_prefix="incident-bucket-logs/" 43 | ) 44 | 45 | # Create Lambda function to query CloudTrail events 46 | lookup_cloudtrail_events_function = create_lambda_function(self, "LookupCloudTrailEvents", "lookup_cloudtrail_events") 47 | lookup_cloudtrail_events_function.add_to_role_policy( 48 | statement=iam.PolicyStatement( 49 | actions=[ 50 | "cloudtrail:LookupEvents" 51 | ], 52 | resources=[ 53 | "*", 54 | ] 55 | ) 56 | ) 57 | 58 | # Get Slack configuration from context 59 | slack_token = self.node.try_get_context("slack_api_token") 60 | slack_channel = self.node.try_get_context("slack_channel") 61 | 62 | # Create Lambda function to query Slack events with required environment variables 63 | lookup_slack_events_function = create_lambda_function(self, "LookupSlackEvents", "lookup_slack_events", {"SLACK_TOKEN": slack_token, "SLACK_CHANNEL": slack_channel}, include_dependencies=True) 64 | 65 | # Create Step Functions tasks for the Lambda functions 66 | lookup_cloudtrail_events_function_task = task.LambdaInvoke(self, "LookupCloudTrailEventsTask", lambda_function=lookup_cloudtrail_events_function) 67 | lookup_slack_events_function_task = task.LambdaInvoke(self, "LookupSlackEventsTask", lambda_function=lookup_slack_events_function) 68 | 69 | # Create parallel state to execute CloudTrail and Slack lookups simultaneously 70 | parallel_state = sfn.Parallel(self, "ParallelState", result_path="$.parallelResults") 71 | parallel_state.branch(lookup_cloudtrail_events_function_task) 72 | parallel_state.branch(lookup_slack_events_function_task) 73 | 74 | # Create Lambda function to generate markdown report using Bedrock 75 | create_markdown_report_function = create_lambda_function(self, "CreateMarkdownReport", "create_markdown_report", {"MODEL_ID": "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account)}) 76 | 77 | # Configure the report generation task with specific input/output paths 78 | create_markdown_report_function_task = task.LambdaInvoke( 79 | self, 80 | "CreateMarkdownReportTask", 81 | lambda_function=create_markdown_report_function, 82 | result_path="$.reportResult", # Store Lambda output in reportResult 83 | input_path="$", # Pass everything as input 84 | output_path="$" # Include all fields in the output 85 | ) 86 | # Grant Bedrock permissions to the report generation function 87 | create_markdown_report_function.add_to_role_policy( 88 | statement=iam.PolicyStatement( 89 | actions=[ 90 | "bedrock:InvokeModel" 91 | ], 92 | resources=[ 93 | "arn:aws:bedrock:us-east-1:{}:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0".format(self.account), 94 | "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 95 | "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0", 96 | "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0" 97 | ] 98 | ) 99 | ) 100 | 101 | # Create Lambda function to upload the generated report to S3 102 | upload_markdown_report_function = create_lambda_function(self, "UploadMarkdownReport", "upload_markdown_report", {"S3_BUCKET_NAME": incident_bucket.bucket_name}) 103 | upload_markdown_report_function_task = task.LambdaInvoke(self, "UploadMarkdownReportTask", lambda_function=upload_markdown_report_function) 104 | 105 | # Grant S3 write permissions to the upload function 106 | incident_bucket.grant_write(upload_markdown_report_function) 107 | 108 | # Chain the workflow steps together 109 | definition = parallel_state.next(create_markdown_report_function_task).next(upload_markdown_report_function_task) 110 | 111 | # Create Step Functions state machine with the workflow definition 112 | step_function = sfn.StateMachine( 113 | self, 114 | "GenerateReportWorkflow", 115 | definition_body=sfn.DefinitionBody.from_chainable(definition), 116 | logs=sfn.LogOptions( 117 | destination=logs.LogGroup( 118 | self, 119 | "GenerateReportWorkflowLogs", 120 | retention=logs.RetentionDays.ONE_DAY # Adjust retention as needed 121 | ), 122 | level=sfn.LogLevel.ALL, # Log all events 123 | include_execution_data=True # Include state input/output in logs 124 | ), 125 | tracing_enabled=True 126 | ) 127 | 128 | # Create Lambda function to trigger the report generation workflow 129 | trigger_generate_report_function = create_lambda_function(self, "TriggerGenerateReport", "chatbot_trigger_generate_report", {"STATE_MACHINE_ARN": step_function.state_machine_arn} ) 130 | 131 | # Grant CloudWatch permissions to the trigger function 132 | trigger_generate_report_function.add_to_role_policy( 133 | statement=iam.PolicyStatement( 134 | actions=[ 135 | "cloudwatch:DescribeAlarmHistory" 136 | ], 137 | resources=[ 138 | "*", 139 | ] 140 | ) 141 | ) 142 | 143 | # Grant permission to start the Step Functions workflow 144 | step_function.grant_start_execution(trigger_generate_report_function) 145 | 146 | # Create Bedrock knowledge base for incident reports 147 | knowledge_base = bedrock.KnowledgeBase(self, 'IncidentReportsKnowledgeBase', 148 | embeddings_model= bedrock.BedrockFoundationModel.TITAN_EMBED_TEXT_V2_1024, 149 | instruction='Use this knowledge base to lookup previous incidents to create a dynamic runbook' 150 | ) 151 | 152 | # Configure S3 as data source for the knowledge base 153 | data_source = bedrock.S3DataSource(self, 'IncidentReportDataSource', 154 | bucket= incident_bucket, 155 | knowledge_base=knowledge_base, 156 | data_source_name='incident_reports', 157 | chunking_strategy= bedrock.ChunkingStrategy.FIXED_SIZE, 158 | ) 159 | 160 | NagSuppressions.add_stack_suppressions( 161 | self, 162 | suppressions=[ 163 | { 164 | "id": "AwsSolutions-L1", 165 | "reason": "Custom resource Lambda functions are automatically generated by CDK with specific runtime requirements", 166 | "applies_to": ["*CustomResourcesFunction*"] 167 | } 168 | ] 169 | ) 170 | 171 | # Create Lambda function to start knowledge base ingestion jobs 172 | start_ingestion_job_function = create_lambda_function(self, "StartIngestionJob", "start_ingestion_job", {"KNOWLEDGE_BASE_ID": knowledge_base.knowledge_base_id, "DATA_SOURCE_ID": data_source.data_source_id}) 173 | 174 | # Grant Bedrock permissions for ingestion 175 | start_ingestion_job_function.add_to_role_policy( 176 | statement=iam.PolicyStatement( 177 | actions=[ 178 | "bedrock:StartIngestionJob" 179 | ], 180 | resources=[ 181 | knowledge_base.knowledge_base_arn 182 | ] 183 | ) 184 | ) 185 | 186 | # Configure S3 event notification to trigger ingestion on new files 187 | incident_bucket.add_event_notification( 188 | s3.EventType.OBJECT_CREATED, 189 | s3n.LambdaDestination(start_ingestion_job_function) 190 | ) 191 | 192 | # Suppress at the stack level for any BucketNotificationsHandler 193 | NagSuppressions.add_stack_suppressions( 194 | self, 195 | suppressions=[ 196 | { 197 | "id": "AwsSolutions-IAM4", 198 | "reason": "This is an automatically created role by CDK for S3 bucket notifications that requires these permissions to function", 199 | "applies_to": ["Resource::*"], 200 | "resource_path": "*BucketNotificationsHandler*" 201 | }, 202 | { 203 | "id": "AwsSolutions-IAM5", 204 | "reason": "This is an automatically created role by CDK for S3 bucket notifications that requires these permissions to function", 205 | "applies_to": ["Resource::*"], 206 | "resource_path": "*BucketNotificationsHandler*" 207 | } 208 | ] 209 | ) 210 | 211 | # Create Lambda function to search previous incidents 212 | chatbot_trigger_search_previous_incidents_function = create_lambda_function(self, "ChatbotTriggerSearchPreviousIncidents", "chatbot_trigger_search_previous_incidents", {"KNOWLEDGE_BASE_ID": knowledge_base.knowledge_base_id, "MODEL_ID": "anthropic.claude-3-haiku-20240307-v1:0"}) 213 | 214 | # Grant Bedrock permissions for searching and generating responses 215 | chatbot_trigger_search_previous_incidents_function.add_to_role_policy( 216 | statement=iam.PolicyStatement( 217 | actions=[ 218 | "bedrock:Retrieve", 219 | "bedrock:RetrieveAndGenerate", 220 | "bedrock:InvokeModel" 221 | ], 222 | resources=[ 223 | "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0", 224 | knowledge_base.knowledge_base_arn 225 | ] 226 | ) 227 | ) 228 | 229 | 230 | def create_lambda_function(self, purpose: str, folder_name: str, environment={}, include_dependencies=False): 231 | """ 232 | Helper function to create Lambda functions with consistent configuration. 233 | 234 | Args: 235 | purpose: Identifier for the Lambda function 236 | folder_name: Name of the folder containing the function code 237 | environment: Dictionary of environment variables 238 | include_dependencies: Whether to include pip dependencies 239 | 240 | Returns: 241 | Lambda function construct 242 | """ 243 | # Define the command to copy assets 244 | command = ["bash", "-c", "cp -au . /asset-output"] 245 | 246 | # If dependencies are required, install them first 247 | if include_dependencies: 248 | command = ["bash", "-c", "pip install -r requirements.txt -t /asset-output && cp -au . /asset-output"] 249 | 250 | # Create and return the Lambda function 251 | lambda_function = _lambda.Function(self, 252 | "{}Lambda".format(purpose), 253 | handler="index.lambda_handler", 254 | runtime=_lambda.Runtime.PYTHON_3_13, 255 | code=_lambda.Code.from_asset( 256 | "./streamline_incident_response/functions/{}".format(folder_name), 257 | bundling=dict( 258 | image=_lambda.Runtime.PYTHON_3_13.bundling_image, 259 | command=command 260 | ) 261 | ), 262 | timeout=Duration.minutes(5), 263 | architecture=_lambda.Architecture.X86_64, 264 | environment=environment 265 | ) 266 | 267 | NagSuppressions.add_resource_suppressions( 268 | lambda_function.role, 269 | suppressions=[ 270 | { 271 | "id": "AwsSolutions-IAM4", 272 | "reason": "Lambda function requires wildcard permissions for logs as name is dynamically generated.", 273 | } 274 | ], 275 | apply_to_children=True 276 | ) 277 | 278 | return lambda_function --------------------------------------------------------------------------------