├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── images ├── dashboard1.jpg ├── dashboard2.jpg ├── quicksight.jpg ├── quicksight_new.png └── rif24architecture.png ├── lambda_functions ├── KMSRead_lambda.py └── lastUsed_lambda.py ├── member-account-kmsread-role.yaml ├── member-account-lastused-lambda.yaml ├── member-account-sample-kms-keys.yaml ├── putDynamo.py ├── security-account-athena.yaml ├── security-account-dynamodb.yaml └── security-account-kmsread-lambda.yaml /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tracking AWS KMS Key Policies using Amazon Quicksight 2 | 3 | ## Architecture 4 | ![Tracking AWS KMS Key Policies using Amazon Quicksight](images/rif24architecture.png) 5 | 6 | ## Overview 7 | 8 | This code talk describes a solution for building a KMS observability platform. 9 | Resources are deployed in one of two account types: 10 | 1. **Security Observability Account** - Hosts the central resources (e.g DynamoDB tables) and dashboarding 11 | 2. **Member Accounts** - Hosts the resources used to enable the scanning of KMS keys and Cloudtrail to be loaded into the dashboarding 12 | 13 | This git repository contains a services of CloudFormation scripts for deploying into these two accounts types. 14 | It consists of the following components: 15 | 16 | ### 1. Security Observability Account 17 | * ***security-account-dynamodb.yaml*** 18 | Deploys the following resources 19 | * *Accounts* DynamoDB table used to store a list of accounts for scanning 20 | * *Last Used* DynamoDB table used to store the date and time of when a KMS key was last used 21 | * ReadOnly DynamoDB IAM Role with a default name of `ReadDynamoDB-Role` 22 | * Write Access DynamoDB IAM role used to Put objects. Default name of `putToDynamoRole` 23 | 24 | * ***security-account-kmsread-lambda.yaml*** 25 | Deploys the following resources: 26 | * *KMSRead* Lambda. This runs from the Security Observability role. It reads the list of accounts from the *accounts* DynamoDB table, assumes a role into all member accounts & regions and then scans all KMS keys and policies before storing them in a CSV in S3 27 | * S3 Bucket used to store the CSV files 28 | * S3 Bucket policy allowing *KMSRead* Lambda to write to the bucket 29 | * IAM role for *KMSRead* Lambda. Has permissions to assume the *XA-KMSRead-Role* role in member accounts and read from the *accounts* DynamoDB table 30 | * EventBridge rule to trigger the Lambda on a cron schedule: every 24 hours 31 | 32 | * ***security-account-athena.yaml*** 33 | Deploys the following resources: 34 | * Athena workgroup 35 | * Requred Athena buckets 36 | * Associated IAM Roles for Athena 37 | * QuickSight IAM Role 38 | 39 | ### 2. Member Accounts 40 | * ***member-account-kmsread-role.yaml*** 41 | Deploys the following resources: 42 | * IAM role with a default name of `XA-KMSRead-Role` that has permissions to read KMS keys and policies. This role is assumed in all member accounts by the *KMSRead* Lambda function in the Security Observability account 43 | * ***member-account-lastused-lambda.yaml*** 44 | Deploys the following resources: 45 | * *LastUsed* Lambda function. Runs in member accounts and scans CloudTrail searching the `kms.amazonaws.com` event source for the last time a KMS key was used. It updates the entry for each KMS keyid found in CloudTrail in the *lastUsed* DynamoDB table in the Security Observability account 46 | * Log Group settings set to delete logs after 90 days 47 | * IAM Role with permissions to assume the `putToDynamoRole` in the Security Observability account (also has permissions to write directly to the table but not currently used as the DynamoDB table has no resource policy) 48 | * EventBridge rule to trigger the Lambda on a cron schedule: also every 24 hours. Should run **before** the KMSRead function. 49 | * ***member-account-sample-kms-keys.yaml*** **OPTIONAL** 50 | 51 | Deploys the following resources: 52 | * Two IAM Users - `Alice` and `Bob` 53 | * Two IAM Roles - `Administrator` and `Developer` 54 | * 11x different KMS keys with varying key policies 55 | * 5x KMS Key aliases 56 | 57 | 58 | 59 | ## Instructions 60 | ### 1. Security Observability Account 61 | Deploy the following: 62 | | Order | filename | Stack / Stackset | Single Region / Multi-Region| 63 | | ----- | ----- | ----- | ----- | 64 | | # 1 | [security-account-dynamodb.yaml](security-account-dynamodb.yaml) | Stack | Single | 65 | | # 2 | [security-account-kmsread-lambda.yaml](security-account-kmsread-lambda.yaml) | Stack | Single | 66 | | # 3 | [security-account-athena.yaml](security-account-athena.yaml) | Stack | Single | 67 | 68 | ### 2. All Member Accounts 69 | Deploy the following as StackSets from your **`Management`** account / [delegated CloudFormation account](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacksets-orgs-delegated-admin.html): 70 | | Order | filename | Stack / Stackset | Single Region / Multi-Region| 71 | | ----- | ----- | ----- | ----- | 72 | | # 1 | [member-account-kmsread-role.yaml](member-account-kmsread-role.yaml) | StackSet | **SINGLE e.g. `us-east-1`** | 73 | | # 2 | [member-account-lastused-lambda.yaml](security-account-lastused-lambda.yaml) | StackSet | **MULTI** | 74 | | # 3 | [member-account-kmsread-role.yaml](member-account-kmsread-role.yaml) | **Stack** (to deploy in Management account) | **SINGLE e.g. `us-east-1`** | 75 | | # 4 | [member-account-lastused-lambda.yaml](security-account-lastused-lambda.yaml) | **Stack** (to deploy in Management account) | **MULTI** | 76 | | **OPTIONAL!** | [member-account-sample-kms-keys.yaml](security-account-sample-kms-keys.yaml) | StackSet | **SINGLE e.g. `us-east-1`** | 77 | 78 | ### 3. Populate `accounts` DynamoDB table 79 | The `accounts` DynamoDB table needs to be populated. The easiest way is to log into the **`Management`** account and run the following command: 80 | 81 | aws organizations list-accounts --query "Accounts[?Status=='ACTIVE'].{accountId:Id, accountEmail:Email}" --output json > ~/accounts.json 82 | 83 | Manually augment the accounts.json with the account names using the `accountName` key to end up with a file that looks similar to this: 84 | 85 | 86 | ```` 87 | [ 88 | { 89 | "accountId": "123456789012", 90 | "accountName": "Development", 91 | "accountEmail": "goffalex+development@amazon.co.uk" 92 | }, 93 | { 94 | "accountId": "987654321098", 95 | "accountName": "SystemTest", 96 | "accountEmail": "goffalex+systest@amazon.co.uk" 97 | } 98 | ] 99 | ```` 100 | 101 | Add the entries using this command: 102 | 103 | python putDynamo.py -f -t -r 104 | 105 | e.g. 106 | 107 | python putDynamo.py -f accounts.json -t accounts -r us-east-1 108 | 109 | 110 | ### 4. Configure Quicksight Dashboard 111 | 112 | #### **4.1. Create the Amazon QuickSight resources** 113 | 114 | ##### **Subscribe to Amazon QuickSight** 115 | 116 | * Sign in to your AWS account and open Amazon QuickSight from the AWS Management Console. It is located under Analytics, and you can find it by searching for "QuickSight". 117 | * Your AWS account number is displayed for verification purposes. Choose Sign up for QuickSight. 118 | * ![Quicksight](images/quicksight_new.png) 119 | * Choose Enterprise. To confirm, choose Continue. A screen titled Create your QuickSight accountappears. 120 | * Configure Authentication as appropriate. ‘Use IAM federated identities & QuickSight-managed users“ 121 | * Under Account info: 122 | * Type in a unique account name for Amazon QuickSight. For example, use `yourname-YYYYMMDD-quicksight` in the QuickSight account name field. Your account name can only contain characters (A–Z and a–z), digits (0–9), and hyphens (-). 123 | * Type in a notification email addess in the Notification email address field. This email receives service and usage notifications. 124 | * Under **‘Allow access and autodiscovery for these resources’** the following should be checked: Amazon Redshift, Amazon RDS, IAM and Amazon Athena checkboxes are checked. 125 | * Click **‘Select S3 buckets’** and add the ‘``kms-read-policy-XXXXXXXXXXXX-REGION``’ and the bucket containing ``‘kmsdashboard-athenaworkgroupbucket-XXXXXXXXXXXX-REGION’``. as well as the ``'kmsdashboard-athena-loggingbucket-XXXXXXXXXXX-REGION-’`` Check the box to the left and also for **‘Write permission for Athena Workgroup’.** Click Finish. 126 | * Deselect "Paginated reports" 127 | * Review the choices you made, then choose Finish. 128 | * Once the Amazon QuickSight account creation process is finished, choose Go to Amazon QuickSight to go to the Amazon QuickSight home page. 129 | 130 | #### **4.2 Create a new Dataset** 131 | 132 | * Open Amazon QuickSight from the AWS Management Console. It is located under Analytics, and you can find it by searching for "QuickSight". On the left pane, choose Datasets. 133 | * Log in to the right region - e.g. ``‘Ireland’`` 134 | * On the top right corner, choose New dataset. 135 | * Under the section FROM NEW DATA SOURCES, choose Athena. 136 | * Under Data source name, type in a name for the data source. For example, enter`kmsdashboard-datasource`. 137 | * Select Athena workgroup `kmsdashboard-athena-workgroup.` 138 | * Click `Validate connection` to test access. 139 | * Choose `Create data source` and wait for the popup to refresh with the new dataset. 140 | * Under Database: contain sets of tables., choose the database `kmsdashboarddatabase` 141 | * Under Tables: choose the data to visualise. Select the `kmsdashboardtable` and chooseEdit/Preview data and wait for data set to be opened. 142 | * In the new view, select `State` and change it from `State` to `String` 143 | * In the new view, select `date` and change it from a `String` to a `Date` 144 | * Enter a data format of `yyyy-MM-dd` and click `Validate` and then `Update` 145 | * In the bottom left, change the ‘Query mode’ from Direct query to ‘SPICE’ 146 | * On the top right corner, save your changes with Save & Publish. 147 | * Then click Publish and Visualize to open the design window 148 | * Click `Cancel` on the pop-up 149 | * REMEMBER TO SET A REFRESH SCHEDULE FOR SPICE DATA 150 | 151 | #### **4.3 Visuals** 152 | 153 | Your final dashboard will look something like this: 154 | ![Dashboard 1](images/dashboard1.jpg) 155 | ![Dashboard 2](images/dashboard2.jpg) 156 | 157 | 158 | ##### **4.3.1 Create QuickSight Analysis - Visual: Count of Records by Concern** 159 | 160 | * (If you didn't follow the steps before, open the analysis from the Quicksight home page.) 161 | * Click on ‘Analyses’ → New Analysis 162 | * Select the ‘``kmsdashboardtable``’ Dataset. 163 | * Check the summary and then click ‘``Use in analysis``’ in the top right. 164 | * Select the `visual` which says "AutoGraph" on "Sheet 1". 165 | * On the bottom left, select as visual type Donut Chart. 166 | * Drag & drop the `‘concern’` field from the left `Fields list` into the `Group/Color` field on the top. 167 | * Make the visual wider and resize the ‘concern’ legend on the right to view the full sentences. 168 | * Visuals 169 | * In the top right of the visual, click the pencil icon to ‘``format visual``’ 170 | * Click ``Data labels`` and check ‘``Show metric``’ to enable the percentages. 171 | * Filter 172 | * Click ‘``Filter'`` in the top left 173 | * Add filter and choose ``‘date’`` 174 | * Click data and choose the following settings`: 175 | * Condition → Equals 176 | * Click ‘``Set a rolling date``, choose ’``today``’ and click ‘``Save``’ 177 | * Actions 178 | * Click the hamburger to the right of the pencil in the top right of the visual and choose ``‘Actions’`` 179 | * Click Quick create → Filter same-sheet visuals 180 | 181 | ##### **4.3.2 Create QuickSight Analysis - Visual: Table of accounts** 182 | 183 | * On the top left, choose Add and select Add visual from the drop down list. 184 | * Shrink it down to small and move it to the right of the first visual 185 | * Select the new `visual`. On the bottom left, select Table from the Visual types 186 | * Choose `account` from the left `Fields list` into the `Group by` field on the top. 187 | * Actions 188 | * Click the hamburger to the right of the pencil in the top right of the visual and choose ``‘Actions’`` 189 | * Click Quick create → Filter same-sheet visuals 190 | 191 | ##### **4.3.3 Create QuickSight Analysis - Visual: Table of data** 192 | 193 | * On the top left, choose Add and select Add visual from the drop down list. 194 | * Shrink it down to small and move it to the right of the first visual 195 | * Select the new `visual`. On the bottom left, select Table from the Visual types 196 | * Choose `region,accountnumber,accountname,alias,keyid,creationdate,concern,sid,principal,principalservice,effect,action,resource,condition,tags,lastusedtime,lastusedaction,lastusedencryptioncontext,lastusedsourceipaddress` and `lastusedusername` from the left `Fields list` into the `Group by` field on the top. 197 | 198 | ##### **4.3.4 Create QuickSight Analysis - Visual: Count of Concern by Date and Concern** 199 | 200 | * On the top left, choose Add and select Add visual from the drop down list. 201 | * Select the new `visual`. On the bottom left, select as visual type `Line chart` 202 | * Add '``date'`` as ``X axis`` 203 | * Add ‘``concern'`` as Value and Colour. 204 | 205 | ##### **4.3.5 Use the QuickSight Dashboards** 206 | 207 | * Click the ‘share’ button in the top right and give your dashboard a name e.g. `KMS Key Dashboard` 208 | 209 | ## Security 210 | 211 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 212 | 213 | ## License 214 | 215 | This library is licensed under the MIT-0 License. See the LICENSE file. 216 | 217 | -------------------------------------------------------------------------------- /images/dashboard1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/tracking-aws-kms-key-policies-using-amazon-quicksight/6b96cf8b18bfb7edf39287ea9356f8640b05fd99/images/dashboard1.jpg -------------------------------------------------------------------------------- /images/dashboard2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/tracking-aws-kms-key-policies-using-amazon-quicksight/6b96cf8b18bfb7edf39287ea9356f8640b05fd99/images/dashboard2.jpg -------------------------------------------------------------------------------- /images/quicksight.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/tracking-aws-kms-key-policies-using-amazon-quicksight/6b96cf8b18bfb7edf39287ea9356f8640b05fd99/images/quicksight.jpg -------------------------------------------------------------------------------- /images/quicksight_new.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/tracking-aws-kms-key-policies-using-amazon-quicksight/6b96cf8b18bfb7edf39287ea9356f8640b05fd99/images/quicksight_new.png -------------------------------------------------------------------------------- /images/rif24architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/tracking-aws-kms-key-policies-using-amazon-quicksight/6b96cf8b18bfb7edf39287ea9356f8640b05fd99/images/rif24architecture.png -------------------------------------------------------------------------------- /lambda_functions/KMSRead_lambda.py: -------------------------------------------------------------------------------- 1 | # KMSRead_lambda function: 2 | # This function reads down the list of accounts in the 'accounts' DynamoDB table. 3 | # For every account is tries to AssumeRole into that account and then read the KMS keys 4 | # For every key it finds it then reads their aliases, tags, creation date via the KMS APIs 5 | # and stores them in a big JSON object. 6 | # It then augments this with the details of when the key was lastUsed from the 7 | # 'lastUsedTable' DynamoDB table 8 | # Once all this is done, each key policy __statement__ is written as a separate line in a CSV 9 | # This is done to make it clearer when 'Concerns' are raised and which policy 10 | # statement they relate to. 11 | # Once all this is complete, the final CSV file is pushed up to S3 with a file 12 | # name of the account and region 13 | 14 | import csv 15 | import json 16 | import logging 17 | import os 18 | from datetime import date, datetime 19 | 20 | # Define Imports 21 | import boto3 22 | import botocore 23 | from botocore.exceptions import ClientError 24 | 25 | # Logging 26 | logger = logging.getLogger(__name__) 27 | FORMAT = ( 28 | "[%(asctime)s:%(levelname)s:%(filename)s:%(lineno)s - %(funcName)10s()] %(message)s" 29 | ) 30 | 31 | logger.setLevel(logging.INFO) 32 | logging.basicConfig(format=FORMAT) 33 | 34 | 35 | # Globals 36 | # DynamoDB table containing list of accounts and mapping IDs 37 | # Table name of accounts. Defaults to 'accounts' 38 | accountsDynamoDBTable = str(os.getenv("ACCOUNTSDYNAMODBTABLE", "accounts")) 39 | 40 | # Account number containing the DynamoDB Table containing 'accounts' table 41 | accountsDynamoDBAccount = str(os.getenv("ACCOUNTSDYNAMODBACCOUNT", "123456789012")) 42 | 43 | # IAM Role assumed by this lambda function with permissions to read 'accounts' DynamoDB table 44 | accountsDynamoDBRole = str(os.getenv("ACCOUNTSDYNAMODBROLE", "ReadDynamoDBRole")) 45 | 46 | # Region containing the DynamoDB Table containing 'accounts' table 47 | accountsDynamoDBRegion = str(os.getenv("ACCOUNTSDYNAMODBREGION", "us-east-1")) 48 | 49 | # DynamoDB table containing the last used data 50 | lastUsedDynamoDBTable = str(os.getenv("LASTUSEDDYNAMODBTABLE", "lastUsedTable")) 51 | 52 | # Account number containing the DynamoDB Table containing 'lastUsedTable' table 53 | lastUsedDynamoDBAccount = str(os.getenv("LASTUSEDDYNAMODBACCOUNT", "123456789012")) 54 | 55 | # IAM Role assumed by this lambda function with permissions to read 'lastUsedTable' table 56 | lastUsedDynamoDBRole = str(os.getenv("LASTUSEDDYNAMODBROLE", "ReadDynamoDBRole")) 57 | 58 | # Region containing the DynamoDB Table containing 'lastUsedTable' table 59 | lastUsedDynamoDBRegion = str(os.getenv("LASTUSEDDYNAMODBREGION", "us-east-1")) 60 | 61 | # Cross-account role assumed by this lambda function in each memeber account. 62 | # This role is deployed by the member-account-kmsread-role.yaml CloudFormation 63 | # and has permissions to read and gather details of KMS keys. 64 | xaKMSReadRole = str(os.getenv("XAKMSREADROLE", "XA-KMSRead-Role")) 65 | 66 | 67 | # Read S3 bucket name containing the CSV files 68 | destination_s3_bucket = str( 69 | os.getenv("DEST_BUCKET", "kms-read-policy-123456789012-us-east-1") 70 | ) 71 | 72 | # Read active regions from env variable to loop through once we assume into 73 | # member account 74 | regions_string = str(os.getenv("REGIONS", "eu-west-1,eu-west-2,us-east-1")) 75 | 76 | 77 | # Gather all the keys. These are used to drive the policy and alias searches 78 | def getKeys(kms): 79 | response = kms.list_keys() 80 | return response["Keys"] 81 | 82 | 83 | # Gather the list of policies containing all the key policy statements 84 | def getKeyPolicies(kms, keyid): 85 | try: 86 | response = kms.list_key_policies(KeyId=keyid) 87 | return response["PolicyNames"] 88 | except botocore.exceptions.ClientError as error: 89 | if error.response["Error"]["Code"] == "AccessDeniedException": 90 | logger.warning( 91 | f"Unable to get Policies: {error.response['Error']['Message']}" 92 | ) 93 | else: 94 | logger.error(f"ERROR RESPONSE: {error.response}") 95 | raise error 96 | 97 | 98 | # Read all the account from the 'accounts' table so we know what to 99 | # loop through 100 | def get_accounts( 101 | accountsDynamoDBAccount: str, 102 | accountsDynamoDBRole: str = "ReadDynamoDBRole", 103 | accountsDynamoDBTable: str = "accounts", 104 | region: str = "us-east-1", 105 | ): 106 | # Get assumed role credentials for dynamodb read 107 | try: 108 | session = getAssumedRoleSession(accountsDynamoDBAccount, accountsDynamoDBRole) 109 | 110 | dynamodb = session.resource("dynamodb", region_name=region) 111 | table = dynamodb.Table(accountsDynamoDBTable) 112 | 113 | accounts = table.scan(ProjectionExpression="accountId, accountName") 114 | 115 | return accounts["Items"] 116 | 117 | except Exception as e: 118 | logger.error("Issue assuming the XA role: " + str(e)) 119 | 120 | 121 | # Gather key policies based on initial list of keys 122 | # Pass in the 'kms' session so we don't need to re-assume any roles anywhere 123 | def getPolicy(kms, keyId, policy): 124 | response = kms.get_key_policy(KeyId=keyId, PolicyName=policy) 125 | return json.loads(response["Policy"]) 126 | 127 | 128 | # Get the creation date of the key 129 | def getCreationDate(kms, keyId): 130 | response = kms.describe_key(KeyId=keyId) 131 | return response["KeyMetadata"]["CreationDate"].strftime("%Y-%m-%d %H:%M:%S") 132 | 133 | 134 | # Get all the tags for the keys 135 | def getTag(kms, keyId): 136 | try: 137 | response = kms.list_resource_tags(KeyId=keyId) 138 | return response["Tags"] 139 | except botocore.exceptions.ClientError as error: 140 | if error.response["Error"]["Code"] == "AccessDeniedException": 141 | logger.warning(f"Unable to get Tags: {error.response['Error']['Message']}") 142 | else: 143 | logger.error(f"ERROR RESPONSE: {error.response}") 144 | raise error 145 | 146 | 147 | # Get the alias the kms key 148 | def getAliases(kms, keyId): 149 | response = kms.list_aliases(KeyId=keyId) 150 | return response["Aliases"] 151 | 152 | 153 | # Get the last used dates from lastUsedTable 154 | def getLastUsed( 155 | keyid, 156 | lastUsedDynamoDBAccount, 157 | lastUsedDynamoDBRole, 158 | lastUsedDynamoDBTable, 159 | lastUsedDynamoDBRegion, 160 | ): 161 | try: 162 | session = getAssumedRoleSession(lastUsedDynamoDBAccount, lastUsedDynamoDBRole) 163 | dynamodb = session.resource("dynamodb", region_name=lastUsedDynamoDBRegion) 164 | table = dynamodb.Table(lastUsedDynamoDBTable) 165 | response = table.get_item(Key={"keyID": keyid}) 166 | if "Item" in response: 167 | return response["Item"] 168 | else: 169 | return None 170 | 171 | except Exception as e: 172 | logger.error("Issue getting the last used events: " + str(e)) 173 | 174 | return None 175 | 176 | 177 | # The main orchestrator function. This pulls everything together 178 | def getEverythingJson(kms): 179 | kms_keys = getKeys(kms) 180 | 181 | keyMap = {"kms_keys": []} 182 | 183 | for kms_key in kms_keys: 184 | # Create an empty dict to hold everything 185 | kms_key_object = {} 186 | # Create a key using the KMS keyId 187 | kms_key_object["KeyId"] = kms_key["KeyId"] 188 | # Create an empty list for all the aliases just in case 189 | # we don't find any. 190 | 191 | # Create a temporary list to hold the aliaes 192 | list_of_aliases = [] 193 | # Create a temporary list to hold the policies 194 | list_of_policies = [] 195 | 196 | keyid = kms_key["KeyId"] 197 | 198 | # Now grab all the aliases passing in our kms session and the keyId 199 | kms_aliases = getAliases(kms, keyid) 200 | 201 | # Did we find any aliases? 202 | if kms_aliases: 203 | # Loop through all the aliases that we found (might just be one) 204 | for ind in range(len(kms_aliases)): 205 | # Add the alias to the list of aliases 206 | list_of_aliases.append(kms_aliases[ind]["AliasName"]) 207 | 208 | # Add them to the key object (even if list_of_aliases is empty) 209 | kms_key_object["Aliases"] = list_of_aliases 210 | 211 | # Now grab all the policies passing in our kms session and the keyId 212 | kms_key_policies = getKeyPolicies(kms, keyid) 213 | 214 | # Did we find any key policies? 215 | # At this stage we just want all the policies. We're not digging 216 | # into them yet. 217 | if kms_key_policies: 218 | # If so, grab the policy details for the specificed keyId 219 | for policy in kms_key_policies: 220 | key_policy = getPolicy(kms, keyid, policy) 221 | # Append this policy to the list of policies 222 | list_of_policies.append(key_policy) 223 | 224 | kms_key_object["Policies"] = list_of_policies 225 | 226 | creation_date = getCreationDate(kms, keyid) 227 | kms_key_object["CreationDate"] = creation_date 228 | 229 | key_tag = getTag(kms, keyid) 230 | kms_key_object["Tags"] = key_tag 231 | 232 | # Now grab the lastUsedDate of the KMS Key from the 'lastUsedTable' DynamoDB table 233 | # NOTE: this table is populated separately by the 'lastUsed_lambda' 234 | kms_last_used = getLastUsed( 235 | kms_key["KeyId"], 236 | lastUsedDynamoDBAccount, 237 | lastUsedDynamoDBRole, 238 | lastUsedDynamoDBTable, 239 | lastUsedDynamoDBRegion, 240 | ) 241 | 242 | # Did we find any lastUsed entries in the DynamoDB table? 243 | if kms_last_used: 244 | # If we did then grab the relevant values from the lastUsed DynamoDB record 245 | if "EventTime" in kms_last_used: 246 | kms_key_object["LastUsedTime"] = kms_last_used["EventTime"] 247 | if "EventName" in kms_last_used: 248 | kms_key_object["LastUsedAction"] = kms_last_used["EventName"] 249 | if "encryptionContext" in kms_last_used: 250 | kms_key_object["LastUsedEncryptionContext"] = kms_last_used[ 251 | "encryptionContext" 252 | ] 253 | if "sourceIPAddress" in kms_last_used: 254 | kms_key_object["LastUsedSourceIPAddress"] = kms_last_used[ 255 | "sourceIPAddress" 256 | ] 257 | if "Username" in kms_last_used: 258 | kms_key_object["LastUsedUsername"] = kms_last_used["Username"] 259 | 260 | # Now we have all entries for the kms_key_object including: 261 | # ["KeyId"], ["Aliases"], ["Policies"], ["Tags"], ["CreationDate"], ["LastUsedTime"] etc. 262 | 263 | # The kms_key_object now contains everything we need for the CSV 264 | 265 | # Add this key to the big list of keys 266 | keyMap["kms_keys"].append(kms_key_object) 267 | return keyMap 268 | 269 | 270 | # Function to assume role 271 | def getAssumedRoleSession(aws_account, role_name="XA-KMSRead-Role"): 272 | role_to_assume_arn = "arn:aws:iam::" + aws_account + ":role/" + role_name 273 | sts_client = boto3.client("sts") 274 | 275 | logged_on_arn = sts_client.get_caller_identity()["Arn"] 276 | logger.debug( 277 | f"Logged on user: '{logged_on_arn}' assuming role '{role_to_assume_arn}" 278 | ) 279 | 280 | try: 281 | response = sts_client.assume_role( 282 | RoleArn=role_to_assume_arn, RoleSessionName=role_name 283 | ) 284 | creds = response["Credentials"] 285 | session = boto3.session.Session( 286 | aws_access_key_id=creds["AccessKeyId"], 287 | aws_secret_access_key=creds["SecretAccessKey"], 288 | aws_session_token=creds["SessionToken"], 289 | ) 290 | return session 291 | except Exception as e: 292 | logger.error( 293 | f"Unable to assume role '{role_name}' in account in '{aws_account}': {e}" 294 | ) 295 | 296 | 297 | # Now generate a CSV based on the keyMap file 298 | def getEverythingToCSV( 299 | accountNumber: str, accountName: str, filename: str, keyMap: dict, region: str 300 | ): 301 | logger.info(f"{len(keyMap['kms_keys'])} KMS keys found in [{region}]") 302 | filename = "/tmp/" + filename 303 | 304 | kmsKeysWithPolicies = [] 305 | 306 | for key in keyMap["kms_keys"]: 307 | # This is where the create the multiple lines for each policy statement 308 | # but repeat the core data such as KeyID, creationDate, last used etc. 309 | 310 | if "Policies" in key: 311 | 312 | # Grab the KeyId 313 | keyId = key["KeyId"] 314 | 315 | # Grab the 1st alias 316 | keyAlias = None 317 | if "Aliases" in key: 318 | logger.debug(f"{len(key['Aliases'])} aliases found for {keyId}.") 319 | # If there are any aliases found, then take the first 320 | if len(key["Aliases"]) > 0: 321 | keyAlias = key["Aliases"][0] 322 | 323 | # Grab the last used details 324 | keyLastUsedTime = None 325 | if "LastUsedTime" in key: 326 | keyLastUsedTime = key["LastUsedTime"] 327 | 328 | keyLastUsedAction = None 329 | if "LastUsedAction" in key: 330 | keyLastUsedAction = key["LastUsedAction"] 331 | 332 | keyLastUsedEncryptionContext = None 333 | if "LastUsedEncryptionContext" in key: 334 | keyLastUsedEncryptionContext = key["LastUsedEncryptionContext"] 335 | 336 | keyLastUsedSourceIPAddress = None 337 | if "LastUsedSourceIPAddress" in key: 338 | keyLastUsedSourceIPAddress = key["LastUsedSourceIPAddress"] 339 | 340 | keyLastUsedUsername = None 341 | if "LastUsedUsername" in key: 342 | keyLastUsedUsername = key["LastUsedUsername"] 343 | 344 | # Process the policies 345 | # NOTE: A key may have multiple statements in a policy. A unique line 346 | # is created for each policy. The aliases and last useds are duplicated for those lines 347 | # but this means that concerns can be generated for each statement rather than a policy overall 348 | for x in range( 349 | len(key["Policies"]) 350 | ): # Probably always one Policy - so x will almost always be 0 351 | # This is where we loop through each Statement in the policy but duplicate the records 352 | for line in grabPolicyStatementDetailsList( 353 | accountNumber, 354 | accountName, 355 | region, 356 | keyId, 357 | keyAlias, 358 | key["Policies"][x]["Statement"], 359 | key["Tags"], 360 | key["CreationDate"], 361 | keyLastUsedTime, 362 | keyLastUsedAction, 363 | keyLastUsedEncryptionContext, 364 | keyLastUsedSourceIPAddress, 365 | keyLastUsedUsername, 366 | ): 367 | kmsKeysWithPolicies.append(line) 368 | 369 | # Write to CSV 370 | header = [ 371 | "Date", 372 | "AccountNumber", 373 | "AccountName", 374 | "Region", 375 | "KeyId", 376 | "Alias", 377 | "Sid", 378 | "Effect", 379 | "Principal", 380 | "Principal Service", 381 | "Action", 382 | "Condition", 383 | "Concern", 384 | "Resource", 385 | "Tags", 386 | "CreationDate", 387 | "LastUsedTime", 388 | "LastUsedAction", 389 | "LastUsedEncryptionContext", 390 | "LastUsedSourceIPAddress", 391 | "LastUsedUsername", 392 | ] 393 | 394 | # Write the actual file from the kmsKeysWithPolicies JSON object 395 | with open(filename, "w") as file: 396 | writer = csv.DictWriter(file, fieldnames=header) 397 | writer.writeheader() 398 | writer.writerows(kmsKeysWithPolicies) 399 | 400 | 401 | # Function to grab the Policy Statement Details 402 | # It will handle multiple statements in the policy 403 | # This function is what generates the actual contents (columns) to 404 | # populate a line of the CSV 405 | def grabPolicyStatementDetailsList( 406 | accountNumber: str, 407 | accountName: str, 408 | region: str, 409 | keyId: str, 410 | keyAlias: str, 411 | policyStatements: json, 412 | Tags: list, 413 | CreationDate: date, 414 | keyLastUsedTime: str, 415 | keyLastUsedAction: str, 416 | keyLastUsedEncryptionContext: str, 417 | keyLastUsedSourceIPAddress: str, 418 | keyLastUsedUsername: str, 419 | ) -> list: 420 | 421 | # Lets create a list of lists! 422 | # This is a collection of entries which will become all of the lines for 423 | # the CSV for the single keyId. 424 | keyListOfLists = [] 425 | 426 | # try: 427 | # Are there some statements in the JSON of the Key Policy? 428 | if policyStatements: 429 | # If there are, loop through them all and create a 'keyList' object which will become 430 | # the line of the CSV 431 | for each in range(len(policyStatements)): 432 | # Add the variables we know exist 433 | keyList = { 434 | "AccountNumber": accountNumber, 435 | "AccountName": accountName, 436 | "Region": region, 437 | "KeyId": keyId, 438 | "Alias": keyAlias, 439 | } 440 | # Add the others that might exist 441 | if "Sid" in policyStatements[each]: 442 | keyList["Sid"] = policyStatements[each]["Sid"] 443 | if "Effect" in policyStatements[each]: 444 | keyList["Effect"] = policyStatements[each]["Effect"] 445 | 446 | # Grab the first Principal 447 | if "Principal" in policyStatements[each]: 448 | keyList["Principal"] = list(policyStatements[each]["Principal"].keys())[ 449 | 0 450 | ] 451 | keyList["Principal Service"] = list( 452 | policyStatements[each]["Principal"].values() 453 | )[0] 454 | 455 | # change to semi-colons to not mangle the CSV 456 | keyList["Principal Service"] = str( 457 | keyList["Principal Service"] 458 | ).replace(",", ";") 459 | 460 | # Check the Actions and change to semi-colons to not mangle the CSV 461 | if "Action" in policyStatements[each]: 462 | keyList["Action"] = str(policyStatements[each]["Action"]).replace( 463 | ",", ";" 464 | ) 465 | if "Resource" in policyStatements[each]: 466 | keyList["Resource"] = policyStatements[each]["Resource"] 467 | if "Condition" in policyStatements[each]: 468 | condition = str(policyStatements[each]["Condition"]) 469 | if isinstance(condition, str): 470 | keyList["Condition"] = condition.replace(",", ";") 471 | keyList["Tags"] = Tags 472 | keyList["Tags"] = str(keyList["Tags"]).replace(",", ";") 473 | keyList["CreationDate"] = CreationDate 474 | keyList["LastUsedTime"] = keyLastUsedTime 475 | keyList["LastUsedAction"] = keyLastUsedAction 476 | keyList["LastUsedEncryptionContext"] = keyLastUsedEncryptionContext 477 | keyList["LastUsedSourceIPAddress"] = keyLastUsedSourceIPAddress 478 | keyList["LastUsedUsername"] = keyLastUsedUsername 479 | 480 | keyList["Date"] = str(datetime.now().strftime("%Y-%m-%d")) 481 | keyList["Concern"] = concernFiller( 482 | principal_service=keyList["Principal Service"], 483 | account_number=accountsDynamoDBAccount, 484 | current_account_number=accountNumber, 485 | action=keyList["Action"], 486 | ) 487 | keyListOfLists.append(keyList) 488 | 489 | return keyListOfLists 490 | 491 | 492 | # Function to push to S3 493 | def pushToS3(filename: str, bucketName: str): 494 | # file to check 495 | file_path = "/tmp/" + filename 496 | 497 | flag = os.path.isfile(file_path) 498 | if flag: 499 | client = boto3.resource("s3") 500 | bucket = client.Bucket(bucketName) 501 | key = filename 502 | try: 503 | bucket.upload_file("/tmp/" + filename, key) 504 | logger.info(f"Successfully uploaded [{filename}] to s3://{bucketName}") 505 | # generate code to handle s3 botoexceptions 506 | 507 | except FileNotFoundError as error: 508 | logger.error(f"{filename} not found!") 509 | except botocore.exceptions.ClientError as error: 510 | logger.error(f"Error Dump: {error}") 511 | errorCode = error.response.get("Error", {}).get("Code") 512 | if errorCode == "AccessDeniedException": 513 | logger.error( 514 | f"Access Denied PUTting {filename} to s3://{bucket}: {error.response['Error']['Message']}" 515 | ) 516 | elif errorCode == "AccessDeniedException": 517 | logger.error( 518 | f"Access Denied PUTting {filename} to s3://{bucket}: {error.response['Error']['Message']}" 519 | ) 520 | elif errorCode == "FileNotFoundError": 521 | logger.error( 522 | f"{filename} not found!: {error.response['Error']['Message']}" 523 | ) 524 | elif errorCode == "S3UploadFailedError": 525 | logger.error( 526 | f"S3UploadFailedError: S3 Upload Failed PUTting {filename} to s3://{bucket}: {error.response['Error']['Message']}" 527 | ) 528 | elif errorCode == "ClientError": 529 | logger.error( 530 | f"ClientError: S3 Upload Failed PUTting {filename} to s3://{bucket}: {error.response['Error']['Message']}" 531 | ) 532 | 533 | else: 534 | logger.error(f"ERROR RESPONSE: {error.response}") 535 | raise error 536 | except client.meta.client.exceptions.BucketAlreadyExists as error: 537 | logger.error( 538 | f"Bucket {err.response['Error']['BucketName']} already exists!" 539 | ) 540 | raise error 541 | except client.meta.client.exceptions.NoSuchBucket as error: 542 | logger.error(f"NoSuchBucket: No such bucket: {bucket}") 543 | raise error 544 | except ClientError as error: 545 | logger.error(f"Unexpected error: {error}") 546 | raise error 547 | 548 | except Exception as error: 549 | logger.error(f"Unexpected error: {error}") 550 | raise error 551 | 552 | else: 553 | logger.info(f"{file_path} not found. Probably because no keys found") 554 | 555 | 556 | # Augment the findings with our own concerns / areas of interest 557 | def concernFiller( 558 | principal_service: str, 559 | account_number: str, 560 | current_account_number: str, 561 | action: str, 562 | ): 563 | # These are the collection of concerns. This is where the 'Concern checks' are 564 | # added to the Key policy statement. 565 | concern_list = [] 566 | concern_list.append(checkManageableThroughIAM(principal_service=principal_service)) 567 | concern_list.append( 568 | checkThirdPartyManaged( 569 | account_number=account_number, current_account_number=current_account_number 570 | ) 571 | ) 572 | concern_list.append(checkKmsPolicy(action=action)) 573 | concern_list.append(checkManageableThroughKMS(principal_service=principal_service)) 574 | return ";".join([x for x in concern_list if x != ""]) 575 | 576 | 577 | # Collection of 'Concern checks'. Used by concernFiller() 578 | def checkManageableThroughIAM(principal_service: str) -> str: 579 | if principal_service[-5:] == ":root": 580 | return "Principal is account" 581 | return "" 582 | 583 | 584 | def checkThirdPartyManaged(account_number: str, current_account_number: str) -> str: 585 | if account_number != current_account_number: 586 | return "External account" 587 | return "" 588 | 589 | 590 | def checkKmsPolicy(action: str) -> str: 591 | if "kms:*" in action: 592 | return "Key policy overly permissive" 593 | return "" 594 | 595 | 596 | def checkManageableThroughKMS(principal_service: str) -> str: 597 | if ":user" in principal_service: 598 | return "Access provided to IAM user" 599 | return "" 600 | 601 | 602 | # Key has permissions but cannot be read (locked down key policy) 603 | def unreadableKey(principal_service: str) -> str: 604 | if principal_service == "": 605 | return "Unreadable key. Key permissions don't allow lambda to read details" 606 | return "" 607 | 608 | 609 | def processAccount( 610 | account_number: str, account_name: str, session: boto3.Session, region: str 611 | ) -> list: 612 | # Create a KMS session... 613 | kms = session.client("kms", region_name=region) 614 | # Grab all the Keys and store them in a JSON object 615 | keyMap = getEverythingJson(kms) 616 | filename = account_number + "-" + region + "-kms-details.csv" 617 | getEverythingToCSV(account_number, account_name, filename, keyMap, region=region) 618 | pushToS3(filename, destination_s3_bucket) 619 | # return keyMap["kms_keys"] 620 | 621 | 622 | def main(): 623 | # Use global variables to avoid passing them around 624 | # regions has been read from environment variables at the beginning 625 | regions = [x.strip() for x in regions_string.strip("[]").split(",")] 626 | 627 | # dynamo_ variables have also been read from environment variables at the beginning 628 | account_ids = get_accounts( 629 | accountsDynamoDBAccount, 630 | accountsDynamoDBRole, 631 | accountsDynamoDBTable, 632 | accountsDynamoDBRegion, 633 | ) 634 | logger.info(f"Accounts: {account_ids}") 635 | 636 | # If we found some accounts in the 'accounts' DynamoDB table... 637 | if account_ids: 638 | # then loop through each account... 639 | for account in account_ids: 640 | account_number = account["accountId"] 641 | account_name = account["accountName"] 642 | logger.info(f"Processing Account Number: {account_number}") 643 | # and assumerole into that account using the cross-account KMS Read Role 644 | session = getAssumedRoleSession(account_number, xaKMSReadRole) 645 | 646 | if session: 647 | # If we successfully AssumeRole into the target account... 648 | # then loop through all the regions defined in the 'regions' env variable 649 | for region in regions: 650 | processAccount(account_number, account_name, session, region) 651 | else: 652 | logger.error(f"Unable to create session for {account_number}") 653 | else: 654 | logger.error(f"No Account IDs found: account_ids={account_ids}") 655 | 656 | 657 | # Run the Lambda 658 | def lambda_handler(event, context): 659 | logging.info("<<<<<<<<<< KMSReadLambda >>>>>>>>>>") 660 | logging.info(f"OS Env Variables: {os.environ}") 661 | logging.info(f"Received Event: {event}") 662 | 663 | main() 664 | 665 | 666 | # Run from CLI 667 | if __name__ == "__main__": 668 | logging.info("<<<<<<<<<< KMSReadLambda >>>>>>>>>>") 669 | main() 670 | -------------------------------------------------------------------------------- /lambda_functions/lastUsed_lambda.py: -------------------------------------------------------------------------------- 1 | # lastUsed_lambda function: 2 | # This function is used to read CloudTrail for KMS Events using a filter of 3 | # EventSource = kms.amazonaws.com. Find the most recent events for each key. 4 | # Write them to DynamoDB 5 | 6 | import json 7 | import logging 8 | import os 9 | from datetime import datetime, timedelta 10 | 11 | import boto3 12 | import botocore 13 | 14 | # Logging 15 | logger = logging.getLogger(__name__) 16 | FORMAT = ( 17 | "[%(asctime)s:%(levelname)s:%(filename)s:%(lineno)s - %(funcName)10s()] %(message)s" 18 | ) 19 | 20 | logger.setLevel(logging.INFO) 21 | logging.basicConfig(format=FORMAT) 22 | 23 | # Globals 24 | # Which region are we scanning? If no env varaible set, default to us-east-1 25 | regionToScan = str(os.getenv("REGIONTOSCAN", "us-east-1")) 26 | # Name of the DynamoDB Table containing lastUsed data. Default to 'lastUsedTable' 27 | dynamoDBTable = str(os.getenv("DYNAMODBTABLE", "lastUsedTable")) 28 | # Account number containing the DynamoDB Table containing lastUsed. 29 | dynamoDBAccount = str(os.getenv("DYNAMODBACCOUNT", "123456789012")) 30 | # IAM Role assumed by Lambdas in member accounts to be able to PUT to DynamoDB table. 31 | dynamoDBRole = str(os.getenv("DYNAMODBROLE", "putToDynamoRole")) 32 | # Region containing the DynamoDB Table containing lastUsed. 33 | dynamoDBRegion = str(os.getenv("DYNAMODBREGION", "us-east-1")) 34 | # Maximum number of results returng in a paging operation to CloudTrail. 35 | maxResults = int(os.getenv("MAXRESULTS", 100)) 36 | # Number of hours to search back in CloudTrail. Default to 24. Set to 2160 for 90 days 37 | numberOfHours = int(os.getenv("NUMBEROFHOURS", 24)) 38 | 39 | 40 | def getLambdaRegion(context): 41 | region = context.invoked_function_arn.split(":")[3] 42 | return region 43 | 44 | 45 | def populateTheObject(event, keyID): 46 | kms_event_object = {} 47 | 48 | kms_event_object["keyID"] = keyID 49 | 50 | if "EventTime" in event: 51 | event["EventTime"] = str(event["EventTime"]) # Convert from datetime to string 52 | kms_event_object["EventTime"] = str(event["EventTime"]) 53 | 54 | if "Username" in event: 55 | kms_event_object["Username"] = event["Username"] 56 | 57 | if "EventName" in event: 58 | kms_event_object["EventName"] = event["EventName"] 59 | 60 | if "resources" in event: 61 | kms_event_object["resources"] = event["resources"] 62 | 63 | if "CloudTrailEvent" in event: 64 | # Turn the JSON into a dictionary 65 | cloudTrailEventDictFromJson = json.loads(event["CloudTrailEvent"]) 66 | 67 | if "requestParameters" in cloudTrailEventDictFromJson: 68 | if "encryptionContext" in cloudTrailEventDictFromJson["requestParameters"]: 69 | kms_event_object["encryptionContext"] = str( 70 | cloudTrailEventDictFromJson["requestParameters"][ 71 | "encryptionContext" 72 | ] 73 | ) 74 | if "resources" in cloudTrailEventDictFromJson: 75 | if "arn" in cloudTrailEventDictFromJson["resources"]: 76 | kms_event_object["resourcesArn"] = cloudTrailEventDictFromJson[ 77 | "resources" 78 | ]["arn"] 79 | kms_event_object["resourcesType"] = cloudTrailEventDictFromJson[ 80 | "resources" 81 | ]["type"] 82 | kms_event_object["resourcesAccountId"] = cloudTrailEventDictFromJson[ 83 | "resources" 84 | ]["accountId"] 85 | 86 | if "eventSource" in cloudTrailEventDictFromJson: 87 | kms_event_object["eventSource"] = cloudTrailEventDictFromJson["eventSource"] 88 | 89 | if "userIdentity" in cloudTrailEventDictFromJson: 90 | if "type" in cloudTrailEventDictFromJson["userIdentity"]: 91 | kms_event_object["userIdentityType"] = cloudTrailEventDictFromJson[ 92 | "userIdentity" 93 | ]["type"] 94 | if "invokedBy" in cloudTrailEventDictFromJson["userIdentity"]: 95 | kms_event_object["userIdentityInvokedBy"] = cloudTrailEventDictFromJson[ 96 | "userIdentity" 97 | ]["invokedBy"] 98 | 99 | if "sourceIPAddress" in cloudTrailEventDictFromJson: 100 | kms_event_object["sourceIPAddress"] = cloudTrailEventDictFromJson[ 101 | "sourceIPAddress" 102 | ] 103 | 104 | return kms_event_object 105 | 106 | 107 | def grabKMSCTEvents(numberOfHours, region): 108 | timeNow = datetime.now() 109 | logger.info(f"Now: {timeNow}") 110 | 111 | # Get the time. Defaults to 24 hours ago 112 | timeToGoBack = timeNow - timedelta(hours=int(numberOfHours)) 113 | logger.info(f"timeToGoBack: {timeToGoBack}") 114 | 115 | cloudtrail = boto3.client("cloudtrail", region_name=region) 116 | 117 | paginator = cloudtrail.get_paginator("lookup_events") 118 | page_iterator = paginator.paginate( 119 | LookupAttributes=[ 120 | {"AttributeKey": "EventSource", "AttributeValue": "kms.amazonaws.com"} 121 | ], 122 | StartTime=timeToGoBack, 123 | MaxResults=int(maxResults), # Is this really used? Defaults to 50 124 | ) 125 | 126 | # What are our definitions of KMS key "last used"? 127 | validActions = [ 128 | "Decrypt", 129 | "Encrypt", 130 | "GenerateDataKeyWithoutPlaintext", 131 | ] 132 | lastUsedEvents = {} 133 | 134 | # kms_events = [] 135 | page_count = 1 # Count how many pages of results there are 136 | events_count = 1 # How many KMS Cloud Trail events did we find? 137 | 138 | logger.info(f"Starting pagination...") 139 | 140 | # Loop through all the pages of CloudTrail events 141 | for page in page_iterator: 142 | logger.info(f"Page Count: {page_count}") 143 | page_count += 1 144 | for event in page["Events"]: 145 | events_count += 1 146 | 147 | # If this is a valid action, then process it 148 | if event["EventName"] in validActions: 149 | # Is there an actual CloudTrailEvent(almost definitely yes) 150 | if "CloudTrailEvent" in event: 151 | # Turn the JSON into a dictionary 152 | cloudTrailEventDictFromJson = json.loads(event["CloudTrailEvent"]) 153 | 154 | # Grab the ARN, resource type and AccountID about the resource(key) from the CloudTrail event 155 | if "resources" in cloudTrailEventDictFromJson: 156 | keyID = cloudTrailEventDictFromJson["resources"][0]["ARN"] 157 | # Split out to return the keyID from the ARN 158 | keyID = keyID.split("/")[1] 159 | 160 | # Get the EventTime 161 | if "EventTime" in event: 162 | EventTime = str(event["EventTime"]) 163 | 164 | ### Now lookup if this is already keyed in the hashmap 165 | 166 | # Do we already have the keyID in the dictionary 167 | if keyID in lastUsedEvents: 168 | # Do we already have a 'lastUsed' time for that keyID? 169 | if "EventTime" in lastUsedEvents[keyID]: 170 | # Is the EventTime newer than the existing one? 171 | if EventTime > lastUsedEvents[keyID]["EventTime"]: 172 | # Create the object in preparation to store in the array and DynamoDB 173 | lastUsedEventObject = populateTheObject( 174 | event=event, keyID=keyID 175 | ) 176 | # Store the object in the array to finally load into DynamoDB 177 | lastUsedEvents[keyID] = lastUsedEventObject 178 | 179 | else: 180 | # Key not found before. 181 | # Let's create the object ready to store in the array to finally load into DynamoDB 182 | lastUsedEventObject = populateTheObject( 183 | event=event, keyID=keyID 184 | ) 185 | # Store the object in the array to finally load into DynamoDB 186 | lastUsedEvents[keyID] = lastUsedEventObject 187 | 188 | logger.info(f"Events Processed: {events_count}") 189 | return lastUsedEvents 190 | 191 | 192 | # Function to assume role 193 | def getAssumedRoleSession(aws_account: str, role_name: str): 194 | role_to_assume_arn = "arn:aws:iam::" + aws_account + ":role/" + role_name 195 | sts_client = boto3.client("sts") 196 | 197 | try: 198 | response = sts_client.assume_role( 199 | RoleArn=role_to_assume_arn, RoleSessionName=role_name 200 | ) 201 | creds = response["Credentials"] 202 | session = boto3.session.Session( 203 | aws_access_key_id=creds["AccessKeyId"], 204 | aws_secret_access_key=creds["SecretAccessKey"], 205 | aws_session_token=creds["SessionToken"], 206 | ) 207 | return session 208 | except Exception as e: 209 | logger.error(f"Unable to assume role in account {aws_account}: {e}") 210 | 211 | 212 | def pushToDynamoDB( 213 | dynamoDB_json: json, 214 | dynamoDBAccount: str, 215 | dynamoDBTable: str, 216 | dynamoDBRole: str, 217 | dynamoDBRegion: str = "us-east-1", 218 | ): 219 | numberOfEntriesSuccessfullyAddedToDynamoDB = 0 220 | 221 | try: 222 | session = getAssumedRoleSession(dynamoDBAccount, dynamoDBRole) 223 | 224 | dynamodb_resource = session.resource("dynamodb", region_name=dynamoDBRegion) 225 | 226 | table = dynamodb_resource.Table(dynamoDBTable) 227 | for item in dynamoDB_json: 228 | 229 | response = add_cloudtrail_item_to_dynamodb(table, dynamoDB_json[item]) 230 | if response["ResponseMetadata"]["HTTPStatusCode"] == 200: 231 | numberOfEntriesSuccessfullyAddedToDynamoDB += 1 232 | logger.info( 233 | f"{numberOfEntriesSuccessfullyAddedToDynamoDB} records successfully added to arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 234 | ) 235 | return numberOfEntriesSuccessfullyAddedToDynamoDB 236 | 237 | except botocore.exceptions.ClientError as err: 238 | logger.error( 239 | "Couldn't add item %s to table %s. Here's why: %s: %s", 240 | item, 241 | table, 242 | err.response["Error"]["Code"], 243 | err.response["Error"]["Message"], 244 | ) 245 | raise 246 | 247 | 248 | def add_cloudtrail_item_to_dynamodb(table, item): 249 | try: 250 | return table.put_item(Item=item) 251 | except botocore.exceptions.ClientError as err: 252 | logger.error( 253 | "Couldn't add item %s to table %s. Here's why: %s: %s", 254 | item, 255 | table, 256 | err.response["Error"]["Code"], 257 | err.response["Error"]["Message"], 258 | ) 259 | raise 260 | 261 | 262 | def lambda_handler(event, context): 263 | # Gather relevant details 264 | region = getLambdaRegion(context) 265 | logger.info(f"Region: {region}") 266 | 267 | kms_events = {} 268 | kms_events = grabKMSCTEvents(numberOfHours, region) 269 | 270 | kms_event_count = len(kms_events) 271 | logger.info(f"{kms_event_count} events found") 272 | 273 | logger.info( 274 | f"Now loading {kms_event_count} events into Dynamo: arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 275 | ) 276 | 277 | successfulRecordCount = pushToDynamoDB( 278 | dynamoDB_json=kms_events, 279 | dynamoDBAccount=dynamoDBAccount, 280 | dynamoDBTable=dynamoDBTable, 281 | dynamoDBRegion=dynamoDBRegion, 282 | dynamoDBRole=dynamoDBRole, 283 | ) 284 | 285 | return { 286 | "statusCode": 200, 287 | "body": json.dumps( 288 | { 289 | "message": f"{kms_event_count} events found. {successfulRecordCount} records successfully added to arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 290 | } 291 | ), 292 | } 293 | 294 | 295 | if __name__ == "__main__": 296 | # Reads regionToScan variable (defaults to 'us-east-1') 297 | kms_events = {} 298 | kms_events = grabKMSCTEvents(numberOfHours, regionToScan) 299 | 300 | kms_event_count = len(kms_events) 301 | logger.info(f"{kms_event_count} events found") 302 | 303 | logger.info( 304 | f"Now loading {kms_event_count} events into Dynamo: arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 305 | ) 306 | 307 | # print(f"kms_events: {kms_events}") 308 | successfulRecordCount = pushToDynamoDB( 309 | dynamoDB_json=kms_events, 310 | dynamoDBAccount=dynamoDBAccount, 311 | dynamoDBTable=dynamoDBTable, 312 | dynamoDBRegion=dynamoDBRegion, 313 | dynamoDBRole=dynamoDBRole, 314 | ) 315 | -------------------------------------------------------------------------------- /member-account-kmsread-role.yaml: -------------------------------------------------------------------------------- 1 | # Author: Alex Goff 2 | # Part 1 of 3 CloudFormation files for member / spoke accounts. 3 | # This template creates the IAM Role that the KMS Read Lambda will assume. 4 | 5 | AWSTemplateFormatVersion: '2010-09-09' 6 | Description: "KMS Keys" 7 | Parameters: 8 | KMSReadLambdaAccount: 9 | Description: AWS Account where the KMS Read Lambda is deployed 10 | Default: 123456789012 11 | Type: String 12 | KMSReadRoleName: 13 | Description: Name of the KMS Read Lambda IAM Role. Needs to match KMSReadRole IAM Policy 14 | Default: XA-KMSRead-Role 15 | Type: String 16 | 17 | Resources: 18 | # KMS Read Lambda IAM Role 19 | KMSReadRole: 20 | Metadata: 21 | cfn_nag: 22 | rules_to_suppress: 23 | - id: W11 24 | reason: "Short lived workshop. More granular permissions not needed" 25 | - id: W28 26 | reason: "Short lived workshop. Resources will not be replaced - only torn down and deleted." 27 | Type: 'AWS::IAM::Role' 28 | Properties: 29 | AssumeRolePolicyDocument: 30 | Version: 2012-10-17 31 | Statement: 32 | - Effect: Allow 33 | Principal: 34 | AWS: !Sub "arn:aws:iam::${KMSReadLambdaAccount}:root" 35 | Action: 36 | sts:AssumeRole 37 | Description: This role allows the central KMS Policy Lambda to read the details of the KMS keys in the account 38 | Path: / 39 | Policies: 40 | - PolicyName: LambdaPolicy 41 | PolicyDocument: 42 | Version: 2012-10-17 43 | Statement: 44 | - Effect: Allow 45 | Action: 46 | - "kms:ListKeys" 47 | - "kms:ListAliases" 48 | Resource: "*" 49 | - Effect: Allow 50 | Action: 51 | - "kms:ListKeyPolicies" 52 | - "kms:GetKeyPolicy" 53 | - "kms:ListResourceTags" 54 | - "kms:DescribeKey" 55 | Resource: "arn:aws:kms:*:*:key/*" 56 | RoleName: !Ref KMSReadRoleName 57 | 58 | -------------------------------------------------------------------------------- /member-account-lastused-lambda.yaml: -------------------------------------------------------------------------------- 1 | # Author: Alex Goff 2 | # Part 2 of 3 CloudFormation files for member / spoke accounts. 3 | # This template deploys the 'lastUsed' Lambda function 4 | 5 | AWSTemplateFormatVersion : "2010-09-09" 6 | Description: Lambda function to search CloudTrail and store last used in DynamoDB 7 | Parameters: 8 | Prefix: 9 | Default: storeLastUsed 10 | Type: String 11 | putToDynamoDBAccount: 12 | Default: 123456789012 13 | Type: String 14 | putToDynamoDBRegion: 15 | Default: us-east-1 16 | Type: String 17 | putToDynamoDBRoleName: 18 | Default: putToDynamoRole 19 | Type: String 20 | lastUsedTableName: 21 | Default: lastUsedTable 22 | Type: String 23 | LambdaRoleName: 24 | Default: lastUsedLambdaRole 25 | Type: String 26 | MaxResults: 27 | Default: 100 28 | Type: String 29 | MaxNumberOfHours: 30 | Description: "This is the number of hours to search back through CloudTrail" 31 | Default: 24 32 | Type: String 33 | 34 | Resources: 35 | 36 | storeLastUsedLambdaCron: 37 | Type: "AWS::Events::Rule" 38 | Properties: 39 | Description: This event rules triggers the lambda on a schedule 40 | Name: !Sub ${Prefix}-Schedule 41 | ScheduleExpression: rate(1 day) 42 | State: ENABLED 43 | Targets: 44 | - 45 | Arn: !GetAtt storeLastUsedLambda.Arn 46 | Id: 1 47 | 48 | InvokeLambda: 49 | Type: "AWS::Lambda::Permission" 50 | Properties: 51 | Action: lambda:InvokeFunction 52 | FunctionName: !GetAtt storeLastUsedLambda.Arn 53 | Principal: events.amazonaws.com 54 | SourceArn: !GetAtt storeLastUsedLambdaCron.Arn 55 | 56 | storeLastUsedLambdaLogGroup: 57 | Metadata: 58 | checkov: 59 | skip: 60 | - id: "CKV_AWS_158" 61 | reason: "No log entries contain sensitive data. No reason for KMS keys" 62 | Type: AWS::Logs::LogGroup 63 | Properties: 64 | LogGroupName: !Sub "/aws/lambda/${storeLastUsedLambda}" 65 | RetentionInDays: 90 66 | 67 | storeLastUsedLambdaRole: 68 | Metadata: 69 | checkov: 70 | skip: 71 | - id: "CKV_AWS_111" 72 | reason: "IAM Policy has been scoped down and has condition keys to further limit" 73 | Type: AWS::IAM::Role 74 | Properties: 75 | RoleName: !Sub ${LambdaRoleName}-${AWS::Region} 76 | AssumeRolePolicyDocument: 77 | Version: "2012-10-17" 78 | Statement: 79 | - Effect: "Allow" 80 | Principal: 81 | Service: 82 | - lambda.amazonaws.com 83 | Action: 84 | - 'sts:AssumeRole' 85 | Path: / 86 | Policies: 87 | - PolicyName: KMSCTtoDynamoDBRead-Policy 88 | PolicyDocument: 89 | Version: "2012-10-17" 90 | Statement: 91 | - Sid: CloudTrailLookup 92 | Effect: Allow 93 | Action: 94 | - cloudtrail:LookupEvents 95 | Resource: '*' 96 | - Sid: CloudWatchWriteEvents 97 | Effect: Allow 98 | Action: 99 | - logs:CreateLogGroup 100 | - logs:CreateLogStream 101 | - logs:PutLogEvents 102 | Resource: '*' 103 | - Sid: DynamoDBPutObject 104 | Effect: Allow 105 | Action: 106 | - dynamodb:PutItem 107 | Resource: !Sub arn:aws:dynamodb:*:${putToDynamoDBAccount}:table/${lastUsedTableName} 108 | - Sid: AssumeRole 109 | Effect: Allow 110 | Action: 111 | - sts:AssumeRole 112 | Resource: 113 | - !Sub 'arn:aws:iam::${putToDynamoDBAccount}:role/${putToDynamoDBRoleName}' 114 | 115 | storeLastUsedLambda: 116 | Metadata: 117 | cfn_nag: 118 | rules_to_suppress: 119 | - id: W58 120 | reason: "Permissions granted in iam-roles.yaml cloudformation" 121 | - id: W89 122 | reason: "No other resources created inside VPCs therefore not needed" 123 | - id: W92 124 | reason: "Short lived workshop. No lambda launch contention therefore not needed" 125 | 126 | checkov: 127 | skip: 128 | - id: "CKV_AWS_117" 129 | reason: "No other resources created inside VPCs therefore not needed" 130 | - id: "CKV_AWS_116" 131 | reason: "No DLQ required" 132 | - id: "CKV_AWS_173" 133 | comment: "No sensitive data" 134 | - id: "CKV_AWS_115" 135 | reason: "PoC code - no lambda launch contention therefore not needed" 136 | - id: "CKV_SECRET_6" 137 | reason: "Role assumption function loads AK/SK from STS call. False positive" 138 | 139 | Type: "AWS::Lambda::Function" 140 | Properties: 141 | Description: Lambda to read through CloudTrail and store lastUsed time for keys 142 | FunctionName: storeLastUsed 143 | Handler: index.lambda_handler 144 | Role: !GetAtt storeLastUsedLambdaRole.Arn 145 | Runtime: python3.12 146 | Architectures: 147 | - arm64 148 | Tags: 149 | - 150 | Key: Project 151 | Value: re:Invent 2024 152 | Environment: 153 | Variables: 154 | DYNAMODBROLE: !Ref putToDynamoDBRoleName 155 | DYNAMODBACCOUNT: !Ref putToDynamoDBAccount 156 | DYNAMODBTABLE: !Ref lastUsedTableName 157 | DYNAMOREGION: !Ref putToDynamoDBRegion 158 | MAXRESULTS: !Ref MaxResults 159 | NUMBEROFHOURS: !Ref MaxNumberOfHours 160 | MemorySize: 256 161 | Timeout: 180 162 | Code: 163 | ZipFile: | 164 | # lastUsed_lambda function: 165 | # This function is used to read CloudTrail for KMS Events using a filter of 166 | # EventSource = kms.amazonaws.com. Find the most recent events for each key. 167 | # Write them to DynamoDB 168 | 169 | import json 170 | import logging 171 | import os 172 | from datetime import datetime, timedelta 173 | 174 | import boto3 175 | import botocore 176 | 177 | # Logging 178 | logger = logging.getLogger(__name__) 179 | FORMAT = ( 180 | "[%(asctime)s:%(levelname)s:%(filename)s:%(lineno)s - %(funcName)10s()] %(message)s" 181 | ) 182 | 183 | logger.setLevel(logging.INFO) 184 | logging.basicConfig(format=FORMAT) 185 | 186 | # Globals 187 | # Which region are we scanning? If no env varaible set, default to us-east-1 188 | regionToScan = str(os.getenv("REGIONTOSCAN", "us-east-1")) 189 | # Name of the DynamoDB Table containing lastUsed data. Default to 'lastUsedTable' 190 | dynamoDBTable = str(os.getenv("DYNAMODBTABLE", "lastUsedTable")) 191 | # Account number containing the DynamoDB Table containing lastUsed. 192 | dynamoDBAccount = str(os.getenv("DYNAMODBACCOUNT", "123456789012")) 193 | # IAM Role assumed by Lambdas in member accounts to be able to PUT to DynamoDB table. 194 | dynamoDBRole = str(os.getenv("DYNAMODBROLE", "putToDynamoRole")) 195 | # Region containing the DynamoDB Table containing lastUsed. 196 | dynamoDBRegion = str(os.getenv("DYNAMODBREGION", "us-east-1")) 197 | # Maximum number of results returng in a paging operation to CloudTrail. 198 | maxResults = int(os.getenv("MAXRESULTS", 100)) 199 | # Number of hours to search back in CloudTrail. Default to 24. Set to 2160 for 90 days 200 | numberOfHours = int(os.getenv("NUMBEROFHOURS", 24)) 201 | 202 | 203 | def getLambdaRegion(context): 204 | region = context.invoked_function_arn.split(":")[3] 205 | return region 206 | 207 | 208 | def populateTheObject(event, keyID): 209 | kms_event_object = {} 210 | 211 | kms_event_object["keyID"] = keyID 212 | 213 | if "EventTime" in event: 214 | event["EventTime"] = str(event["EventTime"]) # Convert from datetime to string 215 | kms_event_object["EventTime"] = str(event["EventTime"]) 216 | 217 | if "Username" in event: 218 | kms_event_object["Username"] = event["Username"] 219 | 220 | if "EventName" in event: 221 | kms_event_object["EventName"] = event["EventName"] 222 | 223 | if "resources" in event: 224 | kms_event_object["resources"] = event["resources"] 225 | 226 | if "CloudTrailEvent" in event: 227 | # Turn the JSON into a dictionary 228 | cloudTrailEventDictFromJson = json.loads(event["CloudTrailEvent"]) 229 | 230 | if "requestParameters" in cloudTrailEventDictFromJson: 231 | if "encryptionContext" in cloudTrailEventDictFromJson["requestParameters"]: 232 | kms_event_object["encryptionContext"] = str( 233 | cloudTrailEventDictFromJson["requestParameters"][ 234 | "encryptionContext" 235 | ] 236 | ) 237 | if "resources" in cloudTrailEventDictFromJson: 238 | if "arn" in cloudTrailEventDictFromJson["resources"]: 239 | kms_event_object["resourcesArn"] = cloudTrailEventDictFromJson[ 240 | "resources" 241 | ]["arn"] 242 | kms_event_object["resourcesType"] = cloudTrailEventDictFromJson[ 243 | "resources" 244 | ]["type"] 245 | kms_event_object["resourcesAccountId"] = cloudTrailEventDictFromJson[ 246 | "resources" 247 | ]["accountId"] 248 | 249 | if "eventSource" in cloudTrailEventDictFromJson: 250 | kms_event_object["eventSource"] = cloudTrailEventDictFromJson["eventSource"] 251 | 252 | if "userIdentity" in cloudTrailEventDictFromJson: 253 | if "type" in cloudTrailEventDictFromJson["userIdentity"]: 254 | kms_event_object["userIdentityType"] = cloudTrailEventDictFromJson[ 255 | "userIdentity" 256 | ]["type"] 257 | if "invokedBy" in cloudTrailEventDictFromJson["userIdentity"]: 258 | kms_event_object["userIdentityInvokedBy"] = cloudTrailEventDictFromJson[ 259 | "userIdentity" 260 | ]["invokedBy"] 261 | 262 | if "sourceIPAddress" in cloudTrailEventDictFromJson: 263 | kms_event_object["sourceIPAddress"] = cloudTrailEventDictFromJson[ 264 | "sourceIPAddress" 265 | ] 266 | 267 | return kms_event_object 268 | 269 | 270 | def grabKMSCTEvents(numberOfHours, region): 271 | timeNow = datetime.now() 272 | logger.info(f"Now: {timeNow}") 273 | 274 | # Get the time. Defaults to 24 hours ago 275 | timeToGoBack = timeNow - timedelta(hours=int(numberOfHours)) 276 | logger.info(f"timeToGoBack: {timeToGoBack}") 277 | 278 | cloudtrail = boto3.client("cloudtrail", region_name=region) 279 | 280 | paginator = cloudtrail.get_paginator("lookup_events") 281 | page_iterator = paginator.paginate( 282 | LookupAttributes=[ 283 | {"AttributeKey": "EventSource", "AttributeValue": "kms.amazonaws.com"} 284 | ], 285 | StartTime=timeToGoBack, 286 | MaxResults=int(maxResults), # Is this really used? Defaults to 50 287 | ) 288 | 289 | # What are our definitions of KMS key "last used"? 290 | validActions = [ 291 | "Decrypt", 292 | "Encrypt", 293 | "GenerateDataKeyWithoutPlaintext", 294 | ] 295 | lastUsedEvents = {} 296 | 297 | # kms_events = [] 298 | page_count = 1 # Count how many pages of results there are 299 | events_count = 1 # How many KMS Cloud Trail events did we find? 300 | 301 | logger.info(f"Starting pagination...") 302 | 303 | # Loop through all the pages of CloudTrail events 304 | for page in page_iterator: 305 | logger.info(f"Page Count: {page_count}") 306 | page_count += 1 307 | for event in page["Events"]: 308 | events_count += 1 309 | 310 | # If this is a valid action, then process it 311 | if event["EventName"] in validActions: 312 | # Is there an actual CloudTrailEvent(almost definitely yes) 313 | if "CloudTrailEvent" in event: 314 | # Turn the JSON into a dictionary 315 | cloudTrailEventDictFromJson = json.loads(event["CloudTrailEvent"]) 316 | 317 | # Grab the ARN, resource type and AccountID about the resource(key) from the CloudTrail event 318 | if "resources" in cloudTrailEventDictFromJson: 319 | keyID = cloudTrailEventDictFromJson["resources"][0]["ARN"] 320 | # Split out to return the keyID from the ARN 321 | keyID = keyID.split("/")[1] 322 | 323 | # Get the EventTime 324 | if "EventTime" in event: 325 | EventTime = str(event["EventTime"]) 326 | 327 | ### Now lookup if this is already keyed in the hashmap 328 | 329 | # Do we already have the keyID in the dictionary 330 | if keyID in lastUsedEvents: 331 | # Do we already have a 'lastUsed' time for that keyID? 332 | if "EventTime" in lastUsedEvents[keyID]: 333 | # Is the EventTime newer than the existing one? 334 | if EventTime > lastUsedEvents[keyID]["EventTime"]: 335 | # Create the object in preparation to store in the array and DynamoDB 336 | lastUsedEventObject = populateTheObject( 337 | event=event, keyID=keyID 338 | ) 339 | # Store the object in the array to finally load into DynamoDB 340 | lastUsedEvents[keyID] = lastUsedEventObject 341 | 342 | else: 343 | # Key not found before. 344 | # Let's create the object ready to store in the array to finally load into DynamoDB 345 | lastUsedEventObject = populateTheObject( 346 | event=event, keyID=keyID 347 | ) 348 | # Store the object in the array to finally load into DynamoDB 349 | lastUsedEvents[keyID] = lastUsedEventObject 350 | 351 | logger.info(f"Events Processed: {events_count}") 352 | return lastUsedEvents 353 | 354 | 355 | # Function to assume role 356 | def getAssumedRoleSession(aws_account: str, role_name: str): 357 | role_to_assume_arn = "arn:aws:iam::" + aws_account + ":role/" + role_name 358 | sts_client = boto3.client("sts") 359 | 360 | try: 361 | response = sts_client.assume_role( 362 | RoleArn=role_to_assume_arn, RoleSessionName=role_name 363 | ) 364 | creds = response["Credentials"] 365 | session = boto3.session.Session( 366 | aws_access_key_id=creds["AccessKeyId"], 367 | aws_secret_access_key=creds["SecretAccessKey"], 368 | aws_session_token=creds["SessionToken"], 369 | ) 370 | return session 371 | except Exception as e: 372 | logger.error(f"Unable to assume role in account {aws_account}: {e}") 373 | 374 | 375 | def pushToDynamoDB( 376 | dynamoDB_json: json, 377 | dynamoDBAccount: str, 378 | dynamoDBTable: str, 379 | dynamoDBRole: str, 380 | dynamoDBRegion: str = "us-east-1", 381 | ): 382 | numberOfEntriesSuccessfullyAddedToDynamoDB = 0 383 | 384 | try: 385 | session = getAssumedRoleSession(dynamoDBAccount, dynamoDBRole) 386 | 387 | dynamodb_resource = session.resource("dynamodb", region_name=dynamoDBRegion) 388 | 389 | table = dynamodb_resource.Table(dynamoDBTable) 390 | for item in dynamoDB_json: 391 | 392 | response = add_cloudtrail_item_to_dynamodb(table, dynamoDB_json[item]) 393 | if response["ResponseMetadata"]["HTTPStatusCode"] == 200: 394 | numberOfEntriesSuccessfullyAddedToDynamoDB += 1 395 | logger.info( 396 | f"{numberOfEntriesSuccessfullyAddedToDynamoDB} records successfully added to arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 397 | ) 398 | return numberOfEntriesSuccessfullyAddedToDynamoDB 399 | 400 | except botocore.exceptions.ClientError as err: 401 | logger.error( 402 | "Couldn't add item %s to table %s. Here's why: %s: %s", 403 | item, 404 | table, 405 | err.response["Error"]["Code"], 406 | err.response["Error"]["Message"], 407 | ) 408 | raise 409 | 410 | 411 | def add_cloudtrail_item_to_dynamodb(table, item): 412 | try: 413 | return table.put_item(Item=item) 414 | except botocore.exceptions.ClientError as err: 415 | logger.error( 416 | "Couldn't add item %s to table %s. Here's why: %s: %s", 417 | item, 418 | table, 419 | err.response["Error"]["Code"], 420 | err.response["Error"]["Message"], 421 | ) 422 | raise 423 | 424 | 425 | def lambda_handler(event, context): 426 | # Gather relevant details 427 | region = getLambdaRegion(context) 428 | logger.info(f"Region: {region}") 429 | 430 | kms_events = {} 431 | kms_events = grabKMSCTEvents(numberOfHours, region) 432 | 433 | kms_event_count = len(kms_events) 434 | logger.info(f"{kms_event_count} events found") 435 | 436 | logger.info( 437 | f"Now loading {kms_event_count} events into Dynamo: arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 438 | ) 439 | 440 | successfulRecordCount = pushToDynamoDB( 441 | dynamoDB_json=kms_events, 442 | dynamoDBAccount=dynamoDBAccount, 443 | dynamoDBTable=dynamoDBTable, 444 | dynamoDBRegion=dynamoDBRegion, 445 | dynamoDBRole=dynamoDBRole, 446 | ) 447 | 448 | return { 449 | "statusCode": 200, 450 | "body": json.dumps( 451 | { 452 | "message": f"{kms_event_count} events found. {successfulRecordCount} records successfully added to arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 453 | } 454 | ), 455 | } 456 | 457 | 458 | if __name__ == "__main__": 459 | # Reads regionToScan variable (defaults to 'us-east-1') 460 | kms_events = {} 461 | kms_events = grabKMSCTEvents(numberOfHours, regionToScan) 462 | 463 | kms_event_count = len(kms_events) 464 | logger.info(f"{kms_event_count} events found") 465 | 466 | logger.info( 467 | f"Now loading {kms_event_count} events into Dynamo: arn:aws:dynamodb:{dynamoDBRegion}:{dynamoDBAccount}:table/{dynamoDBTable}" 468 | ) 469 | 470 | # print(f"kms_events: {kms_events}") 471 | successfulRecordCount = pushToDynamoDB( 472 | dynamoDB_json=kms_events, 473 | dynamoDBAccount=dynamoDBAccount, 474 | dynamoDBTable=dynamoDBTable, 475 | dynamoDBRegion=dynamoDBRegion, 476 | dynamoDBRole=dynamoDBRole, 477 | ) 478 | 479 | Outputs: 480 | storeLastUsedLambdaArn: 481 | Description: "KMS last Used Date to DynamoDB Lambda Function ARN" 482 | Value: !GetAtt storeLastUsedLambda.Arn 483 | kmsCloudTrailDynamoDBFunctionIamRole: 484 | Description: "Implicit IAM Role created for KMS CloudTrail to DynamoDBfunction" 485 | Value: !GetAtt storeLastUsedLambdaRole.Arn 486 | -------------------------------------------------------------------------------- /member-account-sample-kms-keys.yaml: -------------------------------------------------------------------------------- 1 | # Author: Alex Goff 2 | # 3 | # OPTIONAL! Part 3 of 3 CloudFormation files for member / spoke accounts. 4 | # This template deploys some sample KMS keys 5 | AWSTemplateFormatVersion: '2010-09-09' 6 | Description: "Tracking AWS KMS Key Policies using Amazon Quicksight" 7 | Resources: 8 | # IAM Users for KMS Keys 9 | IAMUserAlice: 10 | Type: AWS::IAM::User 11 | Properties: 12 | Path: "/" 13 | IAMUserBob: 14 | Type: AWS::IAM::User 15 | Properties: 16 | Path: "/" 17 | 18 | # User IAM Role for KMS Key Policies 19 | AdministratorRole: 20 | Metadata: 21 | cfn_nag: 22 | rules_to_suppress: 23 | - id: W11 24 | reason: "Short lived workshop. More granular permissions not needed" 25 | - id: W28 26 | reason: "Short lived workshop. Resources will not be replaced - only torn down and deleted." 27 | Type: 'AWS::IAM::Role' 28 | Properties: 29 | ManagedPolicyArns: 30 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 31 | AssumeRolePolicyDocument: 32 | Version: "2012-10-17" 33 | Statement: 34 | - 35 | Effect: "Allow" 36 | Principal: 37 | Service: 38 | - lambda.amazonaws.com 39 | Action: 40 | - 'sts:AssumeRole' 41 | Path: / 42 | DeveloperRole: 43 | Metadata: 44 | cfn_nag: 45 | rules_to_suppress: 46 | - id: W11 47 | reason: "Short lived workshop. More granular permissions not needed" 48 | - id: W28 49 | reason: "Short lived workshop. Resources will not be replaced - only torn down and deleted." 50 | Type: 'AWS::IAM::Role' 51 | Properties: 52 | ManagedPolicyArns: 53 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 54 | AssumeRolePolicyDocument: 55 | Version: "2012-10-17" 56 | Statement: 57 | - 58 | Effect: "Allow" 59 | Principal: 60 | Service: 61 | - lambda.amazonaws.com 62 | Action: 63 | - 'sts:AssumeRole' 64 | Path: / 65 | 66 | # Keys 67 | myKeyWithTag: 68 | Metadata: 69 | cfn_nag: 70 | rules_to_suppress: 71 | - id: F19 72 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 73 | checkov: 74 | skip: 75 | - id: "CKV_AWS_7" 76 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 77 | Type: 'AWS::KMS::Key' 78 | Properties: 79 | KeyPolicy: 80 | Version: 2012-10-17 81 | Id: key-default-2 82 | Statement: 83 | - Sid: Enable IAM User Permissions 84 | Effect: Allow 85 | Principal: 86 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 87 | Action: 'kms:*' 88 | Resource: '*' 89 | Tags: 90 | - Key: Owner 91 | Value: Goffy 92 | RSASigningKey: 93 | Metadata: 94 | cfn_nag: 95 | rules_to_suppress: 96 | - id: F19 97 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 98 | checkov: 99 | skip: 100 | - id: "CKV_AWS_7" 101 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 102 | Type: 'AWS::KMS::Key' 103 | Properties: 104 | Description: RSA-3072 asymmetric KMS key for signing and verification 105 | KeySpec: RSA_3072 106 | KeyUsage: SIGN_VERIFY 107 | KeyPolicy: 108 | Version: 2012-10-17 109 | Id: key-default-3 110 | Statement: 111 | - Sid: Enable IAM User Permissions 112 | Effect: Allow 113 | Principal: 114 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 115 | Action: 'kms:*' 116 | Resource: '*' 117 | - Sid: Allow administration of the key 118 | Effect: Allow 119 | Principal: 120 | AWS: !GetAtt AdministratorRole.Arn 121 | Action: 122 | - 'kms:Create*' 123 | - 'kms:Describe*' 124 | - 'kms:Enable*' 125 | - 'kms:List*' 126 | - 'kms:Put*' 127 | - 'kms:Update*' 128 | - 'kms:Revoke*' 129 | - 'kms:Disable*' 130 | - 'kms:Get*' 131 | - 'kms:Delete*' 132 | - 'kms:ScheduleKeyDeletion' 133 | - 'kms:CancelKeyDeletion' 134 | Resource: '*' 135 | - Sid: Allow use of the key 136 | Effect: Allow 137 | Principal: 138 | AWS: !GetAtt AdministratorRole.Arn 139 | Action: 140 | - 'kms:Sign' 141 | - 'kms:Verify' 142 | - 'kms:DescribeKey' 143 | Resource: '*' 144 | kmsKey1: 145 | Metadata: 146 | cfn_nag: 147 | rules_to_suppress: 148 | - id: F19 149 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 150 | checkov: 151 | skip: 152 | - id: "CKV_AWS_7" 153 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 154 | Type: AWS::KMS::Key 155 | Properties: 156 | Description: An example symmetric encryption KMS key 157 | EnableKeyRotation: true 158 | PendingWindowInDays: 20 159 | KeyPolicy: 160 | Version: 2012-10-17 161 | Id: key-default-1 162 | Statement: 163 | - Sid: Enable IAM User Permissions 164 | Effect: Allow 165 | Principal: 166 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 167 | Action: 'kms:*' 168 | Resource: '*' 169 | - Sid: Allow administration of the key 170 | Effect: Allow 171 | Principal: 172 | AWS: !GetAtt IAMUserAlice.Arn 173 | Action: 174 | - 'kms:Create*' 175 | - 'kms:Describe*' 176 | - 'kms:Enable*' 177 | - 'kms:List*' 178 | - 'kms:Put*' 179 | - 'kms:Update*' 180 | - 'kms:Revoke*' 181 | - 'kms:Disable*' 182 | - 'kms:Get*' 183 | - 'kms:Delete*' 184 | - 'kms:ScheduleKeyDeletion' 185 | - 'kms:CancelKeyDeletion' 186 | Resource: '*' 187 | - Sid: Allow use of the key 188 | Effect: Allow 189 | Principal: 190 | AWS: !GetAtt IAMUserBob.Arn 191 | Action: 192 | - 'kms:DescribeKey' 193 | - 'kms:Encrypt' 194 | - 'kms:Decrypt' 195 | - 'kms:ReEncrypt*' 196 | - 'kms:GenerateDataKey' 197 | - 'kms:GenerateDataKeyWithoutPlaintext' 198 | Resource: '*' 199 | HMACExampleKey: 200 | Metadata: 201 | cfn_nag: 202 | rules_to_suppress: 203 | - id: F19 204 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 205 | checkov: 206 | skip: 207 | - id: "CKV_AWS_7" 208 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 209 | Type: 'AWS::KMS::Key' 210 | Properties: 211 | Description: HMAC_384 key for tokens 212 | KeySpec: HMAC_384 213 | KeyUsage: GENERATE_VERIFY_MAC 214 | KeyPolicy: 215 | Version: 2012-10-17 216 | Id: key-default-1 217 | Statement: 218 | - Sid: Enable IAM User Permissions 219 | Effect: Allow 220 | Principal: 221 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 222 | Action: 'kms:*' 223 | Resource: '*' 224 | - Sid: Allow administration of the key 225 | Effect: Allow 226 | Principal: 227 | AWS: !GetAtt AdministratorRole.Arn 228 | Action: 229 | - 'kms:Create*' 230 | - 'kms:Describe*' 231 | - 'kms:Enable*' 232 | - 'kms:List*' 233 | - 'kms:Put*' 234 | - 'kms:Update*' 235 | - 'kms:Revoke*' 236 | - 'kms:Disable*' 237 | - 'kms:Get*' 238 | - 'kms:Delete*' 239 | - 'kms:ScheduleKeyDeletion' 240 | - 'kms:CancelKeyDeletion' 241 | Resource: '*' 242 | - Sid: Allow use of the key 243 | Effect: Allow 244 | Principal: 245 | AWS: !GetAtt DeveloperRole.Arn 246 | Action: 247 | - 'kms:GenerateMac' 248 | - 'kms:VerifyMac' 249 | - 'kms:DescribeKey' 250 | Resource: '*' 251 | myPrimaryKey: 252 | Metadata: 253 | cfn_nag: 254 | rules_to_suppress: 255 | - id: F19 256 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 257 | checkov: 258 | skip: 259 | - id: "CKV_AWS_7" 260 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 261 | Type: 'AWS::KMS::Key' 262 | Properties: 263 | Description: An example multi-Region primary key 264 | MultiRegion: true 265 | EnableKeyRotation: true 266 | PendingWindowInDays: 10 267 | KeyPolicy: 268 | Version: 2012-10-17 269 | Id: key-default-1 270 | Statement: 271 | - Sid: Enable IAM User Permissions 272 | Effect: Allow 273 | Principal: 274 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 275 | Action: 'kms:*' 276 | Resource: '*' 277 | - Sid: Allow administration of the key 278 | Effect: Allow 279 | Principal: 280 | AWS: !GetAtt IAMUserAlice.Arn 281 | Action: 282 | - 'kms:ReplicateKey' 283 | - 'kms:Create*' 284 | - 'kms:Describe*' 285 | - 'kms:Enable*' 286 | - 'kms:List*' 287 | - 'kms:Put*' 288 | - 'kms:Update*' 289 | - 'kms:Revoke*' 290 | - 'kms:Disable*' 291 | - 'kms:Get*' 292 | - 'kms:Delete*' 293 | - 'kms:ScheduleKeyDeletion' 294 | - 'kms:CancelKeyDeletion' 295 | Resource: '*' 296 | - Sid: Allow use of the key 297 | Effect: Allow 298 | Principal: 299 | AWS: !GetAtt IAMUserBob.Arn 300 | Action: 301 | - 'kms:DescribeKey' 302 | - 'kms:Encrypt' 303 | - 'kms:Decrypt' 304 | - 'kms:ReEncrypt*' 305 | - 'kms:GenerateDataKey' 306 | - 'kms:GenerateDataKeyWithoutPlaintext' 307 | Resource: '*' 308 | MyKey: 309 | Metadata: 310 | cfn_nag: 311 | rules_to_suppress: 312 | - id: F19 313 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 314 | checkov: 315 | skip: 316 | - id: "CKV_AWS_7" 317 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 318 | Type: AWS::KMS::Key 319 | Properties: 320 | Description: "My KMS key" 321 | KeySpec: SYMMETRIC_DEFAULT 322 | KeyUsage: ENCRYPT_DECRYPT 323 | KeyPolicy: 324 | Version: 2012-10-17 325 | Id: MyKeyPolicy 326 | Statement: 327 | - Sid: Enable IAM User Permissions 328 | Effect: Allow 329 | Principal: 330 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 331 | Action: 'kms:*' 332 | Resource: '*' 333 | - Sid: Allow administration of the key 334 | Effect: Allow 335 | Principal: 336 | AWS: !GetAtt AdministratorRole.Arn 337 | Action: 338 | - 'kms:Create*' 339 | - 'kms:Describe*' 340 | - 'kms:Enable*' 341 | - 'kms:List*' 342 | - 'kms:Put*' 343 | - 'kms:Update*' 344 | - 'kms:Revoke*' 345 | - 'kms:Disable*' 346 | - 'kms:Get*' 347 | - 'kms:Delete*' 348 | - 'kms:TagResource' 349 | - 'kms:UntagResource' 350 | - 'kms:ScheduleKeyDeletion' 351 | - 'kms:CancelKeyDeletion' 352 | - 'kms:ReplicateKey' 353 | - 'kms:UpdatePrimaryRegion' 354 | Resource: '*' 355 | 356 | - Sid: Allow use of the key 357 | Effect: Allow 358 | Principal: 359 | AWS: !GetAtt IAMUserAlice.Arn 360 | Action: 361 | - 'kms:Encrypt' 362 | - 'kms:Decrypt' 363 | - 'kms:GenerateDataKey' 364 | - 'kms:GenerateDataKeyWithoutPlaintext' 365 | - 'kms:DescribeKey' 366 | Resource: '*' 367 | UnicornKey: 368 | Metadata: 369 | cfn_nag: 370 | rules_to_suppress: 371 | - id: F19 372 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 373 | checkov: 374 | skip: 375 | - id: "CKV_AWS_7" 376 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 377 | Type: AWS::KMS::Key 378 | Properties: 379 | Description: "Unicorn KMS key" 380 | KeySpec: SYMMETRIC_DEFAULT 381 | KeyUsage: ENCRYPT_DECRYPT 382 | KeyPolicy: 383 | Version: 2012-10-17 384 | Id: MyKeyPolicy 385 | Statement: 386 | - Sid: Enable IAM User Permissions 387 | Effect: Allow 388 | Principal: 389 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 390 | Action: 'kms:*' 391 | Resource: '*' 392 | - Sid: Allow administration of the key 393 | Effect: Allow 394 | Principal: 395 | AWS: !GetAtt AdministratorRole.Arn 396 | Action: 397 | - 'kms:*' 398 | Resource: '*' 399 | 400 | - Sid: Allow use of the key 401 | Effect: Allow 402 | Principal: 403 | AWS: !GetAtt IAMUserAlice.Arn 404 | Action: 405 | - 'kms:Encrypt' 406 | - 'kms:Decrypt' 407 | - 'kms:GenerateDataKey' 408 | - 'kms:GenerateDataKeyWithoutPlaintext' 409 | - 'kms:DescribeKey' 410 | Resource: '*' 411 | UnicornKey2: 412 | Metadata: 413 | cfn_nag: 414 | rules_to_suppress: 415 | - id: F19 416 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 417 | checkov: 418 | skip: 419 | - id: "CKV_AWS_7" 420 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 421 | Type: AWS::KMS::Key 422 | Properties: 423 | Description: "Unicorn 2 KMS key" 424 | KeySpec: SYMMETRIC_DEFAULT 425 | KeyUsage: ENCRYPT_DECRYPT 426 | KeyPolicy: 427 | Version: 2012-10-17 428 | Id: MyKeyPolicy 429 | Statement: 430 | - Sid: Enable IAM User Permissions 431 | Effect: Allow 432 | Principal: 433 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 434 | Action: 435 | - 'kms:Create*' 436 | - 'kms:Describe*' 437 | - 'kms:Enable*' 438 | - 'kms:List*' 439 | - 'kms:Put*' 440 | - 'kms:Update*' 441 | - 'kms:Revoke*' 442 | - 'kms:Disable*' 443 | - 'kms:Get*' 444 | - 'kms:Delete*' 445 | - 'kms:TagResource' 446 | - 'kms:UntagResource' 447 | - 'kms:ScheduleKeyDeletion' 448 | - 'kms:CancelKeyDeletion' 449 | - 'kms:ReplicateKey' 450 | - 'kms:UpdatePrimaryRegion' 451 | Resource: '*' 452 | - Sid: Allow administration of the key 453 | Effect: Allow 454 | Principal: 455 | AWS: !GetAtt AdministratorRole.Arn 456 | Action: 457 | - 'kms:*' 458 | Resource: '*' 459 | 460 | - Sid: Allow use of the key 461 | Effect: Allow 462 | Principal: 463 | AWS: !GetAtt IAMUserBob.Arn 464 | Action: 465 | - 'kms:*' 466 | Resource: '*' 467 | UnicornKey3: 468 | Metadata: 469 | cfn_nag: 470 | rules_to_suppress: 471 | - id: F19 472 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 473 | checkov: 474 | skip: 475 | - id: "CKV_AWS_7" 476 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 477 | Type: AWS::KMS::Key 478 | Properties: 479 | Description: "Unicorn 3 KMS key" 480 | KeySpec: SYMMETRIC_DEFAULT 481 | KeyUsage: ENCRYPT_DECRYPT 482 | KeyPolicy: 483 | Version: 2012-10-17 484 | Id: MyKeyPolicy 485 | Statement: 486 | - Sid: Enable IAM User Permissions 487 | Effect: Allow 488 | Principal: 489 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 490 | Action: 491 | - 'kms:Create*' 492 | - 'kms:Describe*' 493 | - 'kms:Enable*' 494 | - 'kms:List*' 495 | - 'kms:Put*' 496 | - 'kms:Update*' 497 | - 'kms:Revoke*' 498 | - 'kms:Disable*' 499 | - 'kms:Get*' 500 | - 'kms:Delete*' 501 | - 'kms:TagResource' 502 | - 'kms:UntagResource' 503 | - 'kms:ScheduleKeyDeletion' 504 | - 'kms:CancelKeyDeletion' 505 | - 'kms:ReplicateKey' 506 | - 'kms:UpdatePrimaryRegion' 507 | Resource: '*' 508 | - Sid: Allow administration of the key 509 | Effect: Allow 510 | Principal: 511 | AWS: !GetAtt AdministratorRole.Arn 512 | Action: 513 | - 'kms:*' 514 | Resource: '*' 515 | 516 | - Sid: Allow use of the key 517 | Effect: Allow 518 | Principal: 519 | AWS: !GetAtt IAMUserBob.Arn 520 | Action: 521 | - 'kms:*' 522 | Resource: '*' 523 | UnicornRSASigningKey: 524 | Metadata: 525 | cfn_nag: 526 | rules_to_suppress: 527 | - id: F19 528 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 529 | checkov: 530 | skip: 531 | - id: "CKV_AWS_7" 532 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 533 | Type: 'AWS::KMS::Key' 534 | Properties: 535 | Description: Unicorn RSA-3072 asymmetric KMS key for signing and verification 536 | KeySpec: RSA_3072 537 | KeyUsage: SIGN_VERIFY 538 | KeyPolicy: 539 | Version: 2012-10-17 540 | Id: sign-policy-1 541 | Statement: 542 | - Sid: Enable IAM User Permissions 543 | Effect: Allow 544 | Principal: 545 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 546 | Action: 'kms:*' 547 | Resource: '*' 548 | - Sid: Allow administration of the key 549 | Effect: Allow 550 | Principal: 551 | AWS: !GetAtt IAMUserAlice.Arn 552 | Action: 553 | - 'kms:*' 554 | Resource: '*' 555 | - Sid: Allow use of the key 556 | Effect: Allow 557 | Principal: 558 | AWS: !GetAtt IAMUserBob.Arn 559 | Action: 560 | - 'kms:Sign' 561 | - 'kms:Verify' 562 | - 'kms:DescribeKey' 563 | Resource: '*' 564 | UnicornRSASigningKey2: 565 | Metadata: 566 | cfn_nag: 567 | rules_to_suppress: 568 | - id: F19 569 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 570 | checkov: 571 | skip: 572 | - id: "CKV_AWS_7" 573 | reason: "Short lived workshop. Key Rotation intentionally left off this key" 574 | Type: 'AWS::KMS::Key' 575 | Properties: 576 | Description: Unicorn RSA-3072 asymmetric KMS key for signing and verification 577 | KeySpec: RSA_3072 578 | KeyUsage: SIGN_VERIFY 579 | KeyPolicy: 580 | Version: 2012-10-17 581 | Id: sign-policy-2 582 | Statement: 583 | - Sid: Enable IAM User Permissions 584 | Effect: Allow 585 | Principal: 586 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 587 | Action: 'kms:*' 588 | Resource: '*' 589 | - Sid: Allow administration of the key 590 | Effect: Allow 591 | Principal: 592 | AWS: !GetAtt IAMUserAlice.Arn 593 | Action: 594 | - 'kms:*' 595 | Resource: '*' 596 | - Sid: Allow use of the key 597 | Effect: Allow 598 | Principal: 599 | AWS: !GetAtt IAMUserBob.Arn 600 | Action: 601 | - 'kms:*' 602 | Resource: '*' 603 | 604 | # Aliases 605 | myAlias: 606 | Type: 'AWS::KMS::Alias' 607 | Properties: 608 | AliasName: alias/key1 609 | TargetKeyId: !Ref MyKey 610 | kmsKey1Alias: 611 | Type: 'AWS::KMS::Alias' 612 | Properties: 613 | AliasName: alias/Encryptionkey 614 | TargetKeyId: !Ref kmsKey1 615 | myPrimaryKeyAlias: 616 | Type: 'AWS::KMS::Alias' 617 | Properties: 618 | AliasName: alias/Lambdakey 619 | TargetKeyId: !Ref myPrimaryKey 620 | myAlias2: 621 | Type: 'AWS::KMS::Alias' 622 | Properties: 623 | AliasName: alias/RSASigningKEy 624 | TargetKeyId: !Ref RSASigningKey 625 | HMACExampleKeyAlias: 626 | Type: 'AWS::KMS::Alias' 627 | Properties: 628 | AliasName: alias/KeyForHMAC 629 | TargetKeyId: !Ref HMACExampleKey -------------------------------------------------------------------------------- /putDynamo.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import json 3 | 4 | import boto3 5 | 6 | # Arguments 7 | parser = argparse.ArgumentParser("Read JSON File to load into DynamoDB") 8 | parser.add_argument("-f", "--file", help="JSON file", required=True) 9 | parser.add_argument("-t", "--tablename", help="Table name", required=True) 10 | parser.add_argument("-r", "--region", help="Region", required=True) 11 | 12 | 13 | # Parse the arguments 14 | args = vars(parser.parse_args()) 15 | 16 | # Use the arguments 17 | filename = args["file"] 18 | tablename = args["tablename"] 19 | region = args["region"] 20 | 21 | dynamodbclient = boto3.resource("dynamodb", region_name=region) 22 | sample_table = dynamodbclient.Table(tablename) 23 | 24 | with open(filename, "r") as myfile: 25 | data = myfile.read() 26 | 27 | # parse file 28 | obj = json.loads(data) 29 | 30 | print(f"Obj: {obj}") 31 | for item in obj: 32 | print(f"item: {item}") 33 | for key in item.keys(): 34 | print(f"key: {key}") 35 | print(f"value: {item[key]}") 36 | response = sample_table.put_item(Item=item) 37 | -------------------------------------------------------------------------------- /security-account-athena.yaml: -------------------------------------------------------------------------------- 1 | # Author: Alex Goff 2 | # 3 | # Part 3 of 3 CloudFormation files for the Security Observability account 4 | # This template deploys the Athena and Quicksight resources 5 | 6 | AWSTemplateFormatVersion: 2010-09-09 7 | Description: Create Athena database for KMS Dashboard 8 | 9 | Parameters: 10 | Prefix: 11 | Default: kmsdashboard 12 | Type: String 13 | BucketNamePrefix: 14 | Default: kms-read-policy 15 | Type: String 16 | LambdaRoleName: 17 | Default: KMSReadLambdaRole 18 | Type: String 19 | 20 | Resources: 21 | # Set up Athena 22 | executeCreateKMSDashboardDatabase: 23 | Type: Custom::executeAthenaQuery 24 | Properties: 25 | ServiceToken: !GetAtt LambdaFunctionExecuteQuery.Arn 26 | queryoutput: !Join ["",[ "s3://", !Ref S3BucketAthenaWorkGroup, "/athena-results/" ] ] 27 | QueryString: !Sub 'CREATE DATABASE IF NOT EXISTS ${Prefix}database; ' 28 | executeCreateKMSDashboardFindingsTable: 29 | Type: Custom::executeAthenaQuery 30 | DependsOn: executeCreateKMSDashboardDatabase 31 | Properties: 32 | ServiceToken: !GetAtt LambdaFunctionExecuteQuery.Arn 33 | queryoutput: !Join ["",[ "s3://", !Ref S3BucketAthenaWorkGroup, "/athena-results/" ] ] 34 | QueryString: !Join 35 | - '' 36 | - - " CREATE EXTERNAL TABLE IF NOT EXISTS " 37 | - !Sub ${Prefix}database.${Prefix}table 38 | - " ( " 39 | - " `date` string, " 40 | - " `accountnumber` string, " 41 | - " `accountname` string, " 42 | - " `region` string, " 43 | - " `keyid` string, " 44 | - " `alias` string, " 45 | - " `sid` string, " 46 | - " `effect` string, " 47 | - " `principal` string, " 48 | - " `principalservice` string, " 49 | - " `action` string, " 50 | - " `condition` string, " 51 | - " `concern` string, " 52 | - " `resource` string, " 53 | - " `tags` string, " 54 | - " `creationdate` string, " 55 | - " `lastusedtime` string, " 56 | - " `lastusedaction` string, " 57 | - " `lastusedencryptioncontext` string, " 58 | - " `lastusedsourceipaddress` string, " 59 | - " `lastusedusername` string " 60 | - " ) " 61 | - " ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' " 62 | - " WITH SERDEPROPERTIES ('field.delim' = ',') " 63 | - " STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' " 64 | - !Sub " LOCATION 's3://${BucketNamePrefix}-${AWS::AccountId}-${AWS::Region}/' " 65 | - " TBLPROPERTIES ( " 66 | - " 'classification' = 'csv', " 67 | - " 'write.compression' = 'GZIP', " 68 | - " 'skip.header.line.count' = '1' " 69 | - " ); " 70 | LambdaFunctionExecuteQuery: 71 | Metadata: 72 | cfn_nag: 73 | rules_to_suppress: 74 | - id: W89 75 | reason: 'Lambdas only used to stand up Athena. No need to be in VPC' 76 | checkov: 77 | skip: 78 | - id: "CKV_AWS_117" 79 | reason: "No other resources created inside VPCs therefore not needed" 80 | - id: "CKV_AWS_116" 81 | reason: "Lambdas only used to stand up Athena. No need for DLQ" 82 | - id: "CKV_AWS_173" 83 | comment: "No sensitive data in Lambda env variables. No need for encrypting strings" 84 | 85 | 86 | Type: AWS::Lambda::Function 87 | Properties: 88 | ReservedConcurrentExecutions: 3 89 | Code: 90 | ZipFile: | 91 | # SPDX-License-Identifier: MIT-0 92 | import boto3 93 | import time 94 | import os 95 | import cfnresponse 96 | from botocore.exceptions import ClientError 97 | def lambda_handler(event, context): 98 | print(f"context: {context}") 99 | print(f"event: {event}") 100 | if (event['RequestType'] == 'Create' or event['RequestType'] == 'Update'): 101 | try: 102 | client = boto3.client('athena') 103 | query = event['ResourceProperties']['QueryString'] 104 | print(query) 105 | queryoutput = event['ResourceProperties']['queryoutput'] 106 | response = client.start_query_execution(QueryString=query, ResultConfiguration={'OutputLocation': queryoutput}, WorkGroup=os.getenv('athena_workgroup')) 107 | print(response) 108 | cfnresponse.send(event, context, cfnresponse.SUCCESS, {}) 109 | except Exception as ex: 110 | print(ex) 111 | cfnresponse.send(event, context, cfnresponse.FAILED, {}) 112 | else: 113 | cfnresponse.send(event, context, cfnresponse.SUCCESS, {}) 114 | Handler: index.lambda_handler 115 | Role: !GetAtt IAMRoleExecuteQueryAthena.Arn 116 | Runtime: python3.9 117 | MemorySize: 128 118 | Timeout: 200 119 | Description: Lambda for Athena Execute Query 120 | Environment: 121 | Variables: 122 | bucket_name: !Ref S3BucketAthenaWorkGroup 123 | athena_workgroup: !Ref AthenaWorkGroup 124 | AthenaWorkGroup: 125 | Type: AWS::Athena::WorkGroup 126 | Properties: 127 | Description: Athena WorkGroup for KMSDashboard 128 | Name: !Sub '${Prefix}-athena-workgroup' 129 | RecursiveDeleteOption: True 130 | State: ENABLED 131 | WorkGroupConfiguration: 132 | EnforceWorkGroupConfiguration: True 133 | PublishCloudWatchMetricsEnabled: True 134 | RequesterPaysEnabled: False 135 | ResultConfiguration: 136 | OutputLocation: !Join ['', ['s3://', !Ref 'S3BucketAthenaWorkGroup', '/athena-results/']] 137 | EncryptionConfiguration: 138 | EncryptionOption: SSE_S3 139 | LoggingBucket: 140 | Type: 'AWS::S3::Bucket' 141 | Metadata: 142 | cfn_nag: 143 | rules_to_suppress: 144 | - id: W51 145 | reason: "Access intended only within the same account" 146 | - id: W35 147 | reason: "This is a S3 bucket to store access logs from S3BucketAthenaWorkGroup" 148 | checkov: 149 | skip: 150 | - id: "CKV_AWS_18" 151 | reason: "This is the logging bucket! For S3BucketAthenaWorkGroup" 152 | Properties: 153 | BucketName: !Sub "${Prefix}-athena-loggingbucket-${AWS::AccountId}-${AWS::Region}" 154 | AccessControl: LogDeliveryWrite 155 | OwnershipControls: 156 | Rules: 157 | - ObjectOwnership: BucketOwnerPreferred 158 | VersioningConfiguration: 159 | Status: Enabled 160 | PublicAccessBlockConfiguration: 161 | BlockPublicAcls: true 162 | BlockPublicPolicy: true 163 | IgnorePublicAcls: true 164 | RestrictPublicBuckets: true 165 | BucketEncryption: 166 | ServerSideEncryptionConfiguration: 167 | - ServerSideEncryptionByDefault: 168 | SSEAlgorithm: AES256 169 | IAMRoleExecuteQueryAthena: 170 | Type: AWS::IAM::Role 171 | Properties: 172 | ManagedPolicyArns: 173 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 174 | - arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole 175 | AssumeRolePolicyDocument: 176 | Version: '2012-10-17' 177 | Statement: 178 | - Effect: Allow 179 | Principal: 180 | Service: 181 | - lambda.amazonaws.com 182 | Action: 183 | - sts:AssumeRole 184 | Policies: 185 | - PolicyName: getS3Data 186 | PolicyDocument: 187 | Version: '2012-10-17' 188 | Statement: 189 | - Effect: Allow 190 | Action: 191 | - s3:PutObject 192 | - s3:GetBucketLocation 193 | - s3:GetObject 194 | - s3:ListBucket 195 | - s3:ListBucketMultipartUploads 196 | - s3:ListMultipartUploadParts 197 | - s3:AbortMultipartUpload 198 | - s3:PutObject 199 | Resource: 200 | - !Join ['', ['arn:aws:s3:::', !Ref S3BucketAthenaWorkGroup]] 201 | - !Join ['', ['arn:aws:s3:::', !Ref S3BucketAthenaWorkGroup, '/*']] 202 | - !Sub 'arn:aws:s3:::${BucketNamePrefix}-${AWS::AccountId}-${AWS::Region}' 203 | - !Sub 'arn:aws:s3:::${BucketNamePrefix}-${AWS::AccountId}-${AWS::Region}/*' 204 | - Effect: Allow 205 | Action: 206 | - glue:CreateDatabase 207 | - glue:CreateTable 208 | - glue:GetDatabase 209 | - glue:GetDatabases 210 | - glue:GetTables 211 | - glue:GetTable 212 | Resource: 213 | - !Sub arn:aws:glue:*:${AWS::AccountId}:catalog 214 | - !Sub arn:aws:glue:*:${AWS::AccountId}:database/* 215 | - !Sub arn:aws:glue:*:${AWS::AccountId}:table/*/* 216 | - PolicyName: queryAthena 217 | PolicyDocument: 218 | Version: '2012-10-17' 219 | Statement: 220 | - Effect: Allow 221 | Action: 222 | - athena:StartQueryExecution 223 | - athena:GetQueryExecution 224 | - athena:GetQueryResults 225 | - athena:CreateNamedQuery 226 | - athena:CreateWorkGroup 227 | Resource: 228 | - !Sub 'arn:aws:athena:${AWS::Region}:${AWS::AccountId}:workgroup/${AthenaWorkGroup}' 229 | S3BucketAthenaWorkGroup: 230 | Type: AWS::S3::Bucket 231 | Properties: 232 | BucketName: !Sub "${Prefix}-athenaworkgroupbucket-${AWS::AccountId}-${AWS::Region}" 233 | LoggingConfiguration: 234 | DestinationBucketName: !Ref LoggingBucket 235 | AccessControl: Private 236 | PublicAccessBlockConfiguration: 237 | BlockPublicAcls : True 238 | BlockPublicPolicy : True 239 | IgnorePublicAcls : True 240 | RestrictPublicBuckets : True 241 | VersioningConfiguration: 242 | Status: Enabled 243 | BucketEncryption: 244 | ServerSideEncryptionConfiguration: 245 | - ServerSideEncryptionByDefault: 246 | SSEAlgorithm: AES256 247 | OwnershipControls: 248 | Rules: 249 | - ObjectOwnership: BucketOwnerPreferred 250 | S3BucketPolicyAthenaWorkGroup: 251 | Metadata: 252 | cfn_nag: 253 | rules_to_suppress: 254 | - id: F16 255 | reason: "Wildcard allowed by restricted with Conditions to reduce risk" 256 | 257 | Type: AWS::S3::BucketPolicy 258 | Properties: 259 | Bucket: !Ref S3BucketAthenaWorkGroup 260 | PolicyDocument: 261 | Statement: 262 | - 263 | Action: 264 | - s3:GetObject 265 | - s3:ListBucket 266 | - s3:PutObject 267 | Effect: Allow 268 | Resource: 269 | - !Sub arn:aws:s3:::${S3BucketAthenaWorkGroup} 270 | - !Sub arn:aws:s3:::${S3BucketAthenaWorkGroup}/* 271 | Principal: 272 | AWS: !Sub arn:aws:iam::${AWS::AccountId}:root 273 | 274 | # Quicksight Role 275 | QuickSightRole: 276 | Metadata: 277 | cfn_nag: 278 | rules_to_suppress: 279 | - id: W11 280 | reason: "Short lived workshop. More granular permissions not needed" 281 | - id: W28 282 | reason: "Short lived workshop. Resources will not be replaced - only torn down and deleted." 283 | checkov: 284 | skip: 285 | - id: "CKV_AWS_109" 286 | comment: "Short lived workshop. Students are allowed to experiment until workshop expires and is deleted." 287 | - id: "CKV_AWS_110" 288 | comment: "Short lived workshop. Trust Policy means role can only be assumed from by Quicksight service." 289 | Type: 'AWS::IAM::Role' 290 | Properties: 291 | ManagedPolicyArns: 292 | - arn:aws:iam::aws:policy/service-role/AWSQuicksightAthenaAccess 293 | - arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly 294 | - arn:aws:iam::aws:policy/service-role/AWSQuickSightTimestreamPolicy 295 | - arn:aws:iam::aws:policy/service-role/AWSQuickSightSageMakerPolicy 296 | - arn:aws:iam::aws:policy/AWSQuickSightIoTAnalyticsAccess 297 | - arn:aws:iam::aws:policy/service-role/AWSQuickSightElasticsearchPolicy 298 | 299 | AssumeRolePolicyDocument: 300 | Version: '2012-10-17' 301 | Statement: 302 | - Effect: Allow 303 | Principal: 304 | Service: 305 | - quicksight.amazonaws.com 306 | Action: 307 | - sts:AssumeRole 308 | Description: This role is for Quicksight to use 309 | Path: /service-role/ 310 | Policies: 311 | - PolicyName: QuicksightPolicy 312 | PolicyDocument: 313 | Version: 2012-10-17 314 | Statement: 315 | - Sid: AWSQuickSightIAMPolicy 316 | Effect: Allow 317 | Action: 318 | - "iam:List*" 319 | - "iam:CreateRole" 320 | - "iam:CreatePolicy" 321 | - "iam:AttachRolePolicy" 322 | - "iam:CreatePolicyVersion" 323 | - "iam:DeletePolicyVersion" 324 | - "iam:ListAttachedRolePolicies" 325 | - "iam:GetRole" 326 | - "iam:GetPolicy" 327 | - "iam:DetachRolePolicy" 328 | - "iam:GetPolicyVersion" 329 | Resource: "*" 330 | - Sid: AWSQuickSightS3PolicyList 331 | Effect: Allow 332 | Action: 333 | - "s3:ListAllMyBuckets" 334 | - "s3:ListBucket" 335 | Resource: "arn:aws:s3:::*" 336 | - Sid: AWSQuickSightS3Policy 337 | Effect: Allow 338 | Action: 339 | - "s3:Get*" 340 | - "s3:List*" 341 | - "s3:PutObject" 342 | Resource: !Sub "arn:aws:s3:::kms-read-policy-${AWS::AccountId}-${AWS::Region}/*" 343 | RoleName: QuicksightRole 344 | 345 | Outputs: 346 | AthenaCustomResourceLambda: 347 | Description: Athena Custom Resource Lambda 348 | Value: !Ref LambdaFunctionExecuteQuery 349 | Export: 350 | Name: AthenaCustomResourceLambda 351 | AthenaCustomResourceLambdaArn: 352 | Description: Athena Custom Resource Lambda 353 | Value: !GetAtt LambdaFunctionExecuteQuery.Arn 354 | Export: 355 | Name: AthenaCustomResourceLambdaArn 356 | 357 | -------------------------------------------------------------------------------- /security-account-dynamodb.yaml: -------------------------------------------------------------------------------- 1 | # Author: Alex Goff 2 | # 3 | # Part 1 of 3 CloudFormation files for the Security Observability account 4 | # This template deploys the 'lastUsedDate' and 'accounts' DynamoDB tables 5 | AWSTemplateFormatVersion: '2010-09-09' 6 | Description: "KMS Last Used DynamoDB Table" 7 | Parameters: 8 | accountsTableName: 9 | Description: DynamoDB Table for storing a list of valid accounts 10 | Default: accounts 11 | Type: String 12 | readDynamoDBRoleName: 13 | Description: Name of IAM role with permission to read from DynamoDB 14 | Default: ReadDynamoDBRole 15 | Type: String 16 | KMSReadLambdaRoleName: 17 | Description: Name of IAM role in used by KMSRead lambda function 18 | Default: KMSReadLambdaRole 19 | Type: String 20 | lastUsedTableName: 21 | Description: DynamoDB Table for storing KMS last used dates from CloudTrail 22 | Default: lastUsedTable 23 | Type: String 24 | lastUsedDynamoDBLamdaRoleName: 25 | Description: Name of DynamoDB Lambda Role for lastUsedLambdas to assume to. 26 | Default: putToDynamoRole 27 | Type: String 28 | lastUsedLambdaRoleName: 29 | Description: Name of lastUsedLambda Role deployed in member accounts. Needed for AssumeRole Trust Policy 30 | Default: lastUsedLambdaRole 31 | Type: String 32 | Resources: 33 | #DynamoDB Table for a list of accounts 34 | accountDynamoDBTable: 35 | Type: AWS::DynamoDB::Table 36 | Metadata: 37 | checkov: 38 | skip: 39 | - id: "CKV_AWS_119" 40 | reason: "This is proof of concept code. Customers should review their encryption requirements and use CMKs if required" 41 | Properties: 42 | KeySchema: 43 | - AttributeName: accountId #Use accountId as PartitionKey 44 | KeyType: HASH 45 | TableName: !Ref accountsTableName 46 | AttributeDefinitions: 47 | - AttributeName: accountId 48 | AttributeType: "S" 49 | BillingMode: PAY_PER_REQUEST 50 | PointInTimeRecoverySpecification: 51 | PointInTimeRecoveryEnabled: true 52 | SSESpecification: 53 | SSEEnabled: true 54 | 55 | lastUsedDynamoDBTable: 56 | Type: AWS::DynamoDB::Table 57 | Metadata: 58 | checkov: 59 | skip: 60 | - id: "CKV_AWS_119" 61 | reason: "This is proof of concept code. Customers should review their encryption requirements and use CMKs if required" 62 | Properties: 63 | KeySchema: 64 | - AttributeName: keyID #Use KMS keyID as PartitionKey 65 | KeyType: HASH 66 | TableName: !Ref lastUsedTableName 67 | AttributeDefinitions: 68 | - AttributeName: keyID #KMS keyID is a 'string' 69 | AttributeType: "S" 70 | BillingMode: PAY_PER_REQUEST 71 | TimeToLiveSpecification: 72 | AttributeName: TimetoLive 73 | Enabled: True 74 | PointInTimeRecoverySpecification: 75 | PointInTimeRecoveryEnabled: true 76 | SSESpecification: 77 | SSEEnabled: true 78 | 79 | lastUsedDynamoDBRoleForLambdas: 80 | Type: "AWS::IAM::Role" 81 | Metadata: 82 | checkov: 83 | skip: 84 | - id: "CKV_AWS_60" 85 | reason: "IAM role has been scoped down with a condition key to ensure compliance. Customers can further secure by explicitly listing accounts instead of '*'" 86 | Properties: 87 | AssumeRolePolicyDocument: 88 | Version: '2012-10-17' 89 | Statement: 90 | - Effect: Allow 91 | Principal: 92 | AWS: "*" 93 | Action: sts:AssumeRole 94 | Condition: 95 | StringLike: 96 | aws:PrincipalArn: 97 | - !Sub "arn:aws:iam::*:role/${lastUsedLambdaRoleName}-*" 98 | Path: / 99 | Policies: 100 | - PolicyName: DynamoDBPutItem-Policy 101 | PolicyDocument: 102 | Version : "2012-10-17" 103 | Statement: 104 | - Sid: 'DynamoDBPutItem' 105 | Effect: "Allow" 106 | Action: 107 | - "dynamodb:PutItem" 108 | Resource: 109 | !GetAtt lastUsedDynamoDBTable.Arn 110 | RoleName: !Ref lastUsedDynamoDBLamdaRoleName 111 | 112 | DynamoDBRole: 113 | Metadata: 114 | cfn_nag: 115 | rules_to_suppress: 116 | - id: W11 117 | reason: "Short lived workshop. More granular permissions not needed" 118 | - id: W28 119 | reason: "Short lived workshop. Resources will not be replaced - only torn down and deleted." 120 | Type: AWS::IAM::Role 121 | Properties: 122 | RoleName: !Ref readDynamoDBRoleName 123 | AssumeRolePolicyDocument: 124 | Version: '2012-10-17' 125 | Statement: 126 | - Effect: Allow 127 | Principal: 128 | AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" 129 | Action: sts:AssumeRole 130 | Condition: 131 | StringLike: 132 | aws:PrincipalArn: 133 | - !Sub "arn:aws:iam::*:role/${KMSReadLambdaRoleName}" 134 | Path: / 135 | Policies: 136 | - PolicyName: DynDBAccess 137 | PolicyDocument: 138 | Version: "2012-10-17" 139 | Statement: 140 | - Sid: 'ReadDynamo' 141 | Effect: "Allow" 142 | Action: 143 | - "dynamodb:GetItem" 144 | - "dynamodb:Scan" 145 | Resource: !Sub "arn:aws:dynamodb:*:${AWS::AccountId}:table/*" 146 | 147 | Outputs: 148 | lastUsedDynamoDBTableRoleForAssuming: 149 | Description: "KMS CloudTrail to DynamoDB IAM Role - Assumed by Lambdas in other accounts deployed via StackSets" 150 | Value: !GetAtt lastUsedDynamoDBRoleForLambdas.Arn 151 | lastUsedDynamoDBTableTableName: 152 | Description: "Name of Shared DynamoDB Table for storing CloudTrail records from accounts" 153 | Value: !Ref lastUsedTableName 154 | lastUsedDynamoDBTableTableArn: 155 | Description: "Arn of Shared DynamoDB Table for storing CloudTrail records from accounts" 156 | Value: !GetAtt lastUsedDynamoDBTable.Arn -------------------------------------------------------------------------------- /security-account-kmsread-lambda.yaml: -------------------------------------------------------------------------------- 1 | # Author: Alex Goff 2 | # 3 | # Part 2 of 3 CloudFormation files for the Security Observability account 4 | # This template deploys the 'KMSRead' lambda that scans all member accounts. 5 | # Requires the 'kmsread-role' to be deployed 6 | 7 | AWSTemplateFormatVersion: "2010-09-09" 8 | Description: | 9 | This stack creates the S3 bucket and lambda resources for the AWS KMS Policy Read Solution 10 | 11 | Parameters: 12 | kmsPolicyDeletionDays: 13 | Description: 'After how many days to delete KMS Policy CSV files - defaults to 15 months (456 days)' 14 | Default: 456 #15 months 15 | Type: Number 16 | kmsLamdaRoleName: 17 | Description: 'IAM Role name used by the KMS Read Lambda function' 18 | Default: KMSReadLambdaRole 19 | Type: String 20 | kmsReadS3Bucket: 21 | Description: 'Bucket prefix to store CSV files from Lambda function' 22 | Default: kms-read-policy 23 | Type: String 24 | accountsDynamoDBTable: 25 | Description: 'DynamoDB table containing list of account IDs' 26 | Default: accounts 27 | Type: String 28 | dynamoDBAccount: 29 | Description: 'Account where DynamoDB tables are stored' 30 | Type: String 31 | Default: '123456789012' 32 | dynamoDBRole: 33 | Description: 'Role used to assume to read DynamoDB Tables' 34 | Type: String 35 | Default: ReadDynamoDBRole 36 | dynamoDBRegion: 37 | Description: 'Region used to find to DynamoDB Tables' 38 | Type: String 39 | Default: us-east-1 40 | lastUsedDynamoDBTable: 41 | Description: 'DynamoDB table containing last used entries for KMS keys' 42 | Default: lastUsedTable 43 | Type: String 44 | XAKMSReadRole: 45 | Description: 'CrossAccount IAM Role used by KMS Read function to assume into member accounts and read keys' 46 | Default: XA-KMSRead-Role 47 | Type: String 48 | Resources: 49 | KMSPolicyS3Bucket: 50 | Metadata: 51 | checkov: 52 | skip: 53 | - id: "CKV_AWS_18" 54 | reason: "This is proof of concept code. Customers should review their logging requirements and enable access logging if required" 55 | 56 | Type: "AWS::S3::Bucket" 57 | Properties: 58 | BucketName: !Sub "${kmsReadS3Bucket}-${AWS::AccountId}-${AWS::Region}" 59 | BucketEncryption: 60 | ServerSideEncryptionConfiguration: 61 | - ServerSideEncryptionByDefault: 62 | SSEAlgorithm: AES256 63 | PublicAccessBlockConfiguration: 64 | BlockPublicAcls: true 65 | BlockPublicPolicy: true 66 | IgnorePublicAcls: true 67 | RestrictPublicBuckets: true 68 | VersioningConfiguration: 69 | Status: Enabled 70 | LifecycleConfiguration: 71 | Rules: 72 | - Id: Lifecycle rule for managing old KMS CSVs 73 | Prefix: logs 74 | Status: Enabled 75 | Transitions: 76 | - TransitionInDays: 30 77 | StorageClass: STANDARD_IA 78 | - TransitionInDays: 90 79 | StorageClass: GLACIER 80 | ExpirationInDays: !Ref kmsPolicyDeletionDays 81 | KMSPolicyS3BucketPolicy: 82 | Type: AWS::S3::BucketPolicy 83 | Properties: 84 | Bucket: !Ref KMSPolicyS3Bucket 85 | PolicyDocument: 86 | Statement: 87 | - 88 | Action: 89 | - s3:GetObject 90 | - s3:ListBucket 91 | - s3:PutObject 92 | Effect: Allow 93 | Resource: 94 | - !Sub arn:aws:s3:::${KMSPolicyS3Bucket} 95 | - !Sub arn:aws:s3:::${KMSPolicyS3Bucket}/* 96 | Principal: 97 | AWS: 98 | - !Sub arn:aws:iam::${AWS::AccountId}:role/${kmsLamdaRoleName} 99 | LambdaRole: 100 | Metadata: 101 | checkov: 102 | skip: 103 | - id: "CKV_AWS_111" 104 | reason: "This is proof of concept code. Allows creation of log groups" 105 | Type: "AWS::IAM::Role" 106 | Properties: 107 | AssumeRolePolicyDocument: 108 | Version: "2012-10-17" 109 | Statement: 110 | - 111 | Effect: "Allow" 112 | Principal: 113 | Service: 114 | - lambda.amazonaws.com 115 | Action: 116 | - 'sts:AssumeRole' 117 | Path: / 118 | Policies: 119 | - PolicyName: KMSRead-Policy 120 | PolicyDocument: 121 | Version : "2012-10-17" 122 | Statement: 123 | - 124 | Effect: "Allow" 125 | Action: 126 | - "logs:CreateLogGroup" 127 | - "logs:CreateLogStream" 128 | - "logs:PutLogEvents" 129 | Resource: "*" 130 | - 131 | Effect: "Allow" 132 | Action: 133 | - "s3:List*" 134 | - "dynamodb:List*" 135 | Resource: "*" 136 | - 137 | Effect: "Allow" 138 | Action: 139 | - "sts:AssumeRole" 140 | Resource: 141 | - !Sub "arn:aws:iam::*:role/${XAKMSReadRole}" 142 | - !Sub 'arn:aws:iam::${dynamoDBAccount}:role/${dynamoDBRole}' 143 | - 144 | Effect: "Allow" 145 | Action: 146 | - "s3:GetObject" 147 | Resource: 148 | - !Sub 'arn:aws:s3:::${kmsReadS3Bucket}-${AWS::AccountId}-${AWS::Region}' 149 | - !Sub 'arn:aws:s3:::${kmsReadS3Bucket}-${AWS::AccountId}-${AWS::Region}/*' 150 | RoleName: !Ref kmsLamdaRoleName 151 | 152 | LambdaCron: 153 | Type: "AWS::Events::Rule" 154 | Properties: 155 | Description: This event rules triggers the lambda on a schedule 156 | Name: KMSRead-Schedule 157 | ScheduleExpression: rate(1 day) 158 | State: ENABLED 159 | Targets: 160 | - 161 | Arn: !GetAtt KMSReadPolicyLambda.Arn 162 | Id: 1 163 | 164 | KMSReadPolicyLambda: 165 | Type: "AWS::Lambda::Function" 166 | Metadata: 167 | checkov: 168 | skip: 169 | - id: "CKV_AWS_115" 170 | reason: "This is proof of concept code. Review need for concurrent execution limits as needed" 171 | - id: "CKV_AWS_117" 172 | reason: "No other resources created inside VPCs therefore not needed" 173 | - id: "CKV_AWS_116" 174 | reason: "Proof of Concept code. Customers can determine if DLQs are required." 175 | - id: "CKV_AWS_173" 176 | reason: "Proof of Concept code. No senstive data stored in Lambda environment variables." 177 | 178 | 179 | Properties: 180 | Description: Lambda to get query KMS Policies on all KMS keys 181 | FunctionName: KMSRead 182 | Handler: index.lambda_handler 183 | Role: !GetAtt LambdaRole.Arn 184 | Runtime: python3.12 185 | Architectures: 186 | - arm64 187 | Tags: 188 | - 189 | Key: Project 190 | Value: re:Invent 2024 191 | Environment: 192 | Variables: 193 | DEST_BUCKET: !Sub "${kmsReadS3Bucket}-${AWS::AccountId}-${AWS::Region}" 194 | REGIONS: "eu-west-1,eu-west-2,us-east-1" 195 | ACCOUNTSDYNAMODBTABLE: !Ref accountsDynamoDBTable 196 | ACCOUNTSDYNAMODBACCOUNT: !Ref dynamoDBAccount 197 | ACCOUNTSDYNAMODBROLE: !Ref dynamoDBRole 198 | ACCOUNTSDYNAMODBREGION: !Ref dynamoDBRegion 199 | LASTUSEDDYNAMODBTABLE: !Ref lastUsedDynamoDBTable 200 | LASTUSEDDYNAMODBACCOUNT: !Ref dynamoDBAccount 201 | LASTUSEDDYNAMODBROLE: !Ref dynamoDBRole 202 | LASTUSEDDYNAMODBREGION: !Ref dynamoDBRegion 203 | XAKMSREADROLE: !Ref XAKMSReadRole 204 | MemorySize: 256 205 | Timeout: 180 206 | Code: 207 | ZipFile: | 208 | # KMSRead_lambda function: 209 | # This function reads down the list of accounts in the 'accounts' DynamoDB table. 210 | # For every account is tries to AssumeRole into that account and then read the KMS keys 211 | # For every key it finds it then reads their aliases, tags, creation date via the KMS APIs 212 | # and stores them in a big JSON object. 213 | # It then augments this with the details of when the key was lastUsed from the 214 | # 'lastUsedTable' DynamoDB table 215 | # Once all this is done, each key policy __statement__ is written as a separate line in a CSV 216 | # This is done to make it clearer when 'Concerns' are raised and which policy 217 | # statement they relate to. 218 | # Once all this is complete, the final CSV file is pushed up to S3 with a file 219 | # name of the account and region 220 | 221 | import csv 222 | import json 223 | import logging 224 | import os 225 | from datetime import date, datetime 226 | 227 | # Define Imports 228 | import boto3 229 | import botocore 230 | 231 | # Logging 232 | logger = logging.getLogger(__name__) 233 | FORMAT = ( 234 | "[%(asctime)s:%(levelname)s:%(filename)s:%(lineno)s - %(funcName)10s()] %(message)s" 235 | ) 236 | 237 | logger.setLevel(logging.INFO) 238 | logging.basicConfig(format=FORMAT) 239 | 240 | 241 | # Globals 242 | # DynamoDB table containing list of accounts and mapping IDs 243 | # Table name of accounts. Defaults to 'accounts' 244 | accountsDynamoDBTable = str(os.getenv("ACCOUNTSDYNAMODBTABLE", "accounts")) 245 | 246 | # Account number containing the DynamoDB Table containing 'accounts' table 247 | accountsDynamoDBAccount = str(os.getenv("ACCOUNTSDYNAMODBACCOUNT", "123456789012")) 248 | 249 | # IAM Role assumed by this lambda function with permissions to read 'accounts' DynamoDB table 250 | accountsDynamoDBRole = str(os.getenv("ACCOUNTSDYNAMODBROLE", "ReadDynamoDBRole")) 251 | 252 | # Region containing the DynamoDB Table containing 'accounts' table 253 | accountsDynamoDBRegion = str(os.getenv("ACCOUNTSDYNAMODBREGION", "us-east-1")) 254 | 255 | # DynamoDB table containing the last used data 256 | lastUsedDynamoDBTable = str(os.getenv("LASTUSEDDYNAMODBTABLE", "lastUsedTable")) 257 | 258 | # Account number containing the DynamoDB Table containing 'lastUsedTable' table 259 | lastUsedDynamoDBAccount = str(os.getenv("LASTUSEDDYNAMODBACCOUNT", "123456789012")) 260 | 261 | # IAM Role assumed by this lambda function with permissions to read 'lastUsedTable' table 262 | lastUsedDynamoDBRole = str(os.getenv("LASTUSEDDYNAMODBROLE", "ReadDynamoDBRole")) 263 | 264 | # Region containing the DynamoDB Table containing 'lastUsedTable' table 265 | lastUsedDynamoDBRegion = str(os.getenv("LASTUSEDDYNAMODBREGION", "us-east-1")) 266 | 267 | # Cross-account role assumed by this lambda function in each memeber account. 268 | # This role is deployed by the member-account-kmsread-role.yaml CloudFormation 269 | # and has permissions to read and gather details of KMS keys. 270 | xaKMSReadRole = str(os.getenv("XAKMSREADROLE", "XA-KMSRead-Role")) 271 | 272 | 273 | # Read S3 bucket name containing the CSV files 274 | destination_s3_bucket = str( 275 | os.getenv("DEST_BUCKET", "kms-read-policy-123456789012-us-east-1") 276 | ) 277 | 278 | # Read active regions from env variable to loop through once we assume into 279 | # member account 280 | regions_string = str(os.getenv("REGIONS", "eu-west-1,eu-west-2,us-east-1")) 281 | 282 | 283 | # Gather all the keys. These are used to drive the policy and alias searches 284 | def getKeys(kms): 285 | response = kms.list_keys() 286 | return response["Keys"] 287 | 288 | 289 | # Gather the list of policies containing all the key policy statements 290 | def getKeyPolicies(kms, keyid): 291 | try: 292 | response = kms.list_key_policies(KeyId=keyid) 293 | return response["PolicyNames"] 294 | except botocore.exceptions.ClientError as error: 295 | if error.response["Error"]["Code"] == "AccessDeniedException": 296 | logger.warning( 297 | f"Unable to get Policies: {error.response['Error']['Message']}" 298 | ) 299 | else: 300 | logger.error(f"ERROR RESPONSE: {error.response}") 301 | raise 302 | 303 | 304 | # Read all the account from the 'accounts' table so we know what to 305 | # loop through 306 | def get_accounts( 307 | accountsDynamoDBAccount: str, 308 | accountsDynamoDBRole: str = "ReadDynamoDBRole", 309 | accountsDynamoDBTable: str = "accounts", 310 | region: str = "us-east-1", 311 | ): 312 | # Get assumed role credentials for dynamodb read 313 | try: 314 | session = getAssumedRoleSession(accountsDynamoDBAccount, accountsDynamoDBRole) 315 | 316 | dynamodb = session.resource("dynamodb", region_name=region) 317 | table = dynamodb.Table(accountsDynamoDBTable) 318 | 319 | accounts = table.scan(ProjectionExpression="accountId, accountName") 320 | 321 | return accounts["Items"] 322 | 323 | except Exception as e: 324 | logger.error("Issue assuming the XA role: " + str(e)) 325 | 326 | 327 | # Gather key policies based on initial list of keys 328 | # Pass in the 'kms' session so we don't need to re-assume any roles anywhere 329 | def getPolicy(kms, keyId, policy): 330 | response = kms.get_key_policy(KeyId=keyId, PolicyName=policy) 331 | return json.loads(response["Policy"]) 332 | 333 | 334 | # Get the creation date of the key 335 | def getCreationDate(kms, keyId): 336 | response = kms.describe_key(KeyId=keyId) 337 | return response["KeyMetadata"]["CreationDate"].strftime("%Y-%m-%d %H:%M:%S") 338 | 339 | 340 | # Get all the tags for the keys 341 | def getTag(kms, keyId): 342 | response = kms.list_resource_tags(KeyId=keyId) 343 | return response["Tags"] 344 | 345 | 346 | # Get the alias the kms key 347 | def getAliases(kms, keyId): 348 | response = kms.list_aliases(KeyId=keyId) 349 | return response["Aliases"] 350 | 351 | 352 | # Get the last used dates from lastUsedTable 353 | def getLastUsed( 354 | keyid, 355 | lastUsedDynamoDBAccount, 356 | lastUsedDynamoDBRole, 357 | lastUsedDynamoDBTable, 358 | lastUsedDynamoDBRegion, 359 | ): 360 | try: 361 | session = getAssumedRoleSession(lastUsedDynamoDBAccount, lastUsedDynamoDBRole) 362 | dynamodb = session.resource("dynamodb", region_name=lastUsedDynamoDBRegion) 363 | table = dynamodb.Table(lastUsedDynamoDBTable) 364 | response = table.get_item(Key={"keyID": keyid}) 365 | if "Item" in response: 366 | return response["Item"] 367 | else: 368 | return None 369 | 370 | except Exception as e: 371 | logger.error("Issue getting the last used events: " + str(e)) 372 | 373 | return None 374 | 375 | 376 | # The main orchestrator function. This pulls everything together 377 | def getEverythingJson(kms): 378 | kms_keys = getKeys(kms) 379 | 380 | keyMap = {"kms_keys": []} 381 | 382 | for kms_key in kms_keys: 383 | # Create an empty dict to hold everything 384 | kms_key_object = {} 385 | # Create a key using the KMS keyId 386 | kms_key_object["KeyId"] = kms_key["KeyId"] 387 | # Create an empty list for all the aliases just in case 388 | # we don't find any. 389 | 390 | # Create a temporary list to hold the aliaes 391 | list_of_aliases = [] 392 | # Create a temporary list to hold the policies 393 | list_of_policies = [] 394 | 395 | keyid = kms_key["KeyId"] 396 | 397 | # Now grab all the aliases passing in our kms session and the keyId 398 | kms_aliases = getAliases(kms, keyid) 399 | 400 | # Did we find any aliases? 401 | if kms_aliases: 402 | # Loop through all the aliases that we found (might just be one) 403 | for ind in range(len(kms_aliases)): 404 | # Add the alias to the list of aliases 405 | list_of_aliases.append(kms_aliases[ind]["AliasName"]) 406 | 407 | # Add them to the key object (even if list_of_aliases is empty) 408 | kms_key_object["Aliases"] = list_of_aliases 409 | 410 | # Now grab all the policies passing in our kms session and the keyId 411 | kms_key_policies = getKeyPolicies(kms, keyid) 412 | 413 | # Did we find any key policies? 414 | # At this stage we just want all the policies. We're not digging 415 | # into them yet. 416 | if kms_key_policies: 417 | # If so, grab the policy details for the specificed keyId 418 | for policy in kms_key_policies: 419 | key_policy = getPolicy(kms, keyid, policy) 420 | # Append this policy to the list of policies 421 | list_of_policies.append(key_policy) 422 | 423 | kms_key_object["Policies"] = list_of_policies 424 | 425 | creation_date = getCreationDate(kms, keyid) 426 | kms_key_object["CreationDate"] = creation_date 427 | 428 | key_tag = getTag(kms, keyid) 429 | kms_key_object["Tags"] = key_tag 430 | 431 | # Now grab the lastUsedDate of the KMS Key from the 'lastUsedTable' DynamoDB table 432 | # NOTE: this table is populated separately by the 'lastUsed_lambda' 433 | kms_last_used = getLastUsed( 434 | kms_key["KeyId"], 435 | lastUsedDynamoDBAccount, 436 | lastUsedDynamoDBRole, 437 | lastUsedDynamoDBTable, 438 | lastUsedDynamoDBRegion, 439 | ) 440 | 441 | # Did we find any lastUsed entries in the DynamoDB table? 442 | if kms_last_used: 443 | # If we did then grab the relevant values from the lastUsed DynamoDB record 444 | if "EventTime" in kms_last_used: 445 | kms_key_object["LastUsedTime"] = kms_last_used["EventTime"] 446 | if "EventName" in kms_last_used: 447 | kms_key_object["LastUsedAction"] = kms_last_used["EventName"] 448 | if "encryptionContext" in kms_last_used: 449 | kms_key_object["LastUsedEncryptionContext"] = kms_last_used[ 450 | "encryptionContext" 451 | ] 452 | if "sourceIPAddress" in kms_last_used: 453 | kms_key_object["LastUsedSourceIPAddress"] = kms_last_used[ 454 | "sourceIPAddress" 455 | ] 456 | if "Username" in kms_last_used: 457 | kms_key_object["LastUsedUsername"] = kms_last_used["Username"] 458 | 459 | # Now we have all entries for the kms_key_object including: 460 | # ["KeyId"], ["Aliases"], ["Policies"], ["Tags"], ["CreationDate"], ["LastUsedTime"] etc. 461 | 462 | # The kms_key_object now contains everything we need for the CSV 463 | 464 | # Add this key to the big list of keys 465 | keyMap["kms_keys"].append(kms_key_object) 466 | return keyMap 467 | 468 | 469 | # Function to assume role 470 | def getAssumedRoleSession(aws_account, role_name="XA-KMSRead-Role"): 471 | role_to_assume_arn = "arn:aws:iam::" + aws_account + ":role/" + role_name 472 | sts_client = boto3.client("sts") 473 | 474 | logged_on_arn = sts_client.get_caller_identity()["Arn"] 475 | logger.debug( 476 | f"Logged on user: '{logged_on_arn}' assuming role '{role_to_assume_arn}" 477 | ) 478 | 479 | try: 480 | response = sts_client.assume_role( 481 | RoleArn=role_to_assume_arn, RoleSessionName=role_name 482 | ) 483 | creds = response["Credentials"] 484 | session = boto3.session.Session( 485 | aws_access_key_id=creds["AccessKeyId"], 486 | aws_secret_access_key=creds["SecretAccessKey"], 487 | aws_session_token=creds["SessionToken"], 488 | ) 489 | return session 490 | except Exception as e: 491 | logger.error( 492 | f"Unable to assume role '{role_name}' in account in '{aws_account}': {e}" 493 | ) 494 | 495 | 496 | # Now generate a CSV based on the keyMap file 497 | def getEverythingToCSV( 498 | accountNumber: str, accountName: str, filename: str, keyMap: dict, region: str 499 | ): 500 | logger.info(f"{len(keyMap['kms_keys'])} KMS keys found in [{region}]") 501 | filename = "/tmp/" + filename 502 | 503 | kmsKeysWithPolicies = [] 504 | 505 | for key in keyMap["kms_keys"]: 506 | # This is where the create the multiple lines for each policy statement 507 | # but repeat the core data such as KeyID, creationDate, last used etc. 508 | 509 | if "Policies" in key: 510 | 511 | # Grab the KeyId 512 | keyId = key["KeyId"] 513 | 514 | # Grab the 1st alias 515 | keyAlias = None 516 | if "Aliases" in key: 517 | logger.debug(f"{len(key['Aliases'])} aliases found for {keyId}.") 518 | # If there are any aliases found, then take the first 519 | if len(key["Aliases"]) > 0: 520 | keyAlias = key["Aliases"][0] 521 | 522 | # Grab the last used details 523 | keyLastUsedTime = None 524 | if "LastUsedTime" in key: 525 | keyLastUsedTime = key["LastUsedTime"] 526 | 527 | keyLastUsedAction = None 528 | if "LastUsedAction" in key: 529 | keyLastUsedAction = key["LastUsedAction"] 530 | 531 | keyLastUsedEncryptionContext = None 532 | if "LastUsedEncryptionContext" in key: 533 | keyLastUsedEncryptionContext = key["LastUsedEncryptionContext"] 534 | 535 | keyLastUsedSourceIPAddress = None 536 | if "LastUsedSourceIPAddress" in key: 537 | keyLastUsedSourceIPAddress = key["LastUsedSourceIPAddress"] 538 | 539 | keyLastUsedUsername = None 540 | if "LastUsedUsername" in key: 541 | keyLastUsedUsername = key["LastUsedUsername"] 542 | 543 | # Process the policies 544 | # NOTE: A key may have multiple statements in a policy. A unique line 545 | # is created for each policy. The aliases and last useds are duplicated for those lines 546 | # but this means that concerns can be generated for each statement rather than a policy overall 547 | for x in range( 548 | len(key["Policies"]) 549 | ): # Probably always one Policy - so x will almost always be 0 550 | # This is where we loop through each Statement in the policy but duplicate the records 551 | for line in grabPolicyStatementDetailsList( 552 | accountNumber, 553 | accountName, 554 | region, 555 | keyId, 556 | keyAlias, 557 | key["Policies"][x]["Statement"], 558 | key["Tags"], 559 | key["CreationDate"], 560 | keyLastUsedTime, 561 | keyLastUsedAction, 562 | keyLastUsedEncryptionContext, 563 | keyLastUsedSourceIPAddress, 564 | keyLastUsedUsername, 565 | ): 566 | kmsKeysWithPolicies.append(line) 567 | 568 | # Write to CSV 569 | header = [ 570 | "Date", 571 | "AccountNumber", 572 | "AccountName", 573 | "Region", 574 | "KeyId", 575 | "Alias", 576 | "Sid", 577 | "Effect", 578 | "Principal", 579 | "Principal Service", 580 | "Action", 581 | "Condition", 582 | "Concern", 583 | "Resource", 584 | "Tags", 585 | "CreationDate", 586 | "LastUsedTime", 587 | "LastUsedAction", 588 | "LastUsedEncryptionContext", 589 | "LastUsedSourceIPAddress", 590 | "LastUsedUsername", 591 | ] 592 | 593 | # Write the actual file from the kmsKeysWithPolicies JSON object 594 | with open(filename, "w") as file: 595 | writer = csv.DictWriter(file, fieldnames=header) 596 | writer.writeheader() 597 | writer.writerows(kmsKeysWithPolicies) 598 | 599 | 600 | # Function to grab the Policy Statement Details 601 | # It will handle multiple statements in the policy 602 | # This function is what generates the actual contents (columns) to 603 | # populate a line of the CSV 604 | def grabPolicyStatementDetailsList( 605 | accountNumber: str, 606 | accountName: str, 607 | region: str, 608 | keyId: str, 609 | keyAlias: str, 610 | policyStatements: json, 611 | Tags: list, 612 | CreationDate: date, 613 | keyLastUsedTime: str, 614 | keyLastUsedAction: str, 615 | keyLastUsedEncryptionContext: str, 616 | keyLastUsedSourceIPAddress: str, 617 | keyLastUsedUsername: str, 618 | ) -> list: 619 | 620 | # Lets create a list of lists! 621 | # This is a collection of entries which will become all of the lines for 622 | # the CSV for the single keyId. 623 | keyListOfLists = [] 624 | 625 | # try: 626 | # Are there some statements in the JSON of the Key Policy? 627 | if policyStatements: 628 | # If there are, loop through them all and create a 'keyList' object which will become 629 | # the line of the CSV 630 | for each in range(len(policyStatements)): 631 | # Add the variables we know exist 632 | keyList = { 633 | "AccountNumber": accountNumber, 634 | "AccountName": accountName, 635 | "Region": region, 636 | "KeyId": keyId, 637 | "Alias": keyAlias, 638 | } 639 | # Add the others that might exist 640 | if "Sid" in policyStatements[each]: 641 | keyList["Sid"] = policyStatements[each]["Sid"] 642 | if "Effect" in policyStatements[each]: 643 | keyList["Effect"] = policyStatements[each]["Effect"] 644 | 645 | # Grab the first Principal 646 | if "Principal" in policyStatements[each]: 647 | keyList["Principal"] = list(policyStatements[each]["Principal"].keys())[ 648 | 0 649 | ] 650 | keyList["Principal Service"] = list( 651 | policyStatements[each]["Principal"].values() 652 | )[0] 653 | 654 | # change to semi-colons to not mangle the CSV 655 | keyList["Principal Service"] = str( 656 | keyList["Principal Service"] 657 | ).replace(",", ";") 658 | 659 | # Check the Actions and change to semi-colons to not mangle the CSV 660 | if "Action" in policyStatements[each]: 661 | keyList["Action"] = str(policyStatements[each]["Action"]).replace( 662 | ",", ";" 663 | ) 664 | if "Resource" in policyStatements[each]: 665 | keyList["Resource"] = policyStatements[each]["Resource"] 666 | if "Condition" in policyStatements[each]: 667 | condition = str(policyStatements[each]["Condition"]) 668 | if isinstance(condition, str): 669 | keyList["Condition"] = condition.replace(",", ";") 670 | keyList["Tags"] = Tags 671 | keyList["Tags"] = str(keyList["Tags"]).replace(",", ";") 672 | keyList["CreationDate"] = CreationDate 673 | keyList["LastUsedTime"] = keyLastUsedTime 674 | keyList["LastUsedAction"] = keyLastUsedAction 675 | keyList["LastUsedEncryptionContext"] = keyLastUsedEncryptionContext 676 | keyList["LastUsedSourceIPAddress"] = keyLastUsedSourceIPAddress 677 | keyList["LastUsedUsername"] = keyLastUsedUsername 678 | 679 | keyList["Date"] = str(datetime.now().strftime("%Y-%m-%d")) 680 | keyList["Concern"] = concernFiller( 681 | principal_service=keyList["Principal Service"], 682 | account_number=accountsDynamoDBAccount, 683 | current_account_number=accountNumber, 684 | action=keyList["Action"], 685 | ) 686 | keyListOfLists.append(keyList) 687 | 688 | return keyListOfLists 689 | 690 | 691 | # Function to push to S3 692 | def pushToS3(filename: str, bucketName: str): 693 | # file to check 694 | file_path = "/tmp/" + filename 695 | 696 | flag = os.path.isfile(file_path) 697 | if flag: 698 | s3 = boto3.resource("s3") 699 | bucket = s3.Bucket(bucketName) 700 | key = filename 701 | try: 702 | bucket.upload_file("/tmp/" + filename, key) 703 | logger.info(f"Successfully uploaded [{filename}] to s3://{bucketName}") 704 | except FileNotFoundError as error: 705 | logger.error(f"{filename} not found!") 706 | except botocore.exceptions.ClientError as error: 707 | if error.response["Error"]["Code"] == "AccessDeniedException": 708 | logger.error( 709 | f"Access Denied PUTting {filename} to s3://{bucket}: {error.response['Error']['Message']}" 710 | ) 711 | elif error.response["Error"]["Code"] == "FileNotFoundError": 712 | logger.error( 713 | f"{filename} not found!: {error.response['Error']['Message']}" 714 | ) 715 | else: 716 | logger.error(f"ERROR RESPONSE: {error.response}") 717 | raise 718 | 719 | else: 720 | logger.info(f"{file_path} not found. Probably because no keys found") 721 | 722 | 723 | # Augment the findings with our own concerns / areas of interest 724 | def concernFiller( 725 | principal_service: str, 726 | account_number: str, 727 | current_account_number: str, 728 | action: str, 729 | ): 730 | # These are the collection of concerns. This is where the 'Concern checks' are 731 | # added to the Key policy statement. 732 | concern_list = [] 733 | concern_list.append(checkManageableThroughIAM(principal_service=principal_service)) 734 | concern_list.append( 735 | checkThirdPartyManaged( 736 | account_number=account_number, current_account_number=current_account_number 737 | ) 738 | ) 739 | concern_list.append(checkKmsPolicy(action=action)) 740 | concern_list.append(checkManageableThroughKMS(principal_service=principal_service)) 741 | return ";".join([x for x in concern_list if x != ""]) 742 | 743 | 744 | # Collection of 'Concern checks'. Used by concernFiller() 745 | def checkManageableThroughIAM(principal_service: str) -> str: 746 | if principal_service[-5:] == ":root": 747 | return "Principal is account" 748 | return "" 749 | 750 | 751 | def checkThirdPartyManaged(account_number: str, current_account_number: str) -> str: 752 | if account_number != current_account_number: 753 | return "External account" 754 | return "" 755 | 756 | 757 | def checkKmsPolicy(action: str) -> str: 758 | if "kms:*" in action: 759 | return "Key policy overly permissive" 760 | return "" 761 | 762 | 763 | def checkManageableThroughKMS(principal_service: str) -> str: 764 | if ":user" in principal_service: 765 | return "Access provided to IAM user" 766 | return "" 767 | 768 | 769 | # Key has permissions but cannot be read (locked down key policy) 770 | def unreadableKey(principal_service: str) -> str: 771 | if principal_service == "": 772 | return "Unreadable key. Key permissions don't allow lambda to read details" 773 | return "" 774 | 775 | 776 | def main(): 777 | # Use global variables to avoid passing them around 778 | # regions has been read from environment variables at the beginning 779 | regions = [x.strip() for x in regions_string.strip("[]").split(",")] 780 | 781 | # dynamo_ variables have also been read from environment variables at the beginning 782 | account_ids = get_accounts( 783 | accountsDynamoDBAccount, 784 | accountsDynamoDBRole, 785 | accountsDynamoDBTable, 786 | accountsDynamoDBRegion, 787 | ) 788 | logger.info(f"Accounts: {account_ids}") 789 | 790 | # If we found some accounts in the 'accounts' DynamoDB table... 791 | if account_ids: 792 | # then loop through each account... 793 | for account in account_ids: 794 | account_number = account["accountId"] 795 | account_name = account["accountName"] 796 | logger.info(f"Processing Account Number: {account_number}") 797 | # and assumerole into that account using the cross-account KMS Read Role 798 | session = getAssumedRoleSession(account_number, xaKMSReadRole) 799 | 800 | if session: 801 | # If we successfully AssumeRole into the target account... 802 | # then loop through all the regions defined in the 'regions' env variable 803 | for region in regions: 804 | # Create a KMS session... 805 | kms = session.client("kms", region_name=region) 806 | # Grab all the Keys and store them in a JSON object 807 | keyMap = getEverythingJson(kms) 808 | filename = account_number + "-" + region + "-kms-details.csv" 809 | getEverythingToCSV( 810 | account_number, account_name, filename, keyMap, region 811 | ) 812 | pushToS3(filename, destination_s3_bucket) 813 | 814 | else: 815 | logger.error(f"Unable to create session for {account_number}") 816 | else: 817 | logger.error(f"No Account IDs found: account_ids={account_ids}") 818 | 819 | 820 | # Run the Lambda 821 | def lambda_handler(event, context): 822 | logging.info("<<<<<<<<<< KMSReadLambda >>>>>>>>>>") 823 | logging.info(f"OS Env Variables: {os.environ}") 824 | logging.info(f"Received Event: {event}") 825 | 826 | main() 827 | 828 | 829 | # Run from CLI 830 | if __name__ == "__main__": 831 | logging.info("<<<<<<<<<< KMSReadLambda >>>>>>>>>>") 832 | main() 833 | 834 | KMSReadPolicyLambdaLogGroup: 835 | Type: AWS::Logs::LogGroup 836 | Metadata: 837 | checkov: 838 | skip: 839 | - id: "CKV_AWS_158" 840 | reason: "No sensitve data in CloudWatch logs" 841 | Properties: 842 | LogGroupName: !Sub "/aws/lambda/${KMSReadPolicyLambda}" 843 | RetentionInDays: 90 844 | 845 | 846 | InvokeLambda: 847 | Type: "AWS::Lambda::Permission" 848 | Properties: 849 | Action: lambda:InvokeFunction 850 | FunctionName: !GetAtt KMSReadPolicyLambda.Arn 851 | Principal: events.amazonaws.com 852 | SourceArn: !GetAtt LambdaCron.Arn 853 | --------------------------------------------------------------------------------