├── .buildspec.yml ├── .flake8 ├── .github └── workflows │ └── ci.yml ├── .gitignore ├── .pre-commit-config.yaml ├── .pylintrc ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── central-aggregator-cf-template.yaml ├── docs ├── development.md ├── hub-spoke-topology.md ├── images │ ├── hub-spoke-separate-master-aggregator.png │ ├── hub-spoke-single-master-aggregator.png │ └── support-cases-aggregator-pipeline.png ├── member_acc_role_setup.md └── org_master_role_setup.md ├── member-acc-cf-template.yaml ├── org-master-cf-template.yaml ├── requirements.txt ├── run_cloudformation.sh ├── src ├── aws_common_utils_layer.py ├── cloudtrail_process.py └── support_cases_aggregator.py └── tests ├── __init__.py ├── test_aws_common_utils_layer.py └── test_support_case_aggregator.py /.buildspec.yml: -------------------------------------------------------------------------------- 1 | version: 0.2 2 | phases: 3 | build: 4 | commands: 5 | - pre-commit run --all-files 6 | -------------------------------------------------------------------------------- /.flake8: -------------------------------------------------------------------------------- 1 | [flake8] 2 | max-line-length = 80 3 | ignore = E501, C812, W503 4 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: ci 2 | 3 | on: 4 | push: 5 | branches: 6 | - master 7 | pull_request: 8 | 9 | jobs: 10 | build: 11 | runs-on: ubuntu-latest 12 | steps: 13 | - name: Checkout 14 | uses: actions/checkout@v2 15 | - name: Set up Python 3.8 16 | uses: actions/setup-python@v1 17 | with: 18 | python-version: 3.8 19 | - name: Lint with flake8 20 | run: | 21 | pip install flake8 22 | flake8 . 23 | - name: Test with pytest 24 | run: | 25 | pip install -r requirements.txt 26 | pip install pytest 27 | pytest -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Pycharm 10 | .idea/ 11 | 12 | # Distribution / packaging 13 | .Python 14 | build/ 15 | develop-eggs/ 16 | dist/ 17 | downloads/ 18 | eggs/ 19 | .eggs/ 20 | lib/ 21 | lib64/ 22 | parts/ 23 | sdist/ 24 | var/ 25 | wheels/ 26 | *.egg-info/ 27 | .installed.cfg 28 | *.egg 29 | MANIFEST 30 | 31 | # PyInstaller 32 | # Usually these files are written by a python script from a template 33 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 34 | *.manifest 35 | *.spec 36 | 37 | # Installer logs 38 | pip-log.txt 39 | pip-delete-this-directory.txt 40 | 41 | # Unit test / coverage reports 42 | htmlcov/ 43 | .tox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | 53 | # Translations 54 | *.mo 55 | *.pot 56 | 57 | # Django stuff: 58 | *.log 59 | .static_storage/ 60 | .media/ 61 | local_settings.py 62 | 63 | # Flask stuff: 64 | instance/ 65 | .webassets-cache 66 | 67 | # Scrapy stuff: 68 | .scrapy 69 | 70 | # Sphinx documentation 71 | docs/_build/ 72 | 73 | # PyBuilder 74 | target/ 75 | 76 | # Jupyter Notebook 77 | .ipynb_checkpoints 78 | 79 | # pyenv 80 | .python-version 81 | 82 | # celery beat schedule file 83 | celerybeat-schedule 84 | 85 | # SageMath parsed files 86 | *.sage.py 87 | 88 | # Environments 89 | .env 90 | .venv 91 | env/ 92 | venv/ 93 | ENV/ 94 | env.bak/ 95 | venv.bak/ 96 | 97 | # Spyder project settings 98 | .spyderproject 99 | .spyproject 100 | 101 | # Rope project settings 102 | .ropeproject 103 | 104 | # mkdocs documentation 105 | /site 106 | 107 | # mypy 108 | .mypy_cache/ 109 | 110 | # DS_Store on mac 111 | *.DS_Store 112 | 113 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | exclude: ^(buildspec.yml|.pre-commit-config.yaml)$ 2 | fail_fast: true 3 | repos: 4 | - repo: https://github.com/ambv/black 5 | rev: stable 6 | hooks: 7 | - id: black 8 | - repo: https://github.com/pre-commit/pre-commit-hooks 9 | rev: v2.0.0 10 | hooks: 11 | - id: check-case-conflict 12 | - id: end-of-file-fixer 13 | - id: mixed-line-ending 14 | args: 15 | - --fix=lf 16 | - id: trailing-whitespace 17 | - id: flake8 18 | additional_dependencies: 19 | - flake8-bugbear>=19.3.0 20 | - flake8-builtins>=1.4.1 21 | - flake8-commas>=2.0.0 22 | - flake8-comprehensions>=2.1.0 23 | - flake8-debugger>=3.1.0 24 | - flake8-pep3101>=1.2.1 25 | # language_version: python3.6 26 | - id: pretty-format-json 27 | args: 28 | - --autofix 29 | - --indent=4 30 | - --no-sort-keys 31 | - id: check-merge-conflict 32 | # - id: check-yaml # doesn't work with CloudFormation templates/intrinsics, should use cfn-lint instead 33 | # language_version: python3.6 34 | - repo: https://github.com/pre-commit/pygrep-hooks 35 | rev: v1.3.0 36 | hooks: 37 | - id: python-check-blanket-noqa 38 | - id: python-check-mock-methods 39 | - id: python-no-log-warn 40 | - repo: https://github.com/PyCQA/bandit 41 | rev: f5a6f0ca62 # TODO: update once a release > 1.5.1 hits with this change in 42 | hooks: 43 | - id: bandit 44 | files: "^src/" 45 | # have to skip B101, contract tests use it and there's no way to skip for specific files 46 | # have to skip B322, as there is no way to indicate the codebase is Python 3 only (input only vulnerable in Py2) 47 | args: ["--skip", "B101,B322"] 48 | - repo: local 49 | hooks: 50 | - id: pylint-local 51 | name: pylint-local 52 | description: Run pylint in the local virtualenv 53 | entry: pylint "src/" "tests/" 54 | language: system 55 | # ignore all files, run on hard-coded modules instead 56 | pass_filenames: false 57 | always_run: true 58 | -------------------------------------------------------------------------------- /.pylintrc: -------------------------------------------------------------------------------- 1 | [BASIC] 2 | good-names= 3 | i, 4 | e, 5 | logger, 6 | k, 7 | v, 8 | s3, 9 | 10 | disable=C0330 11 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to the Support Cases Aggregator 2 | 3 | Thanks for taking the time to contribute! 4 | 5 | We love it when users contribute their innovations back to the project, or when they discover bugged functionality and want others to know about it. 6 | 7 | This repository is optimized for forking, so please ensure code submitted remains generic (e.g. don't specify a bucket name). 8 | 9 | Small pull requests will be reviewed within 7 days. Large pull requests will be reviewed when possible. All pull requests are greatly appreciated. 10 | 11 | ### Pull Request Template 12 | 13 | We ask you refer to the `Atom` project [PULL_REQUEST_TEMPLATE](https://github.com/atom/atom/blob/master/PULL_REQUEST_TEMPLATE.md) for formatting your pull request. Their formats are generic and broadly applicable to PR's we expect to be opened against this project. 14 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Snap Inc. 4 | 5 | All rights reserved. 6 | 7 | Permission is hereby granted, free of charge, to any person obtaining a copy 8 | of this software and associated documentation files (the "Software"), to deal 9 | in the Software without restriction, including without limitation the rights 10 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | copies of the Software, and to permit persons to whom the Software is 12 | furnished to do so, subject to the following conditions: 13 | 14 | The above copyright notice and this permission notice shall be included in all 15 | copies or substantial portions of the Software. 16 | 17 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AWS Support Cases Aggregator 2 | _AWS Support Cases aggregation for a multi-account organization_ 3 | 4 | [![ci](https://github.com/Snapchat/aws-support-tickets-aggregator/workflows/ci/badge.svg?branch=master)](https://github.com/Snapchat/aws-support-tickets-aggregator/actions?query=workflow%3Aci+branch%3Amaster) 5 | 6 | This is a simple CloudFormation-based serverless pipeline for collecting support case information from all users across an AWS Organization into a single database. 7 | 8 | This allows users to easily discover and monitor all Support Cases within an organization. See our [AWS Blog Post](https://aws.amazon.com/blogs/mt/) for details. 9 | 10 | ### AWS Support Cases Aggregator Architecture 11 | ![aws-support-cases-aggregator-pipeline-diagram](docs/images/support-cases-aggregator-pipeline.png) 12 | 13 | 14 | ### Topology - Hub and Spoke 15 | This service uses a Hub (central aggregator account) and Spoke (member accounts) model. 16 | The aggregator can be configured in two ways: 17 | 18 | 1. Aggregator account is [AWS Organizations master account](docs/hub-spoke-topology.md#using-org-master-as-central-aggregator-account) 19 | 20 | 2. Aggregator account is [NOT AWS Organizations master account](docs/hub-spoke-topology.md#using-separate-org-master-and-central-aggregator-accounts) 21 | 22 | 23 | ### Setup 24 | #### Central Aggregator Account (Hub) 25 | Ensure you have permissions for creating requisite AWS resources including IAM roles and policies. 26 | 27 | ##### Prerequisite Steps 28 | 1. Create a S3 bucket in your central aggregator account to hold the CloudFormation stack template. Pass the bucket name to the `--cf_s3_bucket` parameter of the `run_cloudformation.sh` script. 29 | 30 | > The CloudFormation stack must be created in the same region as the aforementioned S3 bucket. 31 | 32 | 2. Choose a free name for your CloudTrail S3 bucket. Pass it to the `--ct_s3_bucket` parameter of the `run_cloudformation.sh` script. 33 | 34 | > Ensure a bucket with the same name does not already exist, or the CloudFormation stack will fail to create. 35 | 36 | > The CloudFormation stack will handle bucket creation. 37 | 38 | 3. (Optional) If your aggregator account will not be your AWS Organizations master account: 39 | * Pass the role arn `arn:aws:iam:::role/OrgListAccountsViewer` to the `--org_role` parameter of the `run_cloudformation.sh` script. 40 | > Note this role can be created in section [AWS Organizations Master Account #2](#aws-organizations-master-account) 41 | 42 | ##### Creating the CloudFormation Stack 43 | Run the shell script: 44 | 45 | ```bash 46 | ./run_cloudformation.sh --profile= --stack_name= --cf_region= --cf_s3_bucket= --ct_s3_bucket= [--template_file=