├── .github ├── PULL_REQUEST_TEMPLATE.md └── workflows │ └── unit-tests.yaml ├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── Pipfile ├── Pipfile.lock ├── README.md ├── build_deploy.sh ├── drainer ├── __init__.py ├── handler.py ├── k8s_utils.py └── requirements.txt ├── k8s_rbac ├── cluster_role.yaml └── cluster_rolebinding.yaml ├── template.yaml └── tests ├── __init__.py ├── drainer ├── __init__.py ├── fixtures │ ├── event.json │ └── kube_config.yaml ├── test_handler.py └── test_k8s_utils.py └── utils.py /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | *Issue #, if available:* 2 | 3 | *Description of changes:* 4 | 5 | 6 | By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. 7 | -------------------------------------------------------------------------------- /.github/workflows/unit-tests.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | name: Unit Tests 4 | on: 5 | push: 6 | branches: 7 | - master 8 | pull_request: 9 | types: 10 | - opened 11 | - edited 12 | - synchronize 13 | jobs: 14 | unit_tests: 15 | name: Unit tests 16 | runs-on: ubuntu-latest 17 | steps: 18 | - uses: actions/checkout@v2 19 | # Setup 20 | - name: Set up Python 3.7 21 | uses: actions/setup-python@v1 22 | with: 23 | python-version: 3.7 24 | - name: Install python dependencies 25 | run: | 26 | pip install pipenv 27 | pipenv install --dev --ignore-pipfile 28 | # Run Tests 29 | - name: Unit tests 30 | run: pipenv run py.test --cov=drainer 31 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .aws-sam 2 | .coverage 3 | .idea 4 | .pytest_cache 5 | htmlcov 6 | packaged.yaml 7 | __pycache__ 8 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check [existing open](https://github.com/aws-samples/amazon-k8s-node-drainer/issues), or [recently closed](https://github.com/aws-samples/amazon-k8s-node-drainer/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/amazon-k8s-node-drainer/labels/help%20wanted) issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](https://github.com/aws-samples/amazon-k8s-node-drainer/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /Pipfile: -------------------------------------------------------------------------------- 1 | [[source]] 2 | name = "pypi" 3 | url = "https://pypi.org/simple" 4 | verify_ssl = true 5 | 6 | [dev-packages] 7 | boto3 = "==1.9.108" 8 | pytest-mock = "==1.10.1" 9 | pyfakefs = "==3.5.8" 10 | moto = "==1.3.7" 11 | freezegun = "==0.3.11" 12 | pytest-cov = "==2.6.1" 13 | 14 | [packages] 15 | kubernetes = "==9.0.0" 16 | pyyaml = "==5.4" 17 | urllib3 = "==1.26.5" 18 | 19 | [requires] 20 | python_version = "3.7" 21 | -------------------------------------------------------------------------------- /Pipfile.lock: -------------------------------------------------------------------------------- 1 | { 2 | "_meta": { 3 | "hash": { 4 | "sha256": "eaa476cf64593a714c88267b4b1102a85236c2853e84790b306e45b02c76adef" 5 | }, 6 | "pipfile-spec": 6, 7 | "requires": { 8 | "python_version": "3.7" 9 | }, 10 | "sources": [ 11 | { 12 | "name": "pypi", 13 | "url": "https://pypi.org/simple", 14 | "verify_ssl": true 15 | } 16 | ] 17 | }, 18 | "default": { 19 | "cachetools": { 20 | "hashes": [ 21 | "sha256:95ef631eeaea14ba2e36f06437f36463aac3a096799e876ee55e5cdccb102590", 22 | "sha256:dce83f2d9b4e1f732a8cd44af8e8fab2dbe46201467fc98b3ef8f269092bf62b" 23 | ], 24 | "markers": "python_version >= '3.7'", 25 | "version": "==5.3.1" 26 | }, 27 | "certifi": { 28 | "hashes": [ 29 | "sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082", 30 | "sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9" 31 | ], 32 | "markers": "python_version >= '3.6'", 33 | "version": "==2023.7.22" 34 | }, 35 | "charset-normalizer": { 36 | "hashes": [ 37 | "sha256:04e57ab9fbf9607b77f7d057974694b4f6b142da9ed4a199859d9d4d5c63fe96", 38 | "sha256:09393e1b2a9461950b1c9a45d5fd251dc7c6f228acab64da1c9c0165d9c7765c", 39 | "sha256:0b87549028f680ca955556e3bd57013ab47474c3124dc069faa0b6545b6c9710", 40 | "sha256:1000fba1057b92a65daec275aec30586c3de2401ccdcd41f8a5c1e2c87078706", 41 | "sha256:1249cbbf3d3b04902ff081ffbb33ce3377fa6e4c7356f759f3cd076cc138d020", 42 | "sha256:1920d4ff15ce893210c1f0c0e9d19bfbecb7983c76b33f046c13a8ffbd570252", 43 | "sha256:193cbc708ea3aca45e7221ae58f0fd63f933753a9bfb498a3b474878f12caaad", 44 | "sha256:1a100c6d595a7f316f1b6f01d20815d916e75ff98c27a01ae817439ea7726329", 45 | "sha256:1f30b48dd7fa1474554b0b0f3fdfdd4c13b5c737a3c6284d3cdc424ec0ffff3a", 46 | "sha256:203f0c8871d5a7987be20c72442488a0b8cfd0f43b7973771640fc593f56321f", 47 | "sha256:246de67b99b6851627d945db38147d1b209a899311b1305dd84916f2b88526c6", 48 | "sha256:2dee8e57f052ef5353cf608e0b4c871aee320dd1b87d351c28764fc0ca55f9f4", 49 | "sha256:2efb1bd13885392adfda4614c33d3b68dee4921fd0ac1d3988f8cbb7d589e72a", 50 | "sha256:2f4ac36d8e2b4cc1aa71df3dd84ff8efbe3bfb97ac41242fbcfc053c67434f46", 51 | "sha256:3170c9399da12c9dc66366e9d14da8bf7147e1e9d9ea566067bbce7bb74bd9c2", 52 | "sha256:3b1613dd5aee995ec6d4c69f00378bbd07614702a315a2cf6c1d21461fe17c23", 53 | "sha256:3bb3d25a8e6c0aedd251753a79ae98a093c7e7b471faa3aa9a93a81431987ace", 54 | "sha256:3bb7fda7260735efe66d5107fb7e6af6a7c04c7fce9b2514e04b7a74b06bf5dd", 55 | "sha256:41b25eaa7d15909cf3ac4c96088c1f266a9a93ec44f87f1d13d4a0e86c81b982", 56 | "sha256:45de3f87179c1823e6d9e32156fb14c1927fcc9aba21433f088fdfb555b77c10", 57 | "sha256:46fb8c61d794b78ec7134a715a3e564aafc8f6b5e338417cb19fe9f57a5a9bf2", 58 | "sha256:48021783bdf96e3d6de03a6e39a1171ed5bd7e8bb93fc84cc649d11490f87cea", 59 | "sha256:4957669ef390f0e6719db3613ab3a7631e68424604a7b448f079bee145da6e09", 60 | "sha256:5e86d77b090dbddbe78867a0275cb4df08ea195e660f1f7f13435a4649e954e5", 61 | "sha256:6339d047dab2780cc6220f46306628e04d9750f02f983ddb37439ca47ced7149", 62 | "sha256:681eb3d7e02e3c3655d1b16059fbfb605ac464c834a0c629048a30fad2b27489", 63 | "sha256:6c409c0deba34f147f77efaa67b8e4bb83d2f11c8806405f76397ae5b8c0d1c9", 64 | "sha256:7095f6fbfaa55defb6b733cfeb14efaae7a29f0b59d8cf213be4e7ca0b857b80", 65 | "sha256:70c610f6cbe4b9fce272c407dd9d07e33e6bf7b4aa1b7ffb6f6ded8e634e3592", 66 | "sha256:72814c01533f51d68702802d74f77ea026b5ec52793c791e2da806a3844a46c3", 67 | "sha256:7a4826ad2bd6b07ca615c74ab91f32f6c96d08f6fcc3902ceeedaec8cdc3bcd6", 68 | "sha256:7c70087bfee18a42b4040bb9ec1ca15a08242cf5867c58726530bdf3945672ed", 69 | "sha256:855eafa5d5a2034b4621c74925d89c5efef61418570e5ef9b37717d9c796419c", 70 | "sha256:8700f06d0ce6f128de3ccdbc1acaea1ee264d2caa9ca05daaf492fde7c2a7200", 71 | "sha256:89f1b185a01fe560bc8ae5f619e924407efca2191b56ce749ec84982fc59a32a", 72 | "sha256:8b2c760cfc7042b27ebdb4a43a4453bd829a5742503599144d54a032c5dc7e9e", 73 | "sha256:8c2f5e83493748286002f9369f3e6607c565a6a90425a3a1fef5ae32a36d749d", 74 | "sha256:8e098148dd37b4ce3baca71fb394c81dc5d9c7728c95df695d2dca218edf40e6", 75 | "sha256:94aea8eff76ee6d1cdacb07dd2123a68283cb5569e0250feab1240058f53b623", 76 | "sha256:95eb302ff792e12aba9a8b8f8474ab229a83c103d74a750ec0bd1c1eea32e669", 77 | "sha256:9bd9b3b31adcb054116447ea22caa61a285d92e94d710aa5ec97992ff5eb7cf3", 78 | "sha256:9e608aafdb55eb9f255034709e20d5a83b6d60c054df0802fa9c9883d0a937aa", 79 | "sha256:a103b3a7069b62f5d4890ae1b8f0597618f628b286b03d4bc9195230b154bfa9", 80 | "sha256:a386ebe437176aab38c041de1260cd3ea459c6ce5263594399880bbc398225b2", 81 | "sha256:a38856a971c602f98472050165cea2cdc97709240373041b69030be15047691f", 82 | "sha256:a401b4598e5d3f4a9a811f3daf42ee2291790c7f9d74b18d75d6e21dda98a1a1", 83 | "sha256:a7647ebdfb9682b7bb97e2a5e7cb6ae735b1c25008a70b906aecca294ee96cf4", 84 | "sha256:aaf63899c94de41fe3cf934601b0f7ccb6b428c6e4eeb80da72c58eab077b19a", 85 | "sha256:b0dac0ff919ba34d4df1b6131f59ce95b08b9065233446be7e459f95554c0dc8", 86 | "sha256:baacc6aee0b2ef6f3d308e197b5d7a81c0e70b06beae1f1fcacffdbd124fe0e3", 87 | "sha256:bf420121d4c8dce6b889f0e8e4ec0ca34b7f40186203f06a946fa0276ba54029", 88 | "sha256:c04a46716adde8d927adb9457bbe39cf473e1e2c2f5d0a16ceb837e5d841ad4f", 89 | "sha256:c0b21078a4b56965e2b12f247467b234734491897e99c1d51cee628da9786959", 90 | "sha256:c1c76a1743432b4b60ab3358c937a3fe1341c828ae6194108a94c69028247f22", 91 | "sha256:c4983bf937209c57240cff65906b18bb35e64ae872da6a0db937d7b4af845dd7", 92 | "sha256:c4fb39a81950ec280984b3a44f5bd12819953dc5fa3a7e6fa7a80db5ee853952", 93 | "sha256:c57921cda3a80d0f2b8aec7e25c8aa14479ea92b5b51b6876d975d925a2ea346", 94 | "sha256:c8063cf17b19661471ecbdb3df1c84f24ad2e389e326ccaf89e3fb2484d8dd7e", 95 | "sha256:ccd16eb18a849fd8dcb23e23380e2f0a354e8daa0c984b8a732d9cfaba3a776d", 96 | "sha256:cd6dbe0238f7743d0efe563ab46294f54f9bc8f4b9bcf57c3c666cc5bc9d1299", 97 | "sha256:d62e51710986674142526ab9f78663ca2b0726066ae26b78b22e0f5e571238dd", 98 | "sha256:db901e2ac34c931d73054d9797383d0f8009991e723dab15109740a63e7f902a", 99 | "sha256:e03b8895a6990c9ab2cdcd0f2fe44088ca1c65ae592b8f795c3294af00a461c3", 100 | "sha256:e1c8a2f4c69e08e89632defbfabec2feb8a8d99edc9f89ce33c4b9e36ab63037", 101 | "sha256:e4b749b9cc6ee664a3300bb3a273c1ca8068c46be705b6c31cf5d276f8628a94", 102 | "sha256:e6a5bf2cba5ae1bb80b154ed68a3cfa2fa00fde979a7f50d6598d3e17d9ac20c", 103 | "sha256:e857a2232ba53ae940d3456f7533ce6ca98b81917d47adc3c7fd55dad8fab858", 104 | "sha256:ee4006268ed33370957f55bf2e6f4d263eaf4dc3cfc473d1d90baff6ed36ce4a", 105 | "sha256:eef9df1eefada2c09a5e7a40991b9fc6ac6ef20b1372abd48d2794a316dc0449", 106 | "sha256:f058f6963fd82eb143c692cecdc89e075fa0828db2e5b291070485390b2f1c9c", 107 | "sha256:f25c229a6ba38a35ae6e25ca1264621cc25d4d38dca2942a7fce0b67a4efe918", 108 | "sha256:f2a1d0fd4242bd8643ce6f98927cf9c04540af6efa92323e9d3124f57727bfc1", 109 | "sha256:f7560358a6811e52e9c4d142d497f1a6e10103d3a6881f18d04dbce3729c0e2c", 110 | "sha256:f779d3ad205f108d14e99bb3859aa7dd8e9c68874617c72354d7ecaec2a054ac", 111 | "sha256:f87f746ee241d30d6ed93969de31e5ffd09a2961a051e60ae6bddde9ec3583aa" 112 | ], 113 | "markers": "python_version >= '3.7'", 114 | "version": "==3.2.0" 115 | }, 116 | "google-auth": { 117 | "hashes": [ 118 | "sha256:164cba9af4e6e4e40c3a4f90a1a6c12ee56f14c0b4868d1ca91b32826ab334ce", 119 | "sha256:d61d1b40897407b574da67da1a833bdc10d5a11642566e506565d1b1a46ba873" 120 | ], 121 | "markers": "python_version >= '3.6'", 122 | "version": "==2.22.0" 123 | }, 124 | "idna": { 125 | "hashes": [ 126 | "sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4", 127 | "sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2" 128 | ], 129 | "markers": "python_version >= '3.5'", 130 | "version": "==3.4" 131 | }, 132 | "kubernetes": { 133 | "hashes": [ 134 | "sha256:a8b0aed55ba946faea660712595a52ae53a8854df773d96f47a63fa0c9d4e3bf", 135 | "sha256:f56137a298cb1453dd908b49dd4169347287c971e8cabd11b32f27570fec314c" 136 | ], 137 | "index": "pypi", 138 | "version": "==9.0.0" 139 | }, 140 | "oauthlib": { 141 | "hashes": [ 142 | "sha256:8139f29aac13e25d502680e9e19963e83f16838d48a0d71c287fe40e7067fbca", 143 | "sha256:9859c40929662bec5d64f34d01c99e093149682a3f38915dc0655d5a633dd918" 144 | ], 145 | "markers": "python_version >= '3.6'", 146 | "version": "==3.2.2" 147 | }, 148 | "pyasn1": { 149 | "hashes": [ 150 | "sha256:87a2121042a1ac9358cabcaf1d07680ff97ee6404333bacca15f76aa8ad01a57", 151 | "sha256:97b7290ca68e62a832558ec3976f15cbf911bf5d7c7039d8b861c2a0ece69fde" 152 | ], 153 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'", 154 | "version": "==0.5.0" 155 | }, 156 | "pyasn1-modules": { 157 | "hashes": [ 158 | "sha256:5bd01446b736eb9d31512a30d46c1ac3395d676c6f3cafa4c03eb54b9925631c", 159 | "sha256:d3ccd6ed470d9ffbc716be08bd90efbd44d0734bc9303818f7336070984a162d" 160 | ], 161 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'", 162 | "version": "==0.3.0" 163 | }, 164 | "python-dateutil": { 165 | "hashes": [ 166 | "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86", 167 | "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9" 168 | ], 169 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 170 | "version": "==2.8.2" 171 | }, 172 | "pyyaml": { 173 | "hashes": [ 174 | "sha256:02c78d77281d8f8d07a255e57abdbf43b02257f59f50cc6b636937d68efa5dd0", 175 | "sha256:0dc9f2eb2e3c97640928dec63fd8dc1dd91e6b6ed236bd5ac00332b99b5c2ff9", 176 | "sha256:124fd7c7bc1e95b1eafc60825f2daf67c73ce7b33f1194731240d24b0d1bf628", 177 | "sha256:26fcb33776857f4072601502d93e1a619f166c9c00befb52826e7b774efaa9db", 178 | "sha256:31ba07c54ef4a897758563e3a0fcc60077698df10180abe4b8165d9895c00ebf", 179 | "sha256:3c49e39ac034fd64fd576d63bb4db53cda89b362768a67f07749d55f128ac18a", 180 | "sha256:52bf0930903818e600ae6c2901f748bc4869c0c406056f679ab9614e5d21a166", 181 | "sha256:5a3f345acff76cad4aa9cb171ee76c590f37394186325d53d1aa25318b0d4a09", 182 | "sha256:5e7ac4e0e79a53451dc2814f6876c2fa6f71452de1498bbe29c0b54b69a986f4", 183 | "sha256:7242790ab6c20316b8e7bb545be48d7ed36e26bbe279fd56f2c4a12510e60b4b", 184 | "sha256:737bd70e454a284d456aa1fa71a0b429dd527bcbf52c5c33f7c8eee81ac16b89", 185 | "sha256:8635d53223b1f561b081ff4adecb828fd484b8efffe542edcfdff471997f7c39", 186 | "sha256:8b818b6c5a920cbe4203b5a6b14256f0e5244338244560da89b7b0f1313ea4b6", 187 | "sha256:8bf38641b4713d77da19e91f8b5296b832e4db87338d6aeffe422d42f1ca896d", 188 | "sha256:a36a48a51e5471513a5aea920cdad84cbd56d70a5057cca3499a637496ea379c", 189 | "sha256:b2243dd033fd02c01212ad5c601dafb44fbb293065f430b0d3dbf03f3254d615", 190 | "sha256:cc547d3ead3754712223abb7b403f0a184e4c3eae18c9bb7fd15adef1597cc4b", 191 | "sha256:cc552b6434b90d9dbed6a4f13339625dc466fd82597119897e9489c953acbc22", 192 | "sha256:f3790156c606299ff499ec44db422f66f05a7363b39eb9d5b064f17bd7d7c47b", 193 | "sha256:f7a21e3d99aa3095ef0553e7ceba36fb693998fbb1226f1392ce33681047465f", 194 | "sha256:fdc6b2cb4b19e431994f25a9160695cc59a4e861710cc6fc97161c5e845fc579" 195 | ], 196 | "index": "pypi", 197 | "version": "==5.4" 198 | }, 199 | "requests": { 200 | "hashes": [ 201 | "sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f", 202 | "sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1" 203 | ], 204 | "markers": "python_version >= '3.7'", 205 | "version": "==2.31.0" 206 | }, 207 | "requests-oauthlib": { 208 | "hashes": [ 209 | "sha256:2577c501a2fb8d05a304c09d090d6e47c306fef15809d102b327cf8364bddab5", 210 | "sha256:75beac4a47881eeb94d5ea5d6ad31ef88856affe2332b9aafb52c6452ccf0d7a" 211 | ], 212 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 213 | "version": "==1.3.1" 214 | }, 215 | "rsa": { 216 | "hashes": [ 217 | "sha256:90260d9058e514786967344d0ef75fa8727eed8a7d2e43ce9f4bcf1b536174f7", 218 | "sha256:e38464a49c6c85d7f1351b0126661487a7e0a14a50f1675ec50eb34d4f20ef21" 219 | ], 220 | "markers": "python_version >= '3.6' and python_version < '4'", 221 | "version": "==4.9" 222 | }, 223 | "setuptools": { 224 | "hashes": [ 225 | "sha256:11e52c67415a381d10d6b462ced9cfb97066179f0e871399e006c4ab101fc85f", 226 | "sha256:baf1fdb41c6da4cd2eae722e135500da913332ab3f2f5c7d33af9b492acb5235" 227 | ], 228 | "markers": "python_version >= '3.7'", 229 | "version": "==68.0.0" 230 | }, 231 | "six": { 232 | "hashes": [ 233 | "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926", 234 | "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254" 235 | ], 236 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 237 | "version": "==1.16.0" 238 | }, 239 | "urllib3": { 240 | "hashes": [ 241 | "sha256:753a0374df26658f99d826cfe40394a686d05985786d946fbe4165b5148f5a7c", 242 | "sha256:a7acd0977125325f516bda9735fa7142b909a8d01e8b2e4c8108d0984e6e0098" 243 | ], 244 | "index": "pypi", 245 | "version": "==1.26.5" 246 | }, 247 | "websocket-client": { 248 | "hashes": [ 249 | "sha256:c951af98631d24f8df89ab1019fc365f2227c0892f12fd150e935607c79dd0dd", 250 | "sha256:f1f9f2ad5291f0225a49efad77abf9e700b6fef553900623060dad6e26503b9d" 251 | ], 252 | "markers": "python_version >= '3.7'", 253 | "version": "==1.6.1" 254 | } 255 | }, 256 | "develop": { 257 | "aws-xray-sdk": { 258 | "hashes": [ 259 | "sha256:72791618feb22eaff2e628462b0d58f398ce8c1bacfa989b7679817ab1fad60c", 260 | "sha256:9e7ba8dd08fd2939376c21423376206bff01d0deaea7d7721c6b35921fed1943" 261 | ], 262 | "version": "==0.95" 263 | }, 264 | "boto": { 265 | "hashes": [ 266 | "sha256:147758d41ae7240dc989f0039f27da8ca0d53734be0eb869ef16e3adcfa462e8", 267 | "sha256:ea0d3b40a2d852767be77ca343b58a9e3a4b00d9db440efb8da74b4e58025e5a" 268 | ], 269 | "version": "==2.49.0" 270 | }, 271 | "boto3": { 272 | "hashes": [ 273 | "sha256:16e093bf505ccf004ea1cab34188af8df1df02c738a6d2f46bc42e7cbda667f8", 274 | "sha256:6f8bf13e39f52a13a1af6eb067723d6cf28b6c09d5b72953d729bff8a88fa0b9" 275 | ], 276 | "index": "pypi", 277 | "version": "==1.9.108" 278 | }, 279 | "botocore": { 280 | "hashes": [ 281 | "sha256:3baf129118575602ada9926f5166d82d02273c250d0feb313fc270944b27c48b", 282 | "sha256:dc080aed4f9b220a9e916ca29ca97a9d37e8e1d296fe89cbaeef929bf0c8066b" 283 | ], 284 | "version": "==1.12.253" 285 | }, 286 | "certifi": { 287 | "hashes": [ 288 | "sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082", 289 | "sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9" 290 | ], 291 | "markers": "python_version >= '3.6'", 292 | "version": "==2023.7.22" 293 | }, 294 | "cffi": { 295 | "hashes": [ 296 | "sha256:00a9ed42e88df81ffae7a8ab6d9356b371399b91dbdf0c3cb1e84c03a13aceb5", 297 | "sha256:03425bdae262c76aad70202debd780501fabeaca237cdfddc008987c0e0f59ef", 298 | "sha256:04ed324bda3cda42b9b695d51bb7d54b680b9719cfab04227cdd1e04e5de3104", 299 | "sha256:0e2642fe3142e4cc4af0799748233ad6da94c62a8bec3a6648bf8ee68b1c7426", 300 | "sha256:173379135477dc8cac4bc58f45db08ab45d228b3363adb7af79436135d028405", 301 | "sha256:198caafb44239b60e252492445da556afafc7d1e3ab7a1fb3f0584ef6d742375", 302 | "sha256:1e74c6b51a9ed6589199c787bf5f9875612ca4a8a0785fb2d4a84429badaf22a", 303 | "sha256:2012c72d854c2d03e45d06ae57f40d78e5770d252f195b93f581acf3ba44496e", 304 | "sha256:21157295583fe8943475029ed5abdcf71eb3911894724e360acff1d61c1d54bc", 305 | "sha256:2470043b93ff09bf8fb1d46d1cb756ce6132c54826661a32d4e4d132e1977adf", 306 | "sha256:285d29981935eb726a4399badae8f0ffdff4f5050eaa6d0cfc3f64b857b77185", 307 | "sha256:30d78fbc8ebf9c92c9b7823ee18eb92f2e6ef79b45ac84db507f52fbe3ec4497", 308 | "sha256:320dab6e7cb2eacdf0e658569d2575c4dad258c0fcc794f46215e1e39f90f2c3", 309 | "sha256:33ab79603146aace82c2427da5ca6e58f2b3f2fb5da893ceac0c42218a40be35", 310 | "sha256:3548db281cd7d2561c9ad9984681c95f7b0e38881201e157833a2342c30d5e8c", 311 | "sha256:3799aecf2e17cf585d977b780ce79ff0dc9b78d799fc694221ce814c2c19db83", 312 | "sha256:39d39875251ca8f612b6f33e6b1195af86d1b3e60086068be9cc053aa4376e21", 313 | "sha256:3b926aa83d1edb5aa5b427b4053dc420ec295a08e40911296b9eb1b6170f6cca", 314 | "sha256:3bcde07039e586f91b45c88f8583ea7cf7a0770df3a1649627bf598332cb6984", 315 | "sha256:3d08afd128ddaa624a48cf2b859afef385b720bb4b43df214f85616922e6a5ac", 316 | "sha256:3eb6971dcff08619f8d91607cfc726518b6fa2a9eba42856be181c6d0d9515fd", 317 | "sha256:40f4774f5a9d4f5e344f31a32b5096977b5d48560c5592e2f3d2c4374bd543ee", 318 | "sha256:4289fc34b2f5316fbb762d75362931e351941fa95fa18789191b33fc4cf9504a", 319 | "sha256:470c103ae716238bbe698d67ad020e1db9d9dba34fa5a899b5e21577e6d52ed2", 320 | "sha256:4f2c9f67e9821cad2e5f480bc8d83b8742896f1242dba247911072d4fa94c192", 321 | "sha256:50a74364d85fd319352182ef59c5c790484a336f6db772c1a9231f1c3ed0cbd7", 322 | "sha256:54a2db7b78338edd780e7ef7f9f6c442500fb0d41a5a4ea24fff1c929d5af585", 323 | "sha256:5635bd9cb9731e6d4a1132a498dd34f764034a8ce60cef4f5319c0541159392f", 324 | "sha256:59c0b02d0a6c384d453fece7566d1c7e6b7bae4fc5874ef2ef46d56776d61c9e", 325 | "sha256:5d598b938678ebf3c67377cdd45e09d431369c3b1a5b331058c338e201f12b27", 326 | "sha256:5df2768244d19ab7f60546d0c7c63ce1581f7af8b5de3eb3004b9b6fc8a9f84b", 327 | "sha256:5ef34d190326c3b1f822a5b7a45f6c4535e2f47ed06fec77d3d799c450b2651e", 328 | "sha256:6975a3fac6bc83c4a65c9f9fcab9e47019a11d3d2cf7f3c0d03431bf145a941e", 329 | "sha256:6c9a799e985904922a4d207a94eae35c78ebae90e128f0c4e521ce339396be9d", 330 | "sha256:70df4e3b545a17496c9b3f41f5115e69a4f2e77e94e1d2a8e1070bc0c38c8a3c", 331 | "sha256:7473e861101c9e72452f9bf8acb984947aa1661a7704553a9f6e4baa5ba64415", 332 | "sha256:8102eaf27e1e448db915d08afa8b41d6c7ca7a04b7d73af6514df10a3e74bd82", 333 | "sha256:87c450779d0914f2861b8526e035c5e6da0a3199d8f1add1a665e1cbc6fc6d02", 334 | "sha256:8b7ee99e510d7b66cdb6c593f21c043c248537a32e0bedf02e01e9553a172314", 335 | "sha256:91fc98adde3d7881af9b59ed0294046f3806221863722ba7d8d120c575314325", 336 | "sha256:94411f22c3985acaec6f83c6df553f2dbe17b698cc7f8ae751ff2237d96b9e3c", 337 | "sha256:98d85c6a2bef81588d9227dde12db8a7f47f639f4a17c9ae08e773aa9c697bf3", 338 | "sha256:9ad5db27f9cabae298d151c85cf2bad1d359a1b9c686a275df03385758e2f914", 339 | "sha256:a0b71b1b8fbf2b96e41c4d990244165e2c9be83d54962a9a1d118fd8657d2045", 340 | "sha256:a0f100c8912c114ff53e1202d0078b425bee3649ae34d7b070e9697f93c5d52d", 341 | "sha256:a591fe9e525846e4d154205572a029f653ada1a78b93697f3b5a8f1f2bc055b9", 342 | "sha256:a5c84c68147988265e60416b57fc83425a78058853509c1b0629c180094904a5", 343 | "sha256:a66d3508133af6e8548451b25058d5812812ec3798c886bf38ed24a98216fab2", 344 | "sha256:a8c4917bd7ad33e8eb21e9a5bbba979b49d9a97acb3a803092cbc1133e20343c", 345 | "sha256:b3bbeb01c2b273cca1e1e0c5df57f12dce9a4dd331b4fa1635b8bec26350bde3", 346 | "sha256:cba9d6b9a7d64d4bd46167096fc9d2f835e25d7e4c121fb2ddfc6528fb0413b2", 347 | "sha256:cc4d65aeeaa04136a12677d3dd0b1c0c94dc43abac5860ab33cceb42b801c1e8", 348 | "sha256:ce4bcc037df4fc5e3d184794f27bdaab018943698f4ca31630bc7f84a7b69c6d", 349 | "sha256:cec7d9412a9102bdc577382c3929b337320c4c4c4849f2c5cdd14d7368c5562d", 350 | "sha256:d400bfb9a37b1351253cb402671cea7e89bdecc294e8016a707f6d1d8ac934f9", 351 | "sha256:d61f4695e6c866a23a21acab0509af1cdfd2c013cf256bbf5b6b5e2695827162", 352 | "sha256:db0fbb9c62743ce59a9ff687eb5f4afbe77e5e8403d6697f7446e5f609976f76", 353 | "sha256:dd86c085fae2efd48ac91dd7ccffcfc0571387fe1193d33b6394db7ef31fe2a4", 354 | "sha256:e00b098126fd45523dd056d2efba6c5a63b71ffe9f2bbe1a4fe1716e1d0c331e", 355 | "sha256:e229a521186c75c8ad9490854fd8bbdd9a0c9aa3a524326b55be83b54d4e0ad9", 356 | "sha256:e263d77ee3dd201c3a142934a086a4450861778baaeeb45db4591ef65550b0a6", 357 | "sha256:ed9cb427ba5504c1dc15ede7d516b84757c3e3d7868ccc85121d9310d27eed0b", 358 | "sha256:fa6693661a4c91757f4412306191b6dc88c1703f780c8234035eac011922bc01", 359 | "sha256:fcd131dd944808b5bdb38e6f5b53013c5aa4f334c5cad0c72742f6eba4b73db0" 360 | ], 361 | "version": "==1.15.1" 362 | }, 363 | "charset-normalizer": { 364 | "hashes": [ 365 | "sha256:04e57ab9fbf9607b77f7d057974694b4f6b142da9ed4a199859d9d4d5c63fe96", 366 | "sha256:09393e1b2a9461950b1c9a45d5fd251dc7c6f228acab64da1c9c0165d9c7765c", 367 | "sha256:0b87549028f680ca955556e3bd57013ab47474c3124dc069faa0b6545b6c9710", 368 | "sha256:1000fba1057b92a65daec275aec30586c3de2401ccdcd41f8a5c1e2c87078706", 369 | "sha256:1249cbbf3d3b04902ff081ffbb33ce3377fa6e4c7356f759f3cd076cc138d020", 370 | "sha256:1920d4ff15ce893210c1f0c0e9d19bfbecb7983c76b33f046c13a8ffbd570252", 371 | "sha256:193cbc708ea3aca45e7221ae58f0fd63f933753a9bfb498a3b474878f12caaad", 372 | "sha256:1a100c6d595a7f316f1b6f01d20815d916e75ff98c27a01ae817439ea7726329", 373 | "sha256:1f30b48dd7fa1474554b0b0f3fdfdd4c13b5c737a3c6284d3cdc424ec0ffff3a", 374 | "sha256:203f0c8871d5a7987be20c72442488a0b8cfd0f43b7973771640fc593f56321f", 375 | "sha256:246de67b99b6851627d945db38147d1b209a899311b1305dd84916f2b88526c6", 376 | "sha256:2dee8e57f052ef5353cf608e0b4c871aee320dd1b87d351c28764fc0ca55f9f4", 377 | "sha256:2efb1bd13885392adfda4614c33d3b68dee4921fd0ac1d3988f8cbb7d589e72a", 378 | "sha256:2f4ac36d8e2b4cc1aa71df3dd84ff8efbe3bfb97ac41242fbcfc053c67434f46", 379 | "sha256:3170c9399da12c9dc66366e9d14da8bf7147e1e9d9ea566067bbce7bb74bd9c2", 380 | "sha256:3b1613dd5aee995ec6d4c69f00378bbd07614702a315a2cf6c1d21461fe17c23", 381 | "sha256:3bb3d25a8e6c0aedd251753a79ae98a093c7e7b471faa3aa9a93a81431987ace", 382 | "sha256:3bb7fda7260735efe66d5107fb7e6af6a7c04c7fce9b2514e04b7a74b06bf5dd", 383 | "sha256:41b25eaa7d15909cf3ac4c96088c1f266a9a93ec44f87f1d13d4a0e86c81b982", 384 | "sha256:45de3f87179c1823e6d9e32156fb14c1927fcc9aba21433f088fdfb555b77c10", 385 | "sha256:46fb8c61d794b78ec7134a715a3e564aafc8f6b5e338417cb19fe9f57a5a9bf2", 386 | "sha256:48021783bdf96e3d6de03a6e39a1171ed5bd7e8bb93fc84cc649d11490f87cea", 387 | "sha256:4957669ef390f0e6719db3613ab3a7631e68424604a7b448f079bee145da6e09", 388 | "sha256:5e86d77b090dbddbe78867a0275cb4df08ea195e660f1f7f13435a4649e954e5", 389 | "sha256:6339d047dab2780cc6220f46306628e04d9750f02f983ddb37439ca47ced7149", 390 | "sha256:681eb3d7e02e3c3655d1b16059fbfb605ac464c834a0c629048a30fad2b27489", 391 | "sha256:6c409c0deba34f147f77efaa67b8e4bb83d2f11c8806405f76397ae5b8c0d1c9", 392 | "sha256:7095f6fbfaa55defb6b733cfeb14efaae7a29f0b59d8cf213be4e7ca0b857b80", 393 | "sha256:70c610f6cbe4b9fce272c407dd9d07e33e6bf7b4aa1b7ffb6f6ded8e634e3592", 394 | "sha256:72814c01533f51d68702802d74f77ea026b5ec52793c791e2da806a3844a46c3", 395 | "sha256:7a4826ad2bd6b07ca615c74ab91f32f6c96d08f6fcc3902ceeedaec8cdc3bcd6", 396 | "sha256:7c70087bfee18a42b4040bb9ec1ca15a08242cf5867c58726530bdf3945672ed", 397 | "sha256:855eafa5d5a2034b4621c74925d89c5efef61418570e5ef9b37717d9c796419c", 398 | "sha256:8700f06d0ce6f128de3ccdbc1acaea1ee264d2caa9ca05daaf492fde7c2a7200", 399 | "sha256:89f1b185a01fe560bc8ae5f619e924407efca2191b56ce749ec84982fc59a32a", 400 | "sha256:8b2c760cfc7042b27ebdb4a43a4453bd829a5742503599144d54a032c5dc7e9e", 401 | "sha256:8c2f5e83493748286002f9369f3e6607c565a6a90425a3a1fef5ae32a36d749d", 402 | "sha256:8e098148dd37b4ce3baca71fb394c81dc5d9c7728c95df695d2dca218edf40e6", 403 | "sha256:94aea8eff76ee6d1cdacb07dd2123a68283cb5569e0250feab1240058f53b623", 404 | "sha256:95eb302ff792e12aba9a8b8f8474ab229a83c103d74a750ec0bd1c1eea32e669", 405 | "sha256:9bd9b3b31adcb054116447ea22caa61a285d92e94d710aa5ec97992ff5eb7cf3", 406 | "sha256:9e608aafdb55eb9f255034709e20d5a83b6d60c054df0802fa9c9883d0a937aa", 407 | "sha256:a103b3a7069b62f5d4890ae1b8f0597618f628b286b03d4bc9195230b154bfa9", 408 | "sha256:a386ebe437176aab38c041de1260cd3ea459c6ce5263594399880bbc398225b2", 409 | "sha256:a38856a971c602f98472050165cea2cdc97709240373041b69030be15047691f", 410 | "sha256:a401b4598e5d3f4a9a811f3daf42ee2291790c7f9d74b18d75d6e21dda98a1a1", 411 | "sha256:a7647ebdfb9682b7bb97e2a5e7cb6ae735b1c25008a70b906aecca294ee96cf4", 412 | "sha256:aaf63899c94de41fe3cf934601b0f7ccb6b428c6e4eeb80da72c58eab077b19a", 413 | "sha256:b0dac0ff919ba34d4df1b6131f59ce95b08b9065233446be7e459f95554c0dc8", 414 | "sha256:baacc6aee0b2ef6f3d308e197b5d7a81c0e70b06beae1f1fcacffdbd124fe0e3", 415 | "sha256:bf420121d4c8dce6b889f0e8e4ec0ca34b7f40186203f06a946fa0276ba54029", 416 | "sha256:c04a46716adde8d927adb9457bbe39cf473e1e2c2f5d0a16ceb837e5d841ad4f", 417 | "sha256:c0b21078a4b56965e2b12f247467b234734491897e99c1d51cee628da9786959", 418 | "sha256:c1c76a1743432b4b60ab3358c937a3fe1341c828ae6194108a94c69028247f22", 419 | "sha256:c4983bf937209c57240cff65906b18bb35e64ae872da6a0db937d7b4af845dd7", 420 | "sha256:c4fb39a81950ec280984b3a44f5bd12819953dc5fa3a7e6fa7a80db5ee853952", 421 | "sha256:c57921cda3a80d0f2b8aec7e25c8aa14479ea92b5b51b6876d975d925a2ea346", 422 | "sha256:c8063cf17b19661471ecbdb3df1c84f24ad2e389e326ccaf89e3fb2484d8dd7e", 423 | "sha256:ccd16eb18a849fd8dcb23e23380e2f0a354e8daa0c984b8a732d9cfaba3a776d", 424 | "sha256:cd6dbe0238f7743d0efe563ab46294f54f9bc8f4b9bcf57c3c666cc5bc9d1299", 425 | "sha256:d62e51710986674142526ab9f78663ca2b0726066ae26b78b22e0f5e571238dd", 426 | "sha256:db901e2ac34c931d73054d9797383d0f8009991e723dab15109740a63e7f902a", 427 | "sha256:e03b8895a6990c9ab2cdcd0f2fe44088ca1c65ae592b8f795c3294af00a461c3", 428 | "sha256:e1c8a2f4c69e08e89632defbfabec2feb8a8d99edc9f89ce33c4b9e36ab63037", 429 | "sha256:e4b749b9cc6ee664a3300bb3a273c1ca8068c46be705b6c31cf5d276f8628a94", 430 | "sha256:e6a5bf2cba5ae1bb80b154ed68a3cfa2fa00fde979a7f50d6598d3e17d9ac20c", 431 | "sha256:e857a2232ba53ae940d3456f7533ce6ca98b81917d47adc3c7fd55dad8fab858", 432 | "sha256:ee4006268ed33370957f55bf2e6f4d263eaf4dc3cfc473d1d90baff6ed36ce4a", 433 | "sha256:eef9df1eefada2c09a5e7a40991b9fc6ac6ef20b1372abd48d2794a316dc0449", 434 | "sha256:f058f6963fd82eb143c692cecdc89e075fa0828db2e5b291070485390b2f1c9c", 435 | "sha256:f25c229a6ba38a35ae6e25ca1264621cc25d4d38dca2942a7fce0b67a4efe918", 436 | "sha256:f2a1d0fd4242bd8643ce6f98927cf9c04540af6efa92323e9d3124f57727bfc1", 437 | "sha256:f7560358a6811e52e9c4d142d497f1a6e10103d3a6881f18d04dbce3729c0e2c", 438 | "sha256:f779d3ad205f108d14e99bb3859aa7dd8e9c68874617c72354d7ecaec2a054ac", 439 | "sha256:f87f746ee241d30d6ed93969de31e5ffd09a2961a051e60ae6bddde9ec3583aa" 440 | ], 441 | "markers": "python_version >= '3.7'", 442 | "version": "==3.2.0" 443 | }, 444 | "coverage": { 445 | "hashes": [ 446 | "sha256:06a9a2be0b5b576c3f18f1a241f0473575c4a26021b52b2a85263a00f034d51f", 447 | "sha256:06fb182e69f33f6cd1d39a6c597294cff3143554b64b9825d1dc69d18cc2fff2", 448 | "sha256:0a5f9e1dbd7fbe30196578ca36f3fba75376fb99888c395c5880b355e2875f8a", 449 | "sha256:0e1f928eaf5469c11e886fe0885ad2bf1ec606434e79842a879277895a50942a", 450 | "sha256:171717c7cb6b453aebac9a2ef603699da237f341b38eebfee9be75d27dc38e01", 451 | "sha256:1e9d683426464e4a252bf70c3498756055016f99ddaec3774bf368e76bbe02b6", 452 | "sha256:201e7389591af40950a6480bd9edfa8ed04346ff80002cec1a66cac4549c1ad7", 453 | "sha256:245167dd26180ab4c91d5e1496a30be4cd721a5cf2abf52974f965f10f11419f", 454 | "sha256:2aee274c46590717f38ae5e4650988d1af340fe06167546cc32fe2f58ed05b02", 455 | "sha256:2e07b54284e381531c87f785f613b833569c14ecacdcb85d56b25c4622c16c3c", 456 | "sha256:31563e97dae5598556600466ad9beea39fb04e0229e61c12eaa206e0aa202063", 457 | "sha256:33d6d3ea29d5b3a1a632b3c4e4f4ecae24ef170b0b9ee493883f2df10039959a", 458 | "sha256:3d376df58cc111dc8e21e3b6e24606b5bb5dee6024f46a5abca99124b2229ef5", 459 | "sha256:419bfd2caae268623dd469eff96d510a920c90928b60f2073d79f8fe2bbc5959", 460 | "sha256:48c19d2159d433ccc99e729ceae7d5293fbffa0bdb94952d3579983d1c8c9d97", 461 | "sha256:49969a9f7ffa086d973d91cec8d2e31080436ef0fb4a359cae927e742abfaaa6", 462 | "sha256:52edc1a60c0d34afa421c9c37078817b2e67a392cab17d97283b64c5833f427f", 463 | "sha256:537891ae8ce59ef63d0123f7ac9e2ae0fc8b72c7ccbe5296fec45fd68967b6c9", 464 | "sha256:54b896376ab563bd38453cecb813c295cf347cf5906e8b41d340b0321a5433e5", 465 | "sha256:58c2ccc2f00ecb51253cbe5d8d7122a34590fac9646a960d1430d5b15321d95f", 466 | "sha256:5b7540161790b2f28143191f5f8ec02fb132660ff175b7747b95dcb77ac26562", 467 | "sha256:5baa06420f837184130752b7c5ea0808762083bf3487b5038d68b012e5937dbe", 468 | "sha256:5e330fc79bd7207e46c7d7fd2bb4af2963f5f635703925543a70b99574b0fea9", 469 | "sha256:61b9a528fb348373c433e8966535074b802c7a5d7f23c4f421e6c6e2f1697a6f", 470 | "sha256:63426706118b7f5cf6bb6c895dc215d8a418d5952544042c8a2d9fe87fcf09cb", 471 | "sha256:6d040ef7c9859bb11dfeb056ff5b3872436e3b5e401817d87a31e1750b9ae2fb", 472 | "sha256:6f48351d66575f535669306aa7d6d6f71bc43372473b54a832222803eb956fd1", 473 | "sha256:7ee7d9d4822c8acc74a5e26c50604dff824710bc8de424904c0982e25c39c6cb", 474 | "sha256:81c13a1fc7468c40f13420732805a4c38a105d89848b7c10af65a90beff25250", 475 | "sha256:8d13c64ee2d33eccf7437961b6ea7ad8673e2be040b4f7fd4fd4d4d28d9ccb1e", 476 | "sha256:8de8bb0e5ad103888d65abef8bca41ab93721647590a3f740100cd65c3b00511", 477 | "sha256:8fa03bce9bfbeeef9f3b160a8bed39a221d82308b4152b27d82d8daa7041fee5", 478 | "sha256:924d94291ca674905fe9481f12294eb11f2d3d3fd1adb20314ba89e94f44ed59", 479 | "sha256:975d70ab7e3c80a3fe86001d8751f6778905ec723f5b110aed1e450da9d4b7f2", 480 | "sha256:976b9c42fb2a43ebf304fa7d4a310e5f16cc99992f33eced91ef6f908bd8f33d", 481 | "sha256:9e31cb64d7de6b6f09702bb27c02d1904b3aebfca610c12772452c4e6c21a0d3", 482 | "sha256:a342242fe22407f3c17f4b499276a02b01e80f861f1682ad1d95b04018e0c0d4", 483 | "sha256:a3d33a6b3eae87ceaefa91ffdc130b5e8536182cd6dfdbfc1aa56b46ff8c86de", 484 | "sha256:a895fcc7b15c3fc72beb43cdcbdf0ddb7d2ebc959edac9cef390b0d14f39f8a9", 485 | "sha256:afb17f84d56068a7c29f5fa37bfd38d5aba69e3304af08ee94da8ed5b0865833", 486 | "sha256:b1c546aca0ca4d028901d825015dc8e4d56aac4b541877690eb76490f1dc8ed0", 487 | "sha256:b29019c76039dc3c0fd815c41392a044ce555d9bcdd38b0fb60fb4cd8e475ba9", 488 | "sha256:b46517c02ccd08092f4fa99f24c3b83d8f92f739b4657b0f146246a0ca6a831d", 489 | "sha256:b7aa5f8a41217360e600da646004f878250a0d6738bcdc11a0a39928d7dc2050", 490 | "sha256:b7b4c971f05e6ae490fef852c218b0e79d4e52f79ef0c8475566584a8fb3e01d", 491 | "sha256:ba90a9563ba44a72fda2e85302c3abc71c5589cea608ca16c22b9804262aaeb6", 492 | "sha256:cb017fd1b2603ef59e374ba2063f593abe0fc45f2ad9abdde5b4d83bd922a353", 493 | "sha256:d22656368f0e6189e24722214ed8d66b8022db19d182927b9a248a2a8a2f67eb", 494 | "sha256:d2c2db7fd82e9b72937969bceac4d6ca89660db0a0967614ce2481e81a0b771e", 495 | "sha256:d39b5b4f2a66ccae8b7263ac3c8170994b65266797fb96cbbfd3fb5b23921db8", 496 | "sha256:d62a5c7dad11015c66fbb9d881bc4caa5b12f16292f857842d9d1871595f4495", 497 | "sha256:e7d9405291c6928619403db1d10bd07888888ec1abcbd9748fdaa971d7d661b2", 498 | "sha256:e84606b74eb7de6ff581a7915e2dab7a28a0517fbe1c9239eb227e1354064dcd", 499 | "sha256:eb393e5ebc85245347950143969b241d08b52b88a3dc39479822e073a1a8eb27", 500 | "sha256:ebba1cd308ef115925421d3e6a586e655ca5a77b5bf41e02eb0e4562a111f2d1", 501 | "sha256:ee57190f24fba796e36bb6d3aa8a8783c643d8fa9760c89f7a98ab5455fbf818", 502 | "sha256:f2f67fe12b22cd130d34d0ef79206061bfb5eda52feb6ce0dba0644e20a03cf4", 503 | "sha256:f6951407391b639504e3b3be51b7ba5f3528adbf1a8ac3302b687ecababf929e", 504 | "sha256:f75f7168ab25dd93110c8a8117a22450c19976afbc44234cbf71481094c1b850", 505 | "sha256:fdec9e8cbf13a5bf63290fc6013d216a4c7232efb51548594ca3631a7f13c3a3" 506 | ], 507 | "markers": "python_version >= '3.7'", 508 | "version": "==7.2.7" 509 | }, 510 | "cryptography": { 511 | "hashes": [ 512 | "sha256:0d09fb5356f975974dbcb595ad2d178305e5050656affb7890a1583f5e02a306", 513 | "sha256:23c2d778cf829f7d0ae180600b17e9fceea3c2ef8b31a99e3c694cbbf3a24b84", 514 | "sha256:3fb248989b6363906827284cd20cca63bb1a757e0a2864d4c1682a985e3dca47", 515 | "sha256:41d7aa7cdfded09b3d73a47f429c298e80796c8e825ddfadc84c8a7f12df212d", 516 | "sha256:42cb413e01a5d36da9929baa9d70ca90d90b969269e5a12d39c1e0d475010116", 517 | "sha256:4c2f0d35703d61002a2bbdcf15548ebb701cfdd83cdc12471d2bae80878a4207", 518 | "sha256:4fd871184321100fb400d759ad0cddddf284c4b696568204d281c902fc7b0d81", 519 | "sha256:5259cb659aa43005eb55a0e4ff2c825ca111a0da1814202c64d28a985d33b087", 520 | "sha256:57a51b89f954f216a81c9d057bf1a24e2f36e764a1ca9a501a6964eb4a6800dd", 521 | "sha256:652627a055cb52a84f8c448185922241dd5217443ca194d5739b44612c5e6507", 522 | "sha256:67e120e9a577c64fe1f611e53b30b3e69744e5910ff3b6e97e935aeb96005858", 523 | "sha256:6af1c6387c531cd364b72c28daa29232162010d952ceb7e5ca8e2827526aceae", 524 | "sha256:6d192741113ef5e30d89dcb5b956ef4e1578f304708701b8b73d38e3e1461f34", 525 | "sha256:7efe8041897fe7a50863e51b77789b657a133c75c3b094e51b5e4b5cec7bf906", 526 | "sha256:84537453d57f55a50a5b6835622ee405816999a7113267739a1b4581f83535bd", 527 | "sha256:8f09daa483aedea50d249ef98ed500569841d6498aa9c9f4b0531b9964658922", 528 | "sha256:95dd7f261bb76948b52a5330ba5202b91a26fbac13ad0e9fc8a3ac04752058c7", 529 | "sha256:a74fbcdb2a0d46fe00504f571a2a540532f4c188e6ccf26f1f178480117b33c4", 530 | "sha256:a983e441a00a9d57a4d7c91b3116a37ae602907a7618b882c8013b5762e80574", 531 | "sha256:ab8de0d091acbf778f74286f4989cf3d1528336af1b59f3e5d2ebca8b5fe49e1", 532 | "sha256:aeb57c421b34af8f9fe830e1955bf493a86a7996cc1338fe41b30047d16e962c", 533 | "sha256:ce785cf81a7bdade534297ef9e490ddff800d956625020ab2ec2780a556c313e", 534 | "sha256:d0d651aa754ef58d75cec6edfbd21259d93810b73f6ec246436a21b7841908de" 535 | ], 536 | "index": "pypi", 537 | "version": "==41.0.3" 538 | }, 539 | "docker": { 540 | "hashes": [ 541 | "sha256:7a79bb439e3df59d0a72621775d600bc8bc8b422d285824cb37103eab91d1ce0", 542 | "sha256:d916a26b62970e7c2f554110ed6af04c7ccff8e9f81ad17d0d40c75637e227fb" 543 | ], 544 | "markers": "python_version >= '3.6'", 545 | "version": "==5.0.3" 546 | }, 547 | "docutils": { 548 | "hashes": [ 549 | "sha256:6c4f696463b79f1fb8ba0c594b63840ebd41f059e92b31957c46b74a4599b6d0", 550 | "sha256:9e4d7ecfc600058e07ba661411a2b7de2fd0fafa17d1a7f7361cd47b1175c827", 551 | "sha256:a2aeea129088da402665e92e0b25b04b073c04b2dce4ab65caaa38b7ce2e1a99" 552 | ], 553 | "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", 554 | "version": "==0.15.2" 555 | }, 556 | "ecdsa": { 557 | "hashes": [ 558 | "sha256:190348041559e21b22a1d65cee485282ca11a6f81d503fddb84d5017e9ed1e49", 559 | "sha256:80600258e7ed2f16b9aa1d7c295bd70194109ad5a30fdee0eaeefef1d4c559dd" 560 | ], 561 | "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", 562 | "version": "==0.18.0" 563 | }, 564 | "exceptiongroup": { 565 | "hashes": [ 566 | "sha256:12c3e887d6485d16943a309616de20ae5582633e0a2eda17f4e10fd61c1e8af5", 567 | "sha256:e346e69d186172ca7cf029c8c1d16235aa0e04035e5750b4b95039e65204328f" 568 | ], 569 | "markers": "python_version < '3.11'", 570 | "version": "==1.1.2" 571 | }, 572 | "freezegun": { 573 | "hashes": [ 574 | "sha256:6cb82b276f83f2acce67f121dc2656f4df26c71e32238334eb071170b892a278", 575 | "sha256:e839b43bfbe8158b4d62bb97e6313d39f3586daf48e1314fb1083d2ef17700da" 576 | ], 577 | "index": "pypi", 578 | "version": "==0.3.11" 579 | }, 580 | "future": { 581 | "hashes": [ 582 | "sha256:34a17436ed1e96697a86f9de3d15a3b0be01d8bc8de9c1dffd59fb8234ed5307" 583 | ], 584 | "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", 585 | "version": "==0.18.3" 586 | }, 587 | "idna": { 588 | "hashes": [ 589 | "sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4", 590 | "sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2" 591 | ], 592 | "markers": "python_version >= '3.5'", 593 | "version": "==3.4" 594 | }, 595 | "importlib-metadata": { 596 | "hashes": [ 597 | "sha256:1aaf550d4f73e5d6783e7acb77aec43d49da8017410afae93822cc9cca98c4d4", 598 | "sha256:cb52082e659e97afc5dac71e79de97d8681de3aa07ff18578330904a9d18e5b5" 599 | ], 600 | "markers": "python_version < '3.8'", 601 | "version": "==6.7.0" 602 | }, 603 | "iniconfig": { 604 | "hashes": [ 605 | "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3", 606 | "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374" 607 | ], 608 | "markers": "python_version >= '3.7'", 609 | "version": "==2.0.0" 610 | }, 611 | "jinja2": { 612 | "hashes": [ 613 | "sha256:31351a702a408a9e7595a8fc6150fc3f43bb6bf7e319770cbc0db9df9437e852", 614 | "sha256:6088930bfe239f0e6710546ab9c19c9ef35e29792895fed6e6e31a023a182a61" 615 | ], 616 | "markers": "python_version >= '3.7'", 617 | "version": "==3.1.2" 618 | }, 619 | "jmespath": { 620 | "hashes": [ 621 | "sha256:b85d0567b8666149a93172712e68920734333c0ce7e89b78b3e987f71e5ed4f9", 622 | "sha256:cdf6525904cc597730141d61b36f2e4b8ecc257c420fa2f4549bac2c2d0cb72f" 623 | ], 624 | "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", 625 | "version": "==0.10.0" 626 | }, 627 | "jsondiff": { 628 | "hashes": [ 629 | "sha256:2d0437782de9418efa34e694aa59f43d7adb1899bd9a793f063867ddba8f7893" 630 | ], 631 | "version": "==1.1.1" 632 | }, 633 | "jsonpickle": { 634 | "hashes": [ 635 | "sha256:032538804795e73b94ead410800ac387fdb6de98f8882ac957fcd247e3a85200", 636 | "sha256:130d8b293ea0add3845de311aaba55e6d706d0bb17bc123bd2c8baf8a39ac77c" 637 | ], 638 | "markers": "python_version >= '3.7'", 639 | "version": "==3.0.1" 640 | }, 641 | "markupsafe": { 642 | "hashes": [ 643 | "sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e", 644 | "sha256:0a4e4a1aff6c7ac4cd55792abf96c915634c2b97e3cc1c7129578aa68ebd754e", 645 | "sha256:10bbfe99883db80bdbaff2dcf681dfc6533a614f700da1287707e8a5d78a8431", 646 | "sha256:134da1eca9ec0ae528110ccc9e48041e0828d79f24121a1a146161103c76e686", 647 | "sha256:1577735524cdad32f9f694208aa75e422adba74f1baee7551620e43a3141f559", 648 | "sha256:1b40069d487e7edb2676d3fbdb2b0829ffa2cd63a2ec26c4938b2d34391b4ecc", 649 | "sha256:282c2cb35b5b673bbcadb33a585408104df04f14b2d9b01d4c345a3b92861c2c", 650 | "sha256:2c1b19b3aaacc6e57b7e25710ff571c24d6c3613a45e905b1fde04d691b98ee0", 651 | "sha256:2ef12179d3a291be237280175b542c07a36e7f60718296278d8593d21ca937d4", 652 | "sha256:338ae27d6b8745585f87218a3f23f1512dbf52c26c28e322dbe54bcede54ccb9", 653 | "sha256:3c0fae6c3be832a0a0473ac912810b2877c8cb9d76ca48de1ed31e1c68386575", 654 | "sha256:3fd4abcb888d15a94f32b75d8fd18ee162ca0c064f35b11134be77050296d6ba", 655 | "sha256:42de32b22b6b804f42c5d98be4f7e5e977ecdd9ee9b660fda1a3edf03b11792d", 656 | "sha256:504b320cd4b7eff6f968eddf81127112db685e81f7e36e75f9f84f0df46041c3", 657 | "sha256:525808b8019e36eb524b8c68acdd63a37e75714eac50e988180b169d64480a00", 658 | "sha256:56d9f2ecac662ca1611d183feb03a3fa4406469dafe241673d521dd5ae92a155", 659 | "sha256:5bbe06f8eeafd38e5d0a4894ffec89378b6c6a625ff57e3028921f8ff59318ac", 660 | "sha256:65c1a9bcdadc6c28eecee2c119465aebff8f7a584dd719facdd9e825ec61ab52", 661 | "sha256:68e78619a61ecf91e76aa3e6e8e33fc4894a2bebe93410754bd28fce0a8a4f9f", 662 | "sha256:69c0f17e9f5a7afdf2cc9fb2d1ce6aabdb3bafb7f38017c0b77862bcec2bbad8", 663 | "sha256:6b2b56950d93e41f33b4223ead100ea0fe11f8e6ee5f641eb753ce4b77a7042b", 664 | "sha256:787003c0ddb00500e49a10f2844fac87aa6ce977b90b0feaaf9de23c22508b24", 665 | "sha256:7ef3cb2ebbf91e330e3bb937efada0edd9003683db6b57bb108c4001f37a02ea", 666 | "sha256:8023faf4e01efadfa183e863fefde0046de576c6f14659e8782065bcece22198", 667 | "sha256:8758846a7e80910096950b67071243da3e5a20ed2546e6392603c096778d48e0", 668 | "sha256:8afafd99945ead6e075b973fefa56379c5b5c53fd8937dad92c662da5d8fd5ee", 669 | "sha256:8c41976a29d078bb235fea9b2ecd3da465df42a562910f9022f1a03107bd02be", 670 | "sha256:8e254ae696c88d98da6555f5ace2279cf7cd5b3f52be2b5cf97feafe883b58d2", 671 | "sha256:9402b03f1a1b4dc4c19845e5c749e3ab82d5078d16a2a4c2cd2df62d57bb0707", 672 | "sha256:962f82a3086483f5e5f64dbad880d31038b698494799b097bc59c2edf392fce6", 673 | "sha256:9dcdfd0eaf283af041973bff14a2e143b8bd64e069f4c383416ecd79a81aab58", 674 | "sha256:aa7bd130efab1c280bed0f45501b7c8795f9fdbeb02e965371bbef3523627779", 675 | "sha256:ab4a0df41e7c16a1392727727e7998a467472d0ad65f3ad5e6e765015df08636", 676 | "sha256:ad9e82fb8f09ade1c3e1b996a6337afac2b8b9e365f926f5a61aacc71adc5b3c", 677 | "sha256:af598ed32d6ae86f1b747b82783958b1a4ab8f617b06fe68795c7f026abbdcad", 678 | "sha256:b076b6226fb84157e3f7c971a47ff3a679d837cf338547532ab866c57930dbee", 679 | "sha256:b7ff0f54cb4ff66dd38bebd335a38e2c22c41a8ee45aa608efc890ac3e3931bc", 680 | "sha256:bfce63a9e7834b12b87c64d6b155fdd9b3b96191b6bd334bf37db7ff1fe457f2", 681 | "sha256:c011a4149cfbcf9f03994ec2edffcb8b1dc2d2aede7ca243746df97a5d41ce48", 682 | "sha256:c9c804664ebe8f83a211cace637506669e7890fec1b4195b505c214e50dd4eb7", 683 | "sha256:ca379055a47383d02a5400cb0d110cef0a776fc644cda797db0c5696cfd7e18e", 684 | "sha256:cb0932dc158471523c9637e807d9bfb93e06a95cbf010f1a38b98623b929ef2b", 685 | "sha256:cd0f502fe016460680cd20aaa5a76d241d6f35a1c3350c474bac1273803893fa", 686 | "sha256:ceb01949af7121f9fc39f7d27f91be8546f3fb112c608bc4029aef0bab86a2a5", 687 | "sha256:d080e0a5eb2529460b30190fcfcc4199bd7f827663f858a226a81bc27beaa97e", 688 | "sha256:dd15ff04ffd7e05ffcb7fe79f1b98041b8ea30ae9234aed2a9168b5797c3effb", 689 | "sha256:df0be2b576a7abbf737b1575f048c23fb1d769f267ec4358296f31c2479db8f9", 690 | "sha256:e09031c87a1e51556fdcb46e5bd4f59dfb743061cf93c4d6831bf894f125eb57", 691 | "sha256:e4dd52d80b8c83fdce44e12478ad2e85c64ea965e75d66dbeafb0a3e77308fcc", 692 | "sha256:fec21693218efe39aa7f8599346e90c705afa52c5b31ae019b2e57e8f6542bb2" 693 | ], 694 | "markers": "python_version >= '3.7'", 695 | "version": "==2.1.3" 696 | }, 697 | "mock": { 698 | "hashes": [ 699 | "sha256:18c694e5ae8a208cdb3d2c20a993ca1a7b0efa258c247a1e565150f477f83744", 700 | "sha256:5e96aad5ccda4718e0a229ed94b2024df75cc2d55575ba5762d31f5767b8767d" 701 | ], 702 | "markers": "python_version >= '3.6'", 703 | "version": "==5.1.0" 704 | }, 705 | "moto": { 706 | "hashes": [ 707 | "sha256:129de2e04cb250d9f8b2c722ec152ed1b5426ef179b4ebb03e9ec36e6eb3fcc5", 708 | "sha256:4df37936ff8d6a4b8229aab347a7b412cd2ca4823ff47bd1362ddfbc6c5e4ecf" 709 | ], 710 | "index": "pypi", 711 | "version": "==1.3.7" 712 | }, 713 | "packaging": { 714 | "hashes": [ 715 | "sha256:994793af429502c4ea2ebf6bf664629d07c1a9fe974af92966e4b8d2df7edc61", 716 | "sha256:a392980d2b6cffa644431898be54b0045151319d1e7ec34f0cfed48767dd334f" 717 | ], 718 | "markers": "python_version >= '3.7'", 719 | "version": "==23.1" 720 | }, 721 | "pluggy": { 722 | "hashes": [ 723 | "sha256:c2fd55a7d7a3863cba1a013e4e2414658b1d07b6bc57b3919e0c63c9abb99849", 724 | "sha256:d12f0c4b579b15f5e054301bb226ee85eeeba08ffec228092f8defbaa3a4c4b3" 725 | ], 726 | "markers": "python_version >= '3.7'", 727 | "version": "==1.2.0" 728 | }, 729 | "pyaml": { 730 | "hashes": [ 731 | "sha256:287c58ad3aca43fd6da216f42b3d89c835aa1c79301948bbd25be9a2bb71def3", 732 | "sha256:da944f15d3fe035946450f1acefd267be6b32878f2c736f8dd98a94b6bff3c0c" 733 | ], 734 | "version": "==23.5.8" 735 | }, 736 | "pycparser": { 737 | "hashes": [ 738 | "sha256:8ee45429555515e1f6b185e78100aea234072576aa43ab53aefcae078162fca9", 739 | "sha256:e644fdec12f7872f86c58ff790da456218b10f863970249516d60a5eaca77206" 740 | ], 741 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 742 | "version": "==2.21" 743 | }, 744 | "pycryptodome": { 745 | "hashes": [ 746 | "sha256:01489bbdf709d993f3058e2996f8f40fee3f0ea4d995002e5968965fa2fe89fb", 747 | "sha256:10da29526a2a927c7d64b8f34592f461d92ae55fc97981aab5bbcde8cb465bb6", 748 | "sha256:12600268763e6fec3cefe4c2dcdf79bde08d0b6dc1813887e789e495cb9f3403", 749 | "sha256:157c9b5ba5e21b375f052ca78152dd309a09ed04703fd3721dce3ff8ecced148", 750 | "sha256:16bfd98dbe472c263ed2821284118d899c76968db1a6665ade0c46805e6b29a4", 751 | "sha256:363dd6f21f848301c2dcdeb3c8ae5f0dee2286a5e952a0f04954b82076f23825", 752 | "sha256:3811e31e1ac3069988f7a1c9ee7331b942e605dfc0f27330a9ea5997e965efb2", 753 | "sha256:422c89fd8df8a3bee09fb8d52aaa1e996120eafa565437392b781abec2a56e14", 754 | "sha256:4604816adebd4faf8810782f137f8426bf45fee97d8427fa8e1e49ea78a52e2c", 755 | "sha256:4944defabe2ace4803f99543445c27dd1edbe86d7d4edb87b256476a91e9ffa4", 756 | "sha256:51eae079ddb9c5f10376b4131be9589a6554f6fd84f7f655180937f611cd99a2", 757 | "sha256:53aee6be8b9b6da25ccd9028caf17dcdce3604f2c7862f5167777b707fbfb6cb", 758 | "sha256:62a1e8847fabb5213ccde38915563140a5b338f0d0a0d363f996b51e4a6165cf", 759 | "sha256:6f4b967bb11baea9128ec88c3d02f55a3e338361f5e4934f5240afcb667fdaec", 760 | "sha256:78d863476e6bad2a592645072cc489bb90320972115d8995bcfbee2f8b209918", 761 | "sha256:795bd1e4258a2c689c0b1f13ce9684fa0dd4c0e08680dcf597cf9516ed6bc0f3", 762 | "sha256:7a3d22c8ee63de22336679e021c7f2386f7fc465477d59675caa0e5706387944", 763 | "sha256:83c75952dcf4a4cebaa850fa257d7a860644c70a7cd54262c237c9f2be26f76e", 764 | "sha256:928078c530da78ff08e10eb6cada6e0dff386bf3d9fa9871b4bbc9fbc1efe024", 765 | "sha256:957b221d062d5752716923d14e0926f47670e95fead9d240fa4d4862214b9b2f", 766 | "sha256:9ad6f09f670c466aac94a40798e0e8d1ef2aa04589c29faa5b9b97566611d1d1", 767 | "sha256:9c8eda4f260072f7dbe42f473906c659dcbadd5ae6159dfb49af4da1293ae380", 768 | "sha256:b1d9701d10303eec8d0bd33fa54d44e67b8be74ab449052a8372f12a66f93fb9", 769 | "sha256:b6a610f8bfe67eab980d6236fdc73bfcdae23c9ed5548192bb2d530e8a92780e", 770 | "sha256:c9adee653fc882d98956e33ca2c1fb582e23a8af7ac82fee75bd6113c55a0413", 771 | "sha256:cb1be4d5af7f355e7d41d36d8eec156ef1382a88638e8032215c215b82a4b8ec", 772 | "sha256:d1497a8cd4728db0e0da3c304856cb37c0c4e3d0b36fcbabcc1600f18504fc54", 773 | "sha256:d20082bdac9218649f6abe0b885927be25a917e29ae0502eaf2b53f1233ce0c2", 774 | "sha256:e8ad74044e5f5d2456c11ed4cfd3e34b8d4898c0cb201c4038fe41458a82ea27", 775 | "sha256:f022a4fd2a5263a5c483a2bb165f9cb27f2be06f2f477113783efe3fe2ad887b", 776 | "sha256:f21efb8438971aa16924790e1c3dba3a33164eb4000106a55baaed522c261acf", 777 | "sha256:fc0a73f4db1e31d4a6d71b672a48f3af458f548059aa05e83022d5f61aac9c08" 778 | ], 779 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", 780 | "version": "==3.18.0" 781 | }, 782 | "pyfakefs": { 783 | "hashes": [ 784 | "sha256:39f1a39ed3b61def03aa4f3c38c4322273b18834e0b3cc0a200b33f36f7a4321", 785 | "sha256:8cd2270d65d3316dd4dc6bb83242df2e0990d27605209bc16e8041bcc0956961" 786 | ], 787 | "index": "pypi", 788 | "version": "==3.5.8" 789 | }, 790 | "pytest": { 791 | "hashes": [ 792 | "sha256:78bf16451a2eb8c7a2ea98e32dc119fd2aa758f1d5d66dbf0a59d69a3969df32", 793 | "sha256:b4bf8c45bd59934ed84001ad51e11b4ee40d40a1229d2c79f9c592b0a3f6bd8a" 794 | ], 795 | "markers": "python_version >= '3.7'", 796 | "version": "==7.4.0" 797 | }, 798 | "pytest-cov": { 799 | "hashes": [ 800 | "sha256:0ab664b25c6aa9716cbf203b17ddb301932383046082c081b9848a0edf5add33", 801 | "sha256:230ef817450ab0699c6cc3c9c8f7a829c34674456f2ed8df1fe1d39780f7c87f" 802 | ], 803 | "index": "pypi", 804 | "version": "==2.6.1" 805 | }, 806 | "pytest-mock": { 807 | "hashes": [ 808 | "sha256:4d0d06d173eecf172703219a71dbd4ade0e13904e6bbce1ce660e2e0dc78b5c4", 809 | "sha256:bfdf02789e3d197bd682a758cae0a4a18706566395fbe2803badcd1335e0173e" 810 | ], 811 | "index": "pypi", 812 | "version": "==1.10.1" 813 | }, 814 | "python-dateutil": { 815 | "hashes": [ 816 | "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86", 817 | "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9" 818 | ], 819 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 820 | "version": "==2.8.2" 821 | }, 822 | "python-jose": { 823 | "hashes": [ 824 | "sha256:391f860dbe274223d73dd87de25e4117bf09e8fe5f93a417663b1f2d7b591165", 825 | "sha256:3b35cdb0e55a88581ff6d3f12de753aa459e940b50fe7ca5aa25149bc94cb37b" 826 | ], 827 | "version": "==2.0.2" 828 | }, 829 | "pytz": { 830 | "hashes": [ 831 | "sha256:1d8ce29db189191fb55338ee6d0387d82ab59f3d00eac103412d64e0ebd0c588", 832 | "sha256:a151b3abb88eda1d4e34a9814df37de2a80e301e68ba0fd856fb9b46bfbbbffb" 833 | ], 834 | "version": "==2023.3" 835 | }, 836 | "pyyaml": { 837 | "hashes": [ 838 | "sha256:02c78d77281d8f8d07a255e57abdbf43b02257f59f50cc6b636937d68efa5dd0", 839 | "sha256:0dc9f2eb2e3c97640928dec63fd8dc1dd91e6b6ed236bd5ac00332b99b5c2ff9", 840 | "sha256:124fd7c7bc1e95b1eafc60825f2daf67c73ce7b33f1194731240d24b0d1bf628", 841 | "sha256:26fcb33776857f4072601502d93e1a619f166c9c00befb52826e7b774efaa9db", 842 | "sha256:31ba07c54ef4a897758563e3a0fcc60077698df10180abe4b8165d9895c00ebf", 843 | "sha256:3c49e39ac034fd64fd576d63bb4db53cda89b362768a67f07749d55f128ac18a", 844 | "sha256:52bf0930903818e600ae6c2901f748bc4869c0c406056f679ab9614e5d21a166", 845 | "sha256:5a3f345acff76cad4aa9cb171ee76c590f37394186325d53d1aa25318b0d4a09", 846 | "sha256:5e7ac4e0e79a53451dc2814f6876c2fa6f71452de1498bbe29c0b54b69a986f4", 847 | "sha256:7242790ab6c20316b8e7bb545be48d7ed36e26bbe279fd56f2c4a12510e60b4b", 848 | "sha256:737bd70e454a284d456aa1fa71a0b429dd527bcbf52c5c33f7c8eee81ac16b89", 849 | "sha256:8635d53223b1f561b081ff4adecb828fd484b8efffe542edcfdff471997f7c39", 850 | "sha256:8b818b6c5a920cbe4203b5a6b14256f0e5244338244560da89b7b0f1313ea4b6", 851 | "sha256:8bf38641b4713d77da19e91f8b5296b832e4db87338d6aeffe422d42f1ca896d", 852 | "sha256:a36a48a51e5471513a5aea920cdad84cbd56d70a5057cca3499a637496ea379c", 853 | "sha256:b2243dd033fd02c01212ad5c601dafb44fbb293065f430b0d3dbf03f3254d615", 854 | "sha256:cc547d3ead3754712223abb7b403f0a184e4c3eae18c9bb7fd15adef1597cc4b", 855 | "sha256:cc552b6434b90d9dbed6a4f13339625dc466fd82597119897e9489c953acbc22", 856 | "sha256:f3790156c606299ff499ec44db422f66f05a7363b39eb9d5b064f17bd7d7c47b", 857 | "sha256:f7a21e3d99aa3095ef0553e7ceba36fb693998fbb1226f1392ce33681047465f", 858 | "sha256:fdc6b2cb4b19e431994f25a9160695cc59a4e861710cc6fc97161c5e845fc579" 859 | ], 860 | "index": "pypi", 861 | "version": "==5.4" 862 | }, 863 | "requests": { 864 | "hashes": [ 865 | "sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f", 866 | "sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1" 867 | ], 868 | "markers": "python_version >= '3.7'", 869 | "version": "==2.31.0" 870 | }, 871 | "responses": { 872 | "hashes": [ 873 | "sha256:205029e1cb334c21cb4ec64fc7599be48b859a0fd381a42443cdd600bfe8b16a", 874 | "sha256:e6fbcf5d82172fecc0aa1860fd91e58cbfd96cee5e96da5b63fa6eb3caa10dd3" 875 | ], 876 | "markers": "python_version >= '3.7'", 877 | "version": "==0.23.3" 878 | }, 879 | "s3transfer": { 880 | "hashes": [ 881 | "sha256:6efc926738a3cd576c2a79725fed9afde92378aa5c6a957e3af010cb019fac9d", 882 | "sha256:b780f2411b824cb541dbcd2c713d0cb61c7d1bcadae204cdddda2b35cef493ba" 883 | ], 884 | "version": "==0.2.1" 885 | }, 886 | "six": { 887 | "hashes": [ 888 | "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926", 889 | "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254" 890 | ], 891 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", 892 | "version": "==1.16.0" 893 | }, 894 | "tomli": { 895 | "hashes": [ 896 | "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc", 897 | "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f" 898 | ], 899 | "markers": "python_version < '3.11'", 900 | "version": "==2.0.1" 901 | }, 902 | "types-pyyaml": { 903 | "hashes": [ 904 | "sha256:7d340b19ca28cddfdba438ee638cd4084bde213e501a3978738543e27094775b", 905 | "sha256:a461508f3096d1d5810ec5ab95d7eeecb651f3a15b71959999988942063bf01d" 906 | ], 907 | "version": "==6.0.12.11" 908 | }, 909 | "typing-extensions": { 910 | "hashes": [ 911 | "sha256:440d5dd3af93b060174bf433bccd69b0babc3b15b1a8dca43789fd7f61514b36", 912 | "sha256:b75ddc264f0ba5615db7ba217daeb99701ad295353c45f9e95963337ceeeffb2" 913 | ], 914 | "markers": "python_version < '3.8'", 915 | "version": "==4.7.1" 916 | }, 917 | "urllib3": { 918 | "hashes": [ 919 | "sha256:753a0374df26658f99d826cfe40394a686d05985786d946fbe4165b5148f5a7c", 920 | "sha256:a7acd0977125325f516bda9735fa7142b909a8d01e8b2e4c8108d0984e6e0098" 921 | ], 922 | "index": "pypi", 923 | "version": "==1.26.5" 924 | }, 925 | "websocket-client": { 926 | "hashes": [ 927 | "sha256:c951af98631d24f8df89ab1019fc365f2227c0892f12fd150e935607c79dd0dd", 928 | "sha256:f1f9f2ad5291f0225a49efad77abf9e700b6fef553900623060dad6e26503b9d" 929 | ], 930 | "markers": "python_version >= '3.7'", 931 | "version": "==1.6.1" 932 | }, 933 | "werkzeug": { 934 | "hashes": [ 935 | "sha256:2e1ccc9417d4da358b9de6f174e3ac094391ea1d4fbef2d667865d819dfd0afe", 936 | "sha256:56433961bc1f12533306c624f3be5e744389ac61d722175d543e1751285da612" 937 | ], 938 | "markers": "python_version >= '3.7'", 939 | "version": "==2.2.3" 940 | }, 941 | "wrapt": { 942 | "hashes": [ 943 | "sha256:02fce1852f755f44f95af51f69d22e45080102e9d00258053b79367d07af39c0", 944 | "sha256:077ff0d1f9d9e4ce6476c1a924a3332452c1406e59d90a2cf24aeb29eeac9420", 945 | "sha256:078e2a1a86544e644a68422f881c48b84fef6d18f8c7a957ffd3f2e0a74a0d4a", 946 | "sha256:0970ddb69bba00670e58955f8019bec4a42d1785db3faa043c33d81de2bf843c", 947 | "sha256:1286eb30261894e4c70d124d44b7fd07825340869945c79d05bda53a40caa079", 948 | "sha256:21f6d9a0d5b3a207cdf7acf8e58d7d13d463e639f0c7e01d82cdb671e6cb7923", 949 | "sha256:230ae493696a371f1dbffaad3dafbb742a4d27a0afd2b1aecebe52b740167e7f", 950 | "sha256:26458da5653aa5b3d8dc8b24192f574a58984c749401f98fff994d41d3f08da1", 951 | "sha256:2cf56d0e237280baed46f0b5316661da892565ff58309d4d2ed7dba763d984b8", 952 | "sha256:2e51de54d4fb8fb50d6ee8327f9828306a959ae394d3e01a1ba8b2f937747d86", 953 | "sha256:2fbfbca668dd15b744418265a9607baa970c347eefd0db6a518aaf0cfbd153c0", 954 | "sha256:38adf7198f8f154502883242f9fe7333ab05a5b02de7d83aa2d88ea621f13364", 955 | "sha256:3a8564f283394634a7a7054b7983e47dbf39c07712d7b177b37e03f2467a024e", 956 | "sha256:3abbe948c3cbde2689370a262a8d04e32ec2dd4f27103669a45c6929bcdbfe7c", 957 | "sha256:3bbe623731d03b186b3d6b0d6f51865bf598587c38d6f7b0be2e27414f7f214e", 958 | "sha256:40737a081d7497efea35ab9304b829b857f21558acfc7b3272f908d33b0d9d4c", 959 | "sha256:41d07d029dd4157ae27beab04d22b8e261eddfc6ecd64ff7000b10dc8b3a5727", 960 | "sha256:46ed616d5fb42f98630ed70c3529541408166c22cdfd4540b88d5f21006b0eff", 961 | "sha256:493d389a2b63c88ad56cdc35d0fa5752daac56ca755805b1b0c530f785767d5e", 962 | "sha256:4ff0d20f2e670800d3ed2b220d40984162089a6e2c9646fdb09b85e6f9a8fc29", 963 | "sha256:54accd4b8bc202966bafafd16e69da9d5640ff92389d33d28555c5fd4f25ccb7", 964 | "sha256:56374914b132c702aa9aa9959c550004b8847148f95e1b824772d453ac204a72", 965 | "sha256:578383d740457fa790fdf85e6d346fda1416a40549fe8db08e5e9bd281c6a475", 966 | "sha256:58d7a75d731e8c63614222bcb21dd992b4ab01a399f1f09dd82af17bbfc2368a", 967 | "sha256:5c5aa28df055697d7c37d2099a7bc09f559d5053c3349b1ad0c39000e611d317", 968 | "sha256:5fc8e02f5984a55d2c653f5fea93531e9836abbd84342c1d1e17abc4a15084c2", 969 | "sha256:63424c681923b9f3bfbc5e3205aafe790904053d42ddcc08542181a30a7a51bd", 970 | "sha256:64b1df0f83706b4ef4cfb4fb0e4c2669100fd7ecacfb59e091fad300d4e04640", 971 | "sha256:74934ebd71950e3db69960a7da29204f89624dde411afbfb3b4858c1409b1e98", 972 | "sha256:75669d77bb2c071333417617a235324a1618dba66f82a750362eccbe5b61d248", 973 | "sha256:75760a47c06b5974aa5e01949bf7e66d2af4d08cb8c1d6516af5e39595397f5e", 974 | "sha256:76407ab327158c510f44ded207e2f76b657303e17cb7a572ffe2f5a8a48aa04d", 975 | "sha256:76e9c727a874b4856d11a32fb0b389afc61ce8aaf281ada613713ddeadd1cfec", 976 | "sha256:77d4c1b881076c3ba173484dfa53d3582c1c8ff1f914c6461ab70c8428b796c1", 977 | "sha256:780c82a41dc493b62fc5884fb1d3a3b81106642c5c5c78d6a0d4cbe96d62ba7e", 978 | "sha256:7dc0713bf81287a00516ef43137273b23ee414fe41a3c14be10dd95ed98a2df9", 979 | "sha256:7eebcdbe3677e58dd4c0e03b4f2cfa346ed4049687d839adad68cc38bb559c92", 980 | "sha256:896689fddba4f23ef7c718279e42f8834041a21342d95e56922e1c10c0cc7afb", 981 | "sha256:96177eb5645b1c6985f5c11d03fc2dbda9ad24ec0f3a46dcce91445747e15094", 982 | "sha256:96e25c8603a155559231c19c0349245eeb4ac0096fe3c1d0be5c47e075bd4f46", 983 | "sha256:9d37ac69edc5614b90516807de32d08cb8e7b12260a285ee330955604ed9dd29", 984 | "sha256:9ed6aa0726b9b60911f4aed8ec5b8dd7bf3491476015819f56473ffaef8959bd", 985 | "sha256:a487f72a25904e2b4bbc0817ce7a8de94363bd7e79890510174da9d901c38705", 986 | "sha256:a4cbb9ff5795cd66f0066bdf5947f170f5d63a9274f99bdbca02fd973adcf2a8", 987 | "sha256:a74d56552ddbde46c246b5b89199cb3fd182f9c346c784e1a93e4dc3f5ec9975", 988 | "sha256:a89ce3fd220ff144bd9d54da333ec0de0399b52c9ac3d2ce34b569cf1a5748fb", 989 | "sha256:abd52a09d03adf9c763d706df707c343293d5d106aea53483e0ec8d9e310ad5e", 990 | "sha256:abd8f36c99512755b8456047b7be10372fca271bf1467a1caa88db991e7c421b", 991 | "sha256:af5bd9ccb188f6a5fdda9f1f09d9f4c86cc8a539bd48a0bfdc97723970348418", 992 | "sha256:b02f21c1e2074943312d03d243ac4388319f2456576b2c6023041c4d57cd7019", 993 | "sha256:b06fa97478a5f478fb05e1980980a7cdf2712015493b44d0c87606c1513ed5b1", 994 | "sha256:b0724f05c396b0a4c36a3226c31648385deb6a65d8992644c12a4963c70326ba", 995 | "sha256:b130fe77361d6771ecf5a219d8e0817d61b236b7d8b37cc045172e574ed219e6", 996 | "sha256:b56d5519e470d3f2fe4aa7585f0632b060d532d0696c5bdfb5e8319e1d0f69a2", 997 | "sha256:b67b819628e3b748fd3c2192c15fb951f549d0f47c0449af0764d7647302fda3", 998 | "sha256:ba1711cda2d30634a7e452fc79eabcadaffedf241ff206db2ee93dd2c89a60e7", 999 | "sha256:bbeccb1aa40ab88cd29e6c7d8585582c99548f55f9b2581dfc5ba68c59a85752", 1000 | "sha256:bd84395aab8e4d36263cd1b9308cd504f6cf713b7d6d3ce25ea55670baec5416", 1001 | "sha256:c99f4309f5145b93eca6e35ac1a988f0dc0a7ccf9ccdcd78d3c0adf57224e62f", 1002 | "sha256:ca1cccf838cd28d5a0883b342474c630ac48cac5df0ee6eacc9c7290f76b11c1", 1003 | "sha256:cd525e0e52a5ff16653a3fc9e3dd827981917d34996600bbc34c05d048ca35cc", 1004 | "sha256:cdb4f085756c96a3af04e6eca7f08b1345e94b53af8921b25c72f096e704e145", 1005 | "sha256:ce42618f67741d4697684e501ef02f29e758a123aa2d669e2d964ff734ee00ee", 1006 | "sha256:d06730c6aed78cee4126234cf2d071e01b44b915e725a6cb439a879ec9754a3a", 1007 | "sha256:d5fe3e099cf07d0fb5a1e23d399e5d4d1ca3e6dfcbe5c8570ccff3e9208274f7", 1008 | "sha256:d6bcbfc99f55655c3d93feb7ef3800bd5bbe963a755687cbf1f490a71fb7794b", 1009 | "sha256:d787272ed958a05b2c86311d3a4135d3c2aeea4fc655705f074130aa57d71653", 1010 | "sha256:e169e957c33576f47e21864cf3fc9ff47c223a4ebca8960079b8bd36cb014fd0", 1011 | "sha256:e20076a211cd6f9b44a6be58f7eeafa7ab5720eb796975d0c03f05b47d89eb90", 1012 | "sha256:e826aadda3cae59295b95343db8f3d965fb31059da7de01ee8d1c40a60398b29", 1013 | "sha256:eef4d64c650f33347c1f9266fa5ae001440b232ad9b98f1f43dfe7a79435c0a6", 1014 | "sha256:f2e69b3ed24544b0d3dbe2c5c0ba5153ce50dcebb576fdc4696d52aa22db6034", 1015 | "sha256:f87ec75864c37c4c6cb908d282e1969e79763e0d9becdfe9fe5473b7bb1e5f09", 1016 | "sha256:fbec11614dba0424ca72f4e8ba3c420dba07b4a7c206c8c8e4e73f2e98f4c559", 1017 | "sha256:fd69666217b62fa5d7c6aa88e507493a34dec4fa20c5bd925e4bc12fce586639" 1018 | ], 1019 | "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", 1020 | "version": "==1.15.0" 1021 | }, 1022 | "xmltodict": { 1023 | "hashes": [ 1024 | "sha256:341595a488e3e01a85a9d8911d8912fd922ede5fecc4dce437eb4b6c8d037e56", 1025 | "sha256:aa89e8fd76320154a40d19a0df04a4695fb9dc5ba977cbb68ab3e4eb225e7852" 1026 | ], 1027 | "markers": "python_version >= '3.4'", 1028 | "version": "==0.13.0" 1029 | }, 1030 | "zipp": { 1031 | "hashes": [ 1032 | "sha256:112929ad649da941c23de50f356a2b5570c954b65150642bccdd66bf194d224b", 1033 | "sha256:48904fc76a60e542af151aded95726c1a5c34ed43ab4134b597665c86d7ad556" 1034 | ], 1035 | "markers": "python_version >= '3.7'", 1036 | "version": "==3.15.0" 1037 | } 1038 | } 1039 | } 1040 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Build Status](https://travis-ci.org/aws-samples/amazon-k8s-node-drainer.svg?branch=master)](https://travis-ci.org/aws-samples/amazon-k8s-node-drainer) 2 | 3 | > *Note* This repository is archived and this code is not maintained anymore. We recommend using the [Karpenter](https://karpenter.sh/) tool for the functionality this repo provided. 4 | 5 | # Amazon EKS Node Drainer [DEPRECATED] 6 | 7 | This sample code provides a means to gracefully terminate nodes of an Amazon Elastic Container Service for Kubernetes 8 | (Amazon EKS) cluster when managed as part of an Amazon EC2 Auto Scaling Group. 9 | 10 | The code provides an AWS Lambda function that integrates as an [Amazon EC2 Auto 11 | Scaling Lifecycle Hook](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html). 12 | When called, the Lambda function calls the Kubernetes API to cordon and evict all evictable pods from the node being 13 | terminated. It will then wait until all pods have been evicted before the Auto Scaling group continues to terminate the 14 | EC2 instance. The lambda may be killed by the function timeout before all evictions complete successfully, in which case 15 | the lifecycle hook may re-execute the lambda to try again. If the lifecycle heartbeat expires then termination of the EC2 16 | instance will continue regardless of whether or not draining was successful. You may need to increase the function and 17 | heartbeat timeouts in template.yaml if you have very long grace periods. 18 | 19 | Using this approach can minimise disruption to the services running in your cluster by allowing Kubernetes to 20 | reschedule the pod prior to the instance being terminated enters the TERMINATING state. It works by using 21 | [Amazon EC2 Auto Scaling Lifecycle Hooks](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html) 22 | to trigger an AWS Lambda function that uses the Kubernetes API to cordon the node and evict the pods. 23 | 24 | NB: The lambda function created assumes that the Amazon EKS cluster's Kubernetes API server endpoint has public access 25 | enabled, if your endpoint only has private access enabled then you must modify the `template.yml` file to ensure the 26 | lambda function is running in the correct VPC and subnet. 27 | 28 | This lambda can also be used against a non-EKS Kubernetes cluster by reading a `kubeconfig` file from an S3 bucket 29 | specified by the `KUBE_CONFIG_BUCKET` and `KUBE_CONFIG_OBJECT` environment variables. If these two variables are passed 30 | in then Drainer function will assume this is a non-EKS cluster and the IAM authenticator signatures will _not_ be added 31 | to Kubernetes API requests. It is recommended to apply the principle of least privilege to the IAM role that governs 32 | access between the Lambda function and S3 bucket. 33 | -------------------------------------------------------------------------------- /build_deploy.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -o nounset 3 | set -o errexit 4 | BUCKET=$1 5 | ASG=$2 6 | CLUSTER=$3 7 | PROFILE=${4:-default} 8 | 9 | sam build --use-container --skip-pull-image --profile ${PROFILE} 10 | sam package --s3-bucket ${BUCKET} --output-template-file packaged.yaml --profile ${PROFILE} 11 | sam deploy --template-file packaged.yaml --stack-name k8s-drainer-${CLUSTER} --capabilities CAPABILITY_IAM --profile ${PROFILE} --parameter-overrides AutoScalingGroup=${ASG} EksCluster=${CLUSTER} 12 | -------------------------------------------------------------------------------- /drainer/__init__.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | 4 | sys.path.append(os.path.dirname(os.path.realpath(__file__))) 5 | -------------------------------------------------------------------------------- /drainer/handler.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import base64 3 | import logging 4 | import os.path 5 | import re 6 | import yaml 7 | 8 | from botocore.signers import RequestSigner 9 | import kubernetes as k8s 10 | from kubernetes.client.rest import ApiException 11 | 12 | from k8s_utils import (abandon_lifecycle_action, cordon_node, node_exists, remove_all_pods) 13 | 14 | logger = logging.getLogger(__name__) 15 | logger.setLevel(logging.DEBUG) 16 | 17 | KUBE_FILEPATH = '/tmp/kubeconfig' 18 | REGION = os.environ['AWS_REGION'] 19 | 20 | eks = boto3.client('eks', region_name=REGION) 21 | ec2 = boto3.client('ec2', region_name=REGION) 22 | asg = boto3.client('autoscaling', region_name=REGION) 23 | s3 = boto3.client('s3', region_name=REGION) 24 | 25 | 26 | def create_kube_config(eks, cluster_name): 27 | """Creates the Kubernetes config file required when instantiating the API client.""" 28 | cluster_info = eks.describe_cluster(name=cluster_name)['cluster'] 29 | certificate = cluster_info['certificateAuthority']['data'] 30 | endpoint = cluster_info['endpoint'] 31 | 32 | kube_config = { 33 | 'apiVersion': 'v1', 34 | 'clusters': [ 35 | { 36 | 'cluster': 37 | { 38 | 'server': endpoint, 39 | 'certificate-authority-data': certificate 40 | }, 41 | 'name': 'k8s' 42 | 43 | }], 44 | 'contexts': [ 45 | { 46 | 'context': 47 | { 48 | 'cluster': 'k8s', 49 | 'user': 'aws' 50 | }, 51 | 'name': 'aws' 52 | }], 53 | 'current-context': 'aws', 54 | 'Kind': 'config', 55 | 'users': [ 56 | { 57 | 'name': 'aws', 58 | 'user': 'lambda' 59 | }] 60 | } 61 | 62 | with open(KUBE_FILEPATH, 'w') as f: 63 | yaml.dump(kube_config, f, default_flow_style=False) 64 | 65 | 66 | def get_bearer_token(cluster, region): 67 | """Creates the authentication to token required by AWS IAM Authenticator. This is 68 | done by creating a base64 encoded string which represents a HTTP call to the STS 69 | GetCallerIdentity Query Request (https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html). 70 | The AWS IAM Authenticator decodes the base64 string and makes the request on behalf of the user. 71 | """ 72 | STS_TOKEN_EXPIRES_IN = 60 73 | session = boto3.session.Session() 74 | 75 | client = session.client('sts', region_name=region) 76 | service_id = client.meta.service_model.service_id 77 | 78 | signer = RequestSigner( 79 | service_id, 80 | region, 81 | 'sts', 82 | 'v4', 83 | session.get_credentials(), 84 | session.events 85 | ) 86 | 87 | params = { 88 | 'method': 'GET', 89 | 'url': 'https://sts.{}.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15'.format(region), 90 | 'body': {}, 91 | 'headers': { 92 | 'x-k8s-aws-id': cluster 93 | }, 94 | 'context': {} 95 | } 96 | 97 | signed_url = signer.generate_presigned_url( 98 | params, 99 | region_name=region, 100 | expires_in=STS_TOKEN_EXPIRES_IN, 101 | operation_name='' 102 | ) 103 | 104 | base64_url = base64.urlsafe_b64encode(signed_url.encode('utf-8')).decode('utf-8') 105 | 106 | # need to remove base64 encoding padding: 107 | # https://github.com/kubernetes-sigs/aws-iam-authenticator/issues/202 108 | return 'k8s-aws-v1.' + re.sub(r'=*', '', base64_url) 109 | 110 | 111 | def _lambda_handler(env, k8s_config, k8s_client, event): 112 | kube_config_bucket = env['kube_config_bucket'] 113 | cluster_name = env['cluster_name'] 114 | 115 | if not os.path.exists(KUBE_FILEPATH): 116 | if kube_config_bucket: 117 | logger.info('No kubeconfig file found. Downloading...') 118 | s3.download_file(kube_config_bucket, env['kube_config_object'], KUBE_FILEPATH) 119 | else: 120 | logger.info('No kubeconfig file found. Generating...') 121 | create_kube_config(eks, cluster_name) 122 | 123 | lifecycle_hook_name = event['detail']['LifecycleHookName'] 124 | auto_scaling_group_name = event['detail']['AutoScalingGroupName'] 125 | 126 | instance_id = event['detail']['EC2InstanceId'] 127 | logger.info('Instance ID: ' + instance_id) 128 | instance = ec2.describe_instances(InstanceIds=[instance_id])['Reservations'][0]['Instances'][0] 129 | 130 | node_name = instance['PrivateDnsName'] 131 | logger.info('Node name: ' + node_name) 132 | 133 | # Configure 134 | k8s_config.load_kube_config(KUBE_FILEPATH) 135 | configuration = k8s_client.Configuration() 136 | if not kube_config_bucket: 137 | configuration.api_key['authorization'] = get_bearer_token(cluster_name, REGION) 138 | configuration.api_key_prefix['authorization'] = 'Bearer' 139 | # API 140 | api = k8s_client.ApiClient(configuration) 141 | v1 = k8s_client.CoreV1Api(api) 142 | 143 | try: 144 | if not node_exists(v1, node_name): 145 | logger.error('Node not found.') 146 | abandon_lifecycle_action(asg, auto_scaling_group_name, lifecycle_hook_name, instance_id) 147 | return 148 | 149 | cordon_node(v1, node_name) 150 | 151 | remove_all_pods(v1, node_name) 152 | 153 | asg.complete_lifecycle_action(LifecycleHookName=lifecycle_hook_name, 154 | AutoScalingGroupName=auto_scaling_group_name, 155 | LifecycleActionResult='CONTINUE', 156 | InstanceId=instance_id) 157 | except ApiException: 158 | logger.exception('There was an error removing the pods from the node {}'.format(node_name)) 159 | abandon_lifecycle_action(asg, auto_scaling_group_name, lifecycle_hook_name, instance_id) 160 | 161 | 162 | def lambda_handler(event, _): 163 | env = { 164 | 'cluster_name': os.environ.get('CLUSTER_NAME'), 165 | 'kube_config_bucket': os.environ.get('KUBE_CONFIG_BUCKET'), 166 | 'kube_config_object': os.environ.get('KUBE_CONFIG_OBJECT') 167 | } 168 | return _lambda_handler(env, k8s.config, k8s.client, event) 169 | -------------------------------------------------------------------------------- /drainer/k8s_utils.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import time 3 | 4 | from kubernetes.client.rest import ApiException 5 | 6 | logger = logging.getLogger(__name__) 7 | logger.setLevel(logging.DEBUG) 8 | 9 | MIRROR_POD_ANNOTATION_KEY = "kubernetes.io/config.mirror" 10 | CONTROLLER_KIND_DAEMON_SET = "DaemonSet" 11 | 12 | 13 | def cordon_node(api, node_name): 14 | """Marks the specified node as unschedulable, which means that no new pods can be launched on the 15 | node by the Kubernetes scheduler. 16 | """ 17 | patch_body = { 18 | 'apiVersion': 'v1', 19 | 'kind': 'Node', 20 | 'metadata': { 21 | 'name': node_name 22 | }, 23 | 'spec': { 24 | 'unschedulable': True 25 | } 26 | } 27 | 28 | api.patch_node(node_name, patch_body) 29 | 30 | 31 | def remove_all_pods(api, node_name, poll=5): 32 | """Removes all Kubernetes pods from the specified node.""" 33 | pods = get_evictable_pods(api, node_name) 34 | 35 | logger.debug('Number of pods to delete: ' + str(len(pods))) 36 | 37 | evict_until_completed(api, pods, poll) 38 | wait_until_empty(api, node_name, poll) 39 | 40 | 41 | def pod_is_evictable(pod): 42 | if pod.metadata.annotations is not None and pod.metadata.annotations.get(MIRROR_POD_ANNOTATION_KEY): 43 | logger.info("Skipping mirror pod {}/{}".format(pod.metadata.namespace, pod.metadata.name)) 44 | return False 45 | if pod.metadata.owner_references is None: 46 | return True 47 | for ref in pod.metadata.owner_references: 48 | if ref.controller is not None and ref.controller: 49 | if ref.kind == CONTROLLER_KIND_DAEMON_SET: 50 | logger.info("Skipping DaemonSet {}/{}".format(pod.metadata.namespace, pod.metadata.name)) 51 | return False 52 | return True 53 | 54 | 55 | def get_evictable_pods(api, node_name): 56 | field_selector = 'spec.nodeName=' + node_name 57 | pods = api.list_pod_for_all_namespaces(watch=False, field_selector=field_selector, include_uninitialized=True) 58 | return [pod for pod in pods.items if pod_is_evictable(pod)] 59 | 60 | 61 | def evict_until_completed(api, pods, poll): 62 | pending = pods 63 | while True: 64 | pending = evict_pods(api, pending) 65 | if (len(pending)) <= 0: 66 | return 67 | time.sleep(poll) 68 | 69 | 70 | def evict_pods(api, pods): 71 | remaining = [] 72 | for pod in pods: 73 | logger.info('Evicting pod {} in namespace {}'.format(pod.metadata.name, pod.metadata.namespace)) 74 | body = { 75 | 'apiVersion': 'policy/v1beta1', 76 | 'kind': 'Eviction', 77 | 'deleteOptions': {}, 78 | 'metadata': { 79 | 'name': pod.metadata.name, 80 | 'namespace': pod.metadata.namespace 81 | } 82 | } 83 | try: 84 | api.create_namespaced_pod_eviction(pod.metadata.name, pod.metadata.namespace, body) 85 | except ApiException as err: 86 | if err.status == 429: 87 | remaining.append(pod) 88 | logger.warning("Pod {}/{} could not be evicted due to disruption budget. Will retry.".format(pod.metadata.namespace, pod.metadata.name)) 89 | else: 90 | logger.exception("Unexpected error adding eviction for pod {}/{}".format(pod.metadata.namespace, pod.metadata.name)) 91 | except: 92 | logger.exception("Unexpected error adding eviction for pod {}/{}".format(pod.metadata.namespace, pod.metadata.name)) 93 | return remaining 94 | 95 | 96 | def wait_until_empty(api, node_name, poll): 97 | logger.info("Waiting for evictions to complete") 98 | while True: 99 | pods = get_evictable_pods(api, node_name) 100 | if len(pods) <= 0: 101 | logger.info("All pods evicted successfully") 102 | return 103 | logger.debug("Still waiting for deletion of the following pods: {}".format(", ".join(map(lambda pod: pod.metadata.namespace + "/" + pod.metadata.name, pods)))) 104 | time.sleep(poll) 105 | 106 | 107 | def node_exists(api, node_name): 108 | """Determines whether the specified node is still part of the cluster.""" 109 | nodes = api.list_node(include_uninitialized=True, pretty=True).items 110 | node = next((n for n in nodes if n.metadata.name == node_name), None) 111 | return False if not node else True 112 | 113 | 114 | def abandon_lifecycle_action(asg_client, auto_scaling_group_name, lifecycle_hook_name, instance_id): 115 | """Completes the lifecycle action with the ABANDON result, which stops any remaining actions, 116 | such as other lifecycle hooks. 117 | """ 118 | asg_client.complete_lifecycle_action(LifecycleHookName=lifecycle_hook_name, 119 | AutoScalingGroupName=auto_scaling_group_name, 120 | LifecycleActionResult='ABANDON', 121 | InstanceId=instance_id) 122 | -------------------------------------------------------------------------------- /drainer/requirements.txt: -------------------------------------------------------------------------------- 1 | kubernetes==9.0.0 2 | -------------------------------------------------------------------------------- /k8s_rbac/cluster_role.yaml: -------------------------------------------------------------------------------- 1 | kind: ClusterRole 2 | apiVersion: rbac.authorization.k8s.io/v1 3 | metadata: 4 | name: lambda-cluster-access 5 | rules: 6 | - apiGroups: [""] 7 | resources: ["pods", "pods/eviction", "nodes"] 8 | verbs: ["create", "list", "patch"] 9 | -------------------------------------------------------------------------------- /k8s_rbac/cluster_rolebinding.yaml: -------------------------------------------------------------------------------- 1 | kind: ClusterRoleBinding 2 | apiVersion: rbac.authorization.k8s.io/v1 3 | metadata: 4 | name: lambda-user-cluster-role-binding 5 | subjects: 6 | - kind: User 7 | name: lambda 8 | apiGroup: rbac.authorization.k8s.io 9 | roleRef: 10 | kind: ClusterRole 11 | name: lambda-cluster-access 12 | apiGroup: rbac.authorization.k8s.io 13 | -------------------------------------------------------------------------------- /template.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Transform: AWS::Serverless-2016-10-31 3 | Description: Gracefully drain k8s nodes when instances are terminated (uksb-1pf6fjp67) 4 | 5 | Parameters: 6 | 7 | AutoScalingGroup: 8 | Type: String 9 | 10 | EksCluster: 11 | Type: String 12 | 13 | Globals: 14 | Function: 15 | Timeout: 300 16 | 17 | Resources: 18 | 19 | LifecycleHook: 20 | Type: AWS::AutoScaling::LifecycleHook 21 | Properties: 22 | AutoScalingGroupName: !Ref AutoScalingGroup 23 | HeartbeatTimeout: 450 24 | LifecycleTransition: autoscaling:EC2_INSTANCE_TERMINATING 25 | 26 | DrainerRole: 27 | Type: AWS::IAM::Role 28 | Properties: 29 | AssumeRolePolicyDocument: 30 | Version: 2012-10-17 31 | Statement: 32 | - Effect: Allow 33 | Principal: 34 | Service: 35 | - lambda.amazonaws.com 36 | Action: 37 | - sts:AssumeRole 38 | Path: / 39 | Policies: 40 | - PolicyName: DrainerPolicies 41 | PolicyDocument: 42 | Version: 2012-10-17 43 | Statement: 44 | - Effect: Allow 45 | Action: 46 | - autoscaling:CompleteLifecycleAction 47 | - ec2:DescribeInstances 48 | - eks:DescribeCluster 49 | - sts:GetCallerIdentity 50 | Resource: '*' 51 | ManagedPolicyArns: 52 | - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 53 | 54 | DrainerFunction: 55 | Type: AWS::Serverless::Function 56 | Properties: 57 | CodeUri: drainer/ 58 | Handler: handler.lambda_handler 59 | Runtime: python3.7 60 | MemorySize: 256 61 | Environment: 62 | Variables: 63 | CLUSTER_NAME: !Ref EksCluster 64 | Role: !GetAtt DrainerRole.Arn 65 | Events: 66 | TerminationEvent: 67 | Type: CloudWatchEvent 68 | Properties: 69 | Pattern: 70 | source: 71 | - aws.autoscaling 72 | detail-type: 73 | - EC2 Instance-terminate Lifecycle Action 74 | detail: 75 | AutoScalingGroupName: 76 | - !Ref AutoScalingGroup 77 | 78 | Permission: 79 | Type: AWS::Lambda::Permission 80 | Properties: 81 | Action: lambda:InvokeFunction 82 | FunctionName: !GetAtt DrainerFunction.Arn 83 | Principal: events.amazonaws.com 84 | 85 | Outputs: 86 | 87 | DrainerRole: 88 | Description: Draining function role ARN 89 | Value: !GetAtt DrainerRole.Arn 90 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-k8s-node-drainer/45c61b8d534f2d267bc321760f6b3bae2c0f6b0a/tests/__init__.py -------------------------------------------------------------------------------- /tests/drainer/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-k8s-node-drainer/45c61b8d534f2d267bc321760f6b3bae2c0f6b0a/tests/drainer/__init__.py -------------------------------------------------------------------------------- /tests/drainer/fixtures/event.json: -------------------------------------------------------------------------------- 1 | { 2 | "version": "0", 3 | "id": "f81844ff-0310-5dec-b611-79db462cb4e5", 4 | "detail-type": "EC2 Instance-terminate Lifecycle Action", 5 | "source": "aws.autoscaling", 6 | "account": "060997615305", 7 | "time": "2019-02-25T20:54:45Z", 8 | "region": "eu-west-1", 9 | "resources": [ 10 | "arn:aws:autoscaling:eu-west-1:060997615305:autoScalingGroup:4b85bc83-d023-4b05-9006-6d7ed039c272:autoScalingGroupName/k8s-worker-nodes1-dev-NodeGroup-1OFOSHBK0L83J" 11 | ], 12 | "detail": { 13 | "LifecycleActionToken": "86d408c1-adff-4460-9fae-cf0fb852196e", 14 | "AutoScalingGroupName": "k8s-worker-nodes-dev-NodeGroup-F49231EK31OA", 15 | "LifecycleHookName": "k8s-drainer-LifecycleHook-DDXJNVV0KBG1", 16 | "EC2InstanceId": "i-036e525e159f62a5d", 17 | "LifecycleTransition": "autoscaling:EC2_INSTANCE_TERMINATING" 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /tests/drainer/fixtures/kube_config.yaml: -------------------------------------------------------------------------------- 1 | Kind: config 2 | apiVersion: v1 3 | clusters: 4 | - cluster: 5 | certificate-authority-data: 84586abd904ef 6 | server: https://test-cluster.amazonaws.com 7 | name: k8s 8 | contexts: 9 | - context: 10 | cluster: k8s 11 | user: aws 12 | name: aws 13 | current-context: aws 14 | users: 15 | - name: aws 16 | user: lambda 17 | -------------------------------------------------------------------------------- /tests/drainer/test_handler.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | 4 | import boto3 5 | import pytest 6 | from freezegun import freeze_time 7 | from kubernetes.client.rest import ApiException 8 | 9 | from tests.utils import dict_to_simple_namespace 10 | 11 | 12 | @pytest.fixture() 13 | def handler(monkeypatch): 14 | monkeypatch.setenv('AWS_REGION', 'eu-west-1') 15 | import drainer.handler as handler 16 | return handler 17 | 18 | 19 | @pytest.fixture() 20 | def mock_eks(mocker): 21 | return mocker.Mock(**{'describe_cluster.return_value': {'cluster': { 22 | 'certificateAuthority': { 23 | 'data': '84586abd904ef' 24 | }, 25 | 'endpoint': 'https://test-cluster.amazonaws.com' 26 | }}}) 27 | 28 | 29 | @pytest.fixture() 30 | def mock_k8s_client_no_nodes(mocker): 31 | 32 | list_node_val = dict_to_simple_namespace({'items': []}) 33 | 34 | mock_api = mocker.Mock(**{'list_node.return_value': list_node_val}) 35 | 36 | class Configuration: 37 | 38 | def __init__(self): 39 | self.api_key = {} 40 | self.api_key_prefix = {} 41 | 42 | return mocker.Mock(**{'CoreV1Api.return_value': mock_api, 43 | 'Configuration.return_value': Configuration()}) 44 | 45 | 46 | @pytest.fixture() 47 | def mock_k8s_client(mocker): 48 | list_pods_val = dict_to_simple_namespace({'items': [ 49 | {'metadata': { 50 | 'uid': 'aaa', 51 | 'name': 'test_pod1', 52 | 'namespace': 'test_ns', 53 | 'annotations': None, 54 | 'owner_references': None 55 | } 56 | }, {'metadata': { 57 | 'uid': 'bbb', 58 | 'name': 'test_pod2', 59 | 'namespace': 'test_ns', 60 | 'annotations': None, 61 | 'owner_references': None 62 | } 63 | } 64 | ]}) 65 | empty_list_pods_val = dict_to_simple_namespace({'items': []}) 66 | 67 | list_node_val = dict_to_simple_namespace({'items': [{'metadata': {'name': 'test_node'}}]}) 68 | 69 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': [list_pods_val, empty_list_pods_val], 70 | 'list_node.return_value': list_node_val, 71 | 'patch_node.return_value': mocker.Mock()} 72 | ) 73 | 74 | class Configuration: 75 | 76 | def __init__(self): 77 | self.api_key = {} 78 | self.api_key_prefix = {} 79 | 80 | return mocker.Mock(**{'CoreV1Api.return_value': mock_api, 81 | 'Configuration.return_value': Configuration()}) 82 | 83 | 84 | @pytest.fixture() 85 | def mock_k8s_client_patch_exception(mocker): 86 | list_node_val = dict_to_simple_namespace({'items': [{'metadata': {'name': 'test_node'}}]}) 87 | 88 | mock_api = mocker.Mock(**{'list_node.return_value': list_node_val, 89 | 'patch_node.side_effect': ApiException()}) 90 | 91 | class Configuration: 92 | 93 | def __init__(self): 94 | self.api_key = {} 95 | self.api_key_prefix = {} 96 | 97 | return mocker.Mock(**{'CoreV1Api.return_value': mock_api, 98 | 'Configuration.return_value': Configuration()}) 99 | 100 | 101 | @pytest.fixture() 102 | def mock_k8s_client_pods_exception(mocker): 103 | list_node_val = dict_to_simple_namespace({'items': [{'metadata': {'name': 'test_node'}}]}) 104 | 105 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': ApiException(), 106 | 'list_node.return_value': list_node_val, 107 | 'patch_node.return_value': mocker.Mock()}) 108 | 109 | class Configuration: 110 | 111 | def __init__(self): 112 | self.api_key = {} 113 | self.api_key_prefix = {} 114 | 115 | return mocker.Mock(**{'CoreV1Api.return_value': mock_api, 116 | 'Configuration.return_value': Configuration()}) 117 | 118 | 119 | @pytest.fixture() 120 | def mock_ec2(mocker): 121 | describe_instances_resp = { 122 | 'Reservations': [ 123 | {'Instances': [ 124 | {'PrivateDnsName': 'test_node'} 125 | ]} 126 | ] 127 | } 128 | 129 | return mocker.Mock(**{'describe_instances.return_value': describe_instances_resp}) 130 | 131 | 132 | @pytest.fixture() 133 | def fake_eks_env(): 134 | return { 135 | 'kube_config_bucket': None, 136 | 'kube_config_object': None, 137 | 'cluster_name': 'test-cluster' 138 | } 139 | 140 | @pytest.fixture() 141 | def fake_noneks_env(): 142 | return { 143 | 'kube_config_bucket': 'bucket', 144 | 'kube_config_object': 'object', 145 | 'cluster_name': 'test-cluster', 146 | } 147 | 148 | 149 | @pytest.fixture() 150 | def patched_handler(fs, monkeypatch, mocker, mock_eks, mock_ec2): 151 | monkeypatch.setenv('AWS_REGION', 'eu-west-1') 152 | 153 | # pyfakefs always initialises a temp dir, on Mac it is /var on Linux it is /tmp 154 | # https://github.com/jmcgeheeiv/pyfakefs/issues/329 155 | if not os.path.exists('/tmp'): 156 | fs.create_dir('/tmp') 157 | 158 | # boto3 won't work if it can't write to this directory 159 | boto_dir = os.path.abspath(os.path.join(os.path.dirname(boto3.__file__), "..")) 160 | fs.add_real_directory(boto_dir) 161 | 162 | import drainer.handler as handler 163 | 164 | monkeypatch.setattr(handler, 's3', mocker.Mock()) 165 | monkeypatch.setattr(handler, 'asg', mocker.Mock()) 166 | monkeypatch.setattr(handler, 'ec2', mock_ec2) 167 | monkeypatch.setattr(handler, 'eks', mock_eks) 168 | 169 | return handler 170 | 171 | 172 | @pytest.fixture() 173 | def patched_main_handler(mocker, mock_eks, mock_ec2, monkeypatch): 174 | monkeypatch.setenv('CLUSTER_NAME', 'test-cluster') 175 | monkeypatch.setenv('KUBE_CONFIG_BUCKET', 'bucket') 176 | monkeypatch.setenv('KUBE_CONFIG_OBJECT', 'object') 177 | 178 | import drainer.handler as handler 179 | 180 | monkeypatch.setattr(handler, '_lambda_handler', mocker.Mock()) 181 | 182 | return handler 183 | 184 | 185 | @pytest.fixture() 186 | def mock_event(fs): 187 | event_json = os.path.join(os.path.dirname(__file__), 'fixtures/event.json') 188 | fs.add_real_file(event_json) 189 | with open(event_json, 'r') as event: 190 | return json.loads(event.read()) 191 | 192 | 193 | @freeze_time("2011-06-21 18:40:00") 194 | def test_get_bearer_token(handler, monkeypatch): 195 | monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'fake') 196 | monkeypatch.setenv('AWS_SECRET_ACCESS_KEY', 'fake') 197 | 198 | expected = 'k8s-aws-v1.aHR0cHM6Ly9zdHMuZXUtd2VzdC0xLmFtYXpvbmF3cy5jb20vP0' \ 199 | 'FjdGlvbj1HZXRDYWxsZXJJZGVudGl0eSZWZXJzaW9uPTIwMTEtMDYtMTUmWC1' \ 200 | 'BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlh' \ 201 | 'bD1mYWtlJTJGMjAxMTA2MjElMkZldS13ZXN0LTElMkZzdHMlMkZhd3M0X3Jlc' \ 202 | 'XVlc3QmWC1BbXotRGF0ZT0yMDExMDYyMVQxODQwMDBaJlgtQW16LUV4cGlyZX' \ 203 | 'M9NjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JTNCeC1rOHMtYXdzLWlkJlg' \ 204 | 'tQW16LVNpZ25hdHVyZT1mYWUxNDA5NDA2OGMzYWUzNmE1MTI3NzY3ZmIwMzE4' \ 205 | 'ZmE5ZjhjZmJjNzJmMTg2N2I2ZDY4MGY3OTc1Y2I5YTcw' 206 | 207 | actual = handler.get_bearer_token('test-cluster', 'eu-west-1') 208 | assert expected == actual 209 | 210 | 211 | def test_create_kube_config(handler, fs, mock_eks): 212 | # pyfakefs always initialises a temp dir, on Mac it is /var on Linux it is /tmp 213 | # https://github.com/jmcgeheeiv/pyfakefs/issues/329 214 | if not os.path.exists('/tmp'): 215 | fs.create_dir('/tmp') 216 | 217 | kube_config_loc = os.path.join(os.path.dirname(__file__), 'fixtures/kube_config.yaml') 218 | fs.add_real_file(kube_config_loc) 219 | 220 | handler.create_kube_config(mock_eks, 'test-cluster') 221 | 222 | assert os.path.exists('/tmp/kubeconfig') is True 223 | 224 | with open('/tmp/kubeconfig', 'r') as actual, open(kube_config_loc, 'r') as expected: 225 | assert actual.read() == expected.read() 226 | 227 | 228 | @freeze_time("2014-04-09 01:30:00") 229 | def test_handler_eks(mocker, monkeypatch, patched_handler, fake_eks_env, mock_k8s_client, mock_event): 230 | monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'fake') 231 | monkeypatch.setenv('AWS_SECRET_ACCESS_KEY', 'fake') 232 | 233 | patched_handler._lambda_handler(fake_eks_env, mocker.Mock(), mock_k8s_client, mock_event) 234 | 235 | bearer_token = 'k8s-aws-v1.aHR0cHM6Ly9zdHMuZXUtd2VzdC0xLmFtYXpvbmF3cy5jb20vP' \ 236 | '0FjdGlvbj1HZXRDYWxsZXJJZGVudGl0eSZWZXJzaW9uPTIwMTEtMDYtMTUmW' \ 237 | 'C1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVud' \ 238 | 'GlhbD1mYWtlJTJGMjAxNDA0MDklMkZldS13ZXN0LTElMkZzdHMlMkZhd3M0X' \ 239 | '3JlcXVlc3QmWC1BbXotRGF0ZT0yMDE0MDQwOVQwMTMwMDBaJlgtQW16LUV4c' \ 240 | 'GlyZXM9NjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JTNCeC1rOHMtYXdzL' \ 241 | 'WlkJlgtQW16LVNpZ25hdHVyZT04OGY5OWViNjc1YjAwNDU4MTYxNDNhMmQ5Y' \ 242 | '2I3YjhhZjE2Y2QyNWZkN2I1Nzg5NDFlMjczNzNhNzU3NTRjM2Ex' 243 | 244 | assert mock_k8s_client.Configuration.return_value.api_key['authorization'] == bearer_token 245 | assert mock_k8s_client.Configuration.return_value.api_key_prefix['authorization'] == 'Bearer' 246 | 247 | mock_k8s_client.CoreV1Api.return_value.patch_node.assert_called_with('test_node', mocker.ANY) 248 | mock_k8s_client.CoreV1Api.return_value.list_pod_for_all_namespaces.assert_called_with(watch=False, 249 | include_uninitialized=True, 250 | field_selector='spec.nodeName=test_node') 251 | assert mock_k8s_client.CoreV1Api.return_value.create_namespaced_pod_eviction.call_count == 2 252 | 253 | patched_handler.asg.complete_lifecycle_action.assert_called_with(LifecycleHookName='k8s-drainer-LifecycleHook-DDXJNVV0KBG1', 254 | AutoScalingGroupName='k8s-worker-nodes-dev-NodeGroup-F49231EK31OA', 255 | LifecycleActionResult='CONTINUE', 256 | InstanceId='i-036e525e159f62a5d') 257 | 258 | 259 | def test_handler_noneks(mocker, monkeypatch, patched_handler, fake_noneks_env, mock_k8s_client, mock_event): 260 | monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'fake') 261 | monkeypatch.setenv('AWS_SECRET_ACCESS_KEY', 'fake') 262 | 263 | patched_handler._lambda_handler(fake_noneks_env, mocker.Mock(), mock_k8s_client, mock_event) 264 | 265 | patched_handler.s3.download_file.assert_called_with('bucket', 'object', '/tmp/kubeconfig') 266 | 267 | assert mock_k8s_client.Configuration.return_value.api_key.get('authorization') == None 268 | assert mock_k8s_client.Configuration.return_value.api_key_prefix.get('authorization') == None 269 | 270 | mock_k8s_client.CoreV1Api.return_value.patch_node.assert_called_with('test_node', mocker.ANY) 271 | mock_k8s_client.CoreV1Api.return_value.list_pod_for_all_namespaces.assert_called_with(watch=False, 272 | include_uninitialized=True, 273 | field_selector='spec.nodeName=test_node') 274 | assert mock_k8s_client.CoreV1Api.return_value.create_namespaced_pod_eviction.call_count == 2 275 | 276 | patched_handler.asg.complete_lifecycle_action.assert_called_with(LifecycleHookName='k8s-drainer-LifecycleHook-DDXJNVV0KBG1', 277 | AutoScalingGroupName='k8s-worker-nodes-dev-NodeGroup-F49231EK31OA', 278 | LifecycleActionResult='CONTINUE', 279 | InstanceId='i-036e525e159f62a5d') 280 | 281 | 282 | def test_handler_no_nodes(mocker, monkeypatch, patched_handler, fake_eks_env, mock_k8s_client_no_nodes, mock_event): 283 | monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'fake') 284 | monkeypatch.setenv('AWS_SECRET_ACCESS_KEY', 'fake') 285 | 286 | patched_handler._lambda_handler(fake_eks_env, mocker.Mock(), mock_k8s_client_no_nodes, mock_event) 287 | 288 | patched_handler.asg.complete_lifecycle_action.assert_called_with(LifecycleHookName='k8s-drainer-LifecycleHook-DDXJNVV0KBG1', 289 | AutoScalingGroupName='k8s-worker-nodes-dev-NodeGroup-F49231EK31OA', 290 | LifecycleActionResult='ABANDON', 291 | InstanceId='i-036e525e159f62a5d') 292 | 293 | 294 | def test_handler_patch_exception(mocker, monkeypatch, patched_handler, fake_eks_env, mock_k8s_client_patch_exception, mock_event): 295 | monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'fake') 296 | monkeypatch.setenv('AWS_SECRET_ACCESS_KEY', 'fake') 297 | 298 | patched_handler._lambda_handler(fake_eks_env, mocker.Mock(), mock_k8s_client_patch_exception, mock_event) 299 | 300 | patched_handler.asg.complete_lifecycle_action.assert_called_with(LifecycleHookName='k8s-drainer-LifecycleHook-DDXJNVV0KBG1', 301 | AutoScalingGroupName='k8s-worker-nodes-dev-NodeGroup-F49231EK31OA', 302 | LifecycleActionResult='ABANDON', 303 | InstanceId='i-036e525e159f62a5d') 304 | 305 | 306 | def test_handler_pods_exception(mocker, monkeypatch, patched_handler, fake_eks_env, mock_k8s_client_pods_exception, mock_event): 307 | monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'fake') 308 | monkeypatch.setenv('AWS_SECRET_ACCESS_KEY', 'fake') 309 | 310 | patched_handler._lambda_handler(fake_eks_env, mocker.Mock(), mock_k8s_client_pods_exception, mock_event) 311 | 312 | patched_handler.asg.complete_lifecycle_action.assert_called_with(LifecycleHookName='k8s-drainer-LifecycleHook-DDXJNVV0KBG1', 313 | AutoScalingGroupName='k8s-worker-nodes-dev-NodeGroup-F49231EK31OA', 314 | LifecycleActionResult='ABANDON', 315 | InstanceId='i-036e525e159f62a5d') 316 | 317 | 318 | def test_main_handler(patched_main_handler, mock_event): 319 | patched_main_handler.lambda_handler(mock_event, {}) 320 | k8s = patched_main_handler.k8s 321 | env = { 322 | 'cluster_name': 'test-cluster', 323 | 'kube_config_bucket': 'bucket', 324 | 'kube_config_object': 'object' 325 | } 326 | 327 | patched_main_handler._lambda_handler.assert_called_with(env, k8s.config, k8s.client, mock_event) 328 | -------------------------------------------------------------------------------- /tests/drainer/test_k8s_utils.py: -------------------------------------------------------------------------------- 1 | from kubernetes.client.rest import ApiException 2 | from mock import call 3 | 4 | from drainer.k8s_utils import (abandon_lifecycle_action, cordon_node, node_exists, remove_all_pods) 5 | from tests.utils import dict_to_simple_namespace 6 | 7 | 8 | def test_node_exists(mocker): 9 | list_nodes_val = dict_to_simple_namespace({'items': [{'metadata': {'name': 'test_node'}}]}) 10 | mock_api = mocker.Mock(**{'list_node.return_value': list_nodes_val}) 11 | 12 | assert node_exists(mock_api, 'test_node') is True 13 | assert node_exists(mock_api, 'nope') is False 14 | 15 | 16 | def test_abandon_lifecycle_action(mocker): 17 | asg_mock = mocker.Mock() 18 | abandon_lifecycle_action(asg_mock, 'asg', 'hook_name', 'instance_id') 19 | 20 | mock_args = {'LifecycleHookName': 'hook_name', 21 | 'AutoScalingGroupName': 'asg', 22 | 'LifecycleActionResult': 'ABANDON', 23 | 'InstanceId': 'instance_id'} 24 | 25 | asg_mock.complete_lifecycle_action.assert_called_with(**mock_args) 26 | 27 | 28 | def test_cordon_node(mocker): 29 | api_mock = mocker.Mock() 30 | cordon_node(api_mock, 'test_node') 31 | 32 | mock_arg = { 33 | 'apiVersion': 'v1', 34 | 'kind': 'Node', 35 | 'metadata': { 36 | 'name': 'test_node' 37 | }, 38 | 'spec': { 39 | 'unschedulable': True 40 | } 41 | } 42 | 43 | api_mock.patch_node.assert_called_with('test_node', mock_arg) 44 | 45 | 46 | def test_remove_all_pods(mocker): 47 | list_pods_val = dict_to_simple_namespace({'items': [ 48 | {'metadata': { 49 | 'uid': 'aaa', 50 | 'name': 'test_pod1', 51 | 'namespace': 'test_ns', 52 | 'annotations': None, 53 | 'owner_references': None 54 | } 55 | }, {'metadata': { 56 | 'uid': 'bbb', 57 | 'name': 'test_pod2', 58 | 'namespace': 'test_ns', 59 | 'annotations': None, 60 | 'owner_references': None 61 | } 62 | } 63 | ]}) 64 | empty_list_pods_val = dict_to_simple_namespace({'items': []}) 65 | 66 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': [list_pods_val, empty_list_pods_val]}) 67 | 68 | remove_all_pods(mock_api, 'test_node') 69 | 70 | mock_api.list_pod_for_all_namespaces.assert_called_with(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node') 71 | 72 | mock_arg = { 73 | 'apiVersion': 'policy/v1beta1', 74 | 'kind': 'Eviction', 75 | 'deleteOptions': {}, 76 | 'metadata': { 77 | 'name': 'test_pod1', 78 | 'namespace': 'test_ns' 79 | } 80 | } 81 | 82 | mock_arg1 = { 83 | 'apiVersion': 'policy/v1beta1', 84 | 'kind': 'Eviction', 85 | 'deleteOptions': {}, 86 | 'metadata': { 87 | 'name': 'test_pod2', 88 | 'namespace': 'test_ns' 89 | } 90 | } 91 | 92 | mock_api.create_namespaced_pod_eviction.assert_any_call('test_pod1', 'test_ns', mock_arg) 93 | mock_api.create_namespaced_pod_eviction.assert_any_call('test_pod2', 'test_ns', mock_arg1) 94 | 95 | 96 | def test_remove_disruption_failure(mocker): 97 | list_pods_val = dict_to_simple_namespace({'items': [ 98 | {'metadata': { 99 | 'uid': 'aaa', 100 | 'name': 'test_pod1', 101 | 'namespace': 'test_ns', 102 | 'annotations': None, 103 | 'owner_references': None 104 | } 105 | } 106 | ]}) 107 | empty_list_pods_val = dict_to_simple_namespace({'items': []}) 108 | 109 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': [list_pods_val, empty_list_pods_val], 110 | 'create_namespaced_pod_eviction.side_effect': [ApiException(status=429), None]} 111 | ) 112 | 113 | remove_all_pods(mock_api, 'test_node', poll=1) 114 | 115 | mock_api.list_pod_for_all_namespaces.assert_called_with(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node') 116 | 117 | mock_arg = { 118 | 'apiVersion': 'policy/v1beta1', 119 | 'kind': 'Eviction', 120 | 'deleteOptions': {}, 121 | 'metadata': { 122 | 'name': 'test_pod1', 123 | 'namespace': 'test_ns' 124 | } 125 | } 126 | 127 | mock_api.create_namespaced_pod_eviction.assert_has_calls([ 128 | call('test_pod1', 'test_ns', mock_arg), 129 | call('test_pod1', 'test_ns', mock_arg)] 130 | ) 131 | 132 | 133 | def test_remove_pending(mocker): 134 | list_pods_val = dict_to_simple_namespace({'items': [ 135 | {'metadata': { 136 | 'uid': 'aaa', 137 | 'name': 'test_pod1', 138 | 'namespace': 'test_ns', 139 | 'annotations': None, 140 | 'owner_references': None 141 | } 142 | } 143 | ]}) 144 | empty_list_pods_val = dict_to_simple_namespace({'items': []}) 145 | 146 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': [list_pods_val, list_pods_val, empty_list_pods_val]}) 147 | 148 | remove_all_pods(mock_api, 'test_node', poll=1) 149 | 150 | mock_api.list_pod_for_all_namespaces.assert_has_calls([ 151 | call(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node'), 152 | call(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node'), 153 | call(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node') 154 | ]) 155 | 156 | mock_arg = { 157 | 'apiVersion': 'policy/v1beta1', 158 | 'kind': 'Eviction', 159 | 'deleteOptions': {}, 160 | 'metadata': { 161 | 'name': 'test_pod1', 162 | 'namespace': 'test_ns' 163 | } 164 | } 165 | 166 | mock_api.create_namespaced_pod_eviction.assert_any_call('test_pod1', 'test_ns', mock_arg) 167 | 168 | 169 | def test_skip_daemonsets(mocker): 170 | list_pods_val = dict_to_simple_namespace({'items': [ 171 | {'metadata': { 172 | 'uid': 'aaa', 173 | 'name': 'test_pod1', 174 | 'namespace': 'test_ns', 175 | 'annotations': None, 176 | 'owner_references': [ 177 | { 178 | 'controller': True, 179 | 'kind': 'DaemonSet' 180 | } 181 | ] 182 | } 183 | }, {'metadata': { 184 | 'uid': 'bbb', 185 | 'name': 'test_pod2', 186 | 'namespace': 'test_ns', 187 | 'annotations': None, 188 | 'owner_references': None 189 | } 190 | } 191 | ]}) 192 | unevictable_pods_val = dict_to_simple_namespace({'items': [ 193 | {'metadata': { 194 | 'uid': 'aaa', 195 | 'name': 'test_pod1', 196 | 'namespace': 'test_ns', 197 | 'annotations': None, 198 | 'owner_references': [ 199 | { 200 | 'controller': True, 201 | 'kind': 'DaemonSet' 202 | } 203 | ] 204 | } 205 | } 206 | ]}) 207 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': [list_pods_val, unevictable_pods_val]}) 208 | 209 | remove_all_pods(mock_api, 'test_node') 210 | 211 | mock_api.list_pod_for_all_namespaces.assert_called_with(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node') 212 | 213 | mock_arg1 = { 214 | 'apiVersion': 'policy/v1beta1', 215 | 'kind': 'Eviction', 216 | 'deleteOptions': {}, 217 | 'metadata': { 218 | 'name': 'test_pod2', 219 | 'namespace': 'test_ns' 220 | } 221 | } 222 | 223 | mock_api.create_namespaced_pod_eviction.assert_any_call('test_pod2', 'test_ns', mock_arg1) 224 | 225 | def test_skip_mirror_pods(mocker): 226 | list_pods_val = dict_to_simple_namespace({'items': [ 227 | {'metadata': { 228 | 'uid': 'aaa', 229 | 'name': 'test_pod1', 230 | 'namespace': 'test_ns', 231 | 'annotations': { 232 | 'kubernetes.io/config.mirror': 'mirror' 233 | }, 234 | 'owner_references': None 235 | } 236 | }, {'metadata': { 237 | 'uid': 'bbb', 238 | 'name': 'test_pod2', 239 | 'namespace': 'test_ns', 240 | 'annotations': None, 241 | 'owner_references': None 242 | } 243 | } 244 | ]}, skip={".items.metadata.annotations": True}) 245 | unevictable_pods_val = dict_to_simple_namespace({'items': [ 246 | {'metadata': { 247 | 'uid': 'aaa', 248 | 'name': 'test_pod1', 249 | 'namespace': 'test_ns', 250 | 'annotations': { 251 | 'kubernetes.io/config.mirror': 'mirror' 252 | }, 253 | 'owner_references': None 254 | } 255 | } 256 | ]}, skip={".items.metadata.annotations": True}) 257 | mock_api = mocker.Mock(**{'list_pod_for_all_namespaces.side_effect': [list_pods_val, unevictable_pods_val]}) 258 | 259 | remove_all_pods(mock_api, 'test_node') 260 | 261 | mock_api.list_pod_for_all_namespaces.assert_called_with(watch=False, include_uninitialized=True, field_selector='spec.nodeName=test_node') 262 | 263 | mock_arg1 = { 264 | 'apiVersion': 'policy/v1beta1', 265 | 'kind': 'Eviction', 266 | 'deleteOptions': {}, 267 | 'metadata': { 268 | 'name': 'test_pod2', 269 | 'namespace': 'test_ns' 270 | } 271 | } 272 | 273 | mock_api.create_namespaced_pod_eviction.assert_any_call('test_pod2', 'test_ns', mock_arg1) 274 | -------------------------------------------------------------------------------- /tests/utils.py: -------------------------------------------------------------------------------- 1 | import collections 2 | 3 | from types import SimpleNamespace 4 | 5 | 6 | def dict_to_simple_namespace(orig_dict, skip={}): 7 | def _dict_to_simple_namespace(d, path=""): 8 | res = {} 9 | for k, v in d.items(): 10 | cur_path = path + "." + k 11 | if skip.get(cur_path): 12 | res[k] = v 13 | elif isinstance(v, list): 14 | res[k] = [SimpleNamespace(**_dict_to_simple_namespace(x, path=cur_path)) for x in v] 15 | elif isinstance(v, collections.abc.Mapping): 16 | res[k] = SimpleNamespace(**_dict_to_simple_namespace(v, path=cur_path)) 17 | else: 18 | res[k] = v 19 | return res 20 | 21 | return SimpleNamespace(**_dict_to_simple_namespace(orig_dict)) 22 | --------------------------------------------------------------------------------