├── .bumpversion.cfg ├── .gitignore ├── LICENSE ├── README.md ├── SECURITY.md ├── changelog.md ├── credstash ├── credstash-migrate-autoversion.py ├── credstash.py ├── credstash_migrate_digests.py ├── integration_tests ├── test_commands.bats ├── test_credstash_lib.py └── test_kms_region.bats ├── optional-requirements.txt ├── requirements.txt ├── setup.cfg ├── setup.py ├── tests ├── expand_wildcard_test.py ├── get_session_test.py ├── key_pair_test.py ├── key_service_test.py └── pad_left_test.py └── tox.ini /.bumpversion.cfg: -------------------------------------------------------------------------------- 1 | [bumpversion] 2 | current_version = 1.17.1 3 | commit = False 4 | tag = False 5 | 6 | [bumpversion:file:setup.py] 7 | search = version = '{current_version}' 8 | replace = version = '{new_version}' 9 | 10 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | 27 | # Installer logs 28 | pip-log.txt 29 | pip-delete-this-directory.txt 30 | 31 | # Unit test / coverage reports 32 | htmlcov/ 33 | .tox/ 34 | .coverage 35 | .coverage.* 36 | .cache 37 | nosetests.xml 38 | coverage.xml 39 | *,cover 40 | .hypothesis/ 41 | 42 | # Translations 43 | *.mo 44 | *.pot 45 | 46 | # Django stuff: 47 | *.log 48 | 49 | # Sphinx documentation 50 | docs/_build/ 51 | 52 | # PyBuilder 53 | target/ 54 | 55 | #Ipython Notebook 56 | .ipynb_checkpoints 57 | 58 | # IDEA 59 | .idea/* 60 | 61 | # Misc 62 | .DS_Store 63 | *swp 64 | *scratch*.py 65 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright 2020 Fugue Inc. 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CredStash 2 | 3 | ## Quick Installation 4 | 0. (Linux only) Install dependencies 5 | 1. `pip install credstash` 6 | 2. Set up a key called credstash in KMS (found in the IAM console) 7 | 3. Make sure you have AWS creds in a place that boto/botocore can read them 8 | 4. `credstash setup` 9 | 10 | ### Linux install-time dependencies 11 | Credstash recently moved from PyCrypto to `cryptography`. `cryptography` uses pre-built binary wheels on OSX and Windows, but does not on Linux. That means that you need to install some dependencies if you want to run credstash on linux. 12 | 13 | For Debian and Ubuntu, the following command will ensure that the required dependencies are installed: 14 | ``` 15 | $ sudo apt-get install build-essential libssl-dev libffi-dev python-dev 16 | ``` 17 | For Fedora and RHEL-derivatives, the following command will ensure that the required dependencies are installed: 18 | ``` 19 | $ sudo yum install gcc libffi-devel python-devel openssl-devel 20 | ``` 21 | 22 | In either case, once you've installed the dependencies, you can do `pip install credstash` as usual. 23 | 24 | See https://cryptography.io/en/latest/installation/ for more information. 25 | 26 | 27 | ## What is this? 28 | Software systems often need access to some shared credential. For example, your web application needs access to a database password, or an API key for some third party service. 29 | 30 | Some organizations build complete credential-management systems, but for most of us, managing these credentials is usually an afterthought. In the best case, people use systems like ansible-vault, which does a pretty good job, but leads to other management issues (like where/how to store the master key). A lot of credential management schemes amount to just SCP'ing a `secrets` file out to the fleet, or in the worst case, burning secrets into the SCM (do a github search on `password`). 31 | 32 | CredStash is a very simple, easy to use credential management and distribution system that uses AWS Key Management Service (KMS) for key wrapping and master-key storage, and DynamoDB for credential storage and sharing. 33 | 34 | ## Compatibility with Other Languages 35 | A number of great projects exist to provide credstash compatability with other languages. Here are the ones that we know about (feel free to open a pull request if you know of another): 36 | 37 | - https://github.com/klamouri/jcredstash (Java) 38 | - https://github.com/adorechic/rcredstash (Ruby) 39 | - https://github.com/kdrakon/scala-credstash (Scala) 40 | - https://github.com/gmo/credstash-php (PHP) 41 | - https://github.com/DavidTanner/nodecredstash (Node.js) 42 | - https://github.com/winebarrel/gcredstash (Go) 43 | - https://github.com/Narochno/Narochno.Credstash (C#) 44 | - https://github.com/republicwireless-open/erlcredstash (Erlang) 45 | - https://github.com/psibi/rucredstash (Rust) 46 | - https://github.com/ouzi-dev/credstash-operator (Kubernetes) 47 | 48 | ## How does it work? 49 | After you complete the steps in the `Setup` section, you will have an encryption key in KMS (in this README, we will refer to that key as the `master key`), and a credential storage table in DDB. 50 | 51 | ### Stashing Secrets 52 | Whenever you want to store/share a credential, such as a database password, you simply run `credstash put [credential-name] [credential-value]`. For example, `credstash put myapp.db.prod supersecretpassword1234`. credstash will go to the KMS and generate a unique data encryption key, which itself is encrypted by the master key (this is called key wrapping). credstash will use the data encryption key to encrypt the credential value. It will then store the encrypted credential, along with the wrapped (encrypted) data encryption key in the credential store in DynamoDB. 53 | 54 | You can also store a credential either by referencing a file or by passing the secret in via `stdin`. To add a secret from a file, instead of passing the secret as an argument pass the filename of the file containing the secret prefixed by the `@` sign. For example, `credstash put myapp.db.prod @secret.txt`. You can also pass the credential via `stdin` by passing the `-` character as the secret argument. For example, `tr -dc '[:alnum:]' < /dev/urandom | fold -w 32 | head -n 1 | credstash put myapp.db.prod -`. 55 | 56 | ### Getting Secrets 57 | When you want to fetch the credential, for example as part of the bootstrap process on your web-server, you simply do `credstash get [credential-name]`. For example, `export DB_PASSWORD=$(credstash get myapp.db.prod)`. When you run `get`, credstash will go and fetch the encrypted credential and the wrapped encryption key from the credential store (DynamoDB). It will then send the wrapped encryption key to KMS, where it is decrypted with the master key. credstash then uses the decrypted data encryption key to decrypt the credential. The credential is printed to `stdout`, so you can use it in scripts or assign it to environment variables. 58 | 59 | ### Controlling and Auditing Secrets 60 | Optionally, you can include any number of [Encryption Context](http://docs.aws.amazon.com/kms/latest/developerguide/encrypt-context.html) key value pairs to associate with the credential. The exact set of encryption context key value pairs that were associated with the credential when it was `put` in DynamoDB must be provided in the `get` request to successfully decrypt the credential. These encryption context key value pairs are useful to provide auditing context to the encryption and decryption operations in your CloudTrail logs. They are also useful for constraining access to a given credstash stored credential by using KMS Key Policy conditions and KMS Grant conditions. Doing so allows you to, for example, make sure that your database servers and web-servers can read the web-server DB user password but your database servers can not read your web-servers TLS/SSL certificate's private key. A `put` request with encryption context would look like `credstash put myapp.db.prod supersecretpassword1234 app.tier=db environment=prod`. In order for your web-servers to read that same credential they would execute a `get` call like `export DB_PASSWORD=$(credstash get myapp.db.prod environment=prod app.tier=db)` 61 | 62 | ### Versioning Secrets 63 | Credentials stored in the credential-store are versioned and immutable. That is, if you `put` a credential called `foo` with a version of `1` and a value of `bar`, then foo version 1 will always have a value of bar, and there is no way in `credstash` to change its value (although you could go fiddle with the bits in DDB, but you shouldn't do that). Credential rotation is handed through versions. Suppose you do `credstash put foo bar`, and then decide later to rotate `foo`, you can put version 2 of `foo` by doing `credstash put foo baz -v `. The next time you do `credstash get foo`, it will return `baz`. You can get specific credential versions as well (with the same `-v` flag). You can fetch a list of all credentials in the credential-store and their versions with the `list` command. 64 | 65 | If you use incrementing integer version numbers (for example, `[1, 2, 3, ...]`), then you can use the `-a` flag with the `put` command to automatically increment the version number. However, because of the lexicographical sorting in DynamoDB, `credstash` will left-pad the version representation with zeros (for example, `[001, 025, 103, ...]`, except to 19 characters, enough to handle `sys.maxint` on 64-bit systems). 66 | 67 | #### Special Note for Those Using Credstash Auto-Versioning Before December 2015 68 | Prior to December 2015, `credstash` auto-versioned with unpadded integers. This resulted in a sorting error once a key hit ten versions. To ensure support for versions that were not numbers (such as dates, build versions, names, etc.), the lexicographical sorting behavior was retained, but the auto-versioning behavior was changed to left-pad integer representations. 69 | 70 | If you've used auto-versioning so far, you should run the `credstash-migrate-autoversion.py` script included in the root of the repository. If you are supplying your own version numbers, you should ensure a lexicographic sort of your versions produces the result you desire. 71 | 72 | ## Dependencies 73 | credstash uses the following AWS services: 74 | * AWS Key Management Service (KMS) - for master key management and key wrapping 75 | * AWS Identity and Access Management - for access control 76 | * Amazon DynamoDB - for credential storage 77 | 78 | ## Setup 79 | ### tl;dr 80 | 1. Set up a key called `credstash` in KMS 81 | 2. Install credstash's python dependencies (or just use pip) 82 | 3. Make sure you have AWS creds in a place that boto/botocore can read them 83 | 4. Run `credstash setup` 84 | 85 | ### Setting up KMS 86 | `credstash` will not currently set up your KMS master key. To create a KMS master key, 87 | 88 | 1. Go to the AWS Console and make sure you are in `us-east-1`. If you want to use a key in a different region, you can pass it in using the `--kms-region` argument. 89 | 2. Go to the KMS Console 90 | 3. Click "Customer managed keys" in the left sidebar 91 | 4. Click "Next" to configure a Symmetric key 92 | 5. For alias, put "credstash" and click "Next". If you want to use a different name, be sure to pass it to credstash with the `-k` flag. 93 | 6. Decide what IAM principals, if any, you want to be able to manage the key. Click "Next". 94 | 6. On the "Key Usage Permissions" screen, pick the IAM users/roles that will be using credstash (you can change your mind later). Click "Next". 95 | 7. Review the key policy and click "Finish". 96 | 8. Done! 97 | 98 | ### Setting up credstash 99 | The easiest thing to do is to just run `pip install credstash`. That will download and install credstash and its dependencies (boto and PyCypto). You can also install credstash with optional YAML support by running `pip install credstash[YAML]` instead. 100 | 101 | The second easiest thing to do is to do `python setup.py install` in the `credstash` directory. 102 | 103 | The python dependencies for credstash are in the `requirements.txt` file. You can install them with `pip install -r requirements.txt`. 104 | 105 | In all cases, you will need a C compiler for building `PyCrypto` (you can install `gcc` by doing `apt-get install gcc` or `yum install gcc`). 106 | 107 | You will need to have AWS credentials accessible to boto/botocore. The easiest thing to do is to run credstash on an EC2 instance with an IAM role. Alternatively, you can put AWS credentials in the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. Or, you can put them in a file (see http://boto.readthedocs.org/en/latest/boto_config_tut.html). 108 | 109 | You can specify the region in which `credstash` should operate by using the `-r` flag, or by setting the `AWS_DEFAULT_REGION` environment variable. Note that the command line flag takes precedence over the environment variable. If you set neither, then `credstash` will operate against us-east-1. 110 | 111 | Once credentials are in place, run `credstash setup`. This will create the DDB table needed for credential storage. 112 | 113 | ### Working with multiple AWS accounts (profiles) 114 | 115 | If you need to work with multiple AWS accounts, an easy thing to do is to set up multiple profiles in your `~/.aws/credentials` file. For example, 116 | 117 | ``` 118 | [dev] 119 | aws_access_key_id = AKIDEXAMPLEASDFASDF 120 | aws_secret_access_key = SKIDEXAMPLE2103429812039423 121 | [prod] 122 | aws_access_key_id= AKIDEXAMPLEASDFASDF 123 | aws_secret_access_key= SKIDEXAMPLE2103429812039423 124 | ``` 125 | 126 | Then, by setting the `AWS_PROFILE` environment variable to the name of the profile, (dev or prod, in this case), you can point credstash at the appropriate account. 127 | 128 | For example: 129 | export AWS_PROFILE=dev ( or AWS_PROFILE=prod ) 130 | 131 | See https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs for more information. 132 | 133 | ## Usage 134 | ``` 135 | usage: credstash [-h] [-r REGION] [--kms-region KMS_REGION] [-t TABLE] 136 | [--log-level LOG_LEVEL] [--log-file LOG_FILE] 137 | [-p PROFILE | -n ARN] 138 | {delete,get,getall,keys,list,put,putall,setup} ... 139 | 140 | A credential/secret storage system 141 | 142 | positional arguments: 143 | {delete,get,getall,keys,list,put,putall,setup} 144 | Try commands like "/Users/Mike/.pyenv/versions/3.6.5/e 145 | nvs/rm/bin/credstash get -h" or "/Users/Mike/.pyenv/ve 146 | rsions/3.6.5/envs/rm/bin/credstash put --help" to get 147 | each sub command's options 148 | delete Delete a credential from the store 149 | get Get a credential from the store 150 | getall Get all credentials from the store 151 | keys List all keys in the store 152 | list list credentials and their versions 153 | put Put a credential into the store 154 | putall Put credentials from json into the store 155 | setup setup the credential store 156 | 157 | optional arguments: 158 | -h, --help show this help message and exit 159 | -r REGION, --region REGION 160 | the AWS region in which to operate. If a region is not 161 | specified, credstash will use the value of the 162 | AWS_DEFAULT_REGION env variable, or if that is not 163 | set, the value in `~/.aws/config`. As a last resort, 164 | it will use us-east-1 165 | --kms-region KMS_REGION 166 | Region the credstash KMS key will be read from, 167 | independent of the region the DDB table is in. If not 168 | specified, the KMS region will follow the same 169 | resolution path as --region. To save the KMS region, 170 | use `credstash setup --save-kms-region KMS_REGION`. 171 | The value in this argument takes precedence any saved 172 | value. 173 | -t TABLE, --table TABLE 174 | DynamoDB table to use for credential storage. If not 175 | specified, credstash will use the value of the 176 | CREDSTASH_DEFAULT_TABLE env variable, or if that is 177 | not set, the value `credential-store` will be used 178 | --log-level LOG_LEVEL 179 | Set the log level, default WARNING 180 | --log-file LOG_FILE Set the log output file, default credstash.log. Errors 181 | are printed to stderr and stack traces are logged to 182 | file 183 | -p PROFILE, --profile PROFILE 184 | Boto config profile to use when connecting to AWS 185 | -n ARN, --arn ARN AWS IAM ARN for AssumeRole 186 | 187 | delete 188 | usage: credstash delete [-h] [-r REGION] [-t TABLE] [-p PROFILE | -n ARN] credential 189 | 190 | positional arguments: 191 | credential the name of the credential to delete 192 | 193 | get 194 | usage: credstash get [-h] [-n] [-v VERSION] [-f {json,csv,dotenv,yaml}] 195 | credential [context [context ...]] 196 | 197 | positional arguments: 198 | credential the name of the credential to get. Using the wildcard 199 | character '*' will search for credentials that match 200 | the pattern 201 | context encryption context key/value pairs associated with the 202 | credential in the form of "key=value" 203 | 204 | optional arguments: 205 | -h, --help show this help message and exit 206 | -n, --noline Don't append newline to returned value (useful in 207 | scripts or with binary files) 208 | -v VERSION, --version VERSION 209 | Get a specific version of the credential (defaults to 210 | the latest version) 211 | -f {json,csv,dotenv,yaml}, --format {json,csv,dotenv,yaml} 212 | Output format. json(default) yaml csv or dotenv. 213 | 214 | getall 215 | usage: credstash getall [-h] [-r REGION] [-t TABLE] [-p PROFILE | -n ARN] [-v VERSION] [-f {json,yaml,csv,dotenv}] 216 | [context [context ...]] 217 | 218 | positional arguments: 219 | context encryption context key/value pairs associated with the 220 | credential in the form of "key=value" 221 | 222 | optional arguments: 223 | -v VERSION, --version VERSION 224 | Get a specific version of the credential (defaults to 225 | the latest version). 226 | -f {json,yaml,csv,dotenv}, --format {json,yaml,csv,dotenv} 227 | Output format. json(default), yaml, csv or dotenv. 228 | 229 | 230 | list 231 | usage: credstash list [-h] [-r REGION] [-t TABLE] [-p PROFILE | -n ARN] 232 | 233 | put 234 | usage: credstash put [-h] [-k KEY] [-c COMMENT] [-v VERSION] [-a] 235 | [-d {SHA,SHA224,SHA256,SHA384,SHA512,MD5}] [-P] 236 | credential [value] [context [context ...]] 237 | 238 | positional arguments: 239 | credential the name of the credential to store 240 | value the value of the credential to store or, if beginning 241 | with the "@" character, the filename of the file 242 | containing the value, or pass "-" to read the value 243 | from stdin 244 | context encryption context key/value pairs associated with the 245 | credential in the form of "key=value" 246 | 247 | optional arguments: 248 | -h, --help show this help message and exit 249 | -k KEY, --key KEY the KMS key-id of the master key to use. See the 250 | README for more information. Defaults to 251 | alias/credstash 252 | -c COMMENT, --comment COMMENT 253 | Include reference information or a comment about value 254 | to be stored. 255 | -v VERSION, --version VERSION 256 | Put a specific version of the credential (update the 257 | credential; defaults to version `1`). 258 | -a, --autoversion Automatically increment the version of the credential 259 | to be stored. This option causes the `-v` flag to be 260 | ignored. (This option will fail if the currently 261 | stored version is not numeric.) 262 | -d {SHA,SHA224,SHA256,SHA384,SHA512,MD5}, --digest {SHA,SHA224,SHA256,SHA384,SHA512,MD5} 263 | the hashing algorithm used to to encrypt the data. 264 | Defaults to SHA256 265 | -P, --prompt Prompt for secret 266 | 267 | 268 | setup 269 | usage: credstash setup [-h] [--save-kms-region SAVE_KMS_REGION] 270 | [--tags [TAGS [TAGS ...]]] 271 | 272 | optional arguments: 273 | -h, --help show this help message and exit 274 | --save-kms-region SAVE_KMS_REGION 275 | Save the region the credstash KMS key will be read 276 | from, independent of the region the DDB table is in. 277 | This value is saved in ~/.credstash 278 | --tags [TAGS [TAGS ...]] 279 | Tags to apply to the Dynamodb Table passed in as a 280 | space sparated list of Key=Value 281 | ``` 282 | ## IAM Policies 283 | 284 | ### Secret Writer 285 | You can put or write secrets to credstash by either using KMS Key Grants, KMS Key Policies, or IAM Policies. If you are using IAM Policies, the following IAM permissions are the minimum required to be able to put or write secrets: 286 | ``` 287 | { 288 | "Version": "2012-10-17", 289 | "Statement": [ 290 | { 291 | "Action": [ 292 | "kms:GenerateDataKey" 293 | ], 294 | "Effect": "Allow", 295 | "Resource": "arn:aws:kms:us-east-1:AWSACCOUNTID:key/KEY-GUID" 296 | }, 297 | { 298 | "Action": [ 299 | "dynamodb:PutItem" 300 | ], 301 | "Effect": "Allow", 302 | "Resource": "arn:aws:dynamodb:us-east-1:AWSACCOUNTID:table/credential-store" 303 | } 304 | ] 305 | } 306 | ``` 307 | If you are using Key Policies or Grants, then the `kms:GenerateDataKey` is not required in the policy for the IAM user/group/role. Replace `AWSACCOUNTID` with the account ID for your table, and replace the KEY-GUID with the identifier for your KMS key (which you can find in the KMS console). 308 | 309 | ### Secret Reader 310 | You can read secrets from credstash with the get or getall actions by either using KMS Key Grants, KMS Key Policies, or IAM Policies. If you are using IAM Policies, the following IAM permissions are the minimum required to be able to get or read secrets: 311 | ``` 312 | { 313 | "Version": "2012-10-17", 314 | "Statement": [ 315 | { 316 | "Action": [ 317 | "kms:Decrypt" 318 | ], 319 | "Effect": "Allow", 320 | "Resource": "arn:aws:kms:us-east-1:AWSACCOUNTID:key/KEY-GUID" 321 | }, 322 | { 323 | "Action": [ 324 | "dynamodb:GetItem", 325 | "dynamodb:Query", 326 | "dynamodb:Scan" 327 | ], 328 | "Effect": "Allow", 329 | "Resource": "arn:aws:dynamodb:us-east-1:AWSACCOUNTID:table/credential-store" 330 | } 331 | ] 332 | } 333 | ``` 334 | If you are using Key Policies or Grants, then the `kms:Decrypt` is not required in the policy for the IAM user/group/role. Replace `AWSACCOUNTID` with the account ID for your table, and replace the KEY-GUID with the identifier for your KMS key (which you can find in the KMS console). Note that the `dynamodb:Scan` permission is not required if you do not use wildcards in your `get`s. 335 | 336 | ### Setup Permissions 337 | In order to run `credstash setup`, you will also need to be able to perform the following DDB operations: 338 | ``` 339 | { 340 | "Version": "2012-10-17", 341 | "Statement": [ 342 | { 343 | "Action": [ 344 | "dynamodb:CreateTable", 345 | "dynamodb:DescribeTable" 346 | ], 347 | "Effect": "Allow", 348 | "Resource": "arn:aws:dynamodb:us-west-2::table/credential-store" 349 | }, 350 | { 351 | "Action": [ 352 | "dynamodb:ListTables" 353 | ], 354 | "Effect": "Allow", 355 | "Resource": "*" 356 | } 357 | ] 358 | } 359 | ``` 360 | 361 | ## Security Notes 362 | Any IAM principal who can get items from the credential store DDB table, and can call KMS.Decrypt, can read stored credentials. 363 | 364 | The target deployment-story for `credstash` is an EC2 instance running with an IAM role that has permissions to read the credential store and use the master key. Since IAM role credentials are vended by the instance metadata service, by default, any user on the system can fetch creds and use them to retrieve credentials. That means that by default, the instance boundary is the security boundary for this system. If you are worried about unauthorized users on your instance, you should take steps to secure access to the Instance Metadata Service (for example, use iptables to block connections to 169.254.169.254 except for privileged users). Also, because credstash is written in python, if an attacker can dump the memory of the credstash process, they may be able to recover credentials. This is a known issue, but again, in the target deployment case, the security boundary is assumed to be the instance boundary. 365 | 366 | ## Developing credstash 367 | 368 | ### Running the tests 369 | 370 | ``` 371 | python -m unittest discover -v tests "*.py" 372 | ``` 373 | 374 | ### Running the integration tests using BATS 375 | 1. The integration tests require a working install of credstash. I recommend not using your primary development/production install. 376 | 2. Download and install BATS: https://github.com/sstephenson/bats 377 | 3. Run the tests: `bats integration_tests/` 378 | 379 | New integration test PRs are welcome! 380 | 381 | ## Frequently Asked Questions (FAQ) 382 | 383 | ### 1. Where is the master key stored? 384 | The master key is stored in AWS Key Management Service (KMS), where it is stored in secure HSM-backed storage. The Master Key never leaves the KMS service. 385 | 386 | ### 2. How is credential rotation handled? 387 | Every credential in the store has a version number. Whenever you want to a credential to a new value, you have to do a `put` with a new credential version. For example, if you have `foo` version 1 in the database, then to update `foo`, you can put version 2. You can either specify the version manually (i.e. `credstash put foo bar -v 2`), or you can use the `-a` flag, which will attempt to autoincrement the version number (for example, `credstash put foo baz -a`). Whenever you do a `get` operation, credstash will fetch the most recent (highest version) version of that credential. So, to do credential rotation, simply put a new version of the credential, and clients fetching the credential will get the new version. 388 | 389 | ### 3. How much do the AWS services needed to run credstash cost? 390 | tl;dr: If you are using less than 25 reads/sec and 25 writes per second on DDB today, it will cost ~$1/month to use credstash. 391 | 392 | The master key in KMS costs $1 per month. 393 | 394 | The credential store DDB table uses 1 provisioned read and 1 provisioned write throughput, along with a small amount of actual storage. This falls well below the free tier for DDB (25 reads and 25 writes per second). If you are already a heavy DDB user and exceed the free tier, the credential store table will cost about $0.53 per month (mostly from the write throughput). 395 | 396 | If you are using credstash heavily and need to increase the provisioned reads/writes, you may incur additional charges. You can estimate your bill using the AWS Simple Monthly Calculator (http://calculator.s3.amazonaws.com/index.html#s=DYNAMODB). 397 | 398 | ### 4. Why DynamoDB for the credential store? Why not S3? 399 | DDB fits the application really well. Having very low latency fetches are really nice if credstash is in the critical path of spinning up an application. Being able to turn throughput up or down based on load and requirements are also great things to have in a config management tool. Also, as credstash gets into more complex credential management functions, the query capabilities of DDB get super handy. 400 | 401 | That said, S3 support may happen someday. 402 | 403 | ### 5. Where can I learn more about use cases and context for something like credstash? 404 | Check out this blog post: http://blog.fugue.it/2015-04-21-aws-kms-secrets.html 405 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | # Security Policy 2 | 3 | For reporting guidelines and general information regarding security at Fugue, 4 | please visit [fugue.co/security](https://fugue.co/security). 5 | -------------------------------------------------------------------------------- /changelog.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## 1.17.1 4 | * Bugfix: #291 Move `kms_region` optional parameter to end of parameter list to preserve existing functionality when parameters are used positionally 5 | 6 | ## 1.17.0 7 | * New: add `--kms-region` argument to set the KMS region independently from the DDB region. This allows the use of DDB tables in multiple regions with the same KMS key, for example, with DDB Global Tables 8 | * New: `get_session()` now supports passing in only the `profile_name` without AKIDs or SAKs (@eisjcormier) 9 | * Bugfix: #273 #274 Disable logging when `credstash` is imported as a library. This allows `credstash` to be used in contexts where writing to the local disk is not allowed, such as AWS Lambda 10 | * Bugfix: #269 Remove incompatible Python 3 code to ensure compatibility with Python 2 11 | * Bugfix: #276 Do not catch errors when `credstash` is imported as a library 12 | 13 | ## 1.16.2 14 | * New: Smarter cached session handling was added, with support for multiple sessions keyed by AKID 15 | * New: Configurable logging was added 16 | * New: @VincentHokie added the ability to pass a custom session to `getAllSecrets` and `listSecrets` 17 | * Bugfix: An empty dict is returned from getall when there are no secrets, rather than an error 18 | * Bugfix: @aerostitch fixed Python 3.8 syntax warnings 19 | * New languages: Links to Erlang and Rust implementations of `credstash` have been added 20 | 21 | ## 1.16.1 22 | * Bugfix: @corrjo fixed a bug in the tagging feature 23 | * Bugfix: @jamebus fixed a bug in `putall` 24 | 25 | ## 1.16.0 26 | * New: @freddyVandalay added a programmatic way to autoversion: `putSecretAutoversion` 27 | * New: @corrjo added the ability to tag the `credstash` DDB table using `credstash setup --tags Tag=Value` 28 | * New: @alkersan added the ability to specify the `credstash` DDB table using an environment variable 29 | * New: @cheethoe added the ability to pass custom dynamodb/kms sessions to `putSecret` 30 | * Bugfix: @dbanttari fixed large deletes and made them more efficient by using `query` instead of `scan` 31 | * Bugfix: Update to pyyaml>=4.2b1 due to security vulnerability in older versions 32 | * Added basic integration tests 33 | 34 | ## 1.15.0 35 | * New: Arthur Burkart added credential comments 36 | * Updated: added tox, and improved packaging 37 | * New: @jimbocoder added a threadpool to `getall` to fetch groups of credentials faster 38 | * New: @a12k added a migration script if you are using old hashing methods 39 | * Bugfix: @jomunoz and @jessemyers removed unsupported hashing methods and bumped the `cryptography` dependency 40 | 41 | ## 1.14.0 42 | 43 | * New: @stephen-164 added -f to `credstash get` for wildcard gets 44 | * New: @mrwacky42 added `credstash keys` 45 | * New: @evanstachowiak added `credstash putall` 46 | * Updated: @gene1wood, @nkhoshini, and @wyattwalter updated the docs 47 | * Bugfix: @pm990320 fixed a bug by adding pagination for large credential stores 48 | * Bugfix: @artburkart fixed a bug where writing csv files did not have proper line separators 49 | * Removed: Python 3.2 removed from build matrix 50 | 51 | ## 1.13.4 52 | * Set upper bound of `cryptography` to 2.1 53 | 54 | ## 1.13.3 55 | * Only fetch the session resource and client once 56 | * README updates for c# and node imlpementations 57 | * python 3.2 removed from build matrix 58 | * fixed hmac checking 59 | * removed build constraint on `cryptography` <2.0 60 | -------------------------------------------------------------------------------- /credstash: -------------------------------------------------------------------------------- 1 | credstash.py -------------------------------------------------------------------------------- /credstash-migrate-autoversion.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import boto3 4 | import credstash 5 | import copy 6 | 7 | 8 | def isInt(s): 9 | try: 10 | int(s) 11 | return True 12 | except ValueError: 13 | return False 14 | 15 | 16 | def updateVersions(region="us-east-1", table="credential-store"): 17 | ''' 18 | do a full-table scan of the credential-store, 19 | and update the version format of every credential if it is an integer 20 | ''' 21 | dynamodb = boto3.resource('dynamodb', region_name=region) 22 | secrets = dynamodb.Table(table) 23 | 24 | response = secrets.scan(ProjectionExpression="#N, version, #K, contents, hmac", 25 | ExpressionAttributeNames={"#N": "name", "#K": "key"}) 26 | 27 | items = response["Items"] 28 | 29 | for old_item in items: 30 | if isInt(old_item['version']): 31 | new_item = copy.copy(old_item) 32 | new_item['version'] = credstash.paddedInt(new_item['version']) 33 | if new_item['version'] != old_item['version']: 34 | secrets.put_item(Item=new_item) 35 | secrets.delete_item(Key={'name': old_item['name'], 'version': old_item['version']}) 36 | else: 37 | print "Skipping item: %s, %s" % (old_item['name'], old_item['version']) 38 | 39 | 40 | if __name__ == "__main__": 41 | updateVersions() 42 | -------------------------------------------------------------------------------- /credstash.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright 2015 Luminal, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | from __future__ import print_function 16 | 17 | import argparse 18 | import codecs 19 | import csv 20 | import json 21 | import operator 22 | import os 23 | import os.path 24 | import sys 25 | import re 26 | import boto3 27 | import botocore.exceptions 28 | import logging 29 | import functools 30 | 31 | try: 32 | from StringIO import StringIO 33 | except ImportError: 34 | from io import StringIO 35 | 36 | try: 37 | import yaml 38 | NO_YAML = False 39 | except ImportError: 40 | NO_YAML = True 41 | 42 | from base64 import b64encode, b64decode 43 | from boto3.dynamodb.conditions import Attr 44 | from getpass import getpass 45 | 46 | from cryptography.hazmat.backends import default_backend 47 | from cryptography.hazmat.primitives import hashes 48 | from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes 49 | from cryptography.hazmat.primitives.hmac import HMAC 50 | from cryptography.hazmat.primitives import constant_time 51 | 52 | from multiprocessing.dummy import Pool as ThreadPool 53 | 54 | _hash_classes = { 55 | 'SHA': hashes.SHA1, 56 | 'SHA224': hashes.SHA224, 57 | 'SHA256': hashes.SHA256, 58 | 'SHA384': hashes.SHA384, 59 | 'SHA512': hashes.SHA512, 60 | 'MD5': hashes.MD5, 61 | } 62 | 63 | DEFAULT_DIGEST = 'SHA256' 64 | HASHING_ALGORITHMS = _hash_classes.keys() 65 | LEGACY_NONCE = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01' 66 | DEFAULT_REGION = "us-east-1" 67 | PAD_LEN = 19 # number of digits in sys.maxint 68 | WILDCARD_CHAR = "*" 69 | THREAD_POOL_MAX_SIZE = 64 70 | CLI_INVOCATION_TYPE = "CLI" 71 | LIB_INVOCATION_TYPE = "LIB" 72 | 73 | logger = logging.getLogger('credstash') 74 | invocation_type = LIB_INVOCATION_TYPE 75 | 76 | def setup_logging(level, log_file): 77 | """setup logging when invoked as a command. logging is not setup when invoked as a lib, 78 | so credstash can be used even if the application does not have local write access. 79 | """ 80 | for h in logger.handlers: 81 | logger.removeHandler(h) 82 | handler = logging.FileHandler(log_file) 83 | formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s') 84 | handler.setFormatter(formatter) 85 | logger.addHandler(handler) 86 | logger.setLevel(level) 87 | 88 | 89 | class KeyService(object): 90 | 91 | def __init__(self, kms, key_id, encryption_context): 92 | self.kms = kms 93 | self.key_id = key_id 94 | self.encryption_context = encryption_context 95 | 96 | def generate_key_data(self, number_of_bytes): 97 | try: 98 | kms_response = self.kms.generate_data_key( 99 | KeyId=self.key_id, EncryptionContext=self.encryption_context, NumberOfBytes=number_of_bytes 100 | ) 101 | except Exception as e: 102 | raise KmsError("Could not generate key using KMS key %s (Details: %s)" % (self.key_id, str(e))) 103 | return kms_response['Plaintext'], kms_response['CiphertextBlob'] 104 | 105 | def decrypt(self, encoded_key): 106 | try: 107 | kms_response = self.kms.decrypt( 108 | CiphertextBlob=encoded_key, 109 | EncryptionContext=self.encryption_context 110 | ) 111 | except botocore.exceptions.ClientError as e: 112 | if e.response["Error"]["Code"] == "InvalidCiphertextException": 113 | if self.encryption_context is None: 114 | msg = ("Could not decrypt hmac key with KMS. The credential may " 115 | "require that an encryption context be provided to decrypt " 116 | "it.") 117 | else: 118 | msg = ("Could not decrypt hmac key with KMS. The encryption " 119 | "context provided may not match the one used when the " 120 | "credential was stored.") 121 | else: 122 | msg = "Decryption error %s" % e 123 | raise KmsError(msg) 124 | return kms_response['Plaintext'] 125 | 126 | 127 | class KmsError(Exception): 128 | 129 | def __init__(self, value=""): 130 | self.value = "KMS ERROR: " + value if value != "" else "KMS ERROR" 131 | 132 | def __str__(self): 133 | return self.value 134 | 135 | 136 | class IntegrityError(Exception): 137 | 138 | def __init__(self, value=""): 139 | self.value = "INTEGRITY ERROR: " + value if value != "" else \ 140 | "INTEGRITY ERROR" 141 | 142 | def __str__(self): 143 | return self.value 144 | 145 | 146 | class ItemNotFound(Exception): 147 | pass 148 | 149 | 150 | class KeyValueToDictionary(argparse.Action): 151 | 152 | def __call__(self, parser, namespace, values, option_string=None): 153 | setattr(namespace, 154 | self.dest, 155 | dict((x[0], x[1]) for x in values)) 156 | 157 | 158 | def printStdErr(s): 159 | sys.stderr.write(str(s)) 160 | sys.stderr.write("\n") 161 | 162 | 163 | def fatal(s): 164 | printStdErr(s) 165 | sys.exit(1) 166 | 167 | 168 | def key_value_pair(string): 169 | output = string.split('=') 170 | if len(output) != 2 or '' in output: 171 | msg = "%r is not the form of \"key=value\"" % string 172 | raise argparse.ArgumentTypeError(msg) 173 | return output 174 | 175 | 176 | def expand_wildcard(string, secrets): 177 | prog = re.compile('^' + string.replace(WILDCARD_CHAR, '.*') + '$') 178 | output = [] 179 | for secret in secrets: 180 | if prog.search(secret) is not None: 181 | output.append(secret) 182 | return output 183 | 184 | 185 | def value_or_filename(string): 186 | # argparse running on old version of python (<2.7) will pass an empty 187 | # string to this function before it passes the actual value. 188 | # If an empty string is passes in, just return an empty string 189 | if string == "": 190 | return "" 191 | 192 | if string == '-': 193 | try: 194 | return sys.stdin.read() 195 | except KeyboardInterrupt: 196 | raise argparse.ArgumentTypeError("Unable to read value from stdin") 197 | elif string[0] == "@": 198 | filename = string[1:] 199 | try: 200 | with open(os.path.expanduser(filename)) as f: 201 | output = f.read() 202 | except IOError: 203 | raise argparse.ArgumentTypeError("Unable to read file %s" % 204 | filename) 205 | else: 206 | output = string 207 | return output 208 | 209 | 210 | def csv_dump(dictionary): 211 | csvfile = StringIO() 212 | csvwriter = csv.writer(csvfile, lineterminator=os.linesep) 213 | for key in dictionary: 214 | csvwriter.writerow([key, dictionary[key]]) 215 | return csvfile.getvalue() 216 | 217 | 218 | def dotenv_dump(dictionary): 219 | dotenv_buffer = StringIO() 220 | for key in dictionary: 221 | dotenv_buffer.write("%s='%s'\n" % (key.upper(), dictionary[key])) 222 | dotenv_buffer.seek(0) 223 | return dotenv_buffer.read() 224 | 225 | 226 | def paddedInt(i): 227 | ''' 228 | return a string that contains `i`, left-padded with 0's up to PAD_LEN digits 229 | ''' 230 | i_str = str(i) 231 | pad = PAD_LEN - len(i_str) 232 | return (pad * "0") + i_str 233 | 234 | 235 | def getHighestVersion(name, region=None, table="credential-store", 236 | **kwargs): 237 | ''' 238 | Return the highest version of `name` in the table 239 | ''' 240 | session = get_session(**kwargs) 241 | 242 | dynamodb = session.resource('dynamodb', region_name=region) 243 | secrets = dynamodb.Table(table) 244 | 245 | response = secrets.query(Limit=1, 246 | ScanIndexForward=False, 247 | ConsistentRead=True, 248 | KeyConditionExpression=boto3.dynamodb.conditions.Key( 249 | "name").eq(name), 250 | ProjectionExpression="version") 251 | 252 | if response["Count"] == 0: 253 | return 0 254 | return response["Items"][0]["version"] 255 | 256 | 257 | def clean_fail(func): 258 | ''' 259 | A decorator to cleanly exit on a failed call to AWS. 260 | catch a `botocore.exceptions.ClientError` raised from an action. 261 | This sort of error is raised if you are targeting a region that 262 | isn't set up (see, `credstash setup`). 263 | 264 | When invoked via the CLI, the wrapper function catches errors 265 | and logs them to file instead of printing them. 266 | 267 | When invoked as a library, the wrapper function is a passthrough, 268 | so users can handle errors as they choose. 269 | ''' 270 | @functools.wraps(func) 271 | def clean_error(*args, **kwargs): 272 | if invocation_type == CLI_INVOCATION_TYPE: 273 | try: 274 | return func(*args, **kwargs) 275 | except botocore.exceptions.ClientError as e: 276 | print(str(e), file=sys.stderr) 277 | logger.exception(e) 278 | sys.exit(1) 279 | except Exception as e: 280 | print(str(e), file=sys.stderr) 281 | logger.exception(e) 282 | sys.exit(1) 283 | else: 284 | return func(*args, **kwargs) 285 | return clean_error 286 | 287 | @clean_fail 288 | def listSecrets(region=None, table="credential-store", session=None, **kwargs): 289 | ''' 290 | do a full-table scan of the credential-store, 291 | and return the names and versions of every credential 292 | ''' 293 | if session is None: 294 | session = get_session(**kwargs) 295 | 296 | dynamodb = session.resource('dynamodb', region_name=region) 297 | secrets = dynamodb.Table(table) 298 | 299 | items = [] 300 | response = {'LastEvaluatedKey': None} 301 | 302 | while 'LastEvaluatedKey' in response: 303 | params = dict( 304 | ProjectionExpression="#N, version, #C", 305 | ExpressionAttributeNames={"#N": "name", "#C": "comment"} 306 | ) 307 | if response['LastEvaluatedKey']: 308 | params['ExclusiveStartKey'] = response['LastEvaluatedKey'] 309 | 310 | response = secrets.scan(**params) 311 | 312 | items.extend(response['Items']) 313 | 314 | return items 315 | 316 | @clean_fail 317 | def putSecret(name, secret, version="", kms_key="alias/credstash", 318 | region=None, table="credential-store", context=None, 319 | digest=DEFAULT_DIGEST, comment="", kms=None, dynamodb=None, 320 | kms_region=None, **kwargs): 321 | ''' 322 | put a secret called `name` into the secret-store, 323 | protected by the key kms_key 324 | ''' 325 | if not context: 326 | context = {} 327 | 328 | if dynamodb is None or kms is None: 329 | session = get_session(**kwargs) 330 | if dynamodb is None: 331 | dynamodb = session.resource('dynamodb', region_name=region) 332 | if kms is None: 333 | kms = session.client('kms', region_name=kms_region or region) 334 | 335 | key_service = KeyService(kms, kms_key, context) 336 | sealed = seal_aes_ctr_legacy( 337 | key_service, 338 | secret, 339 | digest_method=digest, 340 | ) 341 | 342 | secrets = dynamodb.Table(table) 343 | 344 | data = { 345 | 'name': name, 346 | 'version': paddedInt(version), 347 | } 348 | if comment: 349 | data['comment'] = comment 350 | data.update(sealed) 351 | 352 | return secrets.put_item(Item=data, ConditionExpression=Attr('name').not_exists()) 353 | 354 | 355 | def putSecretAutoversion(name, secret, kms_key="alias/credstash", 356 | region=None, table="credential-store", context=None, 357 | digest=DEFAULT_DIGEST, comment="", kms_region=None, **kwargs): 358 | """ 359 | This function put secrets to credstash using autoversioning 360 | :return: 361 | """ 362 | 363 | latest_version = getHighestVersion(name=name, table=table, region=region) 364 | incremented_version = paddedInt(int(latest_version) + 1) 365 | try: 366 | putSecret(name=name, secret=secret, version=incremented_version, 367 | kms_key=kms_key, region=region, kms_region=kms_region, 368 | table=table, context=context, digest=digest, comment=comment, **kwargs) 369 | print("Secret '{0}' has been stored in table {1}".format(name, table)) 370 | except KmsError as e: 371 | fatal(e) 372 | 373 | 374 | def getAllSecrets(version="", region=None, table="credential-store", 375 | context=None, credential=None, session=None, 376 | kms_region=None, **kwargs): 377 | ''' 378 | fetch and decrypt all secrets 379 | ''' 380 | if session is None: 381 | session = get_session(**kwargs) 382 | dynamodb = session.resource('dynamodb', region_name=region) 383 | kms = session.client('kms', region_name=kms_region or region) 384 | secrets = listSecrets(region, table, session, **kwargs) 385 | 386 | # Only return the secrets that match the pattern in `credential` 387 | # This already works out of the box with the CLI get action, 388 | # but that action doesn't support wildcards when using as library 389 | if credential and WILDCARD_CHAR in credential: 390 | names = set(expand_wildcard(credential, 391 | [x["name"] 392 | for x in secrets])) 393 | else: 394 | names = set(x["name"] for x in secrets) 395 | 396 | if len(names) == 0: 397 | return dict() 398 | 399 | pool = ThreadPool(min(len(names), THREAD_POOL_MAX_SIZE)) 400 | results = pool.map( 401 | lambda credential: getSecret( 402 | credential, 403 | version=version, 404 | region=region, 405 | table=table, 406 | context=context, 407 | dynamodb=dynamodb, 408 | kms=kms, 409 | **kwargs 410 | ), names) 411 | pool.close() 412 | pool.join() 413 | return dict(zip(names, results)) 414 | 415 | 416 | 417 | @clean_fail 418 | def getAllAction(args, region, kms_region, **session_params): 419 | secrets = getAllSecrets(args.version, 420 | region=region, 421 | kms_region=kms_region, 422 | table=args.table, 423 | context=args.context, 424 | **session_params) 425 | if args.format == "json": 426 | output_func = json.dumps 427 | output_args = {"sort_keys": True, 428 | "indent": 4, 429 | "separators": (',', ': ')} 430 | elif not NO_YAML and args.format == "yaml": 431 | output_func = yaml.dump 432 | output_args = {"default_flow_style": False} 433 | elif args.format == 'csv': 434 | output_func = csv_dump 435 | output_args = {} 436 | elif args.format == 'dotenv': 437 | output_func = dotenv_dump 438 | output_args = {} 439 | print(output_func(secrets, **output_args)) 440 | 441 | 442 | @clean_fail 443 | def putSecretAction(args, region, kms_region, **session_params): 444 | if args.autoversion: 445 | latestVersion = getHighestVersion(args.credential, 446 | region, 447 | args.table, 448 | **session_params) 449 | try: 450 | version = paddedInt(int(latestVersion) + 1) 451 | except ValueError: 452 | fatal("Can not autoincrement version. The current " 453 | "version: %s is not an int" % latestVersion) 454 | else: 455 | version = args.version 456 | try: 457 | value = args.value 458 | if(args.prompt): 459 | value = getpass("{}: ".format(args.credential)) 460 | if putSecret(args.credential, value, version=version, 461 | kms_key=args.key, region=region, kms_region=kms_region, 462 | table=args.table, context=args.context, digest=args.digest, 463 | comment=args.comment, **session_params): 464 | print("{0} has been stored".format(args.credential)) 465 | except KmsError as e: 466 | fatal(e) 467 | except botocore.exceptions.ClientError as e: 468 | if e.response["Error"]["Code"] == "ConditionalCheckFailedException": 469 | latestVersion = getHighestVersion(args.credential, region, 470 | args.table, 471 | **session_params) 472 | fatal("%s version %s is already in the credential store. " 473 | "Use the -v flag to specify a new version" % 474 | (args.credential, latestVersion)) 475 | else: 476 | fatal(e) 477 | 478 | 479 | @clean_fail 480 | def putAllSecretsAction(args, region, kms_region, **session_params): 481 | credentials = json.loads(args.credentials) 482 | 483 | for credential, value in credentials.items(): 484 | try: 485 | args.credential = credential 486 | args.value = value 487 | args.comment = None 488 | args.prompt = None 489 | putSecretAction(args, region, kms_region, **session_params) 490 | except SystemExit as e: 491 | pass 492 | 493 | 494 | @clean_fail 495 | def getSecretAction(args, region, kms_region, **session_params): 496 | try: 497 | if WILDCARD_CHAR in args.credential: 498 | names = expand_wildcard(args.credential, 499 | [x["name"] 500 | for x 501 | in listSecrets(region=region, 502 | table=args.table, 503 | **session_params)]) 504 | secrets = { 505 | name:getSecret( 506 | name, 507 | version=args.version, 508 | region=region, 509 | kms_region=kms_region, 510 | table=args.table, 511 | context=args.context, 512 | **session_params 513 | ) 514 | for name in names 515 | } 516 | 517 | if args.format == "json": 518 | output_func = json.dumps 519 | output_args = {"sort_keys": True, 520 | "indent": 4, 521 | "separators": (',', ': ')} 522 | elif not NO_YAML and args.format == "yaml": 523 | output_func = yaml.dump 524 | output_args = {"default_flow_style": False} 525 | elif args.format == 'csv': 526 | output_func = csv_dump 527 | output_args = {} 528 | elif args.format == 'dotenv': 529 | output_func = dotenv_dump 530 | output_args = {} 531 | sys.stdout.write(output_func(secrets, **output_args)) 532 | else: 533 | sys.stdout.write(getSecret( 534 | args.credential, 535 | version=args.version, 536 | region=region, 537 | kms_region=kms_region, 538 | table=args.table, 539 | context=args.context, 540 | **session_params 541 | )) 542 | if not args.noline: 543 | sys.stdout.write("\n") 544 | except ItemNotFound as e: 545 | fatal(e) 546 | except KmsError as e: 547 | fatal(e) 548 | except IntegrityError as e: 549 | fatal(e) 550 | 551 | @clean_fail 552 | def getSecret(name, version="", region=None, table="credential-store", context=None, 553 | dynamodb=None, kms=None, kms_region=None, **kwargs): 554 | ''' 555 | fetch and decrypt the secret called `name` 556 | ''' 557 | if not context: 558 | context = {} 559 | 560 | # Can we cache 561 | if dynamodb is None or kms is None: 562 | session = get_session(**kwargs) 563 | if dynamodb is None: 564 | dynamodb = session.resource('dynamodb', region_name=region) 565 | if kms is None: 566 | kms = session.client('kms', region_name=kms_region or region) 567 | 568 | secrets = dynamodb.Table(table) 569 | 570 | if version == "": 571 | # do a consistent fetch of the credential with the highest version 572 | response = secrets.query(Limit=1, 573 | ScanIndexForward=False, 574 | ConsistentRead=True, 575 | KeyConditionExpression=boto3.dynamodb.conditions.Key("name").eq(name)) 576 | if response["Count"] == 0: 577 | raise ItemNotFound("Item {'name': '%s'} couldn't be found." % name) 578 | material = response["Items"][0] 579 | else: 580 | if len(version) < PAD_LEN: 581 | version = paddedInt(int(version)) 582 | response = secrets.get_item(Key={"name": name, "version": version}) 583 | if "Item" not in response: 584 | raise ItemNotFound( 585 | "Item {'name': '%s', 'version': '%s'} couldn't be found." % (name, version)) 586 | material = response["Item"] 587 | 588 | key_service = KeyService(kms, None, context) 589 | 590 | return open_aes_ctr_legacy(key_service, material) 591 | 592 | 593 | @clean_fail 594 | def deleteSecrets(name, region=None, table="credential-store", 595 | **kwargs): 596 | session = get_session(**kwargs) 597 | dynamodb = session.resource('dynamodb', region_name=region) 598 | secrets = dynamodb.Table(table) 599 | 600 | response = {'LastEvaluatedKey': None} 601 | 602 | while 'LastEvaluatedKey' in response: 603 | params = dict( 604 | KeyConditionExpression=boto3.dynamodb.conditions.Key('name').eq(name), 605 | ProjectionExpression="#N, version", 606 | ExpressionAttributeNames={"#N": "name"}, 607 | ) 608 | if response['LastEvaluatedKey']: 609 | params['ExclusiveStartKey'] = response['LastEvaluatedKey'] 610 | 611 | response = secrets.query(**params) 612 | 613 | for secret in response["Items"]: 614 | print("Deleting %s -- version %s" % 615 | (secret["name"], secret["version"])) 616 | secrets.delete_item(Key=secret) 617 | 618 | 619 | def setKmsRegion(args): 620 | """ 621 | set the KMS region independent of the DDB table region 622 | this value is stored in the file ~/.credstash 623 | """ 624 | options = loadConfig() 625 | options['kms-region'] = args.save_kms_region 626 | writeConfig(options) 627 | print("KMS region set to {}".format(args.save_kms_region)) 628 | 629 | 630 | def getKmsRegion(): 631 | options = loadConfig() 632 | return options.get('kms-region') 633 | 634 | 635 | def loadConfig(): 636 | config = os.path.expanduser("~/.credstash") 637 | 638 | try: 639 | with open(config) as f: 640 | options = json.load(f) 641 | except IOError: 642 | options = {} 643 | 644 | return options 645 | 646 | 647 | def writeConfig(options): 648 | config = os.path.expanduser("~/.credstash") 649 | 650 | with open(config, 'w') as f: 651 | json.dump(options, f) 652 | 653 | 654 | @clean_fail 655 | def createDdbTable(region=None, table="credential-store", tags=None, **kwargs): 656 | ''' 657 | create the secret store table in DDB in the specified region 658 | ''' 659 | session = get_session(**kwargs) 660 | dynamodb = session.resource("dynamodb", region_name=region) 661 | if table in (t.name for t in dynamodb.tables.all()): 662 | print("Credential Store table already exists") 663 | return 664 | 665 | print("Creating table...") 666 | dynamodb.create_table( 667 | TableName=table, 668 | KeySchema=[ 669 | { 670 | "AttributeName": "name", 671 | "KeyType": "HASH", 672 | }, 673 | { 674 | "AttributeName": "version", 675 | "KeyType": "RANGE", 676 | } 677 | ], 678 | AttributeDefinitions=[ 679 | { 680 | "AttributeName": "name", 681 | "AttributeType": "S", 682 | }, 683 | { 684 | "AttributeName": "version", 685 | "AttributeType": "S", 686 | }, 687 | ], 688 | ProvisionedThroughput={ 689 | "ReadCapacityUnits": 1, 690 | "WriteCapacityUnits": 1, 691 | } 692 | ) 693 | 694 | print("Waiting for table to be created...") 695 | client = session.client("dynamodb", region_name=region) 696 | 697 | response = client.describe_table(TableName=table) 698 | 699 | client.get_waiter("table_exists").wait(TableName=table) 700 | 701 | print("Adding tags...") 702 | 703 | client.tag_resource( 704 | ResourceArn=response["Table"]["TableArn"], 705 | Tags=[ 706 | { 707 | 'Key': "Name", 708 | 'Value': "credstash" 709 | }, 710 | ] 711 | ) 712 | 713 | if tags: 714 | tagset = [] 715 | for tag in tags: 716 | tagset.append({'Key': tag[0], 'Value': tag[1]}) 717 | client.tag_resource( 718 | ResourceArn=response["Table"]["TableArn"], 719 | Tags=tagset 720 | ) 721 | 722 | print("Table has been created. " 723 | "Go read the README about how to create your KMS key") 724 | 725 | 726 | def get_session(aws_access_key_id=None, aws_secret_access_key=None, 727 | aws_session_token=None, profile_name=None): 728 | if aws_access_key_id is not None: 729 | if aws_access_key_id not in get_session._cached_sessions: 730 | get_session._cached_sessions[aws_access_key_id] = boto3.Session( 731 | aws_access_key_id=aws_access_key_id, 732 | aws_secret_access_key=aws_secret_access_key, 733 | aws_session_token=aws_session_token, 734 | profile_name=profile_name 735 | ) 736 | get_session._last_session = get_session._cached_sessions[aws_access_key_id] 737 | return get_session._cached_sessions[aws_access_key_id] 738 | else: 739 | if get_session._last_session is None: 740 | get_session._last_session = boto3.Session(profile_name=profile_name) 741 | return get_session._last_session 742 | get_session._cached_sessions = {} 743 | get_session._last_session = None 744 | 745 | def reset_sessions(): 746 | get_session._cached_sessions = {} 747 | get_session._last_session = None 748 | 749 | 750 | def get_assumerole_credentials(arn): 751 | sts_client = boto3.client('sts') 752 | # Use client object and pass the role ARN 753 | assumedRoleObject = sts_client.assume_role(RoleArn=arn, 754 | RoleSessionName="AssumeRoleCredstashSession1") 755 | credentials = assumedRoleObject['Credentials'] 756 | return dict(aws_access_key_id=credentials['AccessKeyId'], 757 | aws_secret_access_key=credentials['SecretAccessKey'], 758 | aws_session_token=credentials['SessionToken']) 759 | 760 | 761 | def open_aes_ctr_legacy(key_service, material): 762 | """ 763 | Decrypts secrets stored by `seal_aes_ctr_legacy`. 764 | Assumes that the plaintext is unicode (non-binary). 765 | """ 766 | key = key_service.decrypt(b64decode(material['key'])) 767 | digest_method = material.get('digest', DEFAULT_DIGEST) 768 | ciphertext = b64decode(material['contents']) 769 | if hasattr(material['hmac'], "value"): 770 | hmac = codecs.decode(material['hmac'].value, "hex") 771 | else: 772 | hmac = codecs.decode(material['hmac'], "hex") 773 | return _open_aes_ctr(key, LEGACY_NONCE, ciphertext, hmac, digest_method).decode("utf-8") 774 | 775 | 776 | def seal_aes_ctr_legacy(key_service, secret, digest_method=DEFAULT_DIGEST): 777 | """ 778 | Encrypts `secret` using the key service. 779 | You can decrypt with the companion method `open_aes_ctr_legacy`. 780 | """ 781 | # generate a a 64 byte key. 782 | # Half will be for data encryption, the other half for HMAC 783 | key, encoded_key = key_service.generate_key_data(64) 784 | ciphertext, hmac = _seal_aes_ctr( 785 | secret, key, LEGACY_NONCE, digest_method, 786 | ) 787 | return { 788 | 'key': b64encode(encoded_key).decode('utf-8'), 789 | 'contents': b64encode(ciphertext).decode('utf-8'), 790 | 'hmac': codecs.encode(hmac, "hex_codec"), 791 | 'digest': digest_method, 792 | } 793 | 794 | 795 | def _open_aes_ctr(key, nonce, ciphertext, expected_hmac, digest_method): 796 | data_key, hmac_key = _halve_key(key) 797 | hmac = _get_hmac(hmac_key, ciphertext, digest_method) 798 | # Check the HMAC before we decrypt to verify ciphertext integrity 799 | if not constant_time.bytes_eq(hmac, expected_hmac): 800 | raise IntegrityError("Computed HMAC on %s does not match stored HMAC") 801 | 802 | decryptor = Cipher( 803 | algorithms.AES(data_key), 804 | modes.CTR(nonce), 805 | backend=default_backend() 806 | ).decryptor() 807 | return decryptor.update(ciphertext) + decryptor.finalize() 808 | 809 | 810 | def _seal_aes_ctr(plaintext, key, nonce, digest_method): 811 | data_key, hmac_key = _halve_key(key) 812 | encryptor = Cipher( 813 | algorithms.AES(data_key), 814 | modes.CTR(nonce), 815 | backend=default_backend() 816 | ).encryptor() 817 | 818 | ciphertext = encryptor.update(plaintext.encode("utf-8")) + encryptor.finalize() 819 | return ciphertext, _get_hmac(hmac_key, ciphertext, digest_method) 820 | 821 | 822 | def _get_hmac(key, ciphertext, digest_method): 823 | hmac = HMAC( 824 | key, 825 | get_digest(digest_method), 826 | backend=default_backend() 827 | ) 828 | hmac.update(ciphertext) 829 | return hmac.finalize() 830 | 831 | 832 | def _halve_key(key): 833 | half = len(key) // 2 834 | return key[:half], key[half:] 835 | 836 | 837 | def get_digest(digest): 838 | try: 839 | return _hash_classes[digest]() 840 | except KeyError: 841 | raise ValueError("Could not find " + digest + " in cryptography.hazmat.primitives.hashes") 842 | 843 | 844 | @clean_fail 845 | def list_credentials(region, args, **session_params): 846 | credential_list = listSecrets(region=region, 847 | table=args.table, 848 | **session_params) 849 | if credential_list: 850 | # print list of credential names and versions, 851 | # sorted by name and then by version 852 | max_len = max([len(x["name"]) for x in credential_list]) 853 | for cred in sorted(credential_list, 854 | key=operator.itemgetter("name", "version")): 855 | print("{0:{1}} -- version {2:>} -- comment {3}".format( 856 | cred["name"], max_len, cred["version"], cred.get("comment", ""))) 857 | else: 858 | return 859 | 860 | 861 | @clean_fail 862 | def list_credential_keys(region, args, **session_params): 863 | credential_list = listSecrets(region=region, 864 | table=args.table, 865 | **session_params) 866 | if credential_list: 867 | creds = sorted(set(cred["name"] for cred in credential_list)) 868 | for cred in creds: 869 | print(cred) 870 | else: 871 | return 872 | 873 | 874 | def get_session_params(profile, arn): 875 | params = {} 876 | if profile is None and arn: 877 | params = get_assumerole_credentials(arn) 878 | elif profile: 879 | params = dict(profile_name=profile) 880 | return params 881 | 882 | 883 | def get_parser(): 884 | """get the parsers dict""" 885 | parsers = {} 886 | parsers['super'] = argparse.ArgumentParser( 887 | description="A credential/secret storage system") 888 | 889 | parsers['super'].add_argument("-r", "--region", 890 | help="the AWS region in which to operate. " 891 | "If a region is not specified, credstash " 892 | "will use the value of the " 893 | "AWS_DEFAULT_REGION env variable, " 894 | "or if that is not set, the value in " 895 | "`~/.aws/config`. As a last resort, " 896 | "it will use " + DEFAULT_REGION) 897 | parsers['super'].add_argument("--kms-region", type=str, default=None, 898 | help="Region the credstash KMS key will be read from, " 899 | "independent of the region the DDB table is in. If not specified, " 900 | "the KMS region will follow the same resolution path as --region. " 901 | "To save the KMS region, use `credstash setup --save-kms-region KMS_REGION`. " 902 | "The value in this argument takes precedence any saved value.") 903 | parsers['super'].add_argument("-t", "--table", default=os.environ.get("CREDSTASH_DEFAULT_TABLE", "credential-store"), 904 | help="DynamoDB table to use for credential storage. " 905 | "If not specified, credstash " 906 | "will use the value of the " 907 | "CREDSTASH_DEFAULT_TABLE env variable, " 908 | "or if that is not set, the value " 909 | "`credential-store` will be used") 910 | parsers['super'].add_argument("--log-level", 911 | help="Set the log level, default WARNING", 912 | default='WARNING' 913 | ) 914 | parsers['super'].add_argument("--log-file", 915 | help="Set the log output file, default credstash.log. Errors are " 916 | "printed to stderr and stack traces are logged to file", 917 | default='credstash.log' 918 | ) 919 | 920 | role_parse = parsers['super'].add_mutually_exclusive_group() 921 | role_parse.add_argument("-p", "--profile", default=None, 922 | help="Boto config profile to use when " 923 | "connecting to AWS") 924 | role_parse.add_argument("-n", "--arn", default=None, 925 | help="AWS IAM ARN for AssumeRole") 926 | subparsers = parsers['super'].add_subparsers(help='Try commands like ' 927 | '"{name} get -h" or "{name} ' 928 | 'put --help" to get each ' 929 | 'sub command\'s options' 930 | .format(name=sys.argv[0])) 931 | 932 | action = 'delete' 933 | parsers[action] = subparsers.add_parser(action, 934 | help='Delete a credential from the store') 935 | parsers[action].add_argument("credential", type=str, 936 | help="the name of the credential to delete") 937 | parsers[action].set_defaults(action=action) 938 | 939 | action = 'get' 940 | parsers[action] = subparsers.add_parser(action, help="Get a credential " 941 | "from the store") 942 | parsers[action].add_argument("credential", type=str, 943 | help="the name of the credential to get. " 944 | "Using the wildcard character '%s' will " 945 | "search for credentials that match the " 946 | "pattern" % WILDCARD_CHAR) 947 | parsers[action].add_argument("context", type=key_value_pair, 948 | action=KeyValueToDictionary, nargs='*', 949 | help="encryption context key/value pairs " 950 | "associated with the credential in the form " 951 | "of \"key=value\"") 952 | parsers[action].add_argument("-n", "--noline", action="store_true", 953 | help="Don't append newline to returned " 954 | "value (useful in scripts or with " 955 | "binary files)") 956 | parsers[action].add_argument("-v", "--version", default="", 957 | help="Get a specific version of the " 958 | "credential (defaults to the latest version)") 959 | parsers[action].add_argument("-f", "--format", default="json", 960 | choices=["json", "csv", "dotenv"] + 961 | ([] if NO_YAML else ["yaml"]), 962 | help="Output format. json(default) " + 963 | ("" if NO_YAML else "yaml ") + " csv or dotenv.") 964 | parsers[action].set_defaults(action=action) 965 | 966 | action = 'getall' 967 | parsers[action] = subparsers.add_parser(action, 968 | help="Get all credentials from " 969 | "the store") 970 | parsers[action].add_argument("context", type=key_value_pair, 971 | action=KeyValueToDictionary, nargs='*', 972 | help="encryption context key/value pairs " 973 | "associated with the credential in the form " 974 | "of \"key=value\"") 975 | parsers[action].add_argument("-v", "--version", default="", 976 | help="Get a specific version of the " 977 | "credential (defaults to the latest version)") 978 | parsers[action].add_argument("-f", "--format", default="json", 979 | choices=["json", "csv", "dotenv"] + 980 | ([] if NO_YAML else ["yaml"]), 981 | help="Output format. json(default) " + 982 | ("" if NO_YAML else "yaml ") + " csv or dotenv.") 983 | parsers[action].set_defaults(action=action) 984 | 985 | action = 'keys' 986 | parsers[action] = subparsers.add_parser(action, 987 | help="List all keys in the store") 988 | parsers[action].set_defaults(action=action) 989 | 990 | action = 'list' 991 | parsers[action] = subparsers.add_parser(action, 992 | help="list credentials and " 993 | "their versions") 994 | parsers[action].set_defaults(action=action) 995 | 996 | action = 'put' 997 | parsers[action] = subparsers.add_parser(action, 998 | help="Put a credential into " 999 | "the store") 1000 | parsers[action].add_argument("credential", type=str, 1001 | help="the name of the credential to store") 1002 | parsers[action].add_argument("value", type=value_or_filename, 1003 | help="the value of the credential to store " 1004 | "or, if beginning with the \"@\" character, " 1005 | "the filename of the file containing " 1006 | "the value, or pass \"-\" to read the value " 1007 | "from stdin", default="", nargs="?") 1008 | parsers[action].add_argument("context", type=key_value_pair, 1009 | action=KeyValueToDictionary, nargs='*', 1010 | help="encryption context key/value pairs " 1011 | "associated with the credential in the form " 1012 | "of \"key=value\"") 1013 | parsers[action].add_argument("-k", "--key", default="alias/credstash", 1014 | help="the KMS key-id of the master key " 1015 | "to use. See the README for more " 1016 | "information. Defaults to alias/credstash") 1017 | parsers[action].add_argument("-c", "--comment", type=str, 1018 | help="Include reference information or a comment about " 1019 | "value to be stored.") 1020 | parsers[action].add_argument("-v", "--version", default="1", 1021 | help="Put a specific version of the " 1022 | "credential (update the credential; " 1023 | "defaults to version `1`).") 1024 | parsers[action].add_argument("-a", "--autoversion", action="store_true", 1025 | help="Automatically increment the version of " 1026 | "the credential to be stored. This option " 1027 | "causes the `-v` flag to be ignored. " 1028 | "(This option will fail if the currently stored " 1029 | "version is not numeric.)") 1030 | parsers[action].add_argument("-d", "--digest", default=DEFAULT_DIGEST, 1031 | choices=HASHING_ALGORITHMS, 1032 | help="the hashing algorithm used to " 1033 | "to encrypt the data. Defaults to SHA256") 1034 | parsers[action].add_argument("-P", "--prompt", action="store_true", 1035 | help="Prompt for secret") 1036 | parsers[action].set_defaults(action=action) 1037 | 1038 | action = 'putall' 1039 | parsers[action] = subparsers.add_parser(action, 1040 | help="Put credentials from json into " 1041 | "the store") 1042 | parsers[action].add_argument("credentials", type=value_or_filename, 1043 | help="the value of the credential to store " 1044 | "or, if beginning with the \"@\" character, " 1045 | "the filename of the file containing " 1046 | "the values, or pass \"-\" to read the values " 1047 | "from stdin. Should be in json format.", default="") 1048 | parsers[action].add_argument("context", type=key_value_pair, 1049 | action=KeyValueToDictionary, nargs='*', 1050 | help="encryption context key/value pairs " 1051 | "associated with the credential in the form " 1052 | "of \"key=value\"") 1053 | parsers[action].add_argument("-k", "--key", default="alias/credstash", 1054 | help="the KMS key-id of the master key " 1055 | "to use. See the README for more " 1056 | "information. Defaults to alias/credstash") 1057 | parsers[action].add_argument("-v", "--version", default="", 1058 | help="Put a specific version of the " 1059 | "credential (update the credential; " 1060 | "defaults to version `1`).") 1061 | parsers[action].add_argument("-c", "--comment", type=str, 1062 | help="Include reference information or a comment about " 1063 | "value to be stored.") 1064 | parsers[action].add_argument("-a", "--autoversion", action="store_true", 1065 | help="Automatically increment the version of " 1066 | "the credential to be stored. This option " 1067 | "causes the `-v` flag to be ignored. " 1068 | "(This option will fail if the currently stored " 1069 | "version is not numeric.)") 1070 | parsers[action].add_argument("-d", "--digest", default="SHA256", 1071 | choices=HASHING_ALGORITHMS, 1072 | help="the hashing algorithm used to " 1073 | "to encrypt the data. Defaults to SHA256") 1074 | parsers[action].set_defaults(action=action) 1075 | action = 'setup' 1076 | parsers[action] = subparsers.add_parser(action, 1077 | help='setup the credential store') 1078 | parsers[action].add_argument("--save-kms-region", type=str, default=None, 1079 | help="Save the region the credstash KMS key will be read from, " 1080 | "independent of the region the DDB table is in. This value is saved " 1081 | "in ~/.credstash") 1082 | parsers[action].add_argument("--tags", type=key_value_pair, 1083 | help="Tags to apply to the Dynamodb Table " 1084 | "passed in as a space sparated list of Key=Value", nargs="*") 1085 | parsers[action].set_defaults(action=action) 1086 | return parsers 1087 | 1088 | def main(): 1089 | parsers = get_parser() 1090 | args = parsers['super'].parse_args() 1091 | global invocation_type 1092 | invocation_type = CLI_INVOCATION_TYPE 1093 | 1094 | # setup logging 1095 | setup_logging(args.log_level, args.log_file) 1096 | 1097 | # Check for assume role and set session params 1098 | session_params = get_session_params(args.profile, args.arn) 1099 | 1100 | # test for region 1101 | try: 1102 | region = args.region 1103 | session = get_session(**session_params) 1104 | session.resource('dynamodb', region_name=region) 1105 | except botocore.exceptions.NoRegionError: 1106 | if 'AWS_DEFAULT_REGION' not in os.environ: 1107 | region = DEFAULT_REGION 1108 | 1109 | # get KMS region (otherwise it is the same as region) 1110 | kms_region = args.kms_region or getKmsRegion() or region 1111 | 1112 | if "action" in vars(args): 1113 | if args.action == "delete": 1114 | deleteSecrets(args.credential, 1115 | region=region, 1116 | table=args.table, 1117 | **session_params) 1118 | return 1119 | if args.action == "list": 1120 | list_credentials(region, args, **session_params) 1121 | return 1122 | if args.action == "keys": 1123 | list_credential_keys(region, args, **session_params) 1124 | return 1125 | if args.action == "put": 1126 | putSecretAction(args, region, kms_region, **session_params) 1127 | return 1128 | if args.action == "putall": 1129 | putAllSecretsAction(args, region, kms_region, **session_params) 1130 | return 1131 | if args.action == "get": 1132 | getSecretAction(args, region, kms_region, **session_params) 1133 | return 1134 | if args.action == "getall": 1135 | getAllAction(args, region, kms_region, **session_params) 1136 | return 1137 | if args.action == "setup": 1138 | if args.save_kms_region: 1139 | setKmsRegion(args) 1140 | createDdbTable(region=region, table=args.table, 1141 | tags=args.tags, **session_params) 1142 | return 1143 | else: 1144 | parsers['super'].print_help() 1145 | 1146 | if __name__ == '__main__': 1147 | main() 1148 | -------------------------------------------------------------------------------- /credstash_migrate_digests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Updates digests in a credstash dynamodb table. Default behavior is to 3 | # update all deprecated WHIRLPOOL and RIPEMD digests to the default SHA256. 4 | # Clears the way for credstash update removing deprecated hashes. 5 | # 6 | # Usage: AWS_PROFILE=my_profile python3 credstash_migrate_digests.py 7 | 8 | import sys 9 | from subprocess import Popen, PIPE 10 | from collections import defaultdict 11 | from boto3 import resource 12 | import credstash 13 | 14 | 15 | def main(): 16 | UPDATED_DIGEST = 'SHA256' 17 | DIGESTS_TO_UPDATE = ['WHIRLPOOL', 'RIPEMD'] 18 | 19 | keys = defaultdict(lambda:0) 20 | keys_to_update = [] 21 | 22 | dynamodb_resource = resource('dynamodb') 23 | table = dynamodb_resource.Table('credential-store') 24 | response = table.scan() 25 | 26 | items = response['Items'] 27 | 28 | # appending all dynamodb entries to items dict 29 | while True: 30 | if response.get('LastEvaluatedKey'): 31 | response = table.scan(ExclusiveStartKey=response['LastEvaluatedKey']) 32 | items += response['Items'] 33 | else: 34 | break 35 | 36 | # storing latest version of keys with their digests 37 | for i in range(len(items)): 38 | try: 39 | digest = items[i]['digest'] 40 | version = int(items[i]['version']) 41 | key = items[i]['name'] 42 | except: 43 | continue 44 | 45 | if key in keys: 46 | if version > keys[key][0]: 47 | keys[key][0] = version 48 | keys[key][1] = digest 49 | else: 50 | keys[key] = [version, digest] 51 | 52 | # store keys to be updated 53 | for k, v in keys.items(): 54 | if v[1] in DIGESTS_TO_UPDATE: 55 | keys_to_update.append(k) 56 | 57 | # confirms update of digests 58 | if len(keys_to_update): 59 | print('\nThe following keys will be updated to {0}:\n'.format(UPDATED_DIGEST)) 60 | for key in keys_to_update: 61 | print('{0}\n'.format(key)) 62 | confirmed = None 63 | while not confirmed: 64 | val = input('Continue? y/n ') 65 | if val.lower() == 'y' or val.lower() == 'yes': 66 | confirmed = True 67 | elif val.lower() == 'n' or val.lower() == 'no': 68 | print('\nexiting...\n') 69 | sys.exit() 70 | else: 71 | print('\nInvalid input\n') 72 | else: 73 | print('\nNo digests to update!\n') 74 | sys.exit() 75 | 76 | # updating deprecated digests 77 | for key in keys_to_update: 78 | p = Popen(['credstash', 'get', key], stdout=PIPE, stderr=PIPE) 79 | secret, err = p.communicate() 80 | secret = secret[:-1] # removes credstash-added newline for stdout 81 | if not err: 82 | p = Popen(['credstash', 'put', key, secret, '-a', '-d', UPDATED_DIGEST], stdout=PIPE) 83 | update, err = p.communicate() 84 | print('{0} has been updated!\n'.format(key)) 85 | else: 86 | print('Error found, skipping update of {0}. Error: {1}'.format(key, err)) 87 | 88 | if __name__ == '__main__': 89 | main() 90 | -------------------------------------------------------------------------------- /integration_tests/test_commands.bats: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bats 2 | 3 | # test basic CRUD 4 | @test "put secret into credstash" { 5 | credstash put __batstestcred1 secretvalue 6 | } 7 | 8 | @test "read secret from credstash" { 9 | SECRET=$(credstash get __batstestcred1) 10 | [ "$SECRET" = secretvalue ] 11 | } 12 | 13 | @test "add a new version to a secret in credstash" { 14 | credstash put __batstestcred1 secretvalue2 -a 15 | } 16 | 17 | @test "read latest version of a secret from credstash" { 18 | SECRET=$(credstash get __batstestcred1) 19 | [ "$SECRET" = secretvalue2 ] 20 | } 21 | 22 | @test "read previous version of a secret from credstash" { 23 | SECRET=$(credstash get __batstestcred1 -v 1) 24 | [ "$SECRET" = secretvalue ] 25 | } 26 | 27 | @test "delete a secret from credstash" { 28 | credstash delete __batstestcred1 29 | } -------------------------------------------------------------------------------- /integration_tests/test_credstash_lib.py: -------------------------------------------------------------------------------- 1 | # test credstash when imported as a library 2 | # run using `pytest integration_tests/test_credstash_lib.py` 3 | import credstash 4 | import pytest 5 | import botocore.exceptions 6 | 7 | @pytest.yield_fixture 8 | def secret(): 9 | secret = { 10 | 'name': 'test', 11 | 'version': '0000000000000000000', 12 | 'value': 'secret' 13 | } 14 | credstash.putSecret(secret['name'], secret['value']) 15 | try: 16 | yield secret 17 | finally: 18 | credstash.deleteSecrets("test") 19 | 20 | def test_listSecrets(secret): 21 | secrets = credstash.listSecrets() 22 | del secret['value'] 23 | assert secrets == [secret] 24 | 25 | 26 | def test_getSecret(secret): 27 | s = credstash.getSecret(secret['name']) 28 | assert s == secret['value'] 29 | 30 | 31 | def test_getSecret_wrong_region(secret): 32 | try: 33 | credstash.getSecret(secret['name'], region='us-west-2') 34 | except botocore.exceptions.ClientError as e: 35 | if e.response['Error']['Code'] == 'ResourceNotFoundException': 36 | assert True 37 | else: 38 | assert False, "expected botocore ResourceNotFoundException" 39 | 40 | def test_getSecret_nonexistent(): 41 | try: 42 | credstash.getSecret("bad secret") 43 | except credstash.ItemNotFound: 44 | assert True 45 | else: 46 | assert False, "expected credstash.ItemNotFound error" 47 | 48 | 49 | def test_getAllSecrets(secret): 50 | s = credstash.getAllSecrets() 51 | assert s == {secret['name']:secret['value']} 52 | 53 | 54 | def test_getAllSecrets_no_secrets(): 55 | s = credstash.getAllSecrets() 56 | assert s == dict() 57 | 58 | 59 | def test_deleteSecret(secret): 60 | secrets = credstash.listSecrets() 61 | del secret['value'] 62 | assert secrets == [secret] 63 | 64 | credstash.deleteSecrets(secret['name']) 65 | secrets = credstash.listSecrets() 66 | assert secrets == [] -------------------------------------------------------------------------------- /integration_tests/test_kms_region.bats: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bats 2 | 3 | # test basic CRUD with separate KMS and DDB regions 4 | # these tests require a duplicated DDB table in us-east-2 5 | @test "put secret into credstash" { 6 | credstash --region us-east-2 --kms-region us-east-1 put __batstestcred1 secretvalue 7 | } 8 | 9 | @test "read secret from credstash" { 10 | SECRET=$(credstash --region us-east-2 --kms-region us-east-1 get __batstestcred1) 11 | [ "$SECRET" = secretvalue ] 12 | } 13 | 14 | @test "add a new version to a secret in credstash" { 15 | credstash --region us-east-2 --kms-region us-east-1 put __batstestcred1 secretvalue2 -a 16 | } 17 | 18 | @test "read latest version of a secret from credstash" { 19 | SECRET=$(credstash --region us-east-2 --kms-region us-east-1 get __batstestcred1) 20 | [ "$SECRET" = secretvalue2 ] 21 | } 22 | 23 | @test "read previous version of a secret from credstash" { 24 | SECRET=$(credstash --region us-east-2 --kms-region us-east-1 get __batstestcred1 -v 1) 25 | [ "$SECRET" = secretvalue ] 26 | } 27 | 28 | @test "delete a secret from credstash" { 29 | credstash --region us-east-2 --kms-region us-east-1 delete __batstestcred1 30 | } -------------------------------------------------------------------------------- /optional-requirements.txt: -------------------------------------------------------------------------------- 1 | # Optional Dependencies 2 | # To install, run: 3 | # $ pip install -r optional-requirements.txt 4 | 5 | # In order to output `getall` in YAML format you need PyYAML 6 | # pytest is needed to run the tests in test_credstash_lib.py 7 | pyyaml>=4.2b1 8 | pytest>=5.4.1 -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | cryptography>=2.1 2 | boto3>=1.1.1 3 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [flake8] 2 | ignore = T001 3 | max-line-length = 120 4 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from setuptools import setup 3 | 4 | name = 'credstash' 5 | version = '1.17.1' 6 | 7 | setup( 8 | name=name, 9 | version=version, 10 | description='A utility for managing secrets in the cloud using AWS KMS and DynamoDB', 11 | author="Alex Schoof, Mike Lin, et al.", 12 | author_email="mike@fugue.co", 13 | license='Apache2', 14 | url="https://github.com/fugue/credstash", 15 | classifiers=[ 16 | 'Intended Audience :: Developers', 17 | 'Intended Audience :: System Administrators', 18 | 'License :: OSI Approved :: Apache Software License', 19 | ], 20 | scripts=['credstash.py'], 21 | py_modules=['credstash'], 22 | install_requires=[ 23 | 'cryptography>=2.1', 24 | 'boto3>=1.1.1', 25 | ], 26 | extras_require={ 27 | 'YAML': ['PyYAML>=3.10'] 28 | }, 29 | entry_points={ 30 | 'console_scripts': [ 31 | 'credstash = credstash:main' 32 | ] 33 | }, 34 | setup_requires=[ 35 | 'pytest>=5.4.1' 36 | ], 37 | ) 38 | -------------------------------------------------------------------------------- /tests/expand_wildcard_test.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | from credstash import expand_wildcard 3 | 4 | 5 | class TestExpandingWildcard(unittest.TestCase): 6 | secrets_set = ["a", "b", "ab", " a", " b", 7 | "ba", "abc", "a[anyvalue]z", "a b", "aabb"] 8 | secrets_set2 = ["QQQ", "QVQQ", "QVQVQ", 9 | "QQ", "Q", "QQVQ", "QrEQrE", "QErQE"] 10 | 11 | def test_start_regex(self): 12 | self.assertEqual(expand_wildcard("a", self.secrets_set), ["a"]) 13 | 14 | def test_end_regex(self): 15 | self.assertEqual(expand_wildcard("ba", self.secrets_set), ["ba"]) 16 | 17 | def test_exact_match_regex(self): 18 | self.assertEqual(expand_wildcard("abc", self.secrets_set), ["abc"]) 19 | 20 | def test_one_wild_card_with_one_match(self): 21 | self.assertEqual(expand_wildcard( 22 | "a*z", self.secrets_set), ["a[anyvalue]z"]) 23 | 24 | def test_one_wild_card_with_many_matches(self): 25 | self.assertEqual(expand_wildcard( 26 | "a*b", self.secrets_set), ["ab", "a b", "aabb"]) 27 | 28 | def test_two_wild_cards_with_many_matches(self): 29 | self.assertEqual(expand_wildcard( 30 | "Q*Q*Q", self.secrets_set2), ["QQQ", "QVQQ", "QVQVQ", "QQVQ"]) 31 | 32 | def test_three_wild_card_with_many_matches(self): 33 | self.assertEqual(expand_wildcard( 34 | "Q*E*Q*E", self.secrets_set2), ["QrEQrE", "QErQE"]) 35 | -------------------------------------------------------------------------------- /tests/get_session_test.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | from unittest.mock import patch, MagicMock 3 | from credstash import get_session, reset_sessions 4 | 5 | class TestGetSession(unittest.TestCase): 6 | def setUp(self): 7 | reset_sessions() 8 | 9 | @patch('boto3.Session') 10 | def test_get_session_initial_session(self, mock_session): 11 | mock_session.return_value = 'session1' 12 | get_session( 13 | aws_access_key_id='session1' 14 | ) 15 | mock_session.assert_called_once_with( 16 | aws_access_key_id='session1', 17 | aws_secret_access_key=None, 18 | aws_session_token=None, 19 | profile_name=None 20 | ) 21 | 22 | @patch('boto3.Session') 23 | def test_get_session_single_last_session(self, mock_session): 24 | mock_session.return_value = 'session1' 25 | get_session( 26 | aws_access_key_id='session1' 27 | ) 28 | mock_session.assert_called_once_with( 29 | aws_access_key_id='session1', 30 | aws_secret_access_key=None, 31 | aws_session_token=None, 32 | profile_name=None 33 | ) 34 | self.assertEqual(get_session(), 'session1') 35 | 36 | @patch('boto3.Session') 37 | def test_get_session_two_sessions(self, mock_session): 38 | mock_session.side_effect = ['session1', 'session2'] 39 | get_session( 40 | aws_access_key_id='session1' 41 | ) 42 | mock_session.assert_called_with( 43 | aws_access_key_id='session1', 44 | aws_secret_access_key=None, 45 | aws_session_token=None, 46 | profile_name=None 47 | ) 48 | get_session( 49 | aws_access_key_id='session2' 50 | ) 51 | mock_session.assert_called_with( 52 | aws_access_key_id='session2', 53 | aws_secret_access_key=None, 54 | aws_session_token=None, 55 | profile_name=None 56 | ) 57 | self.assertEqual(get_session(), 'session2') 58 | self.assertEqual(get_session(aws_access_key_id='session1'), 'session1') 59 | self.assertEqual(get_session(), 'session1') 60 | self.assertEqual(get_session(aws_access_key_id='session2'), 'session2') 61 | self.assertEqual(get_session(), 'session2') 62 | 63 | @patch('boto3.Session') 64 | def test_get_session_no_params(self, mock_session): 65 | mock_session.return_value = 'defaultsession' 66 | self.assertEqual(get_session(), 'defaultsession') 67 | self.assertEqual(get_session(), 'defaultsession') 68 | mock_session.assert_called_once_with(profile_name=None) 69 | 70 | @patch('boto3.Session') 71 | def test_get_session_specify_profile(self, mock_session): 72 | mock_session.return_value = 'session1' 73 | get_session( 74 | profile_name='profile1' 75 | ) 76 | mock_session.assert_called_once_with( 77 | profile_name='profile1' 78 | ) 79 | -------------------------------------------------------------------------------- /tests/key_pair_test.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import argparse 3 | from credstash import key_value_pair 4 | 5 | 6 | class TestKeyValuePairExtraction(unittest.TestCase): 7 | 8 | def test_key_value_pair_has_two_equals_test(self): 9 | self.assertRaises(argparse.ArgumentTypeError, key_value_pair, "==") 10 | 11 | def test_key_value_pair_has_zero_equals(self): 12 | self.assertRaises(argparse.ArgumentTypeError, key_value_pair, "") 13 | 14 | def test_key_value_pair_has_one_equals(self): 15 | self.assertRaises(argparse.ArgumentTypeError, key_value_pair, "key1=key2=key3") 16 | 17 | def test_key_value_pair_has_both_key_and_value(self): 18 | self.assertRaises(argparse.ArgumentTypeError, key_value_pair, "key=") 19 | self.assertRaises(argparse.ArgumentTypeError, key_value_pair, "=value") 20 | 21 | def test_key_value_pair_has_one_equals_with_values(self): 22 | self.assertEqual(key_value_pair("key1=value1"), ["key1", "value1"]) 23 | -------------------------------------------------------------------------------- /tests/key_service_test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright 2015 Luminal, Inc. 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | from __future__ import print_function 16 | 17 | import unittest 18 | import boto3 19 | from botocore.stub import Stubber 20 | 21 | from credstash import KeyService, KmsError 22 | 23 | class TestKeyService(unittest.TestCase): 24 | def test_generate_key_data_success(self): 25 | kms_client = boto3.client('kms') 26 | key_id = "test" 27 | encryption_context = {} 28 | with Stubber(kms_client) as stubber: 29 | stubber.add_response('generate_data_key', { 30 | 'CiphertextBlob': b'ciphertext', 31 | 'Plaintext': b'plaintext', 32 | 'KeyId': 'string' 33 | }, expected_params = { 34 | 'KeyId': key_id, 35 | 'EncryptionContext': encryption_context, 36 | 'NumberOfBytes': 1 37 | }) 38 | key_service = KeyService(kms_client, key_id, encryption_context) 39 | response = key_service.generate_key_data(1) 40 | self.assertEqual(response[0], b'plaintext') 41 | self.assertEqual(response[1], b'ciphertext') 42 | 43 | def test_generate_key_data_error(self): 44 | kms_client = boto3.client('kms') 45 | key_id = "test" 46 | encryption_context = {} 47 | with Stubber(kms_client) as stubber: 48 | stubber.add_client_error( 49 | 'generate_key_data', 50 | 'KeyUnavailableException', 51 | 'The request was rejected because the specified CMK was not available. The request can be retried.', 52 | 500, 53 | expected_params={ 54 | 'KeyId': key_id, 55 | 'EncryptionContext': encryption_context, 56 | 'NumberOfBytes': 1 57 | }) 58 | key_service = KeyService(kms_client, key_id, encryption_context) 59 | with self.assertRaises(KmsError) as e: 60 | key_service.generate_key_data(1) 61 | self.assertEqual(e, KmsError("Could not generate key using KMS key %s (Details: %s)" % (key_id, 'The request was rejected because the specified CMK was not available. The request can be retried.'))) 62 | 63 | def test_decrypt_success(self): 64 | kms_client = boto3.client('kms') 65 | key_id = "test" 66 | encryption_context = {} 67 | with Stubber(kms_client) as stubber: 68 | stubber.add_response('decrypt', { 69 | 'KeyId': 'key_id', 70 | 'Plaintext': b'plaintext' 71 | }, expected_params = { 72 | 'CiphertextBlob': 'encoded_key', 73 | 'EncryptionContext': encryption_context 74 | }) 75 | key_service = KeyService(kms_client, key_id, encryption_context) 76 | response = key_service.decrypt('encoded_key') 77 | self.assertEqual(response, b'plaintext') 78 | 79 | def test_decrypt_error(self): 80 | kms_client = boto3.client('kms') 81 | key_id = "test" 82 | encryption_context = {} 83 | with Stubber(kms_client) as stubber: 84 | stubber.add_client_error( 85 | 'decrypt', 86 | 'NotFoundException', 87 | 'The request was rejected because the specified entity or resource could not be found.', 88 | 400, 89 | expected_params = { 90 | 'CiphertextBlob': 'encoded_key', 91 | 'EncryptionContext': encryption_context 92 | }) 93 | key_service = KeyService(kms_client, key_id, encryption_context) 94 | with self.assertRaises(KmsError) as e: 95 | response = key_service.decrypt('encoded_key') 96 | self.assertEqual(e, KmsError("Decryption error The request was rejected because the specified entity or resource could not be found.")) 97 | 98 | def test_decrypt_invalid_ciphertext_error_no_context(self): 99 | kms_client = boto3.client('kms') 100 | key_id = "test" 101 | encryption_context = {} 102 | with Stubber(kms_client) as stubber: 103 | stubber.add_client_error( 104 | 'decrypt', 105 | 'InvalidCiphertextException', 106 | 'The request was rejected because the specified ciphertext, or additional authenticated data incorporated into the ciphertext, such as the encryption context, is corrupted, missing, or otherwise invalid.', 107 | 400, 108 | expected_params = { 109 | 'CiphertextBlob': 'encoded_key', 110 | 'EncryptionContext': encryption_context 111 | }) 112 | key_service = KeyService(kms_client, key_id, encryption_context) 113 | with self.assertRaises(KmsError) as e: 114 | msg = ("Could not decrypt hmac key with KMS. The credential may " 115 | "require that an encryption context be provided to decrypt " 116 | "it.") 117 | response = key_service.decrypt('encoded_key') 118 | self.assertEqual(e, KmsError(msg)) 119 | 120 | def test_decrypt_invalid_ciphertext_error_with_context(self): 121 | kms_client = boto3.client('kms') 122 | key_id = "test" 123 | encryption_context = { 124 | 'key': 'value' 125 | } 126 | with Stubber(kms_client) as stubber: 127 | stubber.add_client_error( 128 | 'decrypt', 129 | 'InvalidCiphertextException', 130 | 'The request was rejected because the specified ciphertext, or additional authenticated data incorporated into the ciphertext, such as the encryption context, is corrupted, missing, or otherwise invalid.', 131 | 400, 132 | expected_params = { 133 | 'CiphertextBlob': 'encoded_key', 134 | 'EncryptionContext': encryption_context 135 | }) 136 | key_service = KeyService(kms_client, key_id, encryption_context) 137 | with self.assertRaises(KmsError) as e: 138 | msg = ("Could not decrypt hmac key with KMS. The encryption " 139 | "context provided may not match the one used when the " 140 | "credential was stored.") 141 | response = key_service.decrypt('encoded_key') 142 | self.assertEqual(e, KmsError(msg)) 143 | 144 | 145 | -------------------------------------------------------------------------------- /tests/pad_left_test.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | from credstash import paddedInt 3 | 4 | 5 | class TestPadLeft(unittest.TestCase): 6 | def test_zero(self): 7 | i = 0 8 | self.assertEqual(paddedInt(i), "0" * 19) 9 | 10 | def test_ten(self): 11 | i = 10 12 | self.assertEqual(paddedInt(i), str(i).zfill(19)) 13 | 14 | def test_arbitrary_number(self): 15 | i = 98218329123 16 | self.assertEqual(paddedInt(i), str(i).zfill(19)) 17 | 18 | def test_huge_number(self): 19 | i = 12345678901234567890123 20 | self.assertEqual(paddedInt(i), str(i).zfill(19)) 21 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py36, lint 3 | 4 | [testenv] 5 | commands = 6 | python -m pytest 7 | python setup.py sdist bdist_wheel 8 | deps = 9 | pytest>=5.4.1 10 | setuptools>=17.1 11 | 12 | [testenv:lint] 13 | commands=flake8 --max-line-length 120 openapi 14 | basepython=python3.6 15 | deps= 16 | flake8 17 | --------------------------------------------------------------------------------