├── .gitignore ├── LICENSE ├── README.md ├── examples └── environment_import.sh ├── makefile ├── secretcli ├── __init__.py └── secretcli.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Robert Hafner 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # secretcli 2 | 3 | The secretcli project provides a simple to use command line interface to the [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/). It is capable of uploading or downloading the entire secret as well as working with individual fields. 4 | 5 | ## Installing 6 | 7 | This project is available on [pypi](https://pypi.org/project/secretcli/) and can be installed with pip. 8 | 9 | `pip3 install secretcli` 10 | 11 | ## Usage 12 | 13 | ### Initializing a new Secret 14 | 15 | New secrets are easy to initiate. This will create a new Secret in the AWS Secret Manager and store an empty javascript object as the first version. 16 | 17 | ```bash 18 | $ secretcli init TestSecret 19 | ``` 20 | 21 | ### Working with individual Keys 22 | 23 | Additional Key/Value pairs can be added to the secret using a single command. Behind the scenes this downloads the existing database, updates it with the new key/value pair, and uploads it as the current version. 24 | 25 | ```bash 26 | $ secretcli set TestSecret postgreshost 10.10.10.16 27 | $ secretcli set TestSecret postgresuser postgres 28 | $ secretcli set TestSecret postgrespassword super_secret_string 29 | $ secretcli set TestSecret longstring "This is a string with spaces." 30 | ``` 31 | 32 | Retrieving values is just as simple. This can be useful when trying to use values in bash scripts. 33 | 34 | ```bash 35 | $ secretcli get TestSecret postgreshost 36 | 10.10.10.16 37 | $ secretcli get TestSecret postgresuser 38 | postgres 39 | $ secretcli get TestSecret postgrespassword 40 | super_secret_string 41 | ``` 42 | 43 | Values can also be completely removed from the secret. 44 | 45 | ```bash 46 | $ secretcli get TestSecret postgreshost 47 | 10.10.10.16 48 | $ secretcli remove TestSecret postgreshost 49 | $ secretcli get TestSecret postgreshost 50 | ``` 51 | 52 | To avoid passing the value directly into the console (potentially logging it in places like bash history) the `-s` flag can be passed and the value can be passed in interactively without displaying it. 53 | 54 | 55 | ```bash 56 | $ secretcli set TestSecret postgrespassword -s 57 | Value: 58 | Repeat for confirmation: 59 | $ secretcli get TestSecret postgrespassword 60 | super_secret_string 61 | ``` 62 | 63 | ### Working with entire Files 64 | 65 | The entire Secret can be downloaded as a file. This command works regardless of the format of the file- Secrets that are not managed by `secretcli` can be downloaded using this tool. 66 | 67 | ```bash 68 | $ secretcli download TestSecret ./secret_configuration.json 69 | ``` 70 | 71 | The file can also be uploaded- but be careful, it will be uploaded exactly as is without any verification of the json formatting. 72 | 73 | ```bash 74 | $ secretcli upload TestSecret ./secret_configuration.json 75 | ``` 76 | 77 | ## Datastore Format 78 | 79 | `secretcli` stores data as a JSON Object in an attempt to be as interoperable as possible. Each `key` passed to `secretcli` is represented by a `key` in the JSON Object. 80 | 81 | When storing in AWS Secret Manager `secretcli` uses the `SecretString` field in the AWS Secrets Manager. This allows the database to be viewed in the AWS Console both as a raw string and using the Key/Value table. 82 | -------------------------------------------------------------------------------- /examples/environment_import.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This script loads all key/value pairs. Using a bash shell and running `source ./environment_import.sh SECRET_NAME` 4 | # will make the Secret values available as environmental variables. 5 | 6 | for x in $(secretcli list $1); do 7 | VALUE=$(secretcli get $1 $x) 8 | export $x="$VALUE" 9 | done 10 | -------------------------------------------------------------------------------- /makefile: -------------------------------------------------------------------------------- 1 | SHELL:=/bin/bash 2 | ROOT_DIR:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST)))) 3 | 4 | all: dependencies 5 | 6 | fresh: clean dependencies 7 | 8 | dependencies: 9 | if [ ! -d $(ROOT_DIR)/env ]; then python3 -m venv $(ROOT_DIR)/env; fi 10 | source $(ROOT_DIR)/env/bin/activate; yes w | python -m pip install -e .[dev] 11 | 12 | clean: 13 | rm -rf $(ROOT_DIR)/build; 14 | rm -rf $(ROOT_DIR)/dist; 15 | rm -rf $(ROOT_DIR)/env; 16 | rm -rf $(ROOT_DIR)/*.egg-info; 17 | rm -rf $(ROOT_DIR)/secretcli/*.pyc; 18 | 19 | package: 20 | source $(ROOT_DIR)/env/bin/activate; python setup.py bdist_wheel 21 | -------------------------------------------------------------------------------- /secretcli/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tedivm/secretcli/0157fa69f9523e5a05600b1d871982fc5bbc4898/secretcli/__init__.py -------------------------------------------------------------------------------- /secretcli/secretcli.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import click 3 | import json 4 | import os 5 | import requests 6 | import sys 7 | import yaml 8 | 9 | 10 | def get_secret(secret_name, region=None, raw=False): 11 | """Pull the specific secret down from the AWS Secrets Manager""" 12 | client = get_aws_client(region) 13 | 14 | # Depending on whether the secret was a string or binary, one of these fields will be populated 15 | get_secret_value_response = client.get_secret_value(SecretId=secret_name) 16 | if 'SecretString' in get_secret_value_response: 17 | secret = get_secret_value_response['SecretString'] 18 | else: 19 | secret = get_secret_value_response['SecretBinary'].decode("utf-8") 20 | 21 | if raw: 22 | return secret 23 | return yaml.safe_load(secret) 24 | 25 | def put_secret(secret_name, secret_value, region=None, raw=False, binary=False): 26 | """Save the supplied value as the secret in the AWS Secrets Manager""" 27 | client = get_aws_client(region) 28 | if raw: 29 | secret_string = secret_value 30 | else: 31 | secret_string = json.dumps(secret_value) 32 | 33 | args = { 34 | 'SecretId': secret_name, 35 | 'VersionStages': ['AWSCURRENT'] 36 | } 37 | 38 | if binary: 39 | args['SecretBinary'] = secret_string 40 | else: 41 | args['SecretString'] = secret_string 42 | 43 | response = client.put_secret_value(**args) 44 | 45 | def get_region(): 46 | """Extrapolate the preferred region when one isn't supplied""" 47 | boto3_session = boto3.session.Session() 48 | 49 | # Check for boto3/awscli default region. 50 | if boto3_session.region_name: 51 | return boto3_session.region_name 52 | 53 | # Check for specific environmental variable. 54 | if 'AWS_SECRETS_REGION' in os.environ: 55 | return os.environ['AWS_SECRETS_REGION'] 56 | 57 | # If this is being called from an EC2 instance use its region. 58 | r = requests.get('http://169.254.169.254/latest/dynamic/instance-identity/document', timeout=0.2) 59 | r.raise_for_status() 60 | data = r.json() 61 | return data['region'] 62 | 63 | def get_aws_client(region): 64 | """Return an initialized boto3 client pointing at the secretsmanager service""" 65 | if not region: 66 | region = get_region() 67 | 68 | return boto3.client( 69 | service_name='secretsmanager', 70 | region_name=region 71 | ) 72 | 73 | 74 | @click.group() 75 | @click.pass_context 76 | def cli(ctx): 77 | if ctx.parent: 78 | click.echo(ctx.parent.get_help()) 79 | 80 | 81 | @cli.command(short_help="Print the default region used by this application") 82 | def region(): 83 | click.echo(get_region()) 84 | 85 | 86 | @cli.command(short_help="Initialize a new secret in the AWS Secrets Manager") 87 | @click.argument('secret') 88 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 89 | @click.option('-d', '--description', default='') 90 | def init(secret, region, description): 91 | client = get_aws_client(region) 92 | response = client.create_secret( 93 | Name=secret, 94 | Description=description, 95 | SecretString='{}' 96 | ) 97 | 98 | 99 | @cli.command(short_help="Get a specific value from the secret datastore") 100 | @click.argument('secret') 101 | @click.argument('key') 102 | @click.option('-c', '--category', default=None) 103 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 104 | def get(secret, key, category, region): 105 | secret_data = get_secret(secret, region) 106 | if category: 107 | if category in secret_data: 108 | if key in secret_data[category]: 109 | click.echo(secret_data[category][key]) 110 | sys.exit(0) 111 | if key in secret_data: 112 | click.echo(secret_data[key]) 113 | sys.exit(0) 114 | sys.exit(1) 115 | 116 | 117 | @cli.command(short_help="Set a specific value in the secret datastore") 118 | @click.argument('secret') 119 | @click.argument('key') 120 | @click.argument('value', required=False) 121 | @click.option('-s', '--secure', is_flag=True, default=False, help='Pass value to prompt without displaying it on screen') 122 | @click.option('-c', '--category', default=None) 123 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 124 | def set(secret, key, value, category, secure, region): 125 | if not value: 126 | if secure: 127 | value = click.prompt('Value', hide_input=True, confirmation_prompt=True) 128 | else: 129 | raise click.UsageError("Value must be provided for set command") 130 | 131 | secret_data = get_secret(secret, region) 132 | if category: 133 | if category not in secret_data: 134 | secret_data[category] = {} 135 | secret_data[category][key] = value 136 | else: 137 | secret_data[key] = value 138 | put_secret(secret, secret_data, region) 139 | 140 | 141 | @cli.command(short_help="Remove a key from the secret datastore") 142 | @click.argument('secret') 143 | @click.argument('key') 144 | @click.option('-c', '--category', default=None) 145 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 146 | def remove(secret, key, category, region): 147 | secret_data = get_secret(secret, region) 148 | if category: 149 | if category not in secret_data: 150 | sys.exit(0) 151 | if key in secret_data[category]: 152 | del secret_data[category][key] 153 | elif key not in secret_data: 154 | sys.exit(0) 155 | del secret_data[key] 156 | put_secret(secret, secret_data, region) 157 | 158 | 159 | @cli.command(short_help="Set a specific value in the secret datastore") 160 | @click.argument('secret') 161 | @click.option('-c', '--category', default=None) 162 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 163 | def list(secret, category, region): 164 | secret_data = get_secret(secret, region) 165 | if category: 166 | if category in secret_data: 167 | secret_data = secret_data[category] 168 | else: 169 | sys.exit(1) 170 | for key in secret_data: 171 | click.echo(key) 172 | 173 | 174 | @cli.command(short_help="Upload a replacement secrets file") 175 | @click.argument('secret') 176 | @click.argument('input', type=click.File('rb')) 177 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 178 | @click.option('--binary/--no-binary', default=False, help='If set the file will be uploaded as a binary') 179 | def upload(secret, input, region, binary): 180 | if binary: 181 | content = input.read() 182 | put_secret(secret, content, region, raw=True, binary=True) 183 | else: 184 | content = input.read().decode("utf-8") 185 | put_secret(secret, content, region, raw=True) 186 | 187 | 188 | @cli.command(short_help="Download the entire secrets file") 189 | @click.argument('secret') 190 | @click.argument('output', type=click.File('wb'), required=False) 191 | @click.option('-r', '--region', default=None, help='Specify which AWS Region the secret is stored in') 192 | def download(secret, output, region): 193 | contents = get_secret(secret, region, raw=True) 194 | if not output: 195 | click.echo(contents) 196 | else: 197 | output.write(contents.encode('utf-8')) 198 | output.close() 199 | 200 | 201 | if __name__ == '__main__': 202 | cli() 203 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # Always prefer setuptools over distutils 2 | from setuptools import setup, find_packages 3 | # To use a consistent encoding 4 | from codecs import open 5 | 6 | try: 7 | import pypandoc 8 | long_description = pypandoc.convert('README.md', 'rst') 9 | except(IOError, ImportError): 10 | long_description = open('README.md').read() 11 | 12 | 13 | version = '0.1.4' 14 | setup( 15 | 16 | name = 'secretcli', 17 | 18 | version = version, 19 | packages=find_packages(), 20 | 21 | description = '', 22 | long_description=long_description, 23 | python_requires='>=3', 24 | 25 | author = 'Robert Hafner', 26 | author_email = 'tedivm@tedivm.com', 27 | url = 'https://github.com/tedivm/secretcli', 28 | download_url = "https://github.com/tedivm/secretcli/archive/v%s.tar.gz" % (version), 29 | keywords = '', 30 | 31 | classifiers = [ 32 | 'Development Status :: 4 - Beta', 33 | 'License :: OSI Approved :: MIT License', 34 | 35 | 'Intended Audience :: Developers', 36 | 'Intended Audience :: System Administrators', 37 | 38 | 'Programming Language :: Python :: 3', 39 | 'Environment :: Console', 40 | ], 41 | 42 | install_requires=[ 43 | 'boto3>=1.9,<2.0', 44 | 'click>=6.0,<8.0', 45 | 'requests', 46 | 'pyyaml', 47 | 'urllib3<1.24' 48 | ], 49 | 50 | extras_require={ 51 | 'dev': [ 52 | 'pypandoc', 53 | 'twine', 54 | 'wheel' 55 | ], 56 | }, 57 | 58 | entry_points={ 59 | 'console_scripts': [ 60 | 'secretcli=secretcli.secretcli:cli', 61 | ], 62 | }, 63 | 64 | ) 65 | --------------------------------------------------------------------------------