├── .gitignore ├── README.md ├── pillar └── vault.py └── LICENSE /.gitignore: -------------------------------------------------------------------------------- 1 | # Ignore python byte code 2 | *.pyc 3 | 4 | # Ignore vim swap files 5 | .*.s?? 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # salt-pillar-vault 2 | Saltstack external pillar for Hashicorp Vault with flexible minion targeting 3 | 4 | Requirements 5 | ------------ 6 | * `hvac` python module (>= v0.2.17) 7 | 8 | 9 | Example Configuration 10 | --------------------- 11 | 12 | Your Vault server should be defined in the master config file with the 13 | following options: 14 | 15 | ```yaml 16 | ext_pillar: 17 | - vault: 18 | url: https://vault:8200 19 | config: Path or salt:// URL to vault secret configuration 20 | token: (optional) Explicit token for token authentication 21 | app_id: (optional) Application ID for app-id authentication 22 | user_id: (optional) Explicit User ID for app-id authentication 23 | user_file: (optional) File to read for user-id value 24 | role_id: (optional) Role ID for AppRole authentication 25 | secret_id: (optional) Explicit Secret ID for AppRole authentication 26 | secret_file: (optional) File to read for secret-id value 27 | unset_if_missing: (optional) Leave pillar key unset if Vault secret not found 28 | ``` 29 | 30 | The `url` parameter is the full URL to the Vault API endpoint. 31 | 32 | The `config` parameter is the path or salt:// URL to the secret map YML file to be parsed by the master. 33 | 34 | The `token` parameter is an explicit token to use for authentication, and it 35 | overrides all other authentication methods. 36 | 37 | The `app_id` parameter is an Application ID to use for app-id authentication. 38 | 39 | The `user_id` parameter is an explicit User ID to pair with ``app_id`` for 40 | app-id authentication. 41 | 42 | The `user_file` parameter is the path to a file on the master to read for a 43 | ``user-id`` value if `user_id` is not specified. 44 | 45 | The ``role_id`` parameter is a Role ID to use for AppRole authentication. 46 | 47 | The ``secret_id`` parameter is an explicit Role ID to pair with ``role_id`` for 48 | AppRole authentication. 49 | 50 | The ``secret_file`` parameter is the path to a file on the master to read for a 51 | ``secret-id`` value if ``secret_id`` is not specified. 52 | 53 | The `unset_if_missing` parameter determines behavior when the Vault secret is 54 | missing or otherwise inaccessible. If set to ``True``, the pillar key is left 55 | unset. If set to ``False``, the pillar key is set to ``None``. Default is 56 | ``False`` 57 | 58 | Mapping Vault Secrets to Minions 59 | -------------------------------- 60 | 61 | The `config` parameter, above, is a path to the YML file which will be 62 | used for mapping secrets to minions. The map uses syntax similar to the 63 | top file, and will be processed as a Jinja template: 64 | 65 | ```yaml 66 | 'filter': 67 | 'variable': 'path' 68 | 'variable': 'path?key' 69 | 'filter': 70 | 'variable': 'path?key' 71 | ``` 72 | 73 | Each `filter` is a compound matcher: 74 | https://docs.saltstack.com/en/latest/topics/targeting/compound.html 75 | 76 | `variable` is the name of the variable which will be injected into the 77 | pillar data. 78 | 79 | `path` is the path the desired secret on the Vault server. 80 | 81 | `key` is optional. If specified, only this specific key will be returned 82 | for the secret at `path`. If unspecified, the entire secret json structure 83 | will be returned. 84 | 85 | 86 | ```yaml 87 | 'web*': 88 | 'ssl_cert': '/secret/certs/domain?certificate' 89 | 'ssl_key': '/secret/certs/domain?private_key' 90 | 'db* and G@os.Ubuntu': 91 | 'db_pass': '/secret/passwords/database' 92 | '*': 93 | 'my_key': '/secret/certs/{{ grains.id }}?private_key' 94 | ``` 95 | 96 | Authors 97 | ------- 98 | 99 | - [Derek Moore](http://github.com/redredgroovy) - Author 100 | -------------------------------------------------------------------------------- /pillar/vault.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Use Vault secrets as a Pillar source 4 | 5 | Example Configuration 6 | --------------------- 7 | 8 | The Vault server should be defined in the master config file with the 9 | following options: 10 | 11 | .. code-block:: yaml 12 | 13 | ext_pillar: 14 | - vault: 15 | url: https://vault:8200 16 | config: Local path or salt:// URL to secret configuration file 17 | token: Explicit token for token authentication 18 | app_id: Application ID for app-id authentication 19 | user_id: Explicit User ID for app-id authentication 20 | user_file: File to read for user-id value 21 | role_id: Role ID for AppRole authentication 22 | secret_id: Explicit Secret ID for AppRole authentication 23 | secret_file: File to read for secret-id value 24 | unset_if_missing: Leave pillar key unset if Vault secret not found 25 | 26 | The ``url`` parameter is the full URL to the Vault API endpoint. 27 | 28 | The ``config`` parameter is the path or salt:// URL to the secret map YML file. 29 | 30 | The ``token`` parameter is an explicit token to use for authentication, and it 31 | overrides all other authentication methods. 32 | 33 | The ``app_id`` parameter is an Application ID to use for app-id authentication. 34 | 35 | The ``user_id`` parameter is an explicit User ID to pair with ``app_id`` for 36 | app-id authentication. 37 | 38 | The ``user_file`` parameter is the path to a file on the master to read for a 39 | ``user-id`` value if ``user_id`` is not specified. 40 | 41 | The ``role_id`` parameter is a Role ID to use for AppRole authentication. 42 | 43 | The ``secret_id`` parameter is an explicit Role ID to pair with ``role_id`` for 44 | AppRole authentication. 45 | 46 | The ``secret_file`` parameter is the path to a file on the master to read for a 47 | ``secret-id`` value if ``secret_id`` is not specified. 48 | 49 | The ``unset_if_missing`` parameter determins behavior when the Vault secret is 50 | missing or otherwise inaccessible. If set to ``True``, the pillar key is left 51 | unset. If set to ``False``, the pillar key is set to ``None``. Default is 52 | ``False`` 53 | 54 | Mapping Vault Secrets to Minions 55 | -------------------------------- 56 | 57 | The ``config`` parameter, above, is a path to the YML file which will be 58 | used for mapping secrets to minions. The map uses syntax similar to the 59 | top file: 60 | 61 | .. code-block:: yaml 62 | 63 | 'filter': 64 | 'variable': 'path' 65 | 'variable': 'path?key' 66 | 'filter': 67 | 'variable': 'path?key' 68 | 69 | 70 | Each ``filter`` is a compound matcher: 71 | https://docs.saltstack.com/en/latest/topics/targeting/compound.html 72 | 73 | ``variable`` is the name of the variable which will be injected into the 74 | pillar data. 75 | 76 | ``path`` is the path the desired secret on the Vault server. 77 | 78 | ``key`` is optional. If specified, only this specific key will be returned 79 | for the secret at ``path``. If unspecified, the entire secret json structure 80 | will be returned. 81 | 82 | 83 | .. code-block:: yaml 84 | 85 | 'web*': 86 | 'ssl_cert': '/secret/certs/domain?certificate' 87 | 'ssl_key': '/secret/certs/domain?private_key' 88 | 'db* and G@os.Ubuntu': 89 | 'db_pass': '/secret/passwords/database 90 | 91 | """ 92 | 93 | # Import stock modules 94 | from __future__ import absolute_import 95 | import base64 96 | import logging 97 | import os 98 | import yaml 99 | 100 | # Import salt modules 101 | import salt.loader 102 | import salt.minion 103 | import salt.template 104 | import salt.utils.minions 105 | 106 | # Attempt to import the 'hvac' module 107 | try: 108 | import hvac 109 | HAS_HVAC = True 110 | except ImportError: 111 | HAS_HVAC = False 112 | 113 | # Set up logging 114 | LOG = logging.getLogger(__name__) 115 | 116 | # Default config values 117 | CONF = { 118 | 'url': 'https://vault:8200', 119 | 'config': '/srv/salt/secrets.yml', 120 | 'token': None, 121 | 'app_id': None, 122 | 'user_id': None, 123 | 'user_file': None, 124 | 'role_id': None, 125 | 'secret_id': None, 126 | 'secret_file': None, 127 | 'unset_if_missing': False 128 | } 129 | 130 | def __virtual__(): 131 | """ Only return if hvac is installed 132 | """ 133 | if HAS_HVAC: 134 | return True 135 | else: 136 | LOG.error("Vault pillar requires the 'hvac' python module") 137 | return False 138 | 139 | 140 | def _get_id_from_file(source="/.vault-id"): 141 | """ Reads a UUID from file (default: /.vault-id) 142 | """ 143 | source = os.path.abspath(os.path.expanduser(source)) 144 | LOG.debug("Reading '%s' for user_id", source) 145 | 146 | user_id = "" 147 | 148 | # pylint: disable=invalid-name 149 | if os.path.isfile(source): 150 | fd = open(source, "r") 151 | user_id = fd.read() 152 | fd.close() 153 | 154 | return user_id.rstrip() 155 | 156 | 157 | def _authenticate(conn): 158 | """ Determine the appropriate authentication method and authenticate 159 | for a token, if necesssary. 160 | """ 161 | 162 | # Check for explicit token, first 163 | if CONF["token"]: 164 | conn.token = CONF["token"] 165 | 166 | # Check for explicit AppRole authentication 167 | elif CONF["role_id"]: 168 | if CONF["secret_id"]: 169 | secret_id = CONF["secret_id"] 170 | elif CONF["secret_file"]: 171 | secret_id = _get_id_from_file(source=CONF["secret_file"]) 172 | else: 173 | secret_id = _get_id_from_file() 174 | 175 | # Perform AppRole authentication 176 | result = conn.auth_approle(CONF["role_id"], secret_id) 177 | # Required until https://github.com/ianunruh/hvac/pull/90 178 | # is merged, due in hvac 0.3.0 179 | conn.token = result['auth']['client_token'] 180 | 181 | # Check for explicit app-id authentication 182 | elif CONF["app_id"]: 183 | # Check possible sources for user-id 184 | if CONF["user_id"]: 185 | user_id = CONF["user_id"] 186 | elif CONF["user_file"]: 187 | user_id = _get_id_from_file(source=CONF["user_file"]) 188 | else: 189 | user_id = _get_id_from_file() 190 | 191 | # Perform app-id authentication 192 | conn.auth_app_id(CONF["app_id"], user_id) 193 | 194 | # TODO: Add additional auth methods here 195 | 196 | # Check for token in ENV 197 | elif os.environ.get('VAULT_TOKEN'): 198 | conn.token = os.environ.get('VAULT_TOKEN') 199 | 200 | 201 | def couple(location, conn): 202 | """ 203 | If location is a dictionary, loop over its keys, and call couple() for each key 204 | If location is a string, return the value looked up from vault. 205 | """ 206 | coupled_data = {} 207 | if isinstance(location, basestring): 208 | try: 209 | (path, key) = location.split('?', 1) 210 | except ValueError: 211 | (path, key) = (location, None) 212 | secret = conn.read(path) 213 | if key: 214 | secret = secret["data"].get(key, None) 215 | prefix = "base64:" 216 | if secret and secret.startswith(prefix): 217 | secret = base64.b64decode(secret[len(prefix):]).rstrip() 218 | if secret or not CONF["unset_if_missing"]: 219 | return secret 220 | elif isinstance(location, dict): 221 | for return_key, real_location in location.items(): 222 | coupled_data[return_key] = couple(real_location, conn) 223 | if coupled_data or not CONF["unset_if_missing"]: 224 | return coupled_data 225 | 226 | 227 | 228 | def ext_pillar(minion_id, pillar, *args, **kwargs): 229 | """ Main handler. Compile pillar data for the specified minion ID 230 | """ 231 | vault_pillar = {} 232 | 233 | # Load configuration values 234 | for key in CONF: 235 | if kwargs.get(key, None): 236 | CONF[key] = kwargs.get(key) 237 | 238 | # Resolve salt:// fileserver path, if necessary 239 | if CONF["config"].startswith("salt://"): 240 | local_opts = __opts__.copy() 241 | local_opts["file_client"] = "local" 242 | minion = salt.minion.MasterMinion(local_opts) 243 | CONF["config"] = minion.functions["cp.cache_file"](CONF["config"]) 244 | 245 | # Read the secret map 246 | renderers = salt.loader.render(__opts__, __salt__) 247 | raw_yml = salt.template.compile_template(CONF["config"], renderers, 'jinja', whitelist=[], blacklist=[]) 248 | if raw_yml: 249 | secret_map = yaml.safe_load(raw_yml.getvalue()) or {} 250 | else: 251 | LOG.error("Unable to read secret mappings file '%s'", CONF["config"]) 252 | return vault_pillar 253 | 254 | if not CONF["url"]: 255 | LOG.error("'url' must be specified for Vault configuration") 256 | return vault_pillar 257 | 258 | # Connect and authenticate to Vault 259 | conn = hvac.Client(url=CONF["url"]) 260 | _authenticate(conn) 261 | 262 | # Apply the compound filters to determine which secrets to expose for this minion 263 | ckminions = salt.utils.minions.CkMinions(__opts__) 264 | for filter, secrets in secret_map.items(): 265 | minions = ckminions.check_minions(filter, "compound") 266 | if 'minions' in minions: 267 | # In Salt 2018 this is now in a kwarg 268 | minions = minions['minions'] 269 | if minion_id in minions: 270 | for variable, location in secrets.items(): 271 | return_data = couple(location, conn) 272 | if return_data: 273 | vault_pillar[variable] = return_data 274 | 275 | 276 | return vault_pillar 277 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | --------------------------------------------------------------------------------