├── .dockerignore ├── .gitignore ├── Dockerfile ├── LICENSE ├── README.md └── docker-config-update /.dockerignore: -------------------------------------------------------------------------------- 1 | * 2 | !docker-config-update 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | test 2 | *.swp 3 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | ARG DOCKER_VER=stable 2 | FROM docker:${DOCKER_VER} 3 | 4 | RUN apk add --no-cache jq 5 | 6 | COPY docker-config-update /usr/local/bin/ 7 | 8 | ENTRYPOINT [ "/usr/local/bin/docker-config-update" ] 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2018 Brandon Mitchell 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 4 | 5 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 6 | 7 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Docker Config and Secret Update Tool 2 | 3 | This utility will update configs and secrets in docker based on a local 4 | source file. The configs and secrets are versioned and the version is appended 5 | to the config and secret name. An environment variable file is updated with 6 | the latest version number of the configs and secrets. This file can then be 7 | sourced before deploying a stack in docker to use the latest versions. 8 | 9 | ## The .docker-deploy file 10 | 11 | This file contains the following lines: 12 | 13 | - `CONFIG_LIST=`: space separated list of configs. 14 | - `SECRET_LIST=`: space separated list of secrets. 15 | - `ENV_FILE=`: filename to update with config and secret variables, 16 | defaults to `.env`. If set to an empty string (`""`), updates to the 17 | environment file are skipped. 18 | - `STACK_NAME=`: stack name, used to namespace configs/secrets to 19 | automatically cleanup when the stack is removed. 20 | - For each config name in the list above: 21 | - `CONF_name_SRC_FILE=`: filename to read a config from, name is a variable. 22 | - `CONF_name_SRC_TYPE=`: change from the default "file" type, can be "latest" 23 | to use the most recent version. 24 | - `CONF_name_TGT_NAME=`: name of config to create, appended with a version. 25 | - `CONF_name_TGT_VAR=`: variable to update in environment file. 26 | - For each secret name in the list above: 27 | - `SEC_name_SRC_FILE=`: filename to read a secret from, name is a variable. 28 | - `SEC_name_SRC_TYPE=`: change from the default "file" type, can be "latest" 29 | to use the most recent version, and "random" to randomly initialize a 30 | value. 31 | - `SEC_name_TGT_NAME=`: name of secret to create, appended with a version. 32 | - `SEC_name_TGT_VAR=`: variable to update in environment file. 33 | - `OPT_ONLY_LATEST=`: set to 1 to prevent old versions of a config/secret from 34 | being used, forces creation of a new entry even if old ones match. 35 | - `OPT_PRUNE_UNUSED=`: set to 1 to cleanup unused versions of configs and 36 | secrets. This currently blindly deletes all configs/secrets other than the 37 | active one, ignoring errors from those that are still in use. 38 | 39 | An example file could look like: 40 | 41 | ``` 42 | CONFIG_LIST="app demo" 43 | SECRET_LIST="app passwd" 44 | CONF_app_SRC_FILE="app.conf" 45 | CONF_app_TGT_NAME="app-" 46 | CONF_app_TGT_VAR="app_conf_ver" 47 | CONF_demo_SRC_TYPE=latest 48 | CONF_demo_TGT_NAME="demo-" 49 | CONF_demo_TGT_VAR="demo_conf_ver" 50 | SEC_app_SRC_FILE="app.sec" 51 | SEC_app_TGT_NAME="app-" 52 | SEC_app_TGT_VAR="app_sec_ver" 53 | SEC_passwd_SRC_TYPE=random 54 | SEC_passwd_TGT_NAME="passwd-" 55 | SEC_passwd_TGT_VAR="passwd_ver" 56 | ``` 57 | 58 | ## The .env file 59 | 60 | This file will contain lines with each `CONF_name_TGT_VAR` and 61 | `SEC_name_TGT_VAR` defined in the `.docker-deploy` file (where name is from the 62 | list of configs and secrets). 63 | 64 | ## Using with a compose file 65 | 66 | Your compose file will need to define external configs and secrets. With 67 | version 3.5 of the compose file, you define external configs and secrets with 68 | a name using the following syntax: 69 | 70 | ``` 71 | version: '3.5' 72 | 73 | configs: 74 | app_conf: 75 | external: true 76 | name: app_conf_${app_conf_ver} 77 | secrets: 78 | app_sec: 79 | external: true 80 | name: app_sec_${app_sec_ver} 81 | services: 82 | app: 83 | image: app_image 84 | configs: 85 | - source: app_conf 86 | target: /etc/app.conf 87 | mode: 444 88 | secrets: 89 | - source: app_sec 90 | target: /etc/app.sec 91 | mode: 400 92 | uid: "0" 93 | ``` 94 | 95 | When deploying the stack, you'll want to run: 96 | 97 | ``` 98 | # update the .env file with this script 99 | docker-config-update 100 | # source and export the .env file 101 | set -a && . ./.env && set +a 102 | # deploy the stack with the variables 103 | docker stack deploy -c docker-compose.yml app 104 | ``` 105 | 106 | ## Random secrets 107 | 108 | These are a 32 character string created with: 109 | 110 | ``` 111 | base64 -w 0 4 | # License: MIT 5 | # Source repo: https://github.com/sudo-bmitch/docker-config-update 6 | 7 | # Prereqs: docker, jq, sha256sum 8 | 9 | set -e 10 | 11 | conf_file=".docker-deploy" 12 | ENV_FILE=".env" 13 | 14 | usage() { 15 | echo "Usage: $0 [opts]" 16 | echo " -f file: filename to process, defaults to .docker-deploy" 17 | echo " -h: this help message" 18 | echo " -s stack_name: stack name, overrides STACK_NAME settings in file" 19 | [ "$opt_h" = "1" ] && exit 0 || exit 1 20 | } 21 | 22 | error() { 23 | echo "$*" >&2 24 | exit 1 25 | } 26 | log() { 27 | echo "$*" 28 | } 29 | 30 | while getopts 'f:hs:' option; do 31 | case $option in 32 | f) conf_file="$OPTARG";; 33 | h) opt_h=1;; 34 | s) stack_name="$OPTARG";; 35 | esac 36 | done 37 | shift $(expr $OPTIND - 1) 38 | if [ "$opt_h" = "1" -o $# -gt 0 ]; then 39 | usage 40 | fi 41 | 42 | # Set version in env file 43 | set_var() { 44 | tgt_var="$1" 45 | new_ver="$2" 46 | 47 | # if ENV_FILE is intentionally blanked out, then skip this 48 | if [ -z "${ENV_FILE}" ]; then 49 | return 50 | fi 51 | 52 | # check for current version 53 | cur_ver=$(grep "^${tgt_var}=" "${ENV_FILE}" | cut -f2 -d=) 54 | 55 | # if aleady on the right version, return 56 | if [ "${new_ver}" = "${cur_ver}" ]; then 57 | return 58 | fi 59 | 60 | # log new version being set 61 | log "Updating version: ${tgt_var}=${new_ver}" 62 | 63 | if [ -n "${cur_ver}" ]; then 64 | # if already set, modify 65 | sed -i "s/^${tgt_var}=.*\$/${tgt_var}=${new_ver}/" "${ENV_FILE}" 66 | else 67 | # else append a value to the file 68 | echo "${tgt_var}=${new_ver}" >> "${ENV_FILE}" 69 | fi 70 | } 71 | 72 | # Apply template 73 | apply_template() { 74 | src_file="$1" 75 | if [ -f "${src_file}.tmpl" ]; then 76 | log "Applying template for ${src_file}" 77 | if [ -n "${ENV_FILE}" ]; then 78 | env $(cat "$ENV_FILE" | xargs) envsubst <"${src_file}.tmpl" >"${src_file}" 79 | else 80 | envsubst <"${src_file}.tmpl" >"${src_file}" 81 | fi 82 | fi 83 | } 84 | 85 | # Load the config file 86 | common_opts="" 87 | cd "$(dirname "${conf_file}")" 88 | . "./$(basename "${conf_file}")" 89 | if [ -n "${ENV_FILE}" -a ! -f "${ENV_FILE}" ]; then 90 | :>>"${ENV_FILE}" 91 | chmod 755 "${ENV_FILE}" 92 | fi 93 | 94 | # update the stack name 95 | if [ -n "${stack_name}" ]; then 96 | STACK_NAME="${stack_name}" 97 | fi 98 | if [ -n "${STACK_NAME}" ]; then 99 | common_opts="${common_opts} -l com.docker.stack.namespace=${STACK_NAME}" 100 | fi 101 | 102 | clean_ver_list() { 103 | tgt_name="$1" 104 | only_latest="$2" 105 | if [ "$only_latest" = "1" ]; then 106 | sed "s/^${tgt_name}//" | sort -n | tail -1 107 | else 108 | sed "s/^${tgt_name}//" | sort -n 109 | fi 110 | } 111 | 112 | gen_ver_list() { 113 | proc_type="$1" 114 | tgt_name="$2" 115 | only_latest="$3" 116 | docker ${proc_type} ls --filter "name=${tgt_name}" --format "{{.Name}}" \ 117 | | clean_ver_list "$tgt_name" "$only_latest" 118 | } 119 | 120 | cmp_file_to_ver() { 121 | proc_type="$1" 122 | tgt_name="$2" 123 | cur_ver="$3" 124 | src_file="$4" 125 | 126 | if [ "$proc_type" = "config" ]; then 127 | docker config inspect --format '{{json .Spec.Data}}' "${tgt_name}${cur_ver}" \ 128 | | jq -r . | base64 -d | diff - "${src_file}" >/dev/null 2>&1 \ 129 | && return 0 || return 1 130 | elif [ "$proc_type" = "secret" ]; then 131 | if [ "$src_file" != "$cmp_file_last_name" ]; then 132 | cmp_file_cur_sha256=$(sha256sum "${src_file}" | cut -f1 -d' ') 133 | cmp_file_last_name="${src_file}" 134 | fi 135 | cmp_file_old_sha256=$(docker secret inspect \ 136 | --format '{{index .Spec.Labels "secret-sha256sum"}}' \ 137 | "${tgt_name}${cur_ver}" || echo "") 138 | [ "$cmp_file_cur_sha256" = "$cmp_file_old_sha256" ] && return 0 || return 1 139 | fi 140 | } 141 | 142 | make_new_ver_file() { 143 | proc_type="$1" 144 | tgt_name="$2" 145 | new_ver="$3" 146 | src_file="$4" 147 | 148 | if [ "$proc_type" = "config" ]; then 149 | log "Creating new config ${tgt_name}${new_ver}" 150 | # create a new config 151 | docker config create ${common_opts} "${tgt_name}${new_ver}" "${src_file}" >/dev/null 152 | elif [ "$proc_type" = "secret" ]; then 153 | cur_sha256=$(sha256sum "${src_file}" | cut -f1 -d' ') 154 | docker secret create ${common_opts} -l "secret-sha256sum=${cur_sha256}" \ 155 | "${tgt_name}${new_ver}" "${src_file}" >/dev/null 156 | fi 157 | } 158 | 159 | make_new_ver_value() { 160 | proc_type="$1" 161 | tgt_name="$2" 162 | new_ver="$3" 163 | src_value="$4" 164 | 165 | if [ "$proc_type" = "config" ]; then 166 | log "Creating new config ${tgt_name}${new_ver}" 167 | # create a new config 168 | echo "$src_value" | docker config create ${common_opts} \ 169 | "${tgt_name}${new_ver}" - 170 | elif [ "$proc_type" = "secret" ]; then 171 | cur_sha256=$(echo "$src_value" | sha256sum | cut -f1 -d' ') 172 | echo "$src_value" | docker secret create ${common_opts} \ 173 | -l "secret-sha256sum=${cur_sha256}" "${tgt_name}${new_ver}" - 174 | fi 175 | } 176 | 177 | process() { 178 | proc_type="$1" 179 | proc_list="$2" 180 | if [ "$proc_type" = "config" ]; then 181 | proc_prefix="CONF" 182 | elif [ "$proc_type" = "secret" ]; then 183 | proc_prefix="SEC" 184 | else 185 | error "Unknown object type to process: $proc_type" 186 | fi 187 | 188 | for entry in ${proc_list}; do 189 | # get variables for current conf name 190 | src_type=$(eval echo \"\$${proc_prefix}_${entry}_SRC_TYPE\") 191 | tgt_name=$(eval echo \"\$${proc_prefix}_${entry}_TGT_NAME\") 192 | tgt_var=$(eval echo \"\$${proc_prefix}_${entry}_TGT_VAR\") 193 | if [ -z "$src_type" -o "$src_type" = "file" ]; then 194 | # process a file as the source 195 | src_file=$(eval echo \"\$${proc_prefix}_${entry}_SRC_FILE\") 196 | # use the template if available 197 | apply_template "${src_file}" 198 | if [ ! -r "${src_file}" ]; then 199 | error "Source file does not exist or is not readable for ${proc_type} ${entry}: ${src_file}" 200 | fi 201 | # compare the src_file to any/all existing configs/secrets 202 | unset new_ver latest_ver 203 | for cur_ver in $(gen_ver_list $proc_type "$tgt_name" "${OPT_ONLY_LATEST:-0}"); do 204 | if cmp_file_to_ver $proc_type "$tgt_name" "$cur_ver" "$src_file"; then 205 | new_ver=$cur_ver 206 | break 207 | fi 208 | latest_ver=$cur_ver 209 | done 210 | if [ -z "$new_ver" ]; then 211 | # need to create a new config/secret, first get a new version number 212 | if [ -z "$latest_ver" ]; then 213 | new_ver=1 214 | else 215 | new_ver=$(expr "$latest_ver" + 1) 216 | fi 217 | make_new_ver_file $proc_type "$tgt_name" "$new_ver" "$src_file" 218 | fi 219 | elif [ "$src_type" = "latest" ]; then 220 | # scan list of configs for latest version 221 | new_ver=$(gen_ver_list $proc_type "$tgt_name" "1") 222 | if [ -z "${new_ver}" ]; then 223 | error "Could not find a latest version for ${proc_type} ${entry}" 224 | fi 225 | elif [ "$src_type" = "random" ]; then 226 | latest_ver=$(gen_ver_list $proc_type "$tgt_name" "1") 227 | if [ -z "$latest_ver" ]; then 228 | new_ver=1 229 | new_value=$(base64 -w 0 /dev/null 2>&1; then 247 | log "Cleaned $proc_type ${tgt_name}${cur_ver}" 248 | fi 249 | fi 250 | done 251 | fi 252 | done 253 | } 254 | 255 | process config "${CONFIG_LIST}" 256 | process secret "${SECRET_LIST}" 257 | 258 | 259 | 260 | --------------------------------------------------------------------------------