├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── apply ├── lib ├── push └── run /.gitignore: -------------------------------------------------------------------------------- 1 | units 2 | groups 3 | hosts 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2020, Loic Nageleisen 2 | Copyright (c) 2014, ADHOC-GTI 3 | All rights reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions are met: 7 | * Redistributions of source code must retain the above copyright 8 | notice, this list of conditions and the following disclaimer. 9 | * Redistributions in binary form must reproduce the above copyright 10 | notice, this list of conditions and the following disclaimer in the 11 | documentation and/or other materials provided with the distribution. 12 | * Neither the name of the copyright holders nor the 13 | names of its contributors may be used to endorse or promote products 14 | derived from this software without specific prior written permission. 15 | 16 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 17 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 18 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 19 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE 20 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 21 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 22 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 23 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 24 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 25 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: 2 | 3 | lint: 4 | find apply run push lib -type f -not -iname '*.*' | xargs shellcheck -s bash 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # apply 2 | 3 | Lightweight provisioning tool to apply shell scripts on a local or remote 4 | machine. 5 | 6 | ## Usage 7 | 8 | `push` pushes bash scripts called `unit`s through ssh to execute: 9 | 10 | ./push -v units/update units/sshd units/ssh_authorized_keys root@1.2.3.4 11 | 12 | Units are processed on the target machine by the `run` script. Each unit executes within its own subshell so no variable leak occurs. Each subshell is run with `set -euo pipefail`. Each subshell also sources the contents of `lib` which defines a few convenience functions. 13 | 14 | By writing those unit scripts to be idempotent you can just run them again and again. Units can be aggregated in `group`s, which can themselves reference other groups: 15 | 16 | ./push -v groups/base groups/ruby units/dockerd root@foo.example.com 17 | 18 | Finally, you can define `host`s, which are like groups, only they save you some typing to apply units to multiple targets: 19 | 20 | ./apply -v hosts/foo.example.com hosts/bar.example.com 21 | 22 | The above can be made to process hosts in parallel: 23 | 24 | ./apply -v -p hosts/foo.example.com hosts/bar.example.com 25 | 26 | Since `units/`, `groups/`, and `hosts/`, are just directories and files, autocompletion works immediately and you could get creative with shell expansion for arguments. 27 | 28 | ./apply hosts/{foo,bar}.example.com hosts/test.* 29 | 30 | ## Rationale 31 | 32 | At some point in a previous company we had a lot of individual VPSes set up basically the same way. I was sick of internal documentation that listed step-by-step commands intertwined with descriptions and manual actions, and any attempt at puppet or ansible just blew up because it was something else to learn by the team (believe me, I tried, it just wouldn't stick with anyone). 33 | 34 | So I created `apply`. 35 | 36 | It turned out to be a deceptively simple, down-to-earth experience, immediately accessible, trivially enabled literate coding, and overall extremely useful both to pragmatically set up and maintain those VPSes as well as creating dev environments, or local VMs to test a e.g one-shot unit performing a change or migration. 37 | -------------------------------------------------------------------------------- /apply: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -u 5 | 6 | function usage() { 7 | echo "usage: $(basename "$0") [-v] [-p] [...]" 8 | exit 1 9 | } 10 | 11 | function host_targets() { 12 | local host="$*" 13 | 14 | cat < "${APPLY_ROOT}"/"$host" | tr '\n' ' ' 15 | } 16 | 17 | function push_each() { 18 | local vflag="$1" 19 | local parallelize="$2" 20 | shift 21 | shift 22 | if [[ -n $parallelize ]]; then 23 | # shellcheck disable=SC2086 24 | parallel -j10 --progress --colsep ' ' ./push $vflag 25 | else 26 | # shellcheck disable=SC2086 27 | parallel -j1 -u --colsep ' ' ./push $vflag 28 | fi 29 | } 30 | 31 | if [[ -z "${APPLY_ROOT:-}" ]]; then 32 | APPLY_ROOT="." 33 | fi 34 | 35 | if [[ $# -lt 1 ]]; then 36 | usage 37 | fi 38 | 39 | if [[ "$1" == "-v" ]]; then 40 | vflag='-v' 41 | shift 42 | else 43 | vflag='' 44 | fi 45 | 46 | if [[ "$1" == "-p" ]]; then 47 | pflag='-p' 48 | shift 49 | else 50 | pflag='' 51 | fi 52 | 53 | if [[ $# -lt 1 ]]; then 54 | usage 55 | fi 56 | 57 | hosts=() 58 | while [[ $# -gt 0 ]]; do 59 | hosts+=("$1") 60 | shift 61 | done 62 | 63 | for host in ${hosts[*]}; do 64 | target=${host#*/} 65 | # shellcheck disable=SC2046 66 | echo $(host_targets "$host") root@"$target" 67 | done | push_each "$vflag" "$pflag" 68 | -------------------------------------------------------------------------------- /lib: -------------------------------------------------------------------------------- 1 | # shellcheck shell=bash 2 | # vim: ft=sh 3 | 4 | # Support functions 5 | 6 | function ssh_version() { 7 | ssh -V 2>&1 | perl -ne '/^OpenSSH_(\d+\.\d+)/ and print "$1";' 8 | } 9 | 10 | function ssh_back() { 11 | local remote="$1" 12 | local timeout=60 13 | local port=0 14 | shift 15 | 16 | case $(ssh_version) in 17 | 5.*) 18 | port=55555 19 | # shellcheck disable=SC2029 20 | ssh -f -R "$port":127.0.0.1:22 "$remote" sleep "$timeout" >/dev/null 2>&1 21 | ;; 22 | *) 23 | # shellcheck disable=SC2029 24 | port="$(ssh -f -R 0:127.0.0.1:22 "$remote" sleep "$timeout" 2>&1 >/dev/null | head -1 | perl -ne '/Allocated port (\d+)/ and print "$1"')" 25 | ;; 26 | esac 27 | 28 | local args="-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p $port" 29 | # shellcheck disable=SC2029 30 | [[ -n "$port" ]] && ssh -A "$remote" "SSH_BACK_PORT=$port; SSH_BACK_USER=$USER; SSH_BACK_ARGS='$args'; $*" 2> >(grep -v "Permanently added") 31 | } 32 | 33 | # Fallback functions 34 | 35 | if ! which systemctl >/dev/null; then 36 | function systemctl() { 37 | case "$1" in 38 | start|stop|restart|reload) 39 | service "$2" "$1" 40 | ;; 41 | *) 42 | false 43 | ;; 44 | esac 45 | } 46 | fi 47 | -------------------------------------------------------------------------------- /push: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -u 5 | 6 | function usage() { 7 | echo "usage: $(basename "$0") [-v] [...] " 8 | exit 1 9 | } 10 | 11 | if [[ -z "${APPLY_ROOT:-}" ]]; then 12 | APPLY_ROOT="." 13 | fi 14 | 15 | if [[ $# -lt 1 ]]; then 16 | usage 17 | fi 18 | 19 | if [[ "$1" == "-v" ]]; then 20 | vflag='-v' 21 | shift 22 | else 23 | vflag='' 24 | fi 25 | 26 | if [[ $# -lt 2 ]]; then 27 | usage 28 | fi 29 | 30 | targets=() 31 | while [[ $# -gt 1 ]]; do 32 | targets+=("$1") 33 | shift 34 | done 35 | 36 | remote="$1" 37 | 38 | tmp=$(ssh "$remote" 'mktemp -d') 39 | echo -e -n "\033[33m** pushing to\033[0m $remote:$tmp" 40 | if scp -q -r "${APPLY_ROOT}"/{groups,units} run lib "$remote":"$tmp"; then 41 | echo -e " \033[32mOK\033[0m" 42 | fi 43 | 44 | # shellcheck disable=SC2029 45 | ssh -A "$remote" "cd '$tmp' && ./run $vflag ${targets[*]} && cd; rm -rf '$tmp'" 46 | -------------------------------------------------------------------------------- /run: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | set -u 5 | 6 | function usage() { 7 | echo "usage: $(basename "$0") [-v] [...]" 8 | exit 1 9 | } 10 | 11 | function root_path() { 12 | if [[ -z "${APPLY_ROOT:-}" ]]; then 13 | cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd 14 | else 15 | echo "$APPLY_ROOT" 16 | fi 17 | } 18 | 19 | function read_group() { 20 | local target="$*" 21 | 22 | while read -r item; do 23 | if [[ ! "$item" == '#'* ]]; then 24 | echo "$item" 25 | fi 26 | done < "$(root_path)/$target" 27 | } 28 | 29 | function run_group() { 30 | local target="$*" 31 | 32 | for item in $(read_group "$target"); do 33 | if [[ ! "$item" == '#'* ]]; then 34 | run_pretty "$item" 35 | fi 36 | done 37 | } 38 | 39 | function run_unit() { 40 | /bin/bash -l -c "set -e; set -u; set -o pipefail; source lib; source '$(root_path)/$*'" 41 | } 42 | 43 | function log_file() { 44 | echo -n "$LOG_DIR/" 45 | echo "${1/\//_}.log" 46 | } 47 | 48 | function check_target() { 49 | local target="$*" 50 | 51 | if [[ ! -f "$(root_path)/$target" ]]; then 52 | echo "missing $target" 53 | return 1 54 | fi 55 | 56 | if [[ $target == groups/* ]]; then 57 | for item in $(read_group "$target"); do 58 | check_target "$item" 59 | done 60 | fi 61 | } 62 | 63 | function run_pretty() { 64 | local target="$*" 65 | 66 | case "$target" in 67 | groups/*) 68 | echo -e "\033[33m** \033[34m$target \033[33mprocessing...\033[0m" 69 | run_group "$target" 70 | echo -e "\033[33m** \033[34m$target \033[32mOK\033[0m" 71 | ;; 72 | 73 | units/*) 74 | run_pretty_unit "$target" 75 | ;; 76 | *) 77 | echo "unsupported command: $target" 78 | ;; 79 | esac 80 | } 81 | 82 | function run_pretty_unit() { 83 | local rc 84 | local log_file 85 | local target="$*" 86 | 87 | echo -e -n "\033[33m** \033[34m$target\033[0m" 88 | [[ "$VERBOSE" == "1" ]] && echo -e ": \033[33mstarting...\033[0m" 89 | 90 | log_file=$(log_file "$@") 91 | 92 | if [[ "$VERBOSE" == "1" ]]; then 93 | set +e 94 | run_unit "$@" 95 | rc=$? 96 | set -e 97 | else 98 | set +e 99 | run_unit "$@" >"$log_file" 2>&1 100 | rc=$? 101 | set -e 102 | fi 103 | 104 | if [[ "$VERBOSE" == "1" ]]; then 105 | echo -e -n "\033[33m** \033[34m$target\033[0m: " 106 | else 107 | echo -n " " 108 | fi 109 | 110 | if [[ $rc -eq 0 ]]; then 111 | echo -e "\033[32mOK\033[0m" 112 | else 113 | echo -e "\033[31mFAILED\033[0m" 114 | if [[ "$VERBOSE" == "1" ]]; then 115 | echo "For details, see output above" 1>&2 116 | else 117 | echo "Here are the last lines of output:" 1>&2 118 | tail -n20 "$log_file" 1>&2 119 | echo "For details, see $log_file" 1>&2 120 | fi 121 | exit $rc 122 | fi 123 | } 124 | 125 | if [[ $# -lt 1 ]]; then 126 | usage 127 | fi 128 | 129 | if [[ "$1" == "-v" ]]; then 130 | VERBOSE="1" 131 | shift 132 | else 133 | VERBOSE="0" 134 | fi 135 | 136 | if [[ $# -lt 1 ]]; then 137 | usage 138 | fi 139 | 140 | targets=( "$@" ) 141 | 142 | # pre-flight checks 143 | for target in "${targets[@]}"; do 144 | check_target "$target" 145 | done 146 | 147 | # fly! 148 | LOG_DIR=$(mktemp -d) 149 | for target in "${targets[@]}"; do 150 | run_pretty "$target" 151 | done 152 | --------------------------------------------------------------------------------