├── LICENSE ├── Makefile ├── README.md ├── bin ├── deploy ├── deployer-after-job ├── deployer-autoconfigure ├── deployer-bouncer ├── deployer-checkout ├── deployer-git-ssh ├── deployer-init ├── deployer-inspect ├── deployer-mailer-queue ├── deployer-mailer-service ├── deployer-mailer-worker ├── deployer-manage ├── deployer-perform ├── deployer-queue ├── deployer-service ├── deployer-stage ├── deployer-unmanage ├── deployer-update ├── deployer-worker └── rewind ├── debian ├── changelog ├── compat ├── control ├── copyright ├── postinst ├── prerm └── rules ├── doc ├── mailer.md └── privsep.md ├── etc ├── deployer.example.conf └── service │ ├── bouncer │ ├── bounce │ ├── log │ │ ├── main │ │ ├── run │ │ └── supervise │ ├── root │ ├── run │ ├── setsid │ ├── start │ ├── supervise │ └── trigger.fifo │ ├── deployer-mailer │ ├── bounce │ ├── log │ │ ├── main │ │ ├── run │ │ └── supervise │ ├── root │ ├── run │ ├── setsid │ ├── start │ ├── supervise │ └── trigger.fifo │ └── deployer │ ├── bounce │ ├── log │ ├── main │ ├── run │ └── supervise │ ├── root │ ├── run │ ├── setsid │ ├── start │ ├── supervise │ └── trigger.fifo └── test ├── Makefile ├── bin └── deployer-setup-tests ├── case ├── 0001.initial-deployment ├── 0002.subsequent-deployment ├── 0003.noop-deployment ├── 0004.in-place-deployment ├── 0005.different-branch-deployment ├── 0006.default-branch-deployment ├── 0007.removed-commit ├── 0008.symbolic-switch ├── 0009.initial-post-deployment-fail ├── 0010.subsequent-post-deployment-fail ├── 0011.allow-unit-with-period ├── 0012.reject-unit-with-slash ├── 0013.allow-revision-with-slash ├── 0014.alternate-basename ├── 0015.alternate-deploy-root └── 0016.force-redeployment ├── fail └── .empty ├── out └── .empty ├── pass └── .empty └── work └── .empty /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017, Autoglyph LLC 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | SHELL = bash -o pipefail -c 2 | PREFIX ?= /usr 3 | 4 | 5 | default : check 6 | 7 | check : 8 | @ $(MAKE) -k --no-print-directory -C test ; rc=$$? ; $(MAKE) --no-print-directory -C test report ; exit $$rc 9 | 10 | install : 11 | install -m 0755 -o root -g root -d $(DESTDIR)$(PREFIX)/bin 12 | install -m 0755 -o root -g root -t $(DESTDIR)$(PREFIX)/bin $(CURDIR)/bin/* 13 | install -m 0755 -o root -g root -d $(DESTDIR)/etc 14 | install -m 0640 -o root -g root $(CURDIR)/etc/deployer.example.conf $(DESTDIR)/etc/deployer.conf 15 | install -m 0755 -o root -g root -d $(DESTDIR)/etc/service 16 | tar cf - etc/service --owner root --group root | tar xf - -C $(DESTDIR)/ 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Deployer - Automated Zero Downtime Deployments 2 | 3 | More documentation and packaging to come. 4 | 5 | ### Getting Started 6 | 7 | ```sh 8 | which git 9 | which setlock # from acg/daemontools-encore 10 | which setuidgid # from acg/daemontools-encore 11 | which trigger-listen # from acg/trigger 12 | which trigger-pull # from acg/trigger 13 | which fsq-run # from endcrawl/fsq-run 14 | which shellsafe # from endcrawl/daemontools-extras 15 | make # ensure tests pass 16 | ``` 17 | 18 | ### Usage Notes 19 | 20 | Fear not; many of these manual steps will be automated. 21 | 22 | - First, create a config file for `deployer`: 23 | - For user installs, this is `~/.deployer.conf`. 24 | - For system installs, this is `$PREFIX/etc/deployer.conf`. 25 | - [x] Provide an example conf file to start with. 26 | - [ ] Make this easier for user installs. 27 | 28 | - Make sure the deployer binaries are on your path. 29 | - [ ] Provide binary packages that handle installation. 30 | 31 | - Create the necessary filesystem structure. 32 | - `deployer-init` 33 | 34 | - Set up a `daemontools` service which runs `deployer-service`. 35 | - [ ] Provide something to make this easier. 36 | 37 | - Start managing your first deployment: 38 | 39 | ```sh 40 | deployer-manage foo git@github.com:yourname/foo.git 41 | ``` 42 | 43 | - Push some new commits to the `master` branch of `foo`. 44 | 45 | - Request your first automated deployment: 46 | 47 | ```sh 48 | deploy foo master 49 | ``` 50 | 51 | - Watch it work: 52 | 53 | ```sh 54 | tail -n 100 -f /var/service/deployer/log/main/current 55 | ``` 56 | 57 | -------------------------------------------------------------------------------- /bin/deploy: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} []"; } 4 | 5 | main() 6 | { 7 | if [ $# -lt 1 ]; then 8 | usage 1>&2 9 | exit 100 10 | fi 11 | 12 | unit="$1" ; shift 13 | revision="${1:-_}" 14 | 15 | # Validate unit and revision against unsafe chars. 16 | 17 | test -z "$unit" && barf "missing unit" 18 | test "$unit" = "." && barf "invalid unit: $unit" 19 | test "$unit" = ".." && barf "invalid unit: $unit" 20 | test "$unit" != "${unit##*/}" && barf "invalid unit: $unit" 21 | test -z "$revision" && barf "missing revision" 22 | 23 | # Source config files in. 24 | 25 | safe . deployer-autoconfigure 26 | 27 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 28 | test -z "$DEPLOYER_QUEUE" && barf "missing environment variable: DEPLOYER_QUEUE" 29 | test -z "$DEPLOYER_TRIGGER" && barf "missing environment variable: DEPLOYER_TRIGGER" 30 | 31 | # Verify that the unit is actually a managed deployment. 32 | 33 | test ! -d "${DEPLOYER_UNITS_DIR}/${unit}" && { shout "unit is unmanaged: ${unit}" ; exit 99 ; } 34 | 35 | # Generate the job file name. 36 | 37 | revision_escaped=`safe job_escape "$revision"` || exit $? 38 | jobdatetime=`safe date +%Y%m%d.%H%M%S` || exit $? 39 | jobrandom=`safe hexdump -n 4 -e '1/4 "%08x"' < /dev/urandom` || exit $? 40 | jobname="${revision_escaped}.${jobdatetime}.${jobrandom}" 41 | 42 | # First enqueue a deployer job file into our local queue. 43 | 44 | safe job_enqueue "$DEPLOYER_QUEUE" "$unit" "$jobname" "$DEPLOYER_TRIGGER" 45 | 46 | # Then enqueue deployer job files into queues for other hosts. 47 | 48 | if [ -d "$DEPLOYER_HOSTQUEUES_DIR" ]; then 49 | if [ -f "$DEPLOYER_STATE_DIR/hostid" ]; then 50 | ourhostid=`safe cat "$DEPLOYER_STATE_DIR/hostid"` || exit $? 51 | fi 52 | for hostqueue in "$DEPLOYER_HOSTQUEUES_DIR"/*; do 53 | test -n "$ourhostid" -a "${hostqueue##*/}" = "$ourhostid" && continue 54 | test -d "$hostqueue/tmp" || continue 55 | safe job_enqueue "$hostqueue" "$unit" "$jobname" 56 | done 57 | fi 58 | } 59 | 60 | job_enqueue() { 61 | queue="$1" ; shift 62 | unit="$1" ; shift 63 | jobname="$1" ; shift 64 | trigger="$1" 65 | safe touch "$queue/tmp/$jobname" 66 | safe chmod g+w "$queue/tmp/$jobname" 67 | safe mv "$queue/tmp/$jobname" "$queue/req/$unit/$jobname" 68 | log 2 "enqueued new deployer job ${unit}.${jobname} into $queue" 69 | [ -n "$trigger" ] && trigger-pull "$trigger" >/dev/null 2>&1 70 | return 0 71 | } 72 | 73 | job_escape() { 74 | safe sed -e 's/%/%26/g; s/\//%2f/g; s/\./%2e/g' <&2 ; exit 100 ; } 80 | main "$@" 81 | exit $? 82 | 83 | -------------------------------------------------------------------------------- /bin/deployer-after-job: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # When a job exits 111 (temporary failure), retry by triggering 4 | # queue processing again after a delay. Otherwise pull any upstream 5 | # queue's trigger immediately. 6 | 7 | if [ $FSQ_STATUS -eq 111 ]; then 8 | trigger="$DEPLOYER_TRIGGER" 9 | delay="$DEPLOYER_RETRY_DELAY" 10 | else 11 | trigger="$DEPLOYER_MAILER_TRIGGER" 12 | delay=0 13 | fi 14 | 15 | if [ -n "$trigger" -a -e "$trigger" ]; then 16 | sleep "$delay" && trigger-pull "$trigger" >/dev/null 2>&1 & 17 | fi 18 | 19 | exit 0 20 | 21 | -------------------------------------------------------------------------------- /bin/deployer-autoconfigure: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Source config files from somewhere. 4 | 5 | if [ -n "$DEPLOYER_CONFIG" ]; then 6 | safe . "$DEPLOYER_CONFIG" 7 | else 8 | for conf in /etc/deployer.conf /usr/local/etc/deployer.conf /opt/local/etc/deployer.conf "$HOME"/.deployer.conf; do 9 | if [ -f "$conf" ]; then 10 | safe . "$conf" 11 | fi 12 | done 13 | fi 14 | 15 | -------------------------------------------------------------------------------- /bin/deployer-bouncer: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} "; } 4 | 5 | main() 6 | { 7 | if [ $# -lt 1 ]; then 8 | usage 1>&2 9 | exit 100 10 | fi 11 | 12 | service_root="$1" ; shift 13 | 14 | test -d "$service_root" || barf "service root missing: $service_root" 15 | test -z "$DEPLOYER_BOUNCED_DIR" && barf "missing environment variable: DEPLOYER_BOUNCED_DIR" 16 | test -d "$DEPLOYER_BOUNCED_DIR" || barf "deployer bounced directory missing: $DEPLOYER_BOUNCED_DIR" 17 | 18 | safe cd "$service_root" 19 | for svcname in *; do 20 | test -d "$svcname" || continue 21 | ( 22 | safe cd "$svcname" 23 | test -L root || continue 24 | if [ -e "$DEPLOYER_BOUNCED_DIR/$svcname" -a ! "$DEPLOYER_BOUNCED_DIR/$svcname" -ot root/.deployed ]; then 25 | continue 26 | fi 27 | log 2 "bouncing: $svcname" 28 | if [ -x bounce ]; then 29 | safe ./bounce 30 | else 31 | safe svc -h . 32 | fi 33 | safe touch "$DEPLOYER_BOUNCED_DIR/$svcname" 34 | ) 35 | rc="$?" ; test "$rc" -eq 0 || log 0 "error bouncing $svcname: $rc" 36 | done 37 | 38 | return 0 39 | } 40 | 41 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 42 | main "$@" 43 | exit $? 44 | 45 | -------------------------------------------------------------------------------- /bin/deployer-checkout: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} "; } 4 | 5 | main() 6 | { 7 | if [ $# -lt 2 ]; then 8 | usage 1>&2 9 | exit 100 10 | fi 11 | 12 | git_url="$1" ; shift 13 | branch="$1" ; shift 14 | 15 | # Validate config we use. 16 | 17 | test -z "$GIT_WORK_TREE" && barf "missing environment variable: GIT_WORK_TREE" 18 | test -d "$GIT_WORK_TREE" || barf "deploy directory missing: $GIT_WORK_TREE" 19 | test -z "$GIT_DIR" && barf "missing environment variable: GIT_DIR" 20 | test -z "$DEPLOYER_COMMAND" && barf "missing environment variable: DEPLOYER_COMMAND" 21 | 22 | # If .git/ already exists, assume there's nothing to do. 23 | 24 | test ! -d "$GIT_DIR" || return 0 25 | 26 | # If we use an ssh key, run under ssh-agent, and load it from stdin. 27 | 28 | if [ -n "$DEPLOYER_SSH_KEY" ]; then 29 | test -n "$SSH_AUTH_SOCK" || safe exec ssh-agent "$0" "$git_url" "$branch" "$@" 30 | safe ssh-add /dev/stdin 31 | exec 0&2 ; exit 100 ; } 53 | main "$@" 54 | exit $? 55 | 56 | -------------------------------------------------------------------------------- /bin/deployer-git-ssh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | if [ -n "$DEPLOYER_SSH_KNOWN_HOSTS" ]; then 3 | exec ssh -o "UserKnownHostsFile=$DEPLOYER_SSH_KNOWN_HOSTS" "$@" 4 | else 5 | exec ssh "$@" 6 | fi 7 | -------------------------------------------------------------------------------- /bin/deployer-init: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | main() 4 | { 5 | test -z "$DEPLOYER_QUEUE" && barf "missing environment variable: DEPLOYER_QUEUE" 6 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 7 | test -z "$DEPLOYER_TRIGGER" && barf "missing environment variable: DEPLOYER_TRIGGER" 8 | test -z "$DEPLOYER_STAGE_LOCK" && barf "missing environment variable: DEPLOYER_STAGE_LOCK" 9 | test -z "$DEPLOYER_UNIT_LOCK_DIR" && barf "missing environment variable: DEPLOYER_UNIT_LOCK_DIR" 10 | test -z "$DEPLOYER_BOUNCED_DIR" && barf "missing environment variable: DEPLOYER_BOUNCED_DIR" 11 | test -z "$DEPLOYER_DEPLOY_ROOT" && barf "missing environment variable: DEPLOYER_DEPLOY_ROOT" 12 | test "$DEPLOYER_CONCURRENCY" -ge 0 || barf "invalid environment variable: DEPLOYER_CONCURRENCY" 13 | test "$DEPLOYER_RETRY_DELAY" -ge 0 || barf "invalid environment variable: DEPLOYER_RETRY_DELAY" 14 | 15 | safe mkdir -p "$DEPLOYER_QUEUE" 16 | safe mkdir -p "$DEPLOYER_QUEUE/tmp" 17 | safe mkdir -p "$DEPLOYER_QUEUE/cur" 18 | safe mkdir -p "$DEPLOYER_QUEUE/new" 19 | safe mkdir -p "$DEPLOYER_QUEUE/req" 20 | safe mkdir -p "$DEPLOYER_QUEUE/fail" 21 | safe mkdir -p "$DEPLOYER_UNITS_DIR" 22 | safe mkdir -p "${DEPLOYER_STAGE_LOCK%/*}" 23 | safe mkdir -p "$DEPLOYER_UNIT_LOCK_DIR" 24 | safe mkdir -p "$DEPLOYER_BOUNCED_DIR" 25 | safe mkdir -p "$DEPLOYER_DEPLOY_ROOT/.tmp" 26 | 27 | test -p "$DEPLOYER_TRIGGER" || { 28 | safe mkdir -p "${DEPLOYER_TRIGGER%/*}" 29 | safe mkfifo "$DEPLOYER_TRIGGER" 30 | } 31 | 32 | return 0 33 | } 34 | 35 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 36 | main "$@" 37 | exit $? 38 | 39 | -------------------------------------------------------------------------------- /bin/deployer-inspect: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} [-b] [-a|-i] [-v] "; } 4 | 5 | main() 6 | { 7 | export DEPLOYER_INSPECT_WHAT="${DEPLOYER_INSPECT_WHAT:-active}" 8 | export DEPLOYER_INSPECT_BACKEND="${DEPLOYER_INSPECT_BACKEND:-0}" 9 | export DEPLOYER_INSPECT_VERBOSE="${DEPLOYER_INSPECT_VERBOSE:-0}" 10 | 11 | while [ $# -gt 0 ]; do 12 | case "$1" in 13 | -a) DEPLOYER_INSPECT_WHAT='active' ; shift ;; 14 | -i) DEPLOYER_INSPECT_WHAT='inactive' ; shift ;; 15 | -b) DEPLOYER_INSPECT_BACKEND=1 ; shift ;; 16 | -v) DEPLOYER_INSPECT_VERBOSE=1 ; shift ;; 17 | -h) usage ; exit 0 ;; 18 | -*) barf "unknown option: $1" ;; 19 | *) break ;; 20 | esac 21 | done 22 | 23 | if [ $# -ne 1 ]; then 24 | usage 1>&2 25 | exit 100 26 | fi 27 | 28 | unit="$1" ; shift 29 | 30 | # Validate unit. 31 | 32 | test -z "$unit" && barf "missing unit" 33 | test "$unit" = "." && barf "invalid unit: $unit" 34 | test "$unit" = ".." && barf "invalid unit: $unit" 35 | test "$unit" != "${unit##*/}" && barf "invalid unit: $unit" 36 | 37 | # Automatically configure unless we're in backend mode. 38 | 39 | test "$DEPLOYER_INSPECT_BACKEND" -eq 1 -o -n "$DEPLOYER_DEPLOY_ROOT" || safe . deployer-autoconfigure 40 | 41 | # Validate the global config we use. 42 | 43 | test -z "$DEPLOYER_DEPLOY_ROOT" && barf "missing environment variable: DEPLOYER_DEPLOY_ROOT" 44 | test -d "$DEPLOYER_DEPLOY_ROOT" || barf "deploy root directory missing: $DEPLOYER_DEPLOY_ROOT" 45 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 46 | test -d "$DEPLOYER_UNITS_DIR" || barf "deploy data directory missing: $DEPLOYER_UNITS_DIR" 47 | 48 | # Self-exec to put per-unit settings in environment, unless we've already done that. 49 | 50 | test "$unit" = "$DEPLOYER_UNIT" || safe exec envdir "${DEPLOYER_UNITS_DIR}/${unit}" "$0" "$unit" 51 | 52 | # Validate the per-unit config we use. 53 | 54 | test -z "$DEPLOYER_BASENAME" && barf "missing environment variable: DEPLOYER_BASENAME" 55 | 56 | # Require existing .a & .b directories, or a bare directory if we're in update-in-place mode. 57 | 58 | deploy_root="$DEPLOYER_DEPLOY_ROOT" 59 | basename="$DEPLOYER_BASENAME" 60 | path="${deploy_root}/${basename}" 61 | path_a="${deploy_root}/${basename}.a" 62 | path_b="${deploy_root}/${basename}.b" 63 | 64 | if [ "$DEPLOYER_IN_PLACE" -gt 0 ]; then 65 | test -d "$path" || barf "missing git checkout: $path" 66 | else 67 | test -d "$path_a" || barf "missing git checkout: $path_a" 68 | test -d "$path_b" || barf "missing git checkout: $path_b" 69 | fi 70 | 71 | # Determine the currently active and inactive checkouts for this unit. 72 | 73 | if [ "$DEPLOYER_IN_PLACE" -gt 0 ]; then 74 | target_active="$basename" 75 | target_inactive="$basename" 76 | else 77 | if [ -L "$path" ]; then 78 | target_active=`safe readlink "$path"` || exit $? 79 | elif [ -e "$path" ]; then 80 | barf "non-symlink at: $path" 81 | else 82 | barf "failed to find symlink at: $path" 83 | fi 84 | 85 | if [ "$target_active" = "${basename}.a" ]; then 86 | target_inactive="${basename}.b" 87 | else 88 | target_inactive="${basename}.a" 89 | fi 90 | fi 91 | 92 | # Output our findings. 93 | 94 | case "$DEPLOYER_INSPECT_WHAT" in 95 | active) what="$target_active" ;; 96 | inactive) what="$target_inactive" ;; 97 | *) barf "invalid value for what to inspect" ;; 98 | esac 99 | 100 | # In verbose mode, also output commit hash and branch. 101 | 102 | if [ "$DEPLOYER_INSPECT_VERBOSE" -gt 0 ]; then 103 | export GIT_WORK_TREE="${deploy_root}/${what}" 104 | export GIT_DIR="${deploy_root}/${what}/.git" 105 | branch=`git symbolic-ref HEAD 2>/dev/null` # might fail, that's okay 106 | branch="${branch##refs/heads/}" 107 | branch="${branch:-(detached)}" 108 | info=`safe git log -n 1 --pretty="%h %s" HEAD` || exit $? 109 | what="${what} ${branch} ${info}" 110 | fi 111 | 112 | safe printf "%s\n" "$what" 113 | 114 | return 0 115 | } 116 | 117 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 118 | main "$@" 119 | exit $? 120 | 121 | -------------------------------------------------------------------------------- /bin/deployer-mailer-queue: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | safe test -n "$DEPLOYER_MAILER_QUEUE" 6 | safe cd "$DEPLOYER_MAILER_QUEUE" 7 | exec fsq-run \ 8 | -v \ 9 | -Q new \ 10 | -D cur \ 11 | . deployer-mailer-worker 12 | 13 | -------------------------------------------------------------------------------- /bin/deployer-mailer-service: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | safe test -n "$DEPLOYER_CONFIG" 6 | safe . "$DEPLOYER_CONFIG" 7 | safe test -n "$DEPLOYER_MAILER_TRIGGER" 8 | 9 | exec trigger-listen -v -1 -c 1 "$DEPLOYER_MAILER_TRIGGER" \ 10 | sh -e -c '. "$0" ; cd "$DEPLOYER_MAILER_QUEUE" ; exec "$@"' "$DEPLOYER_CONFIG" deployer-mailer-queue 11 | 12 | -------------------------------------------------------------------------------- /bin/deployer-mailer-worker: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | main() { 4 | # Parse data out of the job file name. 5 | 6 | jobname="$FSQ_JOB" 7 | unit="${jobname%%.*}" ; jobname="${jobname#*.}" 8 | revision="${jobname%%.*}" ; jobname="${jobname#*.}" 9 | jobdate="${jobname%%.*}" ; jobname="${jobname#*.}" 10 | jobtime="${jobname%%.*}" ; jobname="${jobname#*.}" 11 | jobrandom="${jobname%%.*}" ; jobname="${jobname#*.}" 12 | 13 | [ -z "$unit" ] && barf "cannot parse unit from jobfile: $FSQ_JOB" 100 14 | [ -z "$revision" ] && barf "cannot parse revision from jobfile: $FSQ_JOB" 100 15 | 16 | unit=`safe job_unescape "$unit"` || exit $? 17 | revision=`safe job_unescape "$revision"` || exit $? 18 | 19 | # Validate unit against unsafe chars. 20 | 21 | [ "$unit" = "." ] && barf "invalid unit: $unit" 22 | [ "$unit" = ".." ] && barf "invalid unit: $unit" 23 | [ "$unit" != "${unit##*/}" ] && barf "invalid unit: $unit" 24 | 25 | # Validate config we use. 26 | 27 | test -z "$DEPLOYER_MAILER_FROM" && barf "missing environment variable: DEPLOYER_MAILER_FROM" 28 | test -z "$DEPLOYER_MAILER_TO" && barf "missing environment variable: DEPLOYER_MAILER_TO" 29 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 30 | 31 | # Self-exec to put per-unit settings in environment, unless we've already done that. 32 | 33 | test "$unit" = "$DEPLOYER_UNIT" || safe exec envdir "${DEPLOYER_UNITS_DIR}/${unit}" "$0" "$@" 34 | 35 | # Default is to assume failure. 36 | 37 | DEPLOYER_STATUS=100 38 | DEPLOYER_RESULT= 39 | 40 | while IFS="=" read var rest; do 41 | case "$var" in 42 | DEPLOYER_STATUS) DEPLOYER_STATUS="$rest" ;; 43 | DEPLOYER_RESULT) DEPLOYER_RESULT="$rest" ;; 44 | esac 45 | done 46 | 47 | # Parse information from DEPLOYER_RESULT. 48 | # If we didn't encounter it, pick some sensible fallbacks. 49 | 50 | DEPLOYER_RESULT=${DEPLOYER_RESULT:-"$unit - $revision $jobdate $jobtime"} 51 | safe read unit_ checkout branch hash message <&2 ; exit 100 ; } 88 | 89 | main "$@" 90 | 91 | -------------------------------------------------------------------------------- /bin/deployer-manage: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} [-x] [-k path/to/ssh/key] [-c command] [-u user] [-g group] [-b basename] [-r path/to/deply/root] []"; } 4 | 5 | main() 6 | { 7 | DEPLOYER_IN_PLACE=${DEPLOYER_IN_PLACE:-0} 8 | DEPLOYER_COMMAND=${DEPLOYER_COMMAND:-make} 9 | options="" 10 | 11 | while [ $# -gt 0 ]; do 12 | case "$1" in 13 | -*) options="${options}${1#-}" ;; 14 | esac 15 | case "$1" in 16 | -x) DEPLOYER_IN_PLACE=1 ; shift ;; 17 | -k) DEPLOYER_SSH_KEY="$2" ; shift ; shift ;; 18 | -c) DEPLOYER_COMMAND="$2" ; shift ; shift ;; 19 | -u) DEPLOYER_USER="$2" ; shift ; shift ;; 20 | -g) DEPLOYER_GROUP="$2" ; shift ; shift ;; 21 | -b) DEPLOYER_BASENAME="$2" ; shift ; shift ;; 22 | -r) DEPLOYER_DEPLOY_ROOT="$2" ; shift ; shift ;; 23 | -h) usage ; exit 0 ;; 24 | -*) barf "unknown option: $1" ;; 25 | *) break ;; 26 | esac 27 | done 28 | export DEPLOYER_IN_PLACE 29 | export DEPLOYER_SSH_KEY 30 | export DEPLOYER_COMMAND 31 | export DEPLOYER_USER 32 | export DEPLOYER_GROUP 33 | export DEPLOYER_BASENAME 34 | export DEPLOYER_DEPLOY_ROOT 35 | 36 | if [ $# -lt 2 ]; then 37 | usage 1>&2 38 | exit 100 39 | fi 40 | 41 | unit="$1" ; shift 42 | git_url="$1" ; shift 43 | branch="${1:-master}" 44 | 45 | test -z "$unit" && barf "missing unit" 46 | test "$unit" = "." && barf "invalid unit: $unit" 47 | test "$unit" = ".." && barf "invalid unit: $unit" 48 | test "$unit" != "${unit##*/}" && barf "invalid unit: $unit" 49 | test -z "$DEPLOYER_DEPLOY_ROOT" && barf "missing environment variable: DEPLOYER_DEPLOY_ROOT" 50 | test -d "$DEPLOYER_DEPLOY_ROOT" || barf "deploy root directory missing: $DEPLOYER_DEPLOY_ROOT" 51 | test -z "$DEPLOYER_QUEUE" && barf "missing environment variable: DEPLOYER_QUEUE" 52 | test -d "$DEPLOYER_QUEUE" || barf "deploy root directory missing: $DEPLOYER_QUEUE" 53 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 54 | test -d "$DEPLOYER_UNITS_DIR" || barf "deploy data directory missing: $DEPLOYER_UNITS_DIR" 55 | test -z "$DEPLOYER_COMMAND" && barf "missing environment variable: DEPLOYER_COMMAND" 56 | test -z "$DEPLOYER_USER" || DEPLOYER_GROUP=${DEPLOYER_GROUP:-`id -ng "$DEPLOYER_USER"`} || barf "gid error for $DEPLOYER_USER" 57 | test -n "$DEPLOYER_BASENAME" || DEPLOYER_BASENAME="$unit" 58 | 59 | # In privileged mode, some per-unit settings are required. 60 | 61 | if [ `id -u` -eq 0 ]; then 62 | test -z "$DEPLOYER_USER" && barf "missing required per-unit setting in privileged mode: DEPLOYER_USER" 63 | test -z "$DEPLOYER_GROUP" && barf "missing required per-unit setting in privileged mode: DEPLOYER_GROUP" 64 | fi 65 | 66 | # Validate the ssh key path if given. 67 | 68 | test -z "$DEPLOYER_SSH_KEY" || test "${DEPLOYER_SSH_KEY#/}" != "$DEPLOYER_SSH_KEY" || barf "non-absolute path for DEPLOYER_SSH_KEY: $DEPLOYER_SSH_KEY" 69 | 70 | # Create a per-unit directory for settings. 71 | 72 | safe mkdir -p "${DEPLOYER_UNITS_DIR}/${unit}" 73 | 74 | # Store per-unit settings. 75 | 76 | test -z "$DEPLOYER_SSH_KEY" || safe printf "%s" "$DEPLOYER_SSH_KEY" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_SSH_KEY" 77 | test -z "$DEPLOYER_USER" || safe printf "%s" "$DEPLOYER_USER" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_USER" 78 | test -z "$DEPLOYER_GROUP" || safe printf "%s" "$DEPLOYER_GROUP" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_GROUP" 79 | safe printf "%s" "$DEPLOYER_COMMAND" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_COMMAND" 80 | safe printf "%s" "$DEPLOYER_IN_PLACE" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_IN_PLACE" 81 | safe printf "%s" "$DEPLOYER_BASENAME" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_BASENAME" 82 | safe printf "%s" "$unit" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_UNIT" 83 | 84 | # Only override this global setting if -r was specified. 85 | 86 | if [ "${options#*r}" != "$options" ]; then 87 | safe rm -f "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_DEPLOY_ROOT" 88 | safe printf "%s" "$DEPLOYER_DEPLOY_ROOT" > "${DEPLOYER_UNITS_DIR}/${unit}/DEPLOYER_DEPLOY_ROOT" 89 | fi 90 | 91 | # Create a per-unit directory for incoming deployer job requests. 92 | 93 | safe mkdir -p "${DEPLOYER_QUEUE}/req/${unit}" 94 | 95 | # Create foo.a/ and foo.b/ directories (or just foo/ if in-place) under the deploy root. 96 | 97 | basename="$DEPLOYER_BASENAME" 98 | 99 | if [ "$DEPLOYER_IN_PLACE" -gt 0 ]; then 100 | checkouts="$basename" 101 | target_default="$basename" 102 | else 103 | checkouts="${basename}.a ${basename}.b" 104 | target_default="$basename.a" 105 | fi 106 | 107 | for checkout in $checkouts; do 108 | export GIT_WORK_TREE="$DEPLOYER_DEPLOY_ROOT/$checkout" 109 | export GIT_DIR="${GIT_WORK_TREE}/.git" 110 | safe mkdir -p "$GIT_WORK_TREE" 111 | if [ `id -u` -eq 0 ]; then 112 | safe chown "${DEPLOYER_USER}:${DEPLOYER_GROUP}" "$GIT_WORK_TREE" 113 | { 114 | safe chown "${DEPLOYER_USER}:${DEPLOYER_GROUP}" /dev/stdin 115 | safe setuidgid -s "$DEPLOYER_USER" env SSH_AUTH_SOCK="" HOME=`eval echo ~$DEPLOYER_USER` deployer-checkout "$git_url" "$branch" 116 | safe chgrp -R "$DEPLOYER_GROUP" "$GIT_WORK_TREE" 117 | } < "${DEPLOYER_SSH_KEY:-/dev/null}" 118 | else 119 | safe env SSH_AUTH_SOCK="" deployer-checkout "$git_url" "$branch" < "${DEPLOYER_SSH_KEY:-/dev/null}" 120 | fi 121 | done 122 | 123 | # Ensure one of foo.a/ and foo.b/ is active (has the foo symlink pointing to it). 124 | 125 | if [ "$DEPLOYER_IN_PLACE" -eq 0 ]; then 126 | test -e "$DEPLOYER_DEPLOY_ROOT/${unit}" || safe ln -s "$target_default" "$DEPLOYER_DEPLOY_ROOT/${unit}" 127 | fi 128 | 129 | # Report on the current situation. 130 | 131 | log 2 "deployed:" "$unit" `deployer-inspect -b -v -a "$unit"` 132 | 133 | return 0 134 | } 135 | 136 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 137 | main "$@" 138 | exit $? 139 | 140 | -------------------------------------------------------------------------------- /bin/deployer-perform: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} []"; } 4 | 5 | main() 6 | { 7 | if [ $# -lt 2 ]; then 8 | usage 1>&2 9 | exit 100 10 | fi 11 | 12 | unit="$1" ; shift 13 | revision="$1" ; shift 14 | 15 | # Validate unit and revision against unsafe path chars. 16 | 17 | test -z "$unit" && barf "missing unit" 18 | test "$unit" = "." && barf "invalid unit: $unit" 19 | test "$unit" = ".." && barf "invalid unit: $unit" 20 | test "$unit" != "${unit##*/}" && barf "invalid unit: $unit" 21 | test -z "$revision" && barf "missing revision" 22 | 23 | # Validate config we use. 24 | 25 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 26 | test -z "$DEPLOYER_DEPLOY_ROOT" && barf "missing environment variable: DEPLOYER_DEPLOY_ROOT" 27 | test -d "$DEPLOYER_DEPLOY_ROOT" || barf "deploy root directory missing: $DEPLOYER_DEPLOY_ROOT" 28 | 29 | # Verify that the unit is actually a managed deployment. 30 | 31 | test ! -d "${DEPLOYER_UNITS_DIR}/${unit}" && barf "unit is unmanaged: ${unit}" 32 | 33 | # Self-exec to put per-unit settings in environment, unless we've already done that. 34 | 35 | test "$unit" = "$DEPLOYER_UNIT" || safe exec envdir "${DEPLOYER_UNITS_DIR}/${unit}" "$0" "$unit" "$revision" "$@" 36 | 37 | # In privileged mode, some per-unit settings are required. 38 | 39 | if [ `id -u` -eq 0 ]; then 40 | test -z "$DEPLOYER_USER" && barf "missing required per-unit setting in privileged mode: DEPLOYER_USER" 41 | test -z "$DEPLOYER_GROUP" && barf "missing required per-unit setting in privileged mode: DEPLOYER_GROUP" 42 | fi 43 | 44 | # Validate the ssh key path if given. 45 | 46 | test -z "$DEPLOYER_SSH_KEY" || test "${DEPLOYER_SSH_KEY#/}" != "$DEPLOYER_SSH_KEY" || barf "non-absolute path for DEPLOYER_SSH_KEY: $DEPLOYER_SSH_KEY" 47 | 48 | # Run the "bottom half" of this operation. Drop privileges in privileged mode. 49 | 50 | if [ `id -u` -eq 0 ]; then 51 | { 52 | safe chown "${DEPLOYER_USER}:${DEPLOYER_GROUP}" /dev/stdin 53 | setuidgid -s "$DEPLOYER_USER" env SSH_AUTH_SOCK="" HOME=`eval echo ~$DEPLOYER_USER` deployer-update "$unit" "$revision" "$@" || exit $? 54 | } < "${DEPLOYER_SSH_KEY:-/dev/null}" || exit $? 55 | else 56 | env SSH_AUTH_SOCK="" deployer-update "$unit" "$revision" "$@" < "${DEPLOYER_SSH_KEY:-/dev/null}" || exit $? 57 | fi 58 | 59 | # Success. Unless we're in-place, swap active & inactive checkouts via the symlink. 60 | 61 | if [ "$DEPLOYER_IN_PLACE" -gt 0 ]; then 62 | rc=0 63 | else 64 | # Create the new symlink under the .tmp/ dir. 65 | 66 | target_new=`safe deployer-inspect -b -i "$unit"` || exit $? 67 | safe ln -s "$target_new" "${DEPLOYER_DEPLOY_ROOT}/.tmp/${unit}" 68 | 69 | # Atomically move the new symlink into place. 70 | 71 | ( 72 | safe cd "${DEPLOYER_DEPLOY_ROOT}/.tmp" 73 | safe mv "$unit" .. 74 | ) 75 | rc="$?" 76 | fi 77 | 78 | if [ $rc -eq 0 ]; then 79 | inspect=`safe deployer-inspect -b -v -a "$unit"` || exit $? 80 | echo "DEPLOYER_RESULT=$unit $inspect" 81 | log 2 "deployed:" "$unit" "$inspect" 82 | fi 83 | 84 | return $rc 85 | } 86 | 87 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 88 | main "$@" 89 | exit $? 90 | 91 | -------------------------------------------------------------------------------- /bin/deployer-queue: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | safe test -n "$DEPLOYER_QUEUE" 6 | safe deployer-stage 7 | safe cd "$DEPLOYER_QUEUE" 8 | FSQ_UPSTREAM="$DEPLOYER_MAILER_QUEUE" exec fsq-run \ 9 | -v \ 10 | -U \ 11 | -Q new \ 12 | -D cur \ 13 | -A deployer-after-job \ 14 | . deployer-worker 15 | 16 | -------------------------------------------------------------------------------- /bin/deployer-service: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | safe test -n "$DEPLOYER_CONFIG" 6 | safe . "$DEPLOYER_CONFIG" 7 | safe test "$DEPLOYER_CONCURRENCY" -ge 0 8 | safe test -n "$DEPLOYER_TRIGGER" 9 | 10 | exec trigger-listen -v -1 -c "$DEPLOYER_CONCURRENCY" -t "${DEPLOYER_RESCAN_INTERVAL:-4294967295}" "$DEPLOYER_TRIGGER" \ 11 | sh -e -c '. "$0" ; cd "$DEPLOYER_QUEUE" ; exec "$@"' "$DEPLOYER_CONFIG" deployer-queue 12 | 13 | -------------------------------------------------------------------------------- /bin/deployer-stage: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | main() 4 | { 5 | # Upfront validation. 6 | 7 | safe test -n "$DEPLOYER_QUEUE" 8 | safe test -n "$DEPLOYER_STAGE_LOCK" 9 | 10 | # Grab the lock that protects the job-staging operation. 11 | 12 | test "${LOCKED_DEPLOYER_STAGE:-0}" -gt 0 || \ 13 | safe exec setlock "$DEPLOYER_STAGE_LOCK" env LOCKED_DEPLOYER_STAGE=1 "$0" "$@" 14 | 15 | # Move deploy job files from the per-unit request directories into processing. 16 | 17 | safe cd "$DEPLOYER_QUEUE/req" 18 | for jobpath in */*; do 19 | test -f "$jobpath" || continue 20 | unit="${jobpath%/*}" 21 | file="${jobpath##*/}" 22 | unit_escaped=`safe job_escape "$unit"` || exit $? 23 | safe mv "$jobpath" "../new/${unit_escaped}.${file}" 24 | done 25 | } 26 | 27 | job_escape() { 28 | safe sed -e 's/%/%26/g; s/\//%2f/g; s/\./%2e/g' <&2 ; exit 100 ; } 34 | main "$@" 35 | exit $? 36 | 37 | -------------------------------------------------------------------------------- /bin/deployer-unmanage: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} "; } 4 | 5 | main() 6 | { 7 | if [ $# -lt 1 ]; then 8 | usage 1>&2 9 | exit 100 10 | fi 11 | 12 | unit="$1" ; shift 13 | 14 | test -z "$unit" && barf "missing unit" 15 | test "$unit" = "." && barf "invalid unit: $unit" 16 | test "$unit" = ".." && barf "invalid unit: $unit" 17 | test "$unit" != "${unit##*/}" && barf "invalid unit: $unit" 18 | test -z "$DEPLOYER_DEPLOY_ROOT" && barf "missing environment variable: DEPLOYER_DEPLOY_ROOT" 19 | test -d "$DEPLOYER_DEPLOY_ROOT" || barf "deploy root directory missing: $DEPLOYER_DEPLOY_ROOT" 20 | test -z "$DEPLOYER_QUEUE" && barf "missing environment variable: DEPLOYER_QUEUE" 21 | test -d "$DEPLOYER_QUEUE" || barf "deploy root directory missing: $DEPLOYER_QUEUE" 22 | test -z "$DEPLOYER_UNITS_DIR" && barf "missing environment variable: DEPLOYER_UNITS_DIR" 23 | test -d "$DEPLOYER_UNITS_DIR" || barf "deploy data directory missing: $DEPLOYER_UNITS_DIR" 24 | test -z "$DEPLOYER_BASENAME" && barf "missing environment variable: DEPLOYER_BASENAME" 25 | 26 | # Remove the per-unit directory for incoming deployer job requests. 27 | 28 | safe rm -rf "${DEPLOYER_QUEUE}/req/${unit}" 29 | 30 | # Remove the per-unit directory for settings. 31 | 32 | safe rm -rf "${DEPLOYER_UNITS_DIR}/${unit}" 33 | 34 | # Remove the active symlink, if any. 35 | 36 | basename="$DEPLOYER_BASENAME" 37 | path="$DEPLOYER_DEPLOY_ROOT/$basename" 38 | test -L "$path" && safe rm "$path" 39 | 40 | # Remove foo.a/, foo.b/, and foo/ directories. 41 | 42 | for checkout in "${basename}.a" "${basename}.b" "$basename"; do 43 | dir="$DEPLOYER_DEPLOY_ROOT/$checkout" 44 | test -d "$dir" && rm -rf "$dir" 45 | done 46 | 47 | return 0 48 | } 49 | 50 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 51 | main "$@" 52 | exit $? 53 | 54 | -------------------------------------------------------------------------------- /bin/deployer-update: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | usage() { echo "usage: ${0##*/} []"; } 4 | 5 | main() 6 | { 7 | if [ $# -lt 2 ]; then 8 | usage 1>&2 9 | exit 100 10 | fi 11 | 12 | unit="$1" ; shift 13 | revision="$1" ; shift 14 | 15 | # Validate unit and revision against unsafe path chars. 16 | 17 | test -z "$unit" && barf "missing unit" 18 | test "$unit" = "." && barf "invalid unit: $unit" 19 | test "$unit" = ".." && barf "invalid unit: $unit" 20 | test "$unit" != "${unit##*/}" && barf "invalid unit: $unit" 21 | test -z "$revision" && barf "missing revision" 22 | 23 | # Validate config we use. 24 | 25 | test -z "$DEPLOYER_DEPLOY_ROOT" && barf "missing environment variable: DEPLOYER_DEPLOY_ROOT" 26 | test -d "$DEPLOYER_DEPLOY_ROOT" || barf "deploy root directory missing: $DEPLOYER_DEPLOY_ROOT" 27 | test -z "$DEPLOYER_COMMAND" && barf "missing environment variable: DEPLOYER_COMMAND" 28 | 29 | # If we use an ssh key, run under ssh-agent, and load it from stdin. 30 | 31 | if [ -n "$DEPLOYER_SSH_KEY" ]; then 32 | test -n "$SSH_AUTH_SOCK" || safe exec ssh-agent "$0" "$unit" "$revision" "$@" 33 | safe ssh-add /dev/stdin 34 | exec 0/dev/null` # might fail, that's okay 51 | branch_old=${branch_old##refs/heads/} 52 | original_revision="$revision" 53 | 54 | # If revision is _ or @, try to stay on same branch as active checkout. 55 | 56 | if [ "$revision" = "_" -o "$revision" = "@" ]; then 57 | revision="$branch_old" 58 | test -n "$revision" || barf "error defaulting to active branch: detached checkout?" 59 | fi 60 | 61 | # Update the inactive checkout with git. 62 | 63 | export GIT_WORK_TREE="$path_new" 64 | export GIT_DIR="${path_new}/.git" 65 | export GIT_SSH=deployer-git-ssh 66 | safe git fetch 67 | 68 | # Unload any per-unit ssh-key, so it's unavailable to the post-update command. 69 | 70 | test -z "$DEPLOYER_SSH_KEY" || safe ssh-add -D 71 | 72 | # Get the new commit hash. 73 | 74 | commit_new=`safe git rev-parse origin/"$revision"` 75 | test -n "$commit_new" || barf "error getting current revision for: ${target_old}" 76 | 77 | # See if the inactive checkout would differ from the active one. 78 | # If revision was @, override difference check and always redeploy. 79 | 80 | differs=0 81 | if [ "$original_revision" = "@" ]; then 82 | differs=1 83 | elif [ "x"`git cat-file -t "$commit_old" 2>/dev/null` != "xcommit" ]; then 84 | # This commit might not even exist anymore (eg was deleted in force push). 85 | differs=1 86 | elif [ "$commit_old" != "$commit_new" ]; then 87 | # Content hashes differ, so trees must differ. 88 | differs=1 89 | elif [ -n "$branch_old" -a "$branch_old" != "$revision" ]; then 90 | # Content hashes may be identical, but branch names differ, so force a switch. 91 | differs=1 92 | fi 93 | 94 | # Exit with special status if it would be identical to the active checkout. 95 | 96 | if [ "$differs" -eq 0 ]; then 97 | log 2 "no changes for: ${target_new} ${revision}" 98 | return 99 99 | fi 100 | 101 | # XXX workaround: git-stash doesn't honor GIT_WORK_TREE / GIT_DIR as of 1.9.1 102 | log 2 "applying changes to inactive checkout: ${target_new}" 103 | ( safe cd "$GIT_WORK_TREE" && git stash ) 104 | safe git checkout -B "$revision" origin/"$revision" 105 | 106 | # Run the DEPLOYER_COMMAND program (typically just `make`) to do post-update tasks. 107 | # Touch a file to indicate that a new deployment happened. 108 | 109 | ( 110 | safe cd "$path_new" 111 | safe "$DEPLOYER_COMMAND" 1>&2 112 | safe touch .deployed 113 | ) 114 | 115 | return $? 116 | } 117 | 118 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 119 | main "$@" 120 | exit $? 121 | 122 | -------------------------------------------------------------------------------- /bin/deployer-worker: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | main() 4 | { 5 | # Parse data out of the job file name. 6 | 7 | jobname="$FSQ_JOB" 8 | unit="${jobname%%.*}" ; jobname="${jobname#*.}" 9 | revision="${jobname%%.*}" ; jobname="${jobname#*.}" 10 | jobdate="${jobname%%.*}" ; jobname="${jobname#*.}" 11 | jobtime="${jobname%%.*}" ; jobname="${jobname#*.}" 12 | jobrandom="${jobname%%.*}" ; jobname="${jobname#*.}" 13 | 14 | [ -z "$unit" ] && barf "cannot parse unit from jobfile: $FSQ_JOB" 100 15 | [ -z "$revision" ] && barf "cannot parse revision from jobfile: $FSQ_JOB" 100 16 | 17 | unit=`safe job_unescape "$unit"` || exit $? 18 | revision=`safe job_unescape "$revision"` || exit $? 19 | 20 | # Validate unit against unsafe chars. 21 | 22 | [ "$unit" = "." ] && barf "invalid unit: $unit" 23 | [ "$unit" = ".." ] && barf "invalid unit: $unit" 24 | [ "$unit" != "${unit##*/}" ] && barf "invalid unit: $unit" 25 | 26 | # Only one deploy is allowed to run at a time for a given unit. 27 | # Grab the per-unit lock, or fail immediately if already locked. 28 | # On lock success, record it with an env var and re-execute self. 29 | 30 | export LOCKED_UNIT 31 | 32 | if [ -z "$LOCKED_UNIT" ]; then 33 | lockfile="$DEPLOYER_UNIT_LOCK_DIR"/"$unit".lock 34 | LOCKED_UNIT="$lockfile" safe exec setlock -n "$lockfile" "$0" "$@" 35 | else 36 | shout "locked: $LOCKED_UNIT" 37 | fi 38 | 39 | # Got all necessary locks. Invoke the deployer. 40 | 41 | deployer-perform "$unit" "$revision" "$@" 42 | rc=$? 43 | 44 | # On successful deployment, pull the bouncer's trigger. 45 | 46 | [ "$rc" -eq 0 -a -n "$DEPLOYER_BOUNCER_TRIGGER" ] && trigger-pull "$DEPLOYER_BOUNCER_TRIGGER" >/dev/null 2>&1 47 | 48 | return $rc 49 | } 50 | 51 | job_unescape() { 52 | safe sed -e 's/%%/%/g; s/%2f/\//g; s/%2e/\./g' <&1 60 | fi 61 | 62 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 63 | 64 | main "$@" 65 | rc=$? 66 | 67 | echo "DEPLOYER_STATUS=$rc" 68 | 69 | # Ignore an exit code of 99 = there was nothing to deploy. 70 | [ $rc -eq 99 ] && rc=0 71 | 72 | exit $rc 73 | 74 | -------------------------------------------------------------------------------- /bin/rewind: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env perl 2 | # Rewind stdin if it's seekable. 3 | # Apparently neither dd(1) nor shell redirections can do this? 4 | seek STDIN, 0, 0 or die "cannot rewind stdin: $!"; 5 | exit 0; 6 | -------------------------------------------------------------------------------- /debian/changelog: -------------------------------------------------------------------------------- 1 | deployer (1:0.2.1-2) trusty; urgency=low 2 | 3 | * Fix debian package control field that caused apt-get upgrade reinstallation. 4 | 5 | -- Alan Grow Thu, 31 Aug 2017 19:05:29 -0600 6 | 7 | deployer (1:0.2.1) trusty; urgency=low 8 | 9 | * Fix issues with hostqueues. 10 | 11 | -- Alan Grow Thu, 31 Aug 2017 19:05:29 -0600 12 | 13 | deployer (1:0.2.0) trusty; urgency=low 14 | 15 | * Add cluster support. 16 | 17 | -- Alan Grow Thu, 31 Aug 2017 17:54:05 -0600 18 | 19 | deployer (1:0.1.5) trusty; urgency=low 20 | 21 | * Fix bug where bouncer would skip the initial bounces. 22 | 23 | -- Alan Grow Thu, 24 Aug 2017 17:23:22 -0600 24 | 25 | deployer (1:0.1.4) trusty; urgency=low 26 | 27 | * Register and unregister services. 28 | 29 | -- Alan Grow Tue, 22 Aug 2017 18:53:43 -0600 30 | 31 | deployer (1:0.1.3) trusty; urgency=low 32 | 33 | * Fix trigger package dependency. 34 | 35 | -- Alan Grow Thu, 17 Aug 2017 18:12:31 -0600 36 | 37 | deployer (1:0.1.2) trusty; urgency=low 38 | 39 | * Install services. 40 | 41 | -- Alan Grow Fri, 18 Aug 2017 00:02:05 +0000 42 | 43 | deployer (1:0.1.1) trusty; urgency=low 44 | 45 | * Install /etc/deployer.conf. 46 | 47 | -- Alan Grow Thu, 10 Aug 2017 15:02:22 -0600 48 | 49 | deployer (1:0.1.0) trusty; urgency=low 50 | 51 | * Initial debianization. 52 | 53 | -- Alan Grow Thu, 10 Aug 2017 12:31:42 -0600 54 | 55 | -------------------------------------------------------------------------------- /debian/compat: -------------------------------------------------------------------------------- 1 | 7 2 | -------------------------------------------------------------------------------- /debian/control: -------------------------------------------------------------------------------- 1 | Source: deployer 2 | Section: devel 3 | Priority: optional 4 | Maintainer: Alan Grow 5 | Build-Depends: debhelper (>= 8.0.0) 6 | Standards-Version: 3.7.3.0 7 | Vcs-Git: git@github.com:endcrawl/deployer.git 8 | Vcs-Browser: https://github.com/endcrawl/deployer 9 | 10 | Package: deployer 11 | Section: devel 12 | Architecture: all 13 | Depends: daemontools (>= 1:1.13), trigger (>= 0.68), fsq-run (>= 1:0.1.0), daemontools-extra (>= 1:0.1.0) 14 | Description: Automated zero-downtime deployments. 15 | 16 | -------------------------------------------------------------------------------- /debian/copyright: -------------------------------------------------------------------------------- 1 | ../LICENSE -------------------------------------------------------------------------------- /debian/postinst: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -e 3 | 4 | . /etc/deployer.conf 5 | 6 | # Create privsep users. 7 | 8 | getent group bouncer >/dev/null || 9 | groupadd bouncer -g 64222 10 | getent passwd bouncer >/dev/null || 11 | useradd bouncer --system -u 64222 -g bouncer -c "Bouncer Service" -d /nonexistent -s /usr/sbin/nologin 12 | 13 | getent group deployer-mailer >/dev/null || 14 | groupadd deployer-mailer -g 64223 15 | getent passwd deployer-mailer >/dev/null || 16 | useradd deployer-mailer --system -u 64223 -g deployer-mailer -c "Deployer Mailer Service" -d /nonexistent -s /usr/sbin/nologin 17 | 18 | getent group deployer-log >/dev/null || 19 | groupadd deployer-log -g 64224 20 | getent passwd deployer-log >/dev/null || 21 | useradd deployer-log --system -u 64224 -g deployer-log -c "Deployer Logger" -d /nonexistent -s /usr/sbin/nologin 22 | 23 | # Create operator groups. 24 | 25 | getent group deployers >/dev/null || 26 | groupadd -g 622 deployers 27 | 28 | getent group svc-deployer >/dev/null || 29 | groupadd -g 623 svc-deployer 30 | 31 | getent group svc-deployer-mailer >/dev/null || 32 | groupadd -g 624 svc-deployer-mailer 33 | 34 | getent group svc-bouncer >/dev/null || 35 | groupadd -g 625 svc-bouncer 36 | 37 | cat < don't send an email. 12 | - Look for `^DEPLOYER_RESULT=...` and parse the parts on success. If no parts, assume nothing was done => don't send an email. 13 | - [x] How can we scan for these, then seek back to the beginning of the file? 14 | - It's dumb that no standard unix utility rewinds. Had to write a tiny one in perl :/ 15 | - Example: `[deployer] success: myrepo/master -- deadbeef fixed the widget` 16 | - [x] `To:` address should be a per-deployer email group configured in the unit. 17 | - Use `DEPLOYER_MAILER_TO`. 18 | - Global default should be respected. 19 | - If `DEPLOYER_MAILER_TO` isn't defined, then don't send an email at all. 20 | - [x] `From:` line should come from global config. 21 | - Use `DEPLOYER_MAILER_FROM`. 22 | - [x] Email body should be formatted readably. 23 | - Probably okay as is, with `Content-Type: text/plain`. 24 | - [x] Hook deployer successes and failures up to the upstream mailer queue. 25 | - Should probably be an optional thing. Tests don't need this mail infrastructure. 26 | - [x] `deployer-queue` needs to know whether there's an upstream queue, and modify options accordingly. 27 | - [x] `deployer-after-job` needs to pull the upstream trigger. See `xevious/bin/render-queue` for a good example of how to do this. 28 | - [ ] Verify, through experiment if necessary, that the `sendmail` arguments in `deployer-mailer-worker` are real and work. 29 | -------------------------------------------------------------------------------- /doc/privsep.md: -------------------------------------------------------------------------------- 1 | ### Introducing Settings for each Managed Unit 2 | 3 | A managed unit "foo" has its settings stored in a directory `$DEPLOYER_UNITS_DIR/units/foo/`. For system installs, `DEPLOYER_UNITS_DIR` is by default `/var/lib/deployer`. 4 | 5 | For one thing, this allows the `deploy` program to `test -d` for whether a unit is managed, and bail early with an error message (or possibly just ignore) if it isn't. 6 | 7 | 8 | ### Privilege Separation 9 | 10 | When `deployer-queue` is running as root, the `$DEPLOYER_UNITS_DIR/units/foo/DEPLOYER_USER` file determines which user account we drop privileges to before conducting any deployment operations. It's a fatal error if this file is missing and we're running as root. 11 | 12 | Since deployment operations can now happen under one of many user accounts, and, in most setups, the `git fetch` operation needs access to a shared ssh key, this presents a problem: ssh refuses to use an `~/.ssh/id_rsa` file that's group readable. 13 | 14 | So, we use `ssh-agent`. The `DEPLOYER_SSH_KEY` setting tells us where to find the shared ssh keypair, which should be `root:root`. Before switching to `DEPLOYER_USER`, we run `ssh-agent`, we add the keypair, and we make `SSH_AUTH_SOCK` group-accessible to `DEPLOYER_USER`. 15 | 16 | Then we drop privileges to `DEPLOYER_USER`, taking care to preserve `SSH_AUTH_SOCK` in the environment. Once dropped, we may need to set up a particular environment with things like `PATH`, `PYTHONPATH` etc in it. It would be nice if this was done in a standard way (eg `/etc/environment` plus some user-specific dot-file). 17 | 18 | The `ssh-agent` approach has the nice property that `DEPLOYER_USER` accounts don't need access to the ssh keypair directly, and permission to use the keypair is only granted temporarily. Normally these accounts won't have any ssh access. 19 | 20 | We can also unload the keypair from `SSH_AUTH_SOCK` after the `git fetch` operation, so the subsequent post-deploy command (eg `make`) doesn't have access. 21 | 22 | #### Operational Issues 23 | 24 | Privilege separation will also require some refactoring. The `deployer` script will need to be factored into a "top half" and a "bottom half". The top half is running as root. The bottom half runs as `DEPLOYER_USER`. Let's call the bottom half `deployer-update`. It determines what the active and inactive checkouts are, what branch we're using, performs the git fetch, deletes the ssh deploy key from the agent, performs the git checkout, and runs the post-update command (typically just `make`). 25 | 26 | The top half, after initial validation, runs itself under ssh-agent, loads the ssh key, does the chmod + chown on `SSH_AUTH_SOCK`, invokes `deployer-update` as `DEPLOYER_USER`, then updates the active symlink if necessary. 27 | 28 | To avoid other kinds of cross-unit compromises, the `DEPLOYER_DEPLOY_ROOT` directory itself should not be writable by `DEPLOYER_USER`. This prevents unit A from updating the symlink for unit B, and thus replacing it with arbitrary code. 29 | 30 | #### Questions 31 | 32 | - [x] Will ssh-agent let you dump the private key for an identity? 33 | - Nope. Good. 34 | 35 | - [x] After deleting an identity, does ssh-agent provide any way to reload it that doesn't require access to the private key file? 36 | - Nope. Good. 37 | 38 | - [ ] Can `git fetch` be coerced into running arbitrary local commands via `.git/` directory configuration? 39 | - Yes :( 40 | - The `pre-auto-gc` hook could be executed. 41 | - In addition, these git config settings could execute commands: 42 | - `core.gitProxy` 43 | - `core.askpass` 44 | - ...probably a number of others 45 | - Mitigations? 46 | - Hooks: refuse to proceed if any hook scripts are executable? 47 | - Config: check `git config --list` against some kind of whitelist? Tricky. 48 | - Could just say meh, this isn't an issue. So `DEPLOYER_USER` could use the ssh key to get read-only access to the other repositories it already has access to...so what? The root problem is maybe the ssh key grants access to too many things. You could always use a separate ssh deploy key per repository. 49 | - [x] Should we encourage a separate ssh key per repository by making the `-k` option to `deployer-manage` required, although it could perhaps be empty? 50 | - Nah. Some people won't care about this feature, which complicates setups operationally. 51 | - I want to switch us to per-repository deploy keys in github though. That lets us eliminate the deploy bot github user. 52 | 53 | - Would be nice if `ssh-agent` could make an identity single-use. 54 | - It can't though. 55 | 56 | 57 | ### Unprivileged Mode 58 | 59 | When `deployer-queue` isn't running as root, none of the above happens: 60 | 61 | - The `DEPLOYER_USER` setting is ignored. 62 | - We don't attempt to self-exec with dropped privileges. 63 | - We don't attempt to self-exec under ssh-agent. 64 | - We don't attempt to load `DEPLOYER_SSH_KEY` into ssh-agent. 65 | - We don't attempt to chmod the ssh-agent socket. 66 | - We don't attempt to unload `DEPLOYER_SSH_KEY` after the actual update step. 67 | 68 | 69 | ### Interface-Level Changes 70 | 71 | - [x] New option: `deployer-manage -u username` 72 | - Stores `DEPLOYER_USER` setting. Optional, but privileged mode deployments will fail without it. 73 | - [x] New option: `deployer-manage -k path/to/ssh/private/key` 74 | - Stores `DEPLOYER_SSH_KEY` setting. Optional, but git+ssh deployments may fail without it. Pubkey is assumed by ssh-agent to be same path with `.pub` appended. 75 | - [x] Should `deployer-manage -x` store the `DEPLOYER_IN_PLACE` option somewhere? 76 | - Not necessary. There are some settings which are already manifest in the checkouts themselves. No sense in storing them twice. 77 | 78 | -------------------------------------------------------------------------------- /etc/deployer.example.conf: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Set to empty string for system install, absolute path for an alternate install. 4 | # PREFIX="${PREFIX:-$PWD}" 5 | PREFIX="" 6 | 7 | # Full path to the deployer config file. 8 | export DEPLOYER_CONFIG="$PREFIX/etc/deployer.conf" 9 | 10 | # Full path to the directory where settings for each managed unit are stored. 11 | export DEPLOYER_UNITS_DIR="$PREFIX/etc/deployer/units" 12 | 13 | # Full path to the deployer filesystem queue. 14 | export DEPLOYER_QUEUE="$PREFIX/var/queue/deployer" 15 | 16 | # How often to rescan the deployer queue. Empty string means don't rescan. 17 | export DEPLOYER_RESCAN_INTERVAL="" 18 | 19 | # Full path to the deployer service trigger, for trigger-listen / trigger-pull. 20 | export DEPLOYER_TRIGGER="$PREFIX/var/trigger/deployer/trigger.fifo" 21 | 22 | # Full path to the bouncer service trigger, for trigger-listen / trigger-pull. 23 | export DEPLOYER_BOUNCER_TRIGGER="$PREFIX/var/trigger/bouncer/trigger.fifo" 24 | 25 | # Full path to lockfile that protects the queue's /req/* -> /new/ job staging operation. 26 | export DEPLOYER_STAGE_LOCK="$PREFIX/var/locks/deployer/stage.lock" 27 | 28 | # Full path to the directory of deployer per-unit lockfiles. 29 | export DEPLOYER_UNIT_LOCK_DIR="$PREFIX/var/locks/deployer/unit" 30 | 31 | # Full path to the state directory for deployer. 32 | export DEPLOYER_STATE_DIR="$PREFIX/var/lib/deployer" 33 | 34 | # Full path to the per-host deployer queues, if running in a cluster. 35 | export DEPLOYER_HOSTQUEUES_DIR="$DEPLOYER_STATE_DIR/hostqueues" 36 | 37 | # Full path to the directory of bounce files used by deployer-bouncer. 38 | export DEPLOYER_BOUNCED_DIR="$PREFIX/var/lib/bouncer" 39 | 40 | # Full path to the directory where all managed deployments live. 41 | export DEPLOYER_DEPLOY_ROOT="$PREFIX/home/deploys" 42 | 43 | # Full path to the ssh known_hosts file to use (optional). 44 | export DEPLOYER_SSH_KNOWN_HOSTS="" 45 | 46 | # How many different units can be deploying simultaneously. 47 | export DEPLOYER_CONCURRENCY=8 48 | 49 | # When a deployer job fails, because the unit is currently deploying, 50 | # or for other transient reasons, retry the job after this many seconds. 51 | export DEPLOYER_RETRY_DELAY=1 52 | 53 | 54 | # Full path to the deployer-mailer filesystem queue. 55 | # export DEPLOYER_MAILER_QUEUE="$PREFIX/var/queue/deployer-mailer" 56 | export DEPLOYER_MAILER_QUEUE="" 57 | 58 | # Full path to the deployer-mailer service trigger, for trigger-listen / trigger-pull. 59 | # export DEPLOYER_MAILER_TRIGGER="$PREFIX/var/trigger/deployer-mailer/trigger.fifo" 60 | export DEPLOYER_MAILER_TRIGGER="" 61 | 62 | # From: line for deployer job emails. 63 | # export DEPLOYER_MAILER_FROM="Deployer " 64 | export DEPLOYER_MAILER_FROM="" 65 | 66 | # To: line for deployer job emails. 67 | # export DEPLOYER_MAILER_TO="Deploys " 68 | export DEPLOYER_MAILER_TO="" 69 | 70 | -------------------------------------------------------------------------------- /etc/service/bouncer/bounce: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # nothing special to do here 3 | true 4 | -------------------------------------------------------------------------------- /etc/service/bouncer/log/main: -------------------------------------------------------------------------------- 1 | /var/log/bouncer -------------------------------------------------------------------------------- /etc/service/bouncer/log/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | test `id -u` -gt 0 || exec setuidgid -s deployer-log "$0" "$@" 3 | exec multilog t '+*' s1048576 n64 ./main 4 | -------------------------------------------------------------------------------- /etc/service/bouncer/log/supervise: -------------------------------------------------------------------------------- 1 | /var/lib/supervise/bouncer.log -------------------------------------------------------------------------------- /etc/service/bouncer/root: -------------------------------------------------------------------------------- 1 | /var/lib/bouncer -------------------------------------------------------------------------------- /etc/service/bouncer/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -e 3 | exec 2>&1 4 | test `id -u` -gt 0 || exec setuidgid -s bouncer env HOME=`eval echo ~bouncer` "$0" "$@" 5 | cd ./root 6 | export DEPLOYER_CONFIG="/etc/deployer.conf" 7 | . "$DEPLOYER_CONFIG" 8 | exec trigger-listen -v -c 1 "$DEPLOYER_BOUNCER_TRIGGER" deployer-bouncer /service 9 | -------------------------------------------------------------------------------- /etc/service/bouncer/setsid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/etc/service/bouncer/setsid -------------------------------------------------------------------------------- /etc/service/bouncer/start: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | exec enable-svcgroup 3 | -------------------------------------------------------------------------------- /etc/service/bouncer/supervise: -------------------------------------------------------------------------------- 1 | /var/lib/supervise/bouncer -------------------------------------------------------------------------------- /etc/service/bouncer/trigger.fifo: -------------------------------------------------------------------------------- 1 | /var/trigger/bouncer/trigger.fifo -------------------------------------------------------------------------------- /etc/service/deployer-mailer/bounce: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # nothing special to do here 3 | true 4 | -------------------------------------------------------------------------------- /etc/service/deployer-mailer/log/main: -------------------------------------------------------------------------------- 1 | /var/log/deployer-mailer -------------------------------------------------------------------------------- /etc/service/deployer-mailer/log/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | test `id -u` -gt 0 || exec setuidgid deployer-log "$0" "$@" 3 | exec multilog t '+*' s1048576 n64 ./main 4 | -------------------------------------------------------------------------------- /etc/service/deployer-mailer/log/supervise: -------------------------------------------------------------------------------- 1 | /var/lib/supervise/deployer-mailer.log -------------------------------------------------------------------------------- /etc/service/deployer-mailer/root: -------------------------------------------------------------------------------- 1 | /var/lib/deployer-mailer -------------------------------------------------------------------------------- /etc/service/deployer-mailer/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -e 3 | exec 2>&1 4 | test `id -u` -gt 0 || exec setuidgid -s deployer-mailer env HOME=`eval echo ~deployer-mailer` "$0" "$@" 5 | cd ./root 6 | export DEPLOYER_CONFIG="/etc/deployer.conf" 7 | exec deployer-mailer-service 8 | -------------------------------------------------------------------------------- /etc/service/deployer-mailer/setsid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/etc/service/deployer-mailer/setsid -------------------------------------------------------------------------------- /etc/service/deployer-mailer/start: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | exec enable-svcgroup 3 | -------------------------------------------------------------------------------- /etc/service/deployer-mailer/supervise: -------------------------------------------------------------------------------- 1 | /var/lib/supervise/deployer-mailer -------------------------------------------------------------------------------- /etc/service/deployer-mailer/trigger.fifo: -------------------------------------------------------------------------------- 1 | /var/trigger/deployer-mailer/trigger.fifo -------------------------------------------------------------------------------- /etc/service/deployer/bounce: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # nothing special to do here 3 | true 4 | -------------------------------------------------------------------------------- /etc/service/deployer/log/main: -------------------------------------------------------------------------------- 1 | /var/log/deployer -------------------------------------------------------------------------------- /etc/service/deployer/log/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | test `id -u` -gt 0 || exec setuidgid -s deployer-log "$0" "$@" 3 | exec multilog t '+*' s1048576 n64 ./main 4 | -------------------------------------------------------------------------------- /etc/service/deployer/log/supervise: -------------------------------------------------------------------------------- 1 | /var/lib/supervise/deployer.log -------------------------------------------------------------------------------- /etc/service/deployer/root: -------------------------------------------------------------------------------- 1 | /var/lib/deployer -------------------------------------------------------------------------------- /etc/service/deployer/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -e 3 | exec 2>&1 4 | umask 0002 5 | cd ./root 6 | export DEPLOYER_CONFIG="/etc/deployer.conf" 7 | exec deployer-service 8 | -------------------------------------------------------------------------------- /etc/service/deployer/setsid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/etc/service/deployer/setsid -------------------------------------------------------------------------------- /etc/service/deployer/start: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | exec enable-svcgroup 3 | -------------------------------------------------------------------------------- /etc/service/deployer/supervise: -------------------------------------------------------------------------------- 1 | /var/lib/supervise/deployer -------------------------------------------------------------------------------- /etc/service/deployer/trigger.fifo: -------------------------------------------------------------------------------- 1 | /var/trigger/deployer/trigger.fifo -------------------------------------------------------------------------------- /test/Makefile: -------------------------------------------------------------------------------- 1 | SHELL = bash -o pipefail -c 2 | PATH := $(CURDIR)/../bin:$(CURDIR)/bin:$(PATH) 3 | export PATH 4 | 5 | TEST_CASES = $(shell find case -type f | sort) 6 | TEST_OUTPUTS = ${TEST_CASES:case/%=out/%} 7 | 8 | 9 | default : tests 10 | 11 | tests : clean $(TEST_OUTPUTS) 12 | 13 | clean : 14 | @ rm -rf out/* work/* pass/* fail/* 15 | 16 | out/% : case/% 17 | @ printf "[test] %-40s ... " $* 18 | @ if env - PATH="$$PATH" $^ "$(CURDIR)"/work/$* >$@ 2>&1; then \ 19 | printf "pass\n" ; \ 20 | touch pass/$* ; \ 21 | else \ 22 | rc=$$? ; \ 23 | printf "fail\n" ; \ 24 | touch fail/$* ; \ 25 | exit $$rc ; \ 26 | fi 27 | 28 | report : 29 | @ printf "[test results] %d/%d passing\n" \ 30 | `find pass -type f -not -name \*.empty | wc -l` \ 31 | `find case -type f | wc -l` 32 | 33 | -------------------------------------------------------------------------------- /test/bin/deployer-setup-tests: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | export DEPLOYER_CONFIG=/dev/null 6 | export DEPLOYER_QUEUE="$workdir"/queue 7 | export DEPLOYER_TRIGGER="$workdir"/trigger.fifo 8 | export DEPLOYER_UNITS_DIR="$workdir"/units 9 | export DEPLOYER_STAGE_LOCK="$workdir"/lock/stage.lock 10 | export DEPLOYER_UNIT_LOCK_DIR="$workdir"/lock/unit 11 | export DEPLOYER_BOUNCED_DIR="$workdir"/bounced 12 | export DEPLOYER_DEPLOY_ROOT="${DEPLOYER_DEPLOY_ROOT:-$workdir/deploys}" 13 | export DEPLOYER_CONCURRENCY=1 14 | export DEPLOYER_RETRY_DELAY=0 15 | export UNIT="${UNIT:-foo}" 16 | export DEPLOYER_BASENAME="${DEPLOYER_BASENAME:-$UNIT}" 17 | 18 | # In privileged mode (running as root), extra settings are required. 19 | 20 | if [ `id -u` -eq 0 ]; then 21 | export DEPLOYER_USER=root 22 | export DEPLOYER_GROUP=root 23 | fi 24 | 25 | # Deployer setup. 26 | 27 | safe rm -rf "$workdir" 28 | safe mkdir -p "$workdir" 29 | safe deployer-init 30 | 31 | # Setup: create a new git repository. 32 | 33 | safe mkdir -p "$workdir/$UNIT" 34 | ( 35 | safe cd "$workdir/$UNIT" 36 | safe git init . 37 | safe git config user.email "deployer@localhost" 38 | safe git config user.name "Deployer" 39 | safe echo "all : " > Makefile 40 | safe git add Makefile 41 | safe touch 1.txt 42 | safe git add 1.txt 43 | safe git commit -m 1 44 | ) 45 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 46 | 47 | # Setup: create the initial deployer a & b checkouts. 48 | 49 | safe deployer-manage "$UNIT" "$workdir/$UNIT" 50 | 51 | -------------------------------------------------------------------------------- /test/case/0001.initial-deployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe touch 2.txt 14 | safe git add 2.txt 15 | safe git commit -m 2 16 | ) 17 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 18 | 19 | # Test: request a new deployment (inject a job file into the queue). 20 | 21 | safe deploy foo master 22 | 23 | # Test: run a deployment queue pass, which should process all jobs. 24 | 25 | ( 26 | safe cd "$DEPLOYER_QUEUE" 27 | safe deployer-queue 28 | ) 29 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 30 | 31 | # Test: check that the initial deployment worked and got the latest commit. 32 | 33 | ( 34 | safe cd "$DEPLOYER_DEPLOY_ROOT" 35 | safe test -L foo 36 | safe test "`readlink foo`" = "foo.b" 37 | safe test -f foo/2.txt 38 | ) 39 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 40 | 41 | exit 0 42 | 43 | -------------------------------------------------------------------------------- /test/case/0002.subsequent-deployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a 2nd commit to the source git repository. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe touch 2.txt 14 | safe git add 2.txt 15 | safe git commit -m 2 16 | ) 17 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 18 | 19 | # Test: request a new deployment (inject a job file into the queue). 20 | 21 | safe deploy foo master 22 | 23 | # Test: run a deployment queue pass, which should process all jobs. 24 | 25 | ( 26 | safe cd "$DEPLOYER_QUEUE" 27 | safe deployer-queue 28 | ) 29 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 30 | 31 | # Test: make a 3rd commit to the source git repository. 32 | 33 | ( 34 | safe cd "$workdir/foo" 35 | safe touch 3.txt 36 | safe git add 3.txt 37 | safe git commit -m 3 38 | ) 39 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 40 | 41 | # Test: request a new deployment (inject a job file into the queue). 42 | 43 | safe deploy foo master 44 | 45 | # Test: run a deployment queue pass, which should process all jobs. 46 | 47 | ( 48 | safe cd "$DEPLOYER_QUEUE" 49 | safe deployer-queue 50 | ) 51 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 52 | 53 | # Test: check that the second deployment worked and got the latest commit. 54 | 55 | ( 56 | safe cd "$DEPLOYER_DEPLOY_ROOT" 57 | safe test -L foo 58 | safe test "`readlink foo`" = "foo.a" 59 | safe test -f foo/3.txt 60 | ) 61 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 62 | 63 | exit 0 64 | 65 | -------------------------------------------------------------------------------- /test/case/0003.noop-deployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe touch 2.txt 14 | safe git add 2.txt 15 | safe git commit -m 2 16 | ) 17 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 18 | 19 | # Test: request a new deployment (inject a job file into the queue). 20 | 21 | safe deploy foo master 22 | 23 | # Test: run a deployment queue pass, which should process all jobs. 24 | 25 | ( 26 | safe cd "$DEPLOYER_QUEUE" 27 | safe deployer-queue 28 | ) 29 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 30 | 31 | # Test: request a new deployment (inject a job file into the queue). 32 | 33 | safe deploy foo master 34 | 35 | # Test: run a deployment queue pass, which should process all jobs. 36 | 37 | ( 38 | safe cd "$DEPLOYER_QUEUE" 39 | safe deployer-queue 40 | ) 41 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 42 | 43 | # Test: check that the 1st deployment did the correct thing, 44 | # and the 2nd deployment did nothing. 45 | 46 | ( 47 | safe cd "$DEPLOYER_DEPLOY_ROOT" 48 | safe test -L foo 49 | safe test "`readlink foo`" = "foo.b" 50 | safe test -f foo/2.txt 51 | ) 52 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 53 | 54 | exit 0 55 | 56 | -------------------------------------------------------------------------------- /test/case/0004.in-place-deployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | export DEPLOYER_IN_PLACE=1 8 | 9 | safe . deployer-setup-tests 10 | 11 | # Test: check that the in-place deployment setup worked. 12 | 13 | ( 14 | safe cd "$DEPLOYER_DEPLOY_ROOT" 15 | safe test -d foo 16 | ) 17 | 18 | # Test: make a new commit to the source git repository. 19 | 20 | ( 21 | safe cd "$workdir/foo" 22 | safe touch 2.txt 23 | safe git add 2.txt 24 | safe git commit -m 2 25 | ) 26 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 27 | 28 | # Test: request a new deployment (inject a job file into the queue). 29 | 30 | safe deploy foo master 31 | 32 | # Test: run a deployment queue pass, which should process all jobs. 33 | 34 | ( 35 | safe cd "$DEPLOYER_QUEUE" 36 | safe deployer-queue 37 | ) 38 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 39 | 40 | # Test: check that the initial deployment worked, 41 | # got the latest commit, but still points to the same checkout. 42 | 43 | ( 44 | safe cd "$DEPLOYER_DEPLOY_ROOT" 45 | safe test -d foo 46 | safe test -f foo/2.txt 47 | ) 48 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 49 | 50 | exit 0 51 | 52 | -------------------------------------------------------------------------------- /test/case/0005.different-branch-deployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository on a new branch. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe git checkout -b branch1 14 | safe touch 2.txt 15 | safe git add 2.txt 16 | safe git commit -m 2 17 | ) 18 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 19 | 20 | # Test: request a new deployment (inject a job file into the queue). 21 | 22 | safe deploy foo branch1 23 | 24 | # Test: run a deployment queue pass, which should process all jobs. 25 | 26 | ( 27 | safe cd "$DEPLOYER_QUEUE" 28 | safe deployer-queue 29 | ) 30 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 31 | 32 | # Test: check that the deployment did the correct thing, 33 | 34 | ( 35 | safe cd "$DEPLOYER_DEPLOY_ROOT" 36 | safe test -L foo 37 | safe test "`readlink foo`" = "foo.b" 38 | safe test -f foo/2.txt 39 | ) 40 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 41 | 42 | exit 0 43 | 44 | -------------------------------------------------------------------------------- /test/case/0006.default-branch-deployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository on a new branch. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe git checkout -b branch1 14 | safe touch 2.txt 15 | safe git add 2.txt 16 | safe git commit -m 2 17 | ) 18 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 19 | 20 | # Test: request a new deployment on the new branch (inject a job file into the queue). 21 | 22 | safe deploy foo branch1 23 | 24 | # Test: run a deployment queue pass, which should process all jobs. 25 | 26 | ( 27 | safe cd "$DEPLOYER_QUEUE" 28 | safe deployer-queue 29 | ) 30 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 31 | 32 | # Test: make a new commit to the source git repository on the new branch. 33 | 34 | ( 35 | safe cd "$workdir/foo" 36 | safe git checkout branch1 37 | safe touch 3.txt 38 | safe git add 3.txt 39 | safe git commit -m 3 40 | ) 41 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 42 | 43 | # Test: request a new deployment on the default branch (inject a job file into the queue). 44 | 45 | safe deploy foo 46 | 47 | # Test: run a deployment queue pass, which should process all jobs. 48 | 49 | ( 50 | safe cd "$DEPLOYER_QUEUE" 51 | safe deployer-queue 52 | ) 53 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 54 | 55 | # Test: check that the 1st and 2nd deployments did the correct thing. 56 | 57 | ( 58 | safe cd "$DEPLOYER_DEPLOY_ROOT" 59 | safe test -L foo 60 | safe test "`readlink foo`" = "foo.a" 61 | safe test -f foo/3.txt 62 | ) 63 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 64 | 65 | exit 0 66 | 67 | -------------------------------------------------------------------------------- /test/case/0007.removed-commit: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe touch 2.txt 14 | safe git add 2.txt 15 | safe git commit -m 2 16 | ) 17 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 18 | 19 | # Test: request a new deployment (inject a job file into the queue). 20 | 21 | safe deploy foo master 22 | 23 | # Test: run a deployment queue pass, which should process all jobs. 24 | 25 | ( 26 | safe cd "$DEPLOYER_QUEUE" 27 | safe deployer-queue 28 | ) 29 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 30 | 31 | # Test: forcibly remove a commit from the source git repository. 32 | 33 | ( 34 | safe cd "$workdir/foo" 35 | git reset --hard HEAD^ 36 | git -c gc.reflogExpire=0 \ 37 | -c gc.reflogExpireUnreachable=0 \ 38 | -c gc.rerereresolved=0 \ 39 | -c gc.rerereunresolved=0 \ 40 | -c gc.pruneExpire=now gc 41 | ) 42 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 43 | 44 | # Test: request a new deployment (inject a job file into the queue). 45 | 46 | safe deploy foo master 47 | 48 | # Test: run a deployment queue pass, which should process all jobs. 49 | 50 | ( 51 | safe cd "$DEPLOYER_QUEUE" 52 | safe deployer-queue 53 | ) 54 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 55 | 56 | # Test: check that the 1st deployment did the correct thing, 57 | # and the 2nd deployment rolled it back. 58 | 59 | ( 60 | safe cd "$DEPLOYER_DEPLOY_ROOT" 61 | safe test -L foo 62 | safe test "`readlink foo`" = "foo.a" 63 | safe test -f foo/1.txt 64 | safe test ! -f foo/2.txt 65 | ) 66 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 67 | 68 | exit 0 69 | 70 | -------------------------------------------------------------------------------- /test/case/0008.symbolic-switch: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: create a feature branch off master. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe git checkout -b feature 14 | safe touch 2.txt 15 | safe git add 2.txt 16 | safe git commit -m 2 17 | ) 18 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 19 | 20 | # Test: request a new deployment (inject a job file into the queue). 21 | 22 | safe deploy foo feature 23 | 24 | # Test: run a deployment queue pass, which should process all jobs. 25 | 26 | ( 27 | safe cd "$DEPLOYER_QUEUE" 28 | safe deployer-queue 29 | ) 30 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 31 | 32 | # Test: check that the 2nd deployment switched to the feature branch. 33 | 34 | ( 35 | safe cd "$DEPLOYER_DEPLOY_ROOT" 36 | safe test -L foo 37 | safe test "`readlink foo`" = "foo.b" 38 | safe test -f foo/1.txt 39 | safe test -f foo/2.txt 40 | ) 41 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 42 | 43 | # Test: merge the feature branch into master. 44 | 45 | ( 46 | safe cd "$workdir/foo" 47 | safe git checkout master 48 | safe git merge --ff-only feature 49 | ) 50 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 51 | 52 | # Test: request a new deployment which switches back to master. 53 | 54 | safe deploy foo master 55 | 56 | # Test: run a deployment queue pass, which should process all jobs. 57 | 58 | ( 59 | safe cd "$DEPLOYER_QUEUE" 60 | safe deployer-queue 61 | ) 62 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 63 | 64 | # Test: check that the 3rd deployment switched back to the master branch. 65 | 66 | ( 67 | safe cd "$DEPLOYER_DEPLOY_ROOT" 68 | safe test -L foo 69 | safe test "`readlink foo`" = "foo.a" 70 | safe test -f foo/1.txt 71 | safe test -f foo/2.txt 72 | safe cd foo 73 | safe test "`git symbolic-ref HEAD`" = "refs/heads/master" 74 | ) 75 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 76 | 77 | exit 0 78 | 79 | -------------------------------------------------------------------------------- /test/case/0009.initial-post-deployment-fail: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | export DEPLOYER_COMMAND=false 8 | 9 | ( 10 | safe . deployer-setup-tests 11 | ) 12 | rc="$?" 13 | safe test "$rc" -gt 0 14 | 15 | exit 0 16 | 17 | -------------------------------------------------------------------------------- /test/case/0010.subsequent-post-deployment-fail: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: commit a post-deploy breakage to the source git repository. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe printf "all :\n\tfalse\n" > Makefile 14 | safe git add -u Makefile 15 | safe touch 2.txt 16 | safe git add 2.txt 17 | safe git commit -m 2 18 | ) 19 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 20 | 21 | # Test: request a new deployment (inject a job file into the queue). 22 | 23 | safe deploy foo master 24 | 25 | # Test: run a deployment queue pass, which should process all jobs. 26 | # Note: don't check exit status of deployer-queue, it's likely a failure 27 | # because the one job failed, but we don't want to insist on that behavior. 28 | 29 | ( 30 | safe cd "$DEPLOYER_QUEUE" 31 | safe deployer-queue 32 | ) 33 | 34 | # Test: check that the initial deployment failed because the post-deploy failed. 35 | 36 | ( 37 | safe cd "$DEPLOYER_DEPLOY_ROOT" 38 | safe test -L foo 39 | safe test "`readlink foo`" = "foo.a" 40 | safe test ! -f foo/2.txt 41 | ) 42 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 43 | 44 | exit 0 45 | 46 | -------------------------------------------------------------------------------- /test/case/0011.allow-unit-with-period: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | export UNIT="foo.bar" 8 | 9 | safe . deployer-setup-tests 10 | 11 | # Test: make a new commit to the source git repository. 12 | 13 | ( 14 | safe cd "$workdir/$UNIT" 15 | safe touch 2.txt 16 | safe git add 2.txt 17 | safe git commit -m 2 18 | ) 19 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 20 | 21 | # Test: request a new deployment (inject a job file into the queue). 22 | 23 | safe deploy "$UNIT" master 24 | 25 | # Test: run a deployment queue pass, which should process all jobs. 26 | 27 | ( 28 | safe cd "$DEPLOYER_QUEUE" 29 | safe deployer-queue 30 | ) 31 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 32 | 33 | # Test: check that the initial deployment worked and got the latest commit. 34 | 35 | ( 36 | safe cd "$DEPLOYER_DEPLOY_ROOT" 37 | safe test -L "$UNIT" 38 | safe test "`readlink $UNIT`" = "${UNIT}.b" 39 | safe test -f $UNIT/2.txt 40 | ) 41 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 42 | 43 | exit 0 44 | 45 | -------------------------------------------------------------------------------- /test/case/0012.reject-unit-with-slash: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | export DEPLOYER_CONFIG=/dev/null 8 | export DEPLOYER_QUEUE="$workdir"/queue 9 | export DEPLOYER_TRIGGER="$workdir"/trigger.fifo 10 | export DEPLOYER_UNITS_DIR="$workdir"/units 11 | export DEPLOYER_STAGE_LOCK="$workdir"/lock/stage.lock 12 | export DEPLOYER_UNIT_LOCK_DIR="$workdir"/lock/unit 13 | export DEPLOYER_BOUNCED_DIR="$workdir"/bounced 14 | export DEPLOYER_DEPLOY_ROOT="$workdir"/deploys 15 | export DEPLOYER_CONCURRENCY=1 16 | export DEPLOYER_RETRY_DELAY=0 17 | 18 | # In privileged mode (running as root), extra settings are required. 19 | 20 | if [ `id -u` -eq 0 ]; then 21 | export DEPLOYER_USER=root 22 | export DEPLOYER_GROUP=root 23 | fi 24 | 25 | # Deployer setup. 26 | 27 | safe rm -rf "$workdir" 28 | safe mkdir -p "$workdir" 29 | safe deployer-init 30 | 31 | # Setup: create a new git repository. 32 | 33 | safe mkdir -p "$workdir/foobar" 34 | ( 35 | safe cd "$workdir/foobar" 36 | safe git init . 37 | safe git config user.email "deployer@localhost" 38 | safe git config user.name "Deployer" 39 | safe echo "all : " > Makefile 40 | safe git add Makefile 41 | safe touch 1.txt 42 | safe git add 1.txt 43 | safe git commit -m 1 44 | ) 45 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 46 | 47 | # Setup: create the initial deployer a & b checkouts. 48 | 49 | deployer-manage "foo/bar" "$workdir/foobar" && barf "deployer-manage: allowed slash in unit" 50 | deploy "foo/bar" && barf "deploy: allowed slash in unit" 51 | 52 | exit 0 53 | 54 | -------------------------------------------------------------------------------- /test/case/0013.allow-revision-with-slash: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository on a new branch. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe git checkout -b feature/one 14 | safe touch 2.txt 15 | safe git add 2.txt 16 | safe git commit -m 2 17 | ) 18 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 19 | 20 | # Test: request a new deployment (inject a job file into the queue). 21 | 22 | safe deploy foo feature/one 23 | 24 | # Test: run a deployment queue pass, which should process all jobs. 25 | 26 | ( 27 | safe cd "$DEPLOYER_QUEUE" 28 | safe deployer-queue 29 | ) 30 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 31 | 32 | # Test: check that the deployment did the correct thing, 33 | 34 | ( 35 | safe cd "$DEPLOYER_DEPLOY_ROOT" 36 | safe test -L foo 37 | safe test "`readlink foo`" = "foo.b" 38 | safe test -f foo/2.txt 39 | ) 40 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 41 | 42 | exit 0 43 | 44 | -------------------------------------------------------------------------------- /test/case/0014.alternate-basename: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | export DEPLOYER_IN_PLACE=1 8 | export DEPLOYER_BASENAME=bar 9 | 10 | safe . deployer-setup-tests 11 | 12 | # Test: check that the in-place deployment setup worked. 13 | 14 | ( 15 | safe cd "$DEPLOYER_DEPLOY_ROOT" 16 | safe test -d bar 17 | ) 18 | 19 | # Test: make a new commit to the source git repository. 20 | 21 | ( 22 | safe cd "$workdir/foo" 23 | safe touch 2.txt 24 | safe git add 2.txt 25 | safe git commit -m 2 26 | ) 27 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 28 | 29 | # Test: request a new deployment (inject a job file into the queue). 30 | 31 | safe deploy foo master 32 | 33 | # Test: run a deployment queue pass, which should process all jobs. 34 | 35 | ( 36 | safe cd "$DEPLOYER_QUEUE" 37 | safe deployer-queue 38 | ) 39 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 40 | 41 | # Test: check that the initial deployment worked, 42 | # got the latest commit, but still points to the same checkout. 43 | 44 | ( 45 | safe cd "$DEPLOYER_DEPLOY_ROOT" 46 | safe test -d bar 47 | safe test -f bar/2.txt 48 | ) 49 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 50 | 51 | exit 0 52 | 53 | -------------------------------------------------------------------------------- /test/case/0015.alternate-deploy-root: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | export DEPLOYER_IN_PLACE=1 8 | export DEPLOYER_DEPLOY_ROOT="$workdir/custom-deploys" 9 | 10 | safe . deployer-setup-tests 11 | 12 | # Test: check that the in-place deployment setup worked. 13 | 14 | ( 15 | safe cd "$DEPLOYER_DEPLOY_ROOT" 16 | safe test -d foo 17 | ) 18 | 19 | # Test: make a new commit to the source git repository. 20 | 21 | ( 22 | safe cd "$workdir/foo" 23 | safe touch 2.txt 24 | safe git add 2.txt 25 | safe git commit -m 2 26 | ) 27 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 28 | 29 | # Test: request a new deployment (inject a job file into the queue). 30 | 31 | safe deploy foo master 32 | 33 | # Test: run a deployment queue pass, which should process all jobs. 34 | 35 | ( 36 | safe cd "$DEPLOYER_QUEUE" 37 | safe deployer-queue 38 | ) 39 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 40 | 41 | # Test: check that the initial deployment worked, 42 | # got the latest commit, but still points to the same checkout. 43 | 44 | ( 45 | safe cd "$DEPLOYER_DEPLOY_ROOT" 46 | safe test -d foo 47 | safe test -f foo/2.txt 48 | ) 49 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 50 | 51 | exit 0 52 | 53 | -------------------------------------------------------------------------------- /test/case/0016.force-redeployment: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | . shellsafe || { echo "error sourcing shellsafe." 1>&2 ; exit 100 ; } 4 | 5 | workdir="$1" ; shift ; export workdir 6 | 7 | safe . deployer-setup-tests 8 | 9 | # Test: make a new commit to the source git repository on a new branch. 10 | 11 | ( 12 | safe cd "$workdir/foo" 13 | safe git checkout -b branch1 14 | safe touch 2.txt 15 | safe git add 2.txt 16 | safe git commit -m 2 17 | ) 18 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 19 | 20 | head_revision=`( safe cd "$workdir/foo" && safe git rev-parse HEAD )` 21 | safe test -n "$head_revision" 22 | 23 | # Test: request a new deployment on the new branch (inject a job file into the queue). 24 | 25 | safe deploy foo branch1 26 | 27 | # Test: run a deployment queue pass, which should process all jobs. 28 | 29 | ( 30 | safe cd "$DEPLOYER_QUEUE" 31 | safe deployer-queue 32 | ) 33 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 34 | 35 | # Test: request a redeployment on the default branch (inject a job file into the queue). 36 | 37 | safe deploy foo @ 38 | 39 | # Test: run a deployment queue pass, which should process all jobs. 40 | 41 | ( 42 | safe cd "$DEPLOYER_QUEUE" 43 | safe deployer-queue 44 | ) 45 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 46 | 47 | # Test: check that redeployment ran even though nothing had changed. 48 | # Also verify that branch and revision haven't changed. 49 | 50 | ( 51 | safe cd "$DEPLOYER_DEPLOY_ROOT" 52 | safe test -L foo 53 | safe test "`readlink foo`" = "foo.a" 54 | safe cd foo 55 | safe test "`git rev-parse HEAD`" = "$head_revision" 56 | safe test "`git rev-parse --abbrev-ref HEAD`" = "branch1" 57 | ) 58 | rc="$?" ; [ "$rc" -ne 0 ] && exit "$rc" 59 | 60 | exit 0 61 | 62 | -------------------------------------------------------------------------------- /test/fail/.empty: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/test/fail/.empty -------------------------------------------------------------------------------- /test/out/.empty: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/test/out/.empty -------------------------------------------------------------------------------- /test/pass/.empty: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/test/pass/.empty -------------------------------------------------------------------------------- /test/work/.empty: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/endcrawl/deployer/15bcc983bcd2b5cee496b54671f56301d82446c4/test/work/.empty --------------------------------------------------------------------------------