├── .gitignore ├── LICENSE ├── README.md ├── VERSION └── bin ├── mergeq ├── mergeq_ci ├── mergeq_install └── mergeq_remote_install /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | *.rbc 3 | .bundle 4 | .config 5 | .yardoc 6 | Gemfile.lock 7 | InstalledFiles 8 | _yardoc 9 | coverage 10 | doc/ 11 | lib/bundler/man 12 | pkg 13 | rdoc 14 | spec/reports 15 | test/tmp 16 | test/version_tmp 17 | tmp 18 | .mergeq 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2012 Aaron Jensen 2 | 3 | MIT License 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining 6 | a copy of this software and associated documentation files (the 7 | "Software"), to deal in the Software without restriction, including 8 | without limitation the rights to use, copy, modify, merge, publish, 9 | distribute, sublicense, and/or sell copies of the Software, and to 10 | permit persons to whom the Software is furnished to do so, subject to 11 | the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be 14 | included in all copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 19 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 20 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 22 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mergeq 2 | 3 | ![mergeq explanatory comic](http://i.imgur.com/2iDn1qu.png) 4 | 5 | Have you ever broken the build? Have you ever had to wait for a teammate to fix 6 | the build so that you can deploy your change that won't break the build? 7 | 8 | mergeq can help by only allowing branches that pass CI to get merged into 9 | `master` or `develop` or whatever. If your build doesn't pass, the branch you 10 | are trying to merge into doesn't change and no one else is affected. 11 | 12 | mergeq is an implementation of the "pre-tested commit" pattern that is: 13 | 14 | * **Robust**--two people can merge at the same time and, as long as their branches do not conflict with one another, they both get a chance to get merged in safely. 15 | * **Continuous Integration server independent**--as long as your server has permission to push to your repo and can support running only one build at a time, it should work. 16 | * **Flexible**--there are [hooks](#hooking-mergeq) to support safety checks or post build notifications. 17 | * **Easy to setup**--no additional intermediary repository, no additional infratructure 18 | * **Battle tested**--we have been using it for over 3 years with great success. 19 | 20 | If you want strict **pre-reviewed**, as well as pre-tested commits, and you don't mind additional infrastructure, you can check out [Gerrit](https://www.gerritcodereview.com/). 21 | 22 | ## How it works 23 | 24 | Say you have a branch, `feature` that you want to merge into `master`. Instead 25 | of merging directly to `master`, you run `mergeq master`. `mergeq` will fetch 26 | the latest `master` from `origin` and merge your branch into it. You can 27 | resolve any merge conflicts and then `mergeq` will push the merged branch to a 28 | special branch called `merge/master`. Your CI server will pick up changes from 29 | that branch, run the build, and if it passes, push the merge to `master`. If it 30 | fails, it will do nothing else and your failing build won't get in anyone's 31 | way. Merge away! 32 | 33 | ## Remote Installation (quick) 34 | 35 | $ cd your_project 36 | $ bash <(curl -s https://raw.githubusercontent.com/aaronjensen/mergeq/master/bin/mergeq_remote_install) 37 | 38 | You can always re-run this script to upgrade your project's copy of mergeq. 39 | 40 | If you don't trust `curl`, which is totally understandable, just do this: 41 | 42 | $ curl -o mergeq_remote_install https://raw.githubusercontent.com/aaronjensen/mergeq/master/bin/mergeq_remote_install 43 | $ chmod +x mergeq_remote_install 44 | 45 | # open the install script and audit it for security 46 | 47 | $ ./mergeq_remote_install 48 | 49 | Running `mergeq_remote_install` will add a few files, so be sure to commit them to your repo: 50 | 51 | $ git status 52 | $ git add . 53 | $ git commit -m "Add mergeq to the project" 54 | 55 | Next, create branches that you want to make queueable. This example creates a branch 56 | called `staging` for queuing builds. 57 | 58 | $ git push origin master:staging 59 | $ git push origin master:merge/staging 60 | 61 | ## Configuring CI 62 | 63 | There are a few things to take into account when using mergeq on your CI server. 64 | 65 | * Only one build per branch can run at once. With TeamCity, you can "Limit the number of simultaneously running builds" to 1. 66 | * You need to run a build for every push rather than just the most recent. With TeamCity, we do this by disabling VCS build triggering and starting the build automatically via a webhook like this: 67 | 68 | ```bash 69 | curl --insecure "https://user:pass@teamcity.server.com/httpAuth/action.html?add2Queue=$build_id&name=GIT_REF&value=$git_ref" 70 | ``` 71 | 72 | You'll need to add two steps to your CI for mergeq. 73 | 74 | 1. First step will merge before testing. 75 | 76 | `%GIT_REF%` is the sha of the queue commit to merge. This will be the tip of the `merge/staging` branch and look like: 77 | 78 | ``` 79 | cf629af - Queuing merge: feature/mergeq-check-acceptance into integration (8 weeks ago) 80 | ``` 81 | 82 | `%BRANCH%` is the name of the target branch. If the queue branch is `merge/staging`, the target branch is `staging`. 83 | 84 | Your CI script should look something like: 85 | 86 | ```bash 87 | $ git reset --hard %GIT_REF% 88 | $ bin/mergeq_ci merge %BRANCH% 89 | ``` 90 | 91 | 2. Then your CI should run build/tests. 92 | 3. If successful, it should: 93 | 94 | ```bash 95 | $ bin/mergeq_ci push %BRANCH% 96 | ``` 97 | 98 | ## Hooking mergeq 99 | 100 | You'll probably want to hook parts of mergeq as part of your build process. Examples: 101 | 102 | * pausing zeus for the duration of mergeq to avoid churn 103 | * cleaning up branch metadata after a CI build 104 | 105 | All the hooks you write live in `$project_dir/.mergeq/hooks`, and therefore are on 106 | a per-repo basis. 107 | 108 | *`mergeq` hooks:* 109 | 110 | * `before_merge` - this runs before a merge happens 111 | * `after_push` - this runs after your branch is pushed to `merge/$target` 112 | * `after_cleanup` - this is the last thing that runs before `mergeq` exits 113 | 114 | *`mergeq_ci` hooks:* 115 | 116 | * `before_ci_startup` - this runs at the very beginning of the `mergeq_ci` script 117 | * `after_ci_merge` - this runs after CI merges into the target branch 118 | * `before_ci_push` - this runs before CI pushes the target branch back to origin 119 | * `after_ci_push` - this runs after CI pushes the target branch back to origin 120 | 121 | All hooks live in `$project_dir/.mergeq/hooks/$hook_name`, and need to have `chmod +x` 122 | to be executable. 123 | 124 | ## Example `mergeq` hook 125 | 126 | Arguments: 127 | 128 | * $1 - the target branch (`integration`, `master`, etc) 129 | * $2 - the name of the merge branch (`merge/integration`, `merge/master`, etc) 130 | 131 | Rails projects often run something like Zeus to handle fast code loading. Since `mergeq` 132 | does a bunch of git checkouts, we want to pause Zeus for the duration of a `mergeq` so 133 | it doesn't freak out. 134 | 135 | ```bash 136 | # .mergeq/hooks/before_merge 137 | 138 | #!/bin/bash 139 | 140 | target_branch=$1 141 | merge_branch=$2 142 | 143 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 144 | 145 | function zeus_pid { 146 | cat $DIR/../../tmp/zeus.pid 2> /dev/null 147 | } 148 | 149 | function stop_zeus { 150 | pid=$(zeus_pid) 151 | if [[ "$pid" ]]; then 152 | kill -USR1 $pid 153 | fi 154 | } 155 | 156 | stop_zeus 157 | ``` 158 | 159 | ```bash 160 | # .mergeq/hooks/after_cleanup 161 | 162 | #!/bin/bash 163 | 164 | target_branch=$1 165 | merge_branch=$2 166 | 167 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 168 | 169 | function zeus_pid { 170 | cat $DIR/../../tmp/zeus.pid 2> /dev/null 171 | } 172 | 173 | function start_zeus { 174 | pid=$(zeus_pid) 175 | if [[ "$pid" ]]; then 176 | kill -USR2 $pid 177 | fi 178 | } 179 | ``` 180 | 181 | ## Example `mergeq_ci` hook 182 | 183 | Arguments: 184 | 185 | * $1 - the target branch (`integration`, `master`, etc) 186 | 187 | Say we want to delete our feature branch from GitHub after a successful merge+push to 188 | `origin/master`. We can hook `after_ci_push` to achieve this. 189 | 190 | ```bash 191 | # .mergeq/hooks/after_ci_push 192 | 193 | #!/bin/bash 194 | 195 | target_branch=$1 196 | 197 | function delete_feature_branch { 198 | if [ "$target_branch" = "master" ] 199 | then 200 | git ls-remote --heads origin | grep `git rev-parse HEAD^2` | cut -f2 -s | xargs -I {} git push origin :{}; true 201 | fi 202 | } 203 | 204 | delete_feature_branch 205 | ``` 206 | 207 | ## Contributing 208 | 209 | 1. Fork it 210 | 2. Create your feature branch (`git checkout -b my-new-feature`) 211 | 3. Commit your changes (`git commit -am 'Added some feature'`) 212 | 4. Push to the branch (`git push origin my-new-feature`) 213 | 5. Create new Pull Request 214 | -------------------------------------------------------------------------------- /VERSION: -------------------------------------------------------------------------------- 1 | 0.1.1 2 | -------------------------------------------------------------------------------- /bin/mergeq: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # script/mergeq 4 | # 5 | # This starts a "merge" build on your CI server. Rather than allowing untested 6 | # merges, we first run tests on the merge and then push the merge to the 7 | # respective branch. It does this by making use of a queue branch (not 8 | # a real term, don't bother googling it) which is basically a branch 9 | # for which the HEAD and each HEAD^ is a "Queueing merge" commit whose 10 | # HEAD^2 is the the actual merge we will be testing. 11 | # 12 | # For example, this is merge/integration: 13 | # 14 | # * 25bc0e2 - Queuing merge: feature/warning-when-inviting-group-members-not-in-org into integration (5 days ago) 15 | # |\ 16 | # | * 9a6ffe1 - Merge feature/warning-when-inviting-group-members-not-in-org into integration (5 days ago) 17 | # | |\ 18 | # | | * edee6e3 - set html of target node vs text in remote_form_msg (5 days ago) 19 | # | | * 14c44bd - Remove Warning: prefix to warning localeapp copy (6 days ago) 20 | # * | | bc31325 - Queuing merge: feature/hide-activity-feed-from-anonymous-users into integration (6 days ago) 21 | # |\ \ \ 22 | # | * \ \ c3fce08 - Merge feature/hide-activity-feed-from-anonymous-users into integration (6 days ago) 23 | # 24 | # In the above example, the build agent will: 25 | # * take 9a6ffe1 and attempt to merge that to integration 26 | # * run tests 27 | # * if the tests pass, it will push the merge. 28 | # 29 | # Pushing the merge is handled by the mergeq_ci executable. 30 | # 31 | # The script will have the user do the merge locally so that they can resolve 32 | # any merge conflicts. The user must do script/mergeq --continue after resolving 33 | # conflicts and committing the merge. 34 | # 35 | # If new merge conflicts appear on the agent due to another branch being 36 | # merged in ahead of the one they are merging, the build will fail. 37 | 38 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 39 | 40 | target_branch=$1 41 | merge_branch=${2:-"merge/$target_branch"} 42 | red='\033[0;31m' 43 | yellow='\033[0;33m' 44 | green='\033[0;32m' 45 | cyan='\033[0;36m' 46 | blue='\033[0;34m' 47 | default='\033[0m' 48 | 49 | mergeq_dir=".mergeq" 50 | merging_file="$mergeq_dir/merging" 51 | 52 | hooks_dir="$mergeq_dir/hooks" 53 | 54 | function run_hook { 55 | hook_name=$1 56 | hook="$hooks_dir/$hook_name" 57 | 58 | [[ -f $hook ]] || return 0 59 | 60 | status "Running hook: $hook_name..." 61 | 62 | eval "$hook \"$target_branch\" \"$merge_branch\"" 63 | [[ $? -eq 0 ]] || exit $? 64 | } 65 | 66 | function validate_parameters { 67 | if [ -f $merging_file ] ; then 68 | echo -e "${red}It looks like you're in the middle of a merge.${default} 69 | If so, try ${blue}$0 --continue${default} 70 | If not, delete the ${blue}$merging_file${default} file and try again." 71 | exit 1 72 | fi 73 | if [ "$target_branch" = "" ] ; then 74 | print_usage_and_exit 75 | fi 76 | } 77 | 78 | function print_usage_and_exit { 79 | echo -e "Usage: ${blue}$0 [merge-branch]${default}" 80 | exit 1 81 | } 82 | 83 | function status { 84 | echo -e "${cyan}// $1${default}" 85 | } 86 | 87 | function exit_if_local_mods { 88 | if [ ! -z "$(git status --porcelain)" ] ; then 89 | status "Local modifications detected. Cannot push." 90 | git status -s 91 | exit 1 92 | fi 93 | 94 | return 0 95 | } 96 | 97 | function merge_failed { 98 | echo -e "${yellow}Doh. Your merge has conflicts, but don't worry:${default}" 99 | echo 100 | echo 1. Fix your merge conflicts 101 | echo 2. Commit them 102 | echo -e "3. Run ${blue}$0 --continue${default}" 103 | 104 | exit 1 105 | } 106 | 107 | function checkout_target_branch { 108 | status "Checking out $target_branch..." 109 | 110 | git checkout -q origin/$target_branch 111 | git reset --hard 112 | git clean -f 113 | } 114 | 115 | function cleanup { 116 | git checkout -q $branch 117 | rm $merging_file 118 | 119 | run_hook "after_cleanup" 120 | } 121 | 122 | function try_to_merge { 123 | status "Merging $branch into $target_branch" 124 | 125 | git merge --no-ff $branch -m "Merge $branch into $target_branch" || merge_failed 126 | } 127 | 128 | function ensure_mergeq_dir_exists { 129 | mkdir -p $mergeq_dir 130 | } 131 | 132 | function write_temp_file { 133 | status "Writing temp file..." 134 | echo "$branch;$merge_branch;$target_branch" > $merging_file 135 | } 136 | 137 | function start_merge { 138 | status "Starting merge..." 139 | set -e 140 | 141 | exit_if_local_mods 142 | 143 | git fetch origin 144 | run_hook "before_merge" 145 | 146 | branch=`git rev-parse --abbrev-ref HEAD` 147 | 148 | checkout_target_branch 149 | 150 | ensure_mergeq_dir_exists 151 | write_temp_file 152 | 153 | try_to_merge 154 | 155 | continue_merge 156 | } 157 | 158 | function push_failed { 159 | status "Your push failed, someone may have beat you. Try again?" 160 | } 161 | 162 | function exit_if_we_have_already_been_merged { 163 | set +e 164 | git diff --quiet origin/$target_branch 165 | if [ $? -eq 0 ] 166 | then 167 | echo " 168 | ********************************************************** 169 | 170 | This branch has already been merged into $target_branch 171 | 172 | **********************************************************" 173 | cleanup 174 | exit 0 175 | fi 176 | set -e 177 | } 178 | 179 | function push_to_merge_branch { 180 | current=`git rev-parse HEAD` 181 | 182 | status "Merging into $merge_branch" 183 | git checkout -q origin/$merge_branch 184 | git merge --no-ff --no-edit -s ours -m "Queuing merge: $branch into $target_branch" $current 185 | merged=`git rev-parse HEAD` 186 | 187 | # make the merge branch match exactly before committing the merge 188 | # see: http://stackoverflow.com/a/27338013/11229 189 | git checkout -q $current 190 | git reset --soft $merged 191 | 192 | echo $current > .merge 193 | git add . 194 | git commit --amend -C HEAD 195 | 196 | status "Queuing merge by pushing $merge_branch" 197 | git push origin HEAD:refs/heads/$merge_branch 198 | 199 | if [ $? = 0 ] ; then 200 | run_hook "after_push" 201 | else 202 | push_failed 203 | fi 204 | } 205 | 206 | function continue_merge { 207 | exit_if_local_mods 208 | exit_if_we_have_already_been_merged 209 | push_to_merge_branch 210 | 211 | cleanup 212 | echo -e "${green}// Done!${default}" 213 | } 214 | 215 | if [ "$target_branch" = "--continue" ] ; then 216 | if [ -f $merging_file ] ; then 217 | IFS=';' read -ra branches < $merging_file 218 | branch=${branches[0]} 219 | merge_branch=${branches[1]} 220 | target_branch=${branches[2]} 221 | 222 | status "Continuing merge..." 223 | continue_merge 224 | else 225 | echo -e " 226 | ${yellow}**********************************************************${default} 227 | 228 | It doesn't look like you're in the middle of a merge. 229 | Try ${blue}$0 ${default} to start one 230 | 231 | ${yellow}**********************************************************${default}" 232 | exit 1 233 | fi 234 | else 235 | validate_parameters 236 | start_merge 237 | fi 238 | -------------------------------------------------------------------------------- /bin/mergeq_ci: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # script/mergeq_ci 4 | # 5 | # This is the other half of script/mergeq. This script is only run on TeamCity. 6 | # The purpose of this script is to reconcile the attempted merge with where 7 | # the target branch actually is. It does this with some nasty hackery involving 8 | # rewriting the .git/MERGE_HEAD. 9 | # 10 | # This also reconciles BRANCH_CHANGES, delegating to another script to merge 11 | # them into CHANGELOG.md when merging into master 12 | # 13 | # It has two modes, one is to do the merge, run before the build, the other is 14 | # to push, which is run after the build. I'm not really sure why it needs 15 | # to handle the push, I don't think it does anything special. 16 | 17 | set -e 18 | 19 | action=$1 20 | target_branch=$2 21 | 22 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 23 | 24 | mergeq_dir=".mergeq" 25 | hooks_dir="$mergeq_dir/hooks" 26 | 27 | function run_hook { 28 | hook_name=$1 29 | hook="$hooks_dir/$hook_name" 30 | 31 | [[ -f $hook ]] || return 0 32 | 33 | status "Running CI hook: $hook_name..." 34 | 35 | eval "$hook \"$target_branch\"" 36 | [[ $? -eq 0 ]] || exit $? 37 | } 38 | 39 | function print_usage_and_exit { 40 | echo "Usage: $0 " 41 | exit 1 42 | } 43 | 44 | function status { 45 | echo "// $1" 46 | } 47 | 48 | function checkout_target_branch { 49 | status "Checking out $target_branch..." 50 | git fetch origin $target_branch 51 | git checkout -q -f FETCH_HEAD 52 | 53 | status "Cleaning up working directory..." 54 | git reset --hard 55 | git clean -df 56 | } 57 | 58 | function merge_branch_into_target_branch { 59 | # This ends up looking like a new merge regardless 60 | # of whether or not we can fast forward merge. 61 | # and it copies over any merge conflict resolutions. 62 | # It's clearly black magic. 63 | status "Merging into $target_branch..." 64 | git merge --no-ff --no-commit $head 65 | echo `git rev-parse $head^2` > .git/MERGE_HEAD 66 | } 67 | 68 | function commit_merge { 69 | message=`git log -1 --pretty=%s $head` 70 | 71 | status "Committing merge ($message)..." 72 | git commit -C $head --signoff 73 | } 74 | 75 | function reset_and_exit_if_we_have_already_been_merged { 76 | set +e 77 | git diff --quiet FETCH_HEAD 78 | if [ $? -eq 0 ] 79 | then 80 | echo " 81 | ********************************************************** 82 | 83 | This branch has already been merged into $target_branch 84 | 85 | **********************************************************" 86 | git reset --hard FETCH_HEAD 87 | exit 0 88 | fi 89 | set -e 90 | } 91 | 92 | function merge { 93 | head=`git rev-parse HEAD^2` 94 | 95 | checkout_target_branch 96 | 97 | run_hook "before_ci_merge" 98 | merge_branch_into_target_branch 99 | run_hook "after_ci_merge" 100 | 101 | reset_and_exit_if_we_have_already_been_merged 102 | commit_merge 103 | } 104 | 105 | function push { 106 | run_hook "before_ci_push" 107 | 108 | status "Pushing to $target_branch..." 109 | git push origin HEAD:$target_branch 110 | 111 | run_hook "after_ci_push" 112 | } 113 | 114 | function validate_parameters { 115 | if [ "$target_branch" = "" ] ; then 116 | print_usage_and_exit 117 | fi 118 | } 119 | 120 | validate_parameters 121 | 122 | run_hook "before_ci_startup" 123 | 124 | if [ "$action" = "merge" ] ; then 125 | merge 126 | elif [ "$action" = "push" ] ; then 127 | push 128 | else 129 | print_usage_and_exit 130 | fi 131 | -------------------------------------------------------------------------------- /bin/mergeq_install: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | project_dir=$(pwd) 4 | script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )" 5 | 6 | bin_dir="$project_dir/bin" 7 | mergeq_dir="$project_dir/.mergeq" 8 | 9 | function ensure_bin_dir { 10 | mkdir -p $bin_dir 11 | } 12 | 13 | function ensure_mergeq_dir { 14 | mkdir -p $mergeq_dir 15 | } 16 | 17 | function gitignore_merging_file { 18 | gitignore_file=$project_dir/.mergeq/.gitignore 19 | 20 | if [ ! -f $gitignore_file ]; then 21 | touch $gitignore_file 22 | fi 23 | 24 | ignore="merging" 25 | 26 | if ! grep -q $ignore $gitignore_file ; then 27 | echo $ignore >> $gitignore_file 28 | fi 29 | } 30 | 31 | function copy_mergeq_scripts_to_bin { 32 | cp $script_dir/bin/mergeq $bin_dir 33 | cp $script_dir/bin/mergeq_ci $bin_dir 34 | } 35 | 36 | ensure_mergeq_dir 37 | ensure_bin_dir 38 | 39 | copy_mergeq_scripts_to_bin 40 | 41 | gitignore_merging_file 42 | -------------------------------------------------------------------------------- /bin/mergeq_remote_install: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | red='\033[0;31m' 6 | yellow='\033[0;33m' 7 | green='\033[0;32m' 8 | cyan='\033[0;36m' 9 | blue='\033[0;34m' 10 | default='\033[0m' 11 | 12 | function status { 13 | echo -e "${cyan}// $1${default}" 14 | } 15 | 16 | # Ensure we don't have a stale copy of mergeq sitting around. 17 | rm -fr /tmp/mergeq 18 | 19 | status "Fetching mergeq from aaronjensen/mergeq..." 20 | git clone git@github.com:aaronjensen/mergeq.git /tmp/mergeq 21 | echo 22 | 23 | status "Installing mergeq to current project directory" 24 | /tmp/mergeq/bin/mergeq_install 25 | 26 | rm -fr /tmp/mergeq 27 | --------------------------------------------------------------------------------