├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── load_branches_from_remote.sh ├── monorepo_add.sh ├── monorepo_build.sh ├── monorepo_split.sh ├── original_refs_restore.sh ├── original_refs_wipe.sh ├── rewrite_history_from.sh ├── rewrite_history_into.sh ├── tag_refs_backup.sh └── tag_refs_move_to_original.sh /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Thank you for your contributions to Shopsys Monorepo Tools package. 4 | Together we are making Shopsys Platform better. 5 | 6 | This repository is READ-ONLY. 7 | If you want to [report issues](https://github.com/shopsys/shopsys/issues/new) and/or send [pull requests](https://github.com/shopsys/shopsys/compare), 8 | please use the main [Shopsys repository](https://github.com/shopsys/shopsys). 9 | 10 | Please check our [Contribution Guide](https://github.com/shopsys/shopsys/blob/HEAD/CONTRIBUTING.md) before contributing. 11 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018-2023 Shopsys s.r.o., http://www.shopsys.com 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 6 | 7 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 11 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 12 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 13 | WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 14 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Shopsys Monorepo Tools 2 | 3 | [![Mentioned in Awesome Monorepo](https://awesome.re/mentioned-badge.svg)](https://github.com/korfuri/awesome-monorepo) 4 | 5 | **This package is used for splitting our monorepo and we share it with our community as it is. We do not intend to support or develop it any further. Feel free to fork it and adjust for your own need.** 6 | 7 | Tools for building and splitting monolithic repository from existing packages. 8 | You can read about pros and cons of monorepo approach on the [Shopsys Platform Blog](https://blog.shopsys.com/how-to-maintain-multiple-git-repositories-with-ease-61a5e17152e0). 9 | 10 | We created these scripts because we couldn't find a tool that would keep the git history of subpackages unchanged. 11 | 12 | You may need to update your `git` (tested on `2.16.1`). 13 | 14 | This repository is maintained by [shopsys/shopsys](https://github.com/shopsys/shopsys) monorepo, information about changes is in its `CHANGELOG` file. 15 | 16 | ## Quick start 17 | 18 | ### 1. Download 19 | 20 | First download this repository so you can use the tools (eg. into `~/monorepo-tools`). 21 | 22 | ``` 23 | git clone https://github.com/shopsys/monorepo-tools ~/monorepo-tools 24 | ``` 25 | 26 | ### 2. Preparing an empty repository with added remotes 27 | 28 | You have to create a new git repository for your monorepo and add all your existing packages as remotes. 29 | You can add as many remotes as you want. 30 | 31 | In this example we will prepare 3 packages from github for merging into monorepo. 32 | 33 | ``` 34 | git init 35 | git remote add main-repository http://github.com/vendor/main-repository.git 36 | git remote add package-alpha http://github.com/vendor/alpha.git 37 | git remote add package-beta http://github.com/vendor/beta.git 38 | git fetch --all --no-tags 39 | ``` 40 | 41 | ### 3. Building the monorepo 42 | 43 | Then you can build your monorepo using `monorepo_build.sh`. 44 | Just list the names of all your previously added remotes as arguments. 45 | Optionally you can specify a directory where the repository will be located by providing `:`, otherwise remote name will be used. 46 | 47 | The command will rewrite history of all mentioned repositories as if they were developed in separate subdirectories. 48 | 49 | Only branches `master` will be merged together, other branches will be kept only from first package to avoid possible branch name conflicts. 50 | 51 | ``` 52 | ~/monorepo-tools/monorepo_build.sh \ 53 | main-repository package-alpha:packages/alpha package-beta:packages/beta 54 | ``` 55 | 56 | This may take a while, depending on the size of your repositories. 57 | 58 | Now your `master` branch should contain all packages in separate directories. For our example it would mean: 59 | 60 | - **main-repository/** - contains repository _vendor/main-repository_ 61 | - **packages/** 62 | - **alpha/** - contains repository _vendor/alpha_ 63 | - **beta/** - contains repository _vendor/beta_ 64 | 65 | ### 4. Splitting into original repositories 66 | 67 | You should develop all your packages in this repository from now on. 68 | 69 | When you made your changes and would like to update the original repositories use `monorepo_split.sh` with the same arguments as before. 70 | 71 | ``` 72 | ~/monorepo-tools/monorepo_split.sh \ 73 | main-repository package-alpha:packages/alpha package-beta:packages/beta 74 | ``` 75 | 76 | This will push all relevant changes into all of your remotes. 77 | It will split and push your `master` branch along with all tags you added in this repository. 78 | Other branches are not pushed. 79 | 80 | It may again take a while, depending on the size of your monorepo. 81 | 82 | **_Note:_** 83 | _The commits in the split repositories should be identical to those from the original repo, keeping the git history intact._ 84 | _Thus, if you have checked out the original `master` previously, you should be able to fast-forward to the new version after splitting._ 85 | _The only known exception is a signed commit (note that GitHub signs commits made via its web UI by default)._ 86 | _If you have signed commits in your original repository, the split commits will NOT be signed._ 87 | _This will prevent `monorepo_split.sh` from pushing the unsigned commits to the remote._ 88 | _To overcome this you can add [the `--force` flag](https://git-scm.com/docs/git-push#git-push--f) to the `git push` calls in the script, but it may cause unforeseen consequences if you're not sure what you're doing._ 89 | 90 | ### Add a new package into the monorepo 91 | 92 | When you have the monorepo, you may find a reason for adding a new package after some time you already use the monorepo. 93 | In this case, don't use `monorepo_build.sh`, but do following steps: 94 | 95 | - Create a new repository, for example, _vendor/gamma_ 96 | - Add remote into the monorepo `git remote add package-gamma http://github.com/vendor/gamma.git` 97 | - Create a new directory in the monorepo **packages/gamma** 98 | - Add the code and commit it 99 | - Use split tool with the new package 100 | ``` 101 | ~/monorepo-tools/monorepo_split.sh \ 102 | main-repository package-alpha:packages/alpha package-beta:packages/beta package-gamma:packages/gamma 103 | ``` 104 | 105 | ## Reference 106 | 107 | This is just a short description and usage of all the tools in the package. 108 | For detailed information go to the scripts themselves and read the comments. 109 | 110 | ### [monorepo_build.sh](./monorepo_build.sh) 111 | 112 | Build monorepo from specified remotes. The remotes must be already added to your repository and fetched. 113 | 114 | Usage: `monorepo_build.sh [:] [:] ...` 115 | 116 | ### [monorepo_split.sh](./monorepo_split.sh) 117 | 118 | Split monorepo built by `monorepo_build.sh` and push all `master` branches along with all tags into specified remotes. 119 | 120 | Usage: `monorepo_split.sh [:] [:] ...` 121 | 122 | ### [monorepo_add.sh](./monorepo_add.sh) 123 | 124 | Add repositories to an existing monorepo from specified remotes. The remotes must be already added to your repository and fetched. Only master branch will be added from each repo. 125 | 126 | Usage: `monorepo_add.sh [:] [:] ...` 127 | 128 | ### [rewrite_history_into.sh](./rewrite_history_into.sh) 129 | 130 | Rewrite git history (even tags) so that all filepaths are in a specific subdirectory. 131 | 132 | Usage: `rewrite_history_into.sh []` 133 | 134 | ### [rewrite_history_from.sh](./rewrite_history_from.sh) 135 | 136 | Rewrite git history (even tags) so that only commits that made changes in a subdirectory are kept and rewrite all filepaths as if it was root. 137 | 138 | Usage: `rewrite_history_from.sh []` 139 | 140 | ### [original_refs_restore.sh](./original_refs_restore.sh) 141 | 142 | Restore original git history after rewrite. 143 | 144 | Usage: `original_refs_restore.sh` 145 | 146 | ### [original_refs_wipe.sh](./original_refs_wipe.sh) 147 | 148 | Wipe original git history after rewrite. 149 | 150 | Usage: `original_refs_wipe.sh` 151 | 152 | ### [load_branches_from_remote.sh](./load_branches_from_remote.sh) 153 | 154 | Delete all local branches and create all non-remote-tracking branches of a specified remote. 155 | 156 | Usage: `load_branches_from_remote.sh ` 157 | 158 | ### [tag_refs_backup.sh](./tag_refs_backup.sh) 159 | 160 | Backup tag refs into `refs/original-tags/` 161 | 162 | Usage: `tag_refs_backup.sh` 163 | 164 | ### [tag_refs_move_to_original.sh](./tag_refs_move_to_original.sh) 165 | 166 | Move tag refs from `refs/original-tags/` into `refs/original/` 167 | 168 | Usage: `tag_refs_move_to_original.sh` 169 | 170 | ## Contributing 171 | 172 | Thank you for your contributions to Shopsys Monorepo Tools package. 173 | Together we are making Shopsys Platform better. 174 | 175 | This repository is READ-ONLY. 176 | If you want to [report issues](https://github.com/shopsys/shopsys/issues/new) and/or send [pull requests](https://github.com/shopsys/shopsys/compare), 177 | please use the main [Shopsys repository](https://github.com/shopsys/shopsys). 178 | 179 | Please check our [Contribution Guide](https://github.com/shopsys/shopsys/blob/HEAD/CONTRIBUTING.md) before contributing. 180 | 181 | ## Support 182 | 183 | What to do when you are in troubles or need some help? 184 | The best way is to join our [Slack](https://join.slack.com/t/shopsysframework/shared_invite/zt-11wx9au4g-e5pXei73UJydHRQ7nVApAQ). 185 | 186 | If you want to [report issues](https://github.com/shopsys/shopsys/issues/new), please use the main [Shopsys repository](https://github.com/shopsys/shopsys). 187 | -------------------------------------------------------------------------------- /load_branches_from_remote.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Delete all local branches and create all non-remote-tracking branches of a specified remote 4 | # 5 | # Usage: load_branches_from_remote.sh 6 | # 7 | # Example: load_branches_from_remote.sh origin 8 | 9 | REMOTE=$1 10 | echo "Loading all branches from the remote '$REMOTE' (all local branches are deleted)" 11 | # Checking out orphan commit so it 's possible to delete current branch 12 | git checkout --orphan void 13 | # Delete all local branches 14 | for BRANCH in `git branch`; do 15 | git branch -D $BRANCH 16 | done 17 | # Create non-remote-tracking branches from selected remote 18 | for REMOTE_BRANCH in $(git branch -r|grep $REMOTE/); do 19 | BRANCH=${REMOTE_BRANCH/$REMOTE\//} 20 | git branch -q $BRANCH $REMOTE_BRANCH 21 | git branch --unset-upstream $BRANCH 22 | done 23 | git checkout -f master 24 | 25 | -------------------------------------------------------------------------------- /monorepo_add.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Add repositories to a monorepo from specified remotes 4 | # You must first add the remotes by "git remote add " and fetch from them by "git fetch --all" 5 | # It will merge master branches of the monorepo and all remotes together while keeping all current branches in monorepo intact 6 | # If subdirectory is not specified remote name will be used instead 7 | # 8 | # Usage: monorepo_add.sh [:] [:] ... 9 | # 10 | # Example: monorepo_add.sh additional-repository package-gamma:packages/gamma package-delta:packages/delta 11 | 12 | # Check provided arguments 13 | if [ "$#" -lt "1" ]; then 14 | echo 'Please provide at least 1 remote to be added into an existing monorepo' 15 | echo 'Usage: monorepo_add.sh [:] [:] ...' 16 | echo 'Example: monorepo_add.sh additional-repository package-gamma:packages/gamma package-delta:packages/delta' 17 | exit 18 | fi 19 | # Get directory of the other scripts 20 | MONOREPO_SCRIPT_DIR=$(dirname "$0") 21 | # Wipe original refs (possible left-over back-up after rewriting git history) 22 | $MONOREPO_SCRIPT_DIR/original_refs_wipe.sh 23 | for PARAM in $@; do 24 | # Parse parameters in format [:] 25 | PARAM_ARR=(${PARAM//:/ }) 26 | REMOTE=${PARAM_ARR[0]} 27 | SUBDIRECTORY=${PARAM_ARR[1]} 28 | if [ "$SUBDIRECTORY" == "" ]; then 29 | SUBDIRECTORY=$REMOTE 30 | fi 31 | echo "Building branch 'master' of the remote '$REMOTE'" 32 | git checkout --detach $REMOTE/master 33 | $MONOREPO_SCRIPT_DIR/rewrite_history_into.sh $SUBDIRECTORY 34 | MERGE_REFS="$MERGE_REFS $(git rev-parse HEAD)" 35 | # Wipe the back-up of original history 36 | $MONOREPO_SCRIPT_DIR/original_refs_wipe.sh 37 | done 38 | # Merge all master branches 39 | COMMIT_MSG="merge multiple repositories into an existing monorepo"$'\n'$'\n'"- merged using: 'monorepo_add.sh $@'"$'\n'"- see https://github.com/shopsys/monorepo-tools" 40 | git checkout master 41 | echo "Merging refs: $MERGE_REFS" 42 | git merge --no-commit -q $MERGE_REFS --allow-unrelated-histories 43 | echo 'Resolving conflicts using trees of all parents' 44 | for REF in $MERGE_REFS; do 45 | # Add all files from all master branches into index 46 | # "git read-tree" with multiple refs cannot be used as it is limited to 8 refs 47 | git ls-tree -r $REF | git update-index --index-info 48 | done 49 | git commit -m "$COMMIT_MSG" 50 | git reset --hard 51 | 52 | -------------------------------------------------------------------------------- /monorepo_build.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Build monorepo from specified remotes 4 | # You must first add the remotes by "git remote add " and fetch from them by "git fetch --all" 5 | # Final monorepo will contain all branches from the first remote and master branches of all remotes will be merged 6 | # If subdirectory is not specified remote name will be used instead 7 | # 8 | # Usage: monorepo_build.sh [:] [:] ... 9 | # 10 | # Example: monorepo_build.sh main-repository package-alpha:packages/alpha package-beta:packages/beta 11 | 12 | # Check provided arguments 13 | if [ "$#" -lt "2" ]; then 14 | echo 'Please provide at least 2 remotes to be merged into a new monorepo' 15 | echo 'Usage: monorepo_build.sh [:] [:] ...' 16 | echo 'Example: monorepo_build.sh main-repository package-alpha:packages/alpha package-beta:packages/beta' 17 | read -p "Do you want to proceed? (y/n) " yn 18 | case $yn in 19 | [yY] ) echo Proceeding; 20 | break;; 21 | * ) echo exiting...; 22 | exit;; 23 | esac 24 | fi 25 | # Get directory of the other scripts 26 | MONOREPO_SCRIPT_DIR=$(dirname "$0") 27 | # Wipe original refs (possible left-over back-up after rewriting git history) 28 | $MONOREPO_SCRIPT_DIR/original_refs_wipe.sh 29 | for PARAM in $@; do 30 | # Parse parameters in format [:] 31 | PARAM_ARR=(${PARAM//:/ }) 32 | REMOTE=${PARAM_ARR[0]} 33 | SUBDIRECTORY=${PARAM_ARR[1]} 34 | if [ "$SUBDIRECTORY" == "" ]; then 35 | SUBDIRECTORY=$REMOTE 36 | fi 37 | # Rewrite all branches from the first remote, only master branches from others 38 | if [ "$PARAM" == "$1" ]; then 39 | echo "Building all branches of the remote '$REMOTE'" 40 | $MONOREPO_SCRIPT_DIR/load_branches_from_remote.sh $REMOTE 41 | $MONOREPO_SCRIPT_DIR/rewrite_history_into.sh $SUBDIRECTORY --branches 42 | MERGE_REFS='master' 43 | else 44 | echo "Building branch 'master' of the remote '$REMOTE'" 45 | git checkout --detach $REMOTE/master 46 | $MONOREPO_SCRIPT_DIR/rewrite_history_into.sh $SUBDIRECTORY 47 | MERGE_REFS="$MERGE_REFS $(git rev-parse HEAD)" 48 | fi 49 | # Wipe the back-up of original history 50 | $MONOREPO_SCRIPT_DIR/original_refs_wipe.sh 51 | done 52 | # Merge all master branches 53 | COMMIT_MSG="merge multiple repositories into a monorepo"$'\n'$'\n'"- merged using: 'monorepo_build.sh $@'"$'\n'"- see https://github.com/shopsys/monorepo-tools" 54 | git checkout master 55 | echo "Merging refs: $MERGE_REFS" 56 | git merge --no-commit -q $MERGE_REFS --allow-unrelated-histories 57 | echo 'Resolving conflicts using trees of all parents' 58 | for REF in $MERGE_REFS; do 59 | # Add all files from all master branches into index 60 | # "git read-tree" with multiple refs cannot be used as it is limited to 8 refs 61 | git ls-tree -r $REF | git update-index --index-info 62 | done 63 | git commit -m "$COMMIT_MSG" 64 | git reset --hard 65 | 66 | -------------------------------------------------------------------------------- /monorepo_split.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Split monorepo and push all master branches and all tags into specified remotes 4 | # You must first build the monorepo via "monorepo_build" (uses same parameters as "monorepo_split") 5 | # If subdirectory is not specified remote name will be used instead 6 | # 7 | # Usage: monorepo_split.sh [:] [:] ... 8 | # 9 | # Example: monorepo_split.sh main-repository package-alpha:packages/alpha package-beta:packages/beta 10 | 11 | # Check provided arguments 12 | if [ "$#" -lt "1" ]; then 13 | echo 'Please provide at least 1 remote for splitting' 14 | echo 'Usage: monorepo_split.sh [:] [:] ...' 15 | echo 'Example: monorepo_split.sh main-repository package-alpha:packages/alpha package-beta:packages/beta' 16 | exit 17 | fi 18 | # Get directory of the other scripts 19 | MONOREPO_SCRIPT_DIR=$(dirname "$0") 20 | # Wipe original refs (possible left-over back-up after rewriting git history) 21 | $MONOREPO_SCRIPT_DIR/original_refs_wipe.sh 22 | for PARAM in $@; do 23 | # Parse parameters in format [:] 24 | PARAM_ARR=(${PARAM//:/ }) 25 | REMOTE=${PARAM_ARR[0]} 26 | SUBDIRECTORY=${PARAM_ARR[1]} 27 | if [ "$SUBDIRECTORY" == "" ]; then 28 | SUBDIRECTORY=$REMOTE 29 | fi 30 | # Rewrite git history of master branch 31 | echo "Splitting repository for the remote '$REMOTE'" 32 | git checkout master 33 | $MONOREPO_SCRIPT_DIR/rewrite_history_from.sh $SUBDIRECTORY master $(git tag) 34 | if [ $? -eq 0 ]; then 35 | echo "Pushing branch 'master' and all tags into '$REMOTE'" 36 | git push --tags $REMOTE master 37 | else 38 | echo "Splitting repository for the remote '$REMOTE' failed! Not pushing anything into it." 39 | fi 40 | # Restore the original history from back-up 41 | $MONOREPO_SCRIPT_DIR/original_refs_restore.sh 42 | done 43 | 44 | -------------------------------------------------------------------------------- /original_refs_restore.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Restore original git history after rewrite 4 | # 5 | # Usage: original_refs_restore.sh 6 | 7 | echo 'Restoring the original history back-up' 8 | # Original refs after history rewrite are stored in refs/original/ 9 | for ORIGINAL_REF in $(git for-each-ref --format="%(refname)" refs/original/); do 10 | git update-ref "${ORIGINAL_REF#refs/original/}" $ORIGINAL_REF 11 | git update-ref -d $ORIGINAL_REF 12 | done 13 | git reset --hard 14 | 15 | -------------------------------------------------------------------------------- /original_refs_wipe.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Wipe original git history after rewrite 4 | # 5 | # Usage: original_refs_wipe.sh 6 | 7 | echo 'Wiping the original history back-up' 8 | # Original refs after history rewrite are stored in refs/original/ 9 | for ORIGINAL_REF in $(git for-each-ref --format="%(refname)" refs/original/); do 10 | git update-ref -d $ORIGINAL_REF 11 | done 12 | git reset --hard 13 | 14 | -------------------------------------------------------------------------------- /rewrite_history_from.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Rewrite git history so that only commits that made changes in a subdirectory are kept and rewrite all filepaths as if it was root 4 | # You can use arguments for "git rev-list" to specify what commits to rewrite (defaults to rewriting history of the checked-out branch) 5 | # All tags in the provided range will be rewritten as well 6 | # 7 | # Usage: rewrite_history_from.sh [] 8 | # 9 | # Example: rewrite_history_from.sh packages/alpha 10 | # Example: rewrite_history_from.sh main-repository --branches 11 | 12 | SUBDIRECTORY=$1 13 | REV_LIST_PARAMS=`echo ${@:2} | tr " " "\n"` 14 | echo "Rewriting history from a subdirectory '$SUBDIRECTORY'" 15 | 16 | # Get directory of the other scripts 17 | MONOREPO_SCRIPT_DIR=$(dirname "$0") 18 | # Backup all monorepo tag refs because `git filter-branch` doesn't backup tags 19 | $MONOREPO_SCRIPT_DIR/tag_refs_backup.sh 20 | 21 | # Remove tags that don't contain any files in $SUBDIRECTORY 22 | # Removes refs only, because tags are objects that can be used later after `original_refs_restore.sh` 23 | # Removes also tags from $REV_LIST_PARAMS, so filter-branch filters only known tags (if they are present) 24 | for TAG in `git tag` 25 | do 26 | if [ `git ls-tree -r --name-only $TAG | grep -E "^\"*$SUBDIRECTORY/" | wc -l` == "0" ]; then 27 | git update-ref -d refs/tags/$TAG 28 | REV_LIST_PARAMS=`echo "$REV_LIST_PARAMS" | grep -v $TAG` 29 | fi 30 | done 31 | 32 | # All paths in the index that are not prefixed with a subdirectory are removed via "git rm --cached" 33 | # Setting quotepath to false is needed to handle path and file names containing unicode characters 34 | # If there are any files in the index all paths have the subdirectory prefix removed and the index is updated 35 | # Previous index file is replaced by a new one (otherwise each file would be in the index twice) 36 | # Only non-empty are filtered by the commit-filter 37 | # The tags are rewritten as well as commits (the "cat" command will use original name without any change) 38 | if [ $(uname) == "Darwin" ]; then 39 | XARGS_OPTS="-0" 40 | SED_OPTS="-E" 41 | else 42 | XARGS_OPTS="-r -0" 43 | SED_OPTS="-r" 44 | fi 45 | SUBDIRECTORY=$SUBDIRECTORY SUBDIRECTORY_SED=${SUBDIRECTORY//-/\\-} TAB=$'\t' XARGS_OPTS=$XARGS_OPTS SED_OPTS=$SED_OPTS git filter-branch \ 46 | --index-filter ' 47 | git -c core.quotepath=false ls-files | grep -vE "^\"*$SUBDIRECTORY/" | tr "\n" "\0" | xargs $XARGS_OPTS git rm -q --cached 48 | if [ "$(git ls-files)" != "" ]; then 49 | git ls-files -s | sed $SED_OPTS "s-($TAB\"*)$SUBDIRECTORY_SED/-\1-" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE 50 | fi' \ 51 | --commit-filter 'git_commit_non_empty_tree "$@"' \ 52 | --tag-name-filter 'cat' \ 53 | -- $REV_LIST_PARAMS 54 | 55 | # Moves backup monorepo tag refs to the same group with automatic backup from `git filter-branch` 56 | # so tags can be restored by `original_refs_restore.sh` 57 | $MONOREPO_SCRIPT_DIR/tag_refs_move_to_original.sh 58 | -------------------------------------------------------------------------------- /rewrite_history_into.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Rewrite git history so that all filepaths are in a specific subdirectory 4 | # You can use arguments for "git rev-list" to specify what commits to rewrite (defaults to rewriting history of the checked-out branch) 5 | # All tags in the provided range will be rewritten as well 6 | # 7 | # Usage: rewrite_history_into.sh [] 8 | # 9 | # Example: rewrite_history_into.sh packages/alpha 10 | # Example: rewrite_history_into.sh main-repository --branches 11 | 12 | SUBDIRECTORY=$1 13 | REV_LIST_PARAMS=${@:2} 14 | echo "Rewriting history into a subdirectory '$SUBDIRECTORY'" 15 | # All paths in the index are prefixed with a subdirectory and the index is updated 16 | # Previous index file is replaced by a new one (otherwise each file would be in the index twice) 17 | # The tags are rewritten as well as commits (the "cat" command will use original name without any change) 18 | SUBDIRECTORY_SED=${SUBDIRECTORY//-/\\-} TAB=$'\t' git filter-branch \ 19 | --index-filter ' 20 | git ls-files -s | sed "s-$TAB\"*-&$SUBDIRECTORY_SED/-" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && if [ -f "$GIT_INDEX_FILE.new" ]; then mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"; fi' \ 21 | --tag-name-filter 'cat' \ 22 | -- $REV_LIST_PARAMS 23 | -------------------------------------------------------------------------------- /tag_refs_backup.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Backup tag refs, because `git filter-branch` doesn't do it 4 | # Backup into refs/original-tags/ because `git filter-branch` needs /refs/original/ empty 5 | # 6 | # Usage: tag_refs_backup.sh 7 | 8 | for TAG_REF in $(git for-each-ref --format="%(refname)" refs/tags/); do 9 | git update-ref refs/original-tags/$TAG_REF $TAG_REF 10 | done 11 | -------------------------------------------------------------------------------- /tag_refs_move_to_original.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Move tag refs from refs/original-tags/ into refs/original/ 4 | # 5 | # Usage: tag_refs_move_to_original.sh 6 | 7 | for TAG_REF in $(git for-each-ref --format="%(refname)" refs/original-tags/); do 8 | git update-ref refs/original/"${TAG_REF#refs/original-tags/}" $TAG_REF 9 | git update-ref -d $TAG_REF 10 | done 11 | --------------------------------------------------------------------------------