├── .gitignore
├── README.md
├── git-cadd
├── git-cclone
├── git-crecover
├── gitcache-server
├── gitcache-server.conf
└── gitcache.conf


/.gitignore:
--------------------------------------------------------------------------------
1 | .*.sw*
2 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | git-cache
  2 | =========
  3 | 
  4 | Git extension to cache upstream repos on a local server
  5 | 
  6 | <pre>
  7 |        Figure 1: basic principle
  8 |        -------------------------
  9 | 
 10 |           +---------------+
 11 |           |  workstation  |
 12 |           +---v-------^---+
 13 |               |       |
 14 |               |       |fetch
 15 |               |       |
 16 |               |   +---^-------+
 17 |           push|   | git-cache |
 18 |               |   +---^-------+
 19 |               |       |
 20 |               |       |fetch
 21 |               |       |
 22 |           +---v-------^---+
 23 |           |  git server   |
 24 |           +---------------+
 25 | </pre>
 26 | 
 27 | Audience
 28 | --------
 29 | 
 30 | These scripts are for teams working on some git repositories
 31 | staying on external sites. Using git-cache reduces significantly the external bandwith usage due to repeated fetchs.
 32 | 
 33 | From the developer point of view, the cloned repository is usable transparently: pushes occur on the primary upstream server.
 34 | 
 35 | Content
 36 | -------
 37 | 
 38 | The solution comes with some scripts:
 39 | * 'git-cclone', 'git-crecover' and 'git-cadd' to be installed on all clients
 40 | * 'gitccache-server' to be installed on the cache server
 41 | 
 42 | And 2 examples of configuration files.
 43 | * 'gitcache.conf' for clients
 44 | * 'gitcache-server.conf' for servers
 45 | 
 46 | Installing the client scripts
 47 | -----------------------------
 48 | 
 49 | As user, put the scripts 'git-cclone', 'git-crecover' and
 50 | 'git-cadd' in your PATH.
 51 | 
 52 | For example:
 53 | <pre>
 54 | $ git clone https://github.com/kooltux/git-cache.git
 55 | $ ln -s $(pwd)/git-cache/git-cclone ~/bin/
 56 | $ ln -s $(pwd)/git-cache/git-crecover ~/bin
 57 | $ ln -s $(pwd)/git-cache/git-cadd ~/bin
 58 | </pre>
 59 | 
 60 | Create the file (or copy git-cache/gitcache.conf) **~/.gitcache.conf** and edit it to put the 
 61 | definition of the variable **GIT_CACHE_SERVER** as in the example below:
 62 | 
 63 | <pre>
 64 | GIT_CACHE_SERVER=XXXX
 65 | </pre>
 66 | 
 67 | where *XXXX* stands for **git@proxy** or **me@my.company.com** .
 68 | 
 69 | This is how a client can connect to the cache using ssh. Feel free to use ssh-copy-id to install public keys on the cache server:
 70 | 
 71 | <pre>
 72 | $ . ~/.gitcache.conf
 73 | $ ssh-copy-id $GIT_CACHE_SERVER
 74 | </pre>
 75 | 
 76 | Refer to installation of the cache server to get full understanding of what has
 77 | to be set.
 78 | 
 79 | **NOTE:** you can also define the variable **GIT_CACHE_SERVER** in your profile 
 80 | or rc files (.profile or .bashrc or .zshrc or ....)
 81 | 
 82 | Usage on the client
 83 | -------------------
 84 | 
 85 | After installation, the client mainly have the command to clone git repositories
 86 | through the cache. The standard clone is still available as usual.
 87 | 
 88 | ### git-cclone
 89 | 
 90 | To clone a directory use can use either the command 'git-cclone' or the command
 91 | 'git cclone'.
 92 | 
 93 | Usage: **git cclone repository [directory [--mirror|--bare]]**
 94 | 
 95 | **Please note**: Currently, most of the options of 'git clone' aren't available through 'cclone'.
 96 | 
 97 | The effect of the command is:
 98 | - access the git cache server to ensure caching of the repository 
 99 | (it will produce a creation or a fetch on server side);
100 | - clone the cached git;
101 | - configure the cloned project to push on original git server and to fetch on cache.
102 | 
103 | Example:
104 | <pre>
105 | $ git cclone git@github.com:kooltux/git-cache.git git-cache
106 | $ cd git-cache
107 | $ git remote -v
108 | origin	git@gitcache.localdomain:/srv/git/gitcache/github.com//kooltux/git-cache.git (fetch)
109 | origin	https://github.com/kooltux/git-cache (push)
110 | </pre>
111 | 
112 | The client can also use the command 'git-crecover' (that is also 'git crecover').
113 | This command can either recover for a cache repository that was removed from the
114 | cache server or it can merely make a cache entry for an existing git clone (in other words, 
115 | it inserts a fetching cache for a repository that was cloned in a standard way **without
116 | cache**).
117 | 
118 | ### git-cadd
119 | 
120 | It is sometimes interesting to add remote repositories to your current
121 | repository with the command 'git remote add'. The command 'git-cadd'
122 | is the git-cache version of the command 'git remote add' but it inserts the
123 | cache for the added remote. It accepts exactly the same arguments that
124 | 'git remote add'.
125 | 
126 | Usage: **git cadd [options...] name repository**
127 | 
128 | 
129 | ### git-crecover
130 | 
131 | 'git-crecover' is interactive unless called with options. 
132 | 
133 | Usage: **git crecover [name] [-r|--remove|-a|--auto]**
134 | 
135 | The optional argument 'name' identify the remote repository name and
136 | is 'origin' by default.
137 | 
138 | Go to the directory of the cloned repository and call it
139 | with 'git-crecover' or 'git crecover'. It will ask you what to do: REMOVE or RECOVER.
140 | 
141 | **REMOVE:** remove any reference to any caching of git.
142 | 
143 | **RECOVER:** insert the git cache references and ensure that the cached repository exists.
144 | 
145 | Interesting side effect: 'git crecover' can also transform a non-cached clone 
146 | in a cached clone when using the **RECOVER** command.
147 | 
148 | When called with '-r' or '--remove', git-crecover removes any cache reference
149 | for the optionnaly given remote without any prompting.
150 | 
151 | When called with '-a' or '--auto', git-crecover insert the caching 
152 | for the optionnaly given remote without any prompting.
153 | 
154 | 
155 | 
156 | Installation of the cache server
157 | --------------------------------
158 | 
159 | Choose the machine and the account that will be used to store cached repositories
160 | and run the server script.
161 | 
162 | Log on the machine, clone the repo and copy 'gitcache-server' in a directory referenced by $PATH
163 | 
164 | <pre>
165 | $ git clone https://github.com/kooltux/git-cache.git
166 | 
167 | $ ln -s git-cache/gitcache-server ~/bin/
168 | 
169 | $ cp git-cache/gitcache-server.conf ~/.gitcache-server.conf
170 | </pre>
171 | 
172 | Edit the ~/.gitcache-server.conf file and ajust the variables:
173 | * GIT_CACHE_DIR : the cache directory where all git repos will be stored
174 | * GIT_CACHE_SERVER_PRIMARY : (optional) primary cache server for chaining
175 | * GIT_CACHE_LOG : (optional) a log file for gitcache-server activity
176 | 
177 | Using cache standalone (without server)
178 | ---------------------------------------
179 | 
180 | Into your account, you can put the client AND the server too. The server will still be 
181 | accessed with ssh. Here below the examples of configuring your **.profile** or **.bashrc**:
182 | 
183 | <pre>
184 | GIT_CACHE_SERVER=$(id -un)@localhost
185 | GIT_CACHE_DIR=~/.gitcachedir
186 | </pre>
187 | 
188 | Chaining caches
189 | ---------------
190 | 
191 | Using the GIT_CACHE_SERVER_PRIMARY variable, it's possible to chain caches.
192 | 
193 | To use this, install git-cclone and git-crecover on the server too (add them to execution path).
194 | 
195 | When gitcache-server is invoked to cache a new repo, it will try to fetch it from another cache and so on.
196 | 
197 | *GIT_CACHE_SERVER_PRIMARY* has the same syntax as GIT_CACHE_SERVER and indicates the 'next' cache server to fetch from.
198 | 
199 | <pre>
200 |    Figure 2: using a hierarchy of cache servers
201 |    --------------------------------------------
202 |    
203 |     ......................................
204 |     . high bit rate areas                .
205 |     .                                    ...  
206 |     .    +---------------+               . .  
207 |     .    | +---------------+             . ...
208 |     .    +-| +---------------+           . . .
209 |     .      +-|  workstations |           . . .
210 |     .        +---v-------^---+           . . .
211 |     .            |       |               . . .
212 |     .            |       |fetch          . . .
213 |     .            |       |               . . .
214 |     .            |   +---^-------+       . . .
215 |     .        push|   | git-cache |       . . .
216 |     .            |   +---^-------+       . . .
217 |     .            |       |               . . . 
218 |     .............|.......|................ . .
219 |       .          |       |                 . .
220 |       ...........|.v.....|.^................ .
221 |         .        | |     | |                 .
222 |         .........|.|.v...|.|.^................
223 |                  | | |   | | |
224 |                  | | |   | | |
225 |                  | | | +-^-^-^-----+
226 |                  | | | | git-cache |
227 |            pushs | | | +-^---------+
228 |                  | | |   |
229 |                  | | |   | fetch
230 |                  | | |   |
231 |                  | | |   |   medium bitrate area
232 |     .............|.|.|...|.....................
233 |                  | | |   |   slow bitrate area
234 |                  | | |   |
235 |                +-v-v-v---^--+
236 |                | git server |
237 |                +------------+
238 |    
239 | </pre>
240 | 
241 | Security
242 | --------
243 | 
244 | As for any client of SSH, you should be aware of the security holes of ssh.
245 | Please refer to a good lesson about it.
246 | 
247 | FAQ
248 | ---
249 | 
250 | *_Q:_* And for Windows?
251 | 
252 | *_A:_* Hahahahahahahaha
253 | 
254 | *_Q:_* But seriously...
255 | 
256 | *_A:_* Well, fork and let us know the URL to put in place of the laughs.
257 | 
258 | 


--------------------------------------------------------------------------------
/git-cadd:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | # Copyright (c) 2013: José Bollo (jobol), Stéphane Desneux (kooltux)
 4 | # All rights reserved.
 5 | #
 6 | # This program is free software; you can redistribute it and/or modify
 7 | # it under the terms of the GNU General Public License as published by
 8 | # the Free Software Foundation; version 2 of the License.
 9 | #
10 | # This program is distributed in the hope that it will be useful,
11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
13 | # GNU Library General Public License for more details.
14 | #
15 | # You should have received a copy of the GNU General Public License
16 | # along with this program; if not, write to the Free Software
17 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
18 | 
19 | error() { echo "git-cadd ERROR: $*" >&2; exit 1; }
20 | info() { echo "git-cadd $*" >&2; }
21 | 
22 | git branch >/dev/null 2>&1 || error "not in a git repository"
23 | 
24 | [[ -r ~/.gitcache.conf ]] && . ~/.gitcache.conf
25 | 
26 | [[ -z "$GIT_CACHE_SERVER" ]] && error "variable GIT_CACHE_SERVER undefined"
27 | 
28 | declare -a args=("$@")
29 | 
30 | repo="${args[-1]}"
31 | name="${args[-2]}"
32 | 
33 | info "adding remote $name for repo $repo"
34 | 
35 | git remote | grep -q "^$name\$" && error "remote $name already exists, remove it first"
36 | 
37 | info "asking $GIT_CACHE_SERVER a cache for $repo"
38 | cpath=$(ssh -n "$GIT_CACHE_SERVER" gitcache-server --clone "$repo") || error "server failure"
39 | [[ -z "$cpath" ]] && error "can't get a cache"
40 | crepo="${GIT_CACHE_SERVER}:${cpath}"
41 | 
42 | 
43 | info "Upstream repository: $repo"
44 | info "Cache repository: $crepo"
45 | 
46 | args[$((${#args[@]}-1))]="$crepo"
47 | git remote add "${args[@]}" >&2 || error "failure of git remote add ${args[@]}"
48 | 
49 | git remote set-url --push "$name" "$repo" >&2
50 | git config "remote.$name.uploadpack" gitcache-server >&2
51 | 
52 | info "done. Git-cache is active for future fetch operations on remote $name"
53 | 
54 | 


--------------------------------------------------------------------------------
/git-cclone:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | # Copyright (c) 2013: José Bollo (jobol), Stéphane Desneux (kooltux)
 4 | # All rights reserved.
 5 | #
 6 | # This program is free software; you can redistribute it and/or modify
 7 | # it under the terms of the GNU General Public License as published by
 8 | # the Free Software Foundation; version 2 of the License.
 9 | #
10 | # This program is distributed in the hope that it will be useful,
11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
13 | # GNU Library General Public License for more details.
14 | #
15 | # You should have received a copy of the GNU General Public License
16 | # along with this program; if not, write to the Free Software
17 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
18 | 
19 | error() { echo "git-cclone ERROR: $*" >&2; exit 1; }
20 | info() { echo "git-clone $*" >&2; }
21 | 
22 | [[ -r ~/.gitcache.conf ]] && . ~/.gitcache.conf
23 | 
24 | [[ -z "$GIT_CACHE_SERVER" ]] && error "variable GIT_CACHE_SERVER undefined"
25 | 
26 | repo="$1"
27 | dir="${2:-$(basename $repo .git)}"
28 | mirror=$3 # optional --mirror (used by git-crecover) or --bare (can be set by user)
29 | 
30 | [[ -d "$dir" ]] && error "directory $dir exists"
31 | 
32 | info "asking $GIT_CACHE_SERVER a cache for $repo"
33 | cpath=$(ssh -n "$GIT_CACHE_SERVER" gitcache-server --clone "$repo") || error "server failure"
34 | [[ -z "$cpath" ]] && error "can't get a cache"
35 | crepo="${GIT_CACHE_SERVER}:${cpath}"
36 | 
37 | info "Upstream repository: $repo"
38 | info "Cache repository: $crepo"
39 | 
40 | git clone $mirror "$crepo" "$dir" >&2 || error "failure of git clone $mirror $crepo $dir"
41 | 
42 | cd "$dir" || error "can't go to directory $dir"
43 | 
44 | git remote set-url --push origin "$repo" >&2
45 | git config remote.origin.uploadpack gitcache-server >&2
46 | info "done. Git-cache is active for future fetch operations"
47 | 
48 | 


--------------------------------------------------------------------------------
/git-crecover:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | # Copyright (c) 2013: José Bollo (jobol), Stéphane Desneux (kooltux)
 4 | # All rights reserved.
 5 | #
 6 | # This program is free software; you can redistribute it and/or modify
 7 | # it under the terms of the GNU General Public License as published by
 8 | # the Free Software Foundation; version 2 of the License.
 9 | #
10 | # This program is distributed in the hope that it will be useful,
11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
13 | # GNU Library General Public License for more details.
14 | #
15 | # You should have received a copy of the GNU General Public License
16 | # along with this program; if not, write to the Free Software
17 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
18 | 
19 | error() { echo "git-crecover ERROR: $*" >&2; exit 1; }
20 | info() { echo "git-crecover $*" >&2; }
21 | 
22 | git branch >/dev/null 2>&1 || error "not in a git repository"
23 | 
24 | [[ -r ~/.gitcache.conf ]] && . ~/.gitcache.conf
25 | 
26 | origin=
27 | 
28 | setorg() {
29 |     git remote|grep -q "^$1\$" || error "bad remote name $1"
30 |     origin="$1"
31 | }
32 | 
33 | reporg() {
34 |     git remote -v | awk -vn="$origin" '$1==n && $3=="(push)" {print $2}'
35 | }
36 | 
37 | do_recover() {
38 |     local repo=$(reporg)
39 |     info "recovering cache for $repo"
40 |     [[ -z "$GIT_CACHE_SERVER" ]] && error "variable GIT_CACHE_SERVER undefined"
41 |     info "asking $GIT_CACHE_SERVER a cache for $repo"
42 | 
43 |     local cpath=$(ssh -n "$GIT_CACHE_SERVER" gitcache-server --clone "$repo") || error "server failure"
44 |     [[ -z "$cpath" ]] && error "can't get a cache"
45 |     local crepo="${GIT_CACHE_SERVER}:${cpath}"
46 | 
47 |     info "init of git $dir"
48 |     git remote set-url "$origin" "$crepo" >&2
49 |     git remote set-url --push "$origin" "$repo" >&2
50 |     git config "remote.$origin.uploadpack" gitcache-server >&2
51 |     info "done"
52 |     exit 0
53 | }
54 | 
55 | do_remove() {
56 |     local repo=$(reporg)
57 |     info "removing cache reference"
58 |     git config --unset "remote.$origin.uploadpack" >&2
59 |     git remote set-url "$origin" "$repo" >&2
60 |     info "done"
61 |     exit 0
62 | }
63 | 
64 | setorg origin
65 | while [[ $# -gt 0 ]]; do
66 |     case "$1" in
67 | 	-a|--auto) do_recover;;
68 | 	-r|--remove) do_remove;;
69 | 	*) setorg "$1"; shift;;
70 |     esac
71 | done
72 | 
73 | cat << EOC
74 | 
75 | You have two ways of recovering from a cache miss of git-cache.
76 | You can either REMOVE the caching indirection or you can RECOVER
77 | the cache and reset it to the current directory.
78 | 
79 | The target remote is $origin
80 | 
81 | EOC
82 | PS3="What to do? "
83 | select action in REMOVE RECOVER; do
84 |     case $action in
85 | 	REMOVE) do_remove;;
86 | 	RECOVER) do_recover;;
87 |     esac
88 | done
89 | 
90 | 


--------------------------------------------------------------------------------
/gitcache-server:
--------------------------------------------------------------------------------
  1 | #!/bin/bash
  2 | 
  3 | # Copyright (c) 2013: José Bollo (jobol), Stéphane Desneux (kooltux)
  4 | # All rights reserved.
  5 | #
  6 | # This program is free software; you can redistribute it and/or modify
  7 | # it under the terms of the GNU General Public License as published by
  8 | # the Free Software Foundation; version 2 of the License.
  9 | #
 10 | # This program is distributed in the hope that it will be useful,
 11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
 12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 13 | # GNU Library General Public License for more details.
 14 | #
 15 | # You should have received a copy of the GNU General Public License
 16 | # along with this program; if not, write to the Free Software
 17 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 18 | 
 19 | ts_start=$(date +%s)
 20 | op_start=$ts_start
 21 | op_elapsed() { echo $(date +%s) | awk -vts1="$op_start" '{printf("%d",($1-ts1+1));}'; }
 22 | 
 23 | srvid="gitcache-server($(id -un)@$(hostname -f))"
 24 | userip=$(echo $SSH_CONNECTION | awk '{print $1}')
 25 | 
 26 | prefixoutput() { awk -vX="$srvid" -vts1="$ts_start" -vRS='[\r\n]' '{ ts2=strftime("%s"); printf("%s - %ds - %s%s",X,(ts2-ts1+1),$0,RT);fflush();}' >&2; }
 27 | exec 2> >(prefixoutput)
 28 | 
 29 | log() { 
 30 |     [[ -n "$GIT_CACHE_LOG" ]] && echo $(date "+%Y%m%d %H:%M:%S") [$userip] $(op_elapsed)s - "$@" >>$GIT_CACHE_LOG; 
 31 |     op_start=$(date +%s)
 32 | }
 33 | 
 34 | error() { echo "ERROR: $*" >&2; log "ERROR: $*"; exit 1; }
 35 | info() { echo "$*" >&2; }
 36 | 
 37 | [[ -r ~/.gitcache-server.conf ]] && . ~/.gitcache-server.conf
 38 | 
 39 | [[ $# -eq 0 ]] && error "Invalid arguments."
 40 | 
 41 | [[ -z "$GIT_CACHE_DIR" ]] && error "GIT_CACHE_DIR is not defined"
 42 | 
 43 | #####################################################
 44 | # FETCHING (not cloning mean FETCHING)
 45 | #####################################################
 46 | arg2=$2
 47 | if [[ "$1" != "--clone" ]]; then
 48 |     shift $(($# - 1))
 49 |     info using cache directory $1
 50 |     if [[ -d "$1" ]]; then
 51 |         cd "$1" 2>/dev/null || error "directory not found $1"
 52 |         info refreshing cache from $(git ls-remote --get-url origin)
 53 |         # remove the tags to fetch them (for syncing with deleted tags)
 54 |         git tag -l 2>/dev/null | xargs git tag -d >&/dev/null
 55 |         # fetch with prune (for syncing with deleted branches)
 56 |         #git fetch --prune --progress origin +refs/heads/*:refs/remotes/origin/* +refs/changes/*:refs/changes/* >&2
 57 |         git fetch --prune --progress >&2
 58 |         log "fetch => fetch : $1"
 59 |         info uploading changes to client
 60 |         git-upload-pack .
 61 |         log "fetch <= upload : $1"
 62 |         exit
 63 |     else
 64 |         # for any reason, the cache dir has disappeared
 65 |         # rebuild url from directory to clone
 66 |         arg2=$(sed -e "s|$GIT_CACHE_DIR/*||" -e 's|/|:|' <<<$1)
 67 | 	info "Cache directory is not present. Cloning from $arg2"
 68 |     fi
 69 | fi
 70 | 
 71 | #####################################################
 72 | # CLONING (but it may mean fetching...)
 73 | #####################################################
 74 | 
 75 | # get the uri to clone
 76 | uri=$(sed 's/ /%20/g' <<<$arg2)
 77 | info "clone request for $uri"
 78 | 
 79 | # scan the domain and projet parts of the uri
 80 | if [[ $uri =~ ^(.+://)?(.+@)?([^:/]+)(:[0-9]+)?(/.+)$ ]]; then
 81 |     domain=${BASH_REMATCH[3]}
 82 |     project=${BASH_REMATCH[5]}
 83 |     # info "sheme ${BASH_REMATCH[1]} user ${BASH_REMATCH[2]} domain ${BASH_REMATCH[3]} port ${BASH_REMATCH[4]} path ${BASH_REMATCH[5]}"
 84 | elif [[ $uri =~ ^(.+@)?([^:/]+):(.+)$ ]]; then
 85 |     domain=${BASH_REMATCH[2]}
 86 |     project=${BASH_REMATCH[3]}
 87 |     # info "sheme - user ${BASH_REMATCH[1]} domain ${BASH_REMATCH[2]} port - path ${BASH_REMATCH[3]}"
 88 | else
 89 |     error "Bad uri $uri"
 90 | fi
 91 | 
 92 | # enter the cachedir
 93 | if [[ ! -d $GIT_CACHE_DIR ]]; then
 94 |     mkdir -p $GIT_CACHE_DIR || error "can't create $GIT_CACHE_DIR"
 95 | fi
 96 | cd $GIT_CACHE_DIR
 97 | 
 98 | # create the domain/ip root directories
 99 | ip=$(dig +short $domain)
100 | [[ -z "$ip" ]] && error "can't resolve ip for $domain"
101 | 
102 | if [[ ! -d $ip ]]; then
103 |     mkdir -p $ip || error "can't create $(pwd)/$ip"
104 | fi
105 | ln -sf $ip $domain
106 | 
107 | # try to fetch if cached and remove on error
108 | gcdir=$(pwd)/$domain/${project%.git}.git
109 | if [[ -d $gcdir ]]; then
110 |     cd $gcdir
111 |     info "cache hit - refreshing cache from $uri"
112 |     # remove the tags to fetch them (for syncing with deleted tags)
113 |     git tag -l 2>/dev/null | xargs git tag -d >&/dev/null
114 |     # fetch with prune (for syncing with deleted branches)
115 |     #git fetch --prune --progress origin +refs/heads/*:refs/remotes/origin/* +refs/changes/*:refs/changes/* >&2 || { rm -rf $gcdir; info "Fetching cached repo failed"; }
116 |     git fetch --prune --progress >&2 || { rm -rf $gcdir; info "Fetching cached repo failed"; }
117 |     log "clone => fetch : $domain/${project%.git}"
118 | fi
119 | 
120 | # if not cached or had fetch error, clone
121 | if [[ ! -d $gcdir ]]; then
122 |     dir=$(dirname $gcdir)
123 |     [[ -d $dir ]] || mkdir -p $dir || error "can't create directory $dir"
124 |     cd $dir
125 |     edir="$(basename $gcdir)"
126 |     if [[ -n "$GIT_CACHE_SERVER_PRIMARY" ]]; then
127 | 	# clone from a git-cache (chaining)
128 |         export GIT_CACHE_SERVER="$GIT_CACHE_SERVER_PRIMARY"
129 |         info "cache miss - chain-cloning $uri into $gcdir through $GIT_CACHE_SERVER"
130 |         git-cclone "$uri" "$edir" --mirror >&2 || { rm -rf $gcdir; error "chain-cloning failure"; }
131 |         log "clone => cclone : $domain/${project%.git}"
132 |     else
133 | 	# direct clone
134 |         info "cache miss - cloning $uri into $gcdir"
135 |         git clone --progress --mirror "$uri" "$edir" >&2 || { rm -rf $gcdir; error "cloning failure"; }
136 |         log "clone => clone : $domain/${project%.git}"
137 |     fi
138 | fi
139 | 
140 | echo "$gcdir"
141 | exit 0
142 | 
143 | 


--------------------------------------------------------------------------------
/gitcache-server.conf:
--------------------------------------------------------------------------------
 1 | # git-cache server configuration file
 2 | 
 3 | # cache directory
 4 | GIT_CACHE_DIR=~/gitcache
 5 | 
 6 | # primary cache server for chaining (ssh format: [user@]host)
 7 | # GIT_CACHE_SERVER_PRIMARY=git@primarycache.example.com
 8 | 
 9 | # log file (log is disabled if variable is not set)
10 | #GIT_CACHE_LOG=$GIT_CACHE_DIR/gitcache.log
11 | 


--------------------------------------------------------------------------------
/gitcache.conf:
--------------------------------------------------------------------------------
1 | # git-cache configuration file for clients
2 | 
3 | # git-cache server address (ssh format: [user@]host)
4 | GIT_CACHE_SERVER=git@localcache.example.com
5 | 
6 | 


--------------------------------------------------------------------------------