├── .gitignore ├── Dockerfile ├── README.md ├── _entrypoint.sh ├── build-squid ├── _make.sh ├── make.sh └── rules.patch ├── build.sh ├── run.sh └── squid.conf.template /.gitignore: -------------------------------------------------------------------------------- 1 | squid.conf 2 | *.deb -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | 3 | # These must be built with ssl support! Use build-squid to do this... 4 | ADD /squid-common.deb /root/squid-common.deb 5 | ADD /squid.deb /root/squid.deb 6 | 7 | RUN apt-get update && apt-get install -y squid libssl1.0.0 && \ 8 | dpkg -i /root/squid-common.deb && dpkg -i /root/squid.deb && \ 9 | rm -rf /var/lib/apt/lists/* 10 | 11 | ADD /squid.conf /etc/squid/squid.conf 12 | ADD /_entrypoint.sh /sbin/entrypoint.sh 13 | 14 | CMD /sbin/entrypoint.sh -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | docker-registry-cache 2 | ===================== 3 | 4 | This is a pull-through docker private registry cache implemented using a 5 | Squid HTTP proxy. 6 | 7 | **WARNING**: This only works with the modern [docker registry API (v2+)](https://docs.docker.com/registry/spec/api/). This will not work with docker registries that use the legacy v1 API. 8 | 9 | What is this useful for? 10 | ------------------------ 11 | 12 | Bottom line, it's a HTTP cache. 13 | 14 | * Data center A contains your private registry, and DC B is connected to it over a slow link. Run an instance of this cache on DC B's network, and anytime you pull from it the second time will be significantly faster since you don't have to traverse that slow link as much! 15 | * I'm sure there are other usecases.. 16 | 17 | Requirements 18 | ============ 19 | 20 | You must have docker installed. You must create an SSL certificate and key for 21 | your proxy (out of scope for this guide -- google it yourself). 22 | 23 | SSL-enabled squid 24 | ----------------- 25 | 26 | You must have an SSL-enabled version of squid. Currently, Ubuntu's squid 27 | package is not ssl enabled due to licensing issues. To build your own, do this: 28 | 29 | ```console 30 | $ build-squid/make.sh 31 | ``` 32 | 33 | After about 30 minutes or so, you should have a bunch of deb packages in 34 | the `build-squid` directory. 35 | 36 | Build the cache server image 37 | ---------------------------- 38 | 39 | First, copy squid.conf.template to squid.conf, and change the variables. 40 | 41 | * `{{ docker_host }}` is your private docker registry host 42 | * `{{ cache_size }}` should be set to the cache size, in MB. According to the 43 | squid docs, this should not exceed 80% of the disk. For example, 500000 is 44 | 500GB. 45 | 46 | Next, build the image: 47 | 48 | ```console 49 | $ ./build.sh 50 | ``` 51 | 52 | Setup 53 | ===== 54 | 55 | * Create an empty directory for your cache. 56 | * Create a directory for your SSL certificates, and place them in a directory, 57 | with the certificate called 'cert.pem' and the private key called 'key.pem' 58 | 59 | ```console 60 | $ ./run.sh /path/to/ssl /path/to/cache 61 | ``` 62 | 63 | The registry will start, and be listening on port 443. It should restart on 64 | bootup if your docker daemon is running. 65 | 66 | Registry authentication 67 | ----------------------- 68 | 69 | https://github.com/virtuald/docker-registry-cache/pull/7 added a configuration option 70 | to squid that passes basic authentication thru to the remote registry. This should 71 | work without any further configuration. 72 | 73 | It is unknown if other authentication schemes will work with this cache. Feel free 74 | to try it out and leave a note with what you find out! 75 | 76 | Using the cache 77 | =============== 78 | 79 | Once the registry cache starts, you pull from it like you would pull from a 80 | normal docker registry -- but you pull from the cache hostname, not your 81 | original private registry hostname! 82 | 83 | ```console 84 | $ docker pull CACHE_HOST/foo/bar:latest 85 | ``` 86 | 87 | Bugs 88 | ==== 89 | 90 | I'm sure this isn't ideal, but it seems to work. Submit issues and PRs! 91 | 92 | License 93 | ======= 94 | 95 | The contents of this repository are available under the Apache v2 license. 96 | 97 | 98 | Author 99 | ====== 100 | 101 | Dustin Spicuzza (dustin@virtualroadside.com) 102 | -------------------------------------------------------------------------------- /_entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Inspired by docker-squid 3 | 4 | if [[ ! -d /cache/cc ]]; then 5 | mkdir /cache/cc 6 | chown proxy:proxy /cache/cc 7 | /usr/sbin/squid3 -N -z 8 | fi 9 | 10 | echo "Starting squid..." 11 | /usr/sbin/squid3 -NYCd 1 -------------------------------------------------------------------------------- /build-squid/_make.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | 3 | #### ensure deb-src is enabled 4 | sed -i 's/^#\s*\(deb-src.*main.*\)$/\1/g' /etc/apt/sources.list 5 | 6 | cd /build 7 | 8 | apt-get update 9 | apt-get install -y build-essential fakeroot devscripts libssl-dev 10 | apt-get build-dep -y squid 11 | 12 | apt-get source squid 13 | 14 | cd squid3* 15 | patch -p0 < /source/rules.patch 16 | 17 | debuild -us -uc -i -I 18 | 19 | cd .. 20 | cp *.deb /source 21 | -------------------------------------------------------------------------------- /build-squid/make.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | 3 | cd $(dirname $0) 4 | docker run -it -v $(pwd):/source -v /build ubuntu:16.04 /source/_make.sh 5 | 6 | -------------------------------------------------------------------------------- /build-squid/rules.patch: -------------------------------------------------------------------------------- 1 | --- debian/rules 2016-07-07 14:24:20.000000000 -0400 2 | +++ debian/rules.new 2016-07-07 14:25:11.000000000 -0400 3 | @@ -20,6 +20,7 @@ 4 | --sysconfdir=/etc/squid \ 5 | --libexecdir=/usr/lib/squid \ 6 | --mandir=/usr/share/man \ 7 | + --enable-ssl --with-openssl \ 8 | --enable-inline \ 9 | --disable-arch-native \ 10 | --enable-async-io=8 \ 11 | -------------------------------------------------------------------------------- /build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | 3 | cd $(dirname $0) 4 | 5 | if [[ ! -f squid.conf ]]; then 6 | echo "You must copy squid.conf.template AND modify it! See README.md" 7 | exit 1 8 | fi 9 | 10 | if [[ "$(grep -c '{{ docker_host }}' squid.conf)" != "0" ]]; then 11 | echo "You must change {{ docker_host }} in squid.conf" 12 | exit 1 13 | fi 14 | 15 | if [[ "$(grep -c '{{ cache_size }}' squid.conf)" != "0" ]]; then 16 | echo "You must change {{ cache_size }} in squid.conf" 17 | exit 1 18 | fi 19 | 20 | cp build-squid/squid-common*.deb squid-common.deb 21 | cp build-squid/squid_*.deb squid.deb 22 | 23 | docker build -t registry-cache:latest . 24 | -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [[ "$1" == "" ]]; then 4 | echo "Usage: $0 ssl_directory cache_directory" 5 | exit 1 6 | fi 7 | 8 | SSL_DIRECTORY="$1" 9 | CACHE_DIRECTORY="$2" 10 | 11 | docker run -d --restart=always -p 443:443 --name registry-cache \ 12 | -v ${SSL_DIRECTORY}:/etc/ssl/private \ 13 | -v ${CACHE_DIRECTORY}:/cache \ 14 | registry-cache:latest 15 | -------------------------------------------------------------------------------- /squid.conf.template: -------------------------------------------------------------------------------- 1 | https_port 443 accel defaultsite={{ docker_host }} no-vhost cert=/etc/ssl/private/cert.pem key=/etc/ssl/private/key.pem 2 | 3 | # Uncomment this if you're having caching issues... the logfile is 4 | # written to /var/log/squid/cache.log 5 | #debug_options ALL,2 6 | 7 | maximum_object_size 8 GB 8 | range_offset_limit 8 GB 9 | quick_abort_min -1 10 | cache_dir ufs /cache/cc {{ cache_size }} 16 256 11 | 12 | cache_peer {{ docker_host }} parent 443 0 no-query originserver no-digest name=upstream login=PASSTHRU ssl sslflags=DONT_VERIFY_PEER 13 | 14 | acl site dstdomain {{ docker_host }} 15 | http_access allow site 16 | cache_peer_access upstream allow site 17 | cache_peer_access upstream deny all 18 | cache allow site 19 | --------------------------------------------------------------------------------