├── .gitignore ├── Dockerfile ├── README.md ├── backup.py └── docker-compose.yml /.gitignore: -------------------------------------------------------------------------------- 1 | backup 2 | borg 3 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM debian:buster 2 | 3 | RUN apt-get update && \ 4 | apt-get -yq --no-install-recommends install \ 5 | borgbackup \ 6 | python3-docker && \ 7 | apt-get clean && \ 8 | rm -rf /var/tmp/* /tmp/* /var/lib/apt/lists/* && \ 9 | mkdir -p /borg/ssh && \ 10 | ln -s /root/.ssh /borg/ssh 11 | 12 | ENV BORG_BASE_DIR /borg 13 | ENV BORG_CONFIG_DIR /borg/config 14 | ENV BORG_CACHE_DIR /borg/cache 15 | ENV BORG_SECURITY_DIR /borg/config/security 16 | ENV BORG_KEYS_DIR /borg/config/keys 17 | 18 | ENV BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK yes 19 | ENV BORG_RELOCATED_REPO_ACCESS_IS_OK yes 20 | 21 | VOLUME /borg 22 | 23 | COPY ./backup.py /backup.py 24 | ENTRYPOINT [ "/backup.py" ] 25 | CMD [ "backup" ] 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Docker-BorgBackup 2 | Full Automated Container-Backup - Docker Style! 3 | 4 | **WARNING: This image is in ALPHA/POC-State! Don't rely your production backup on it!** 5 | **Also, since this Container could be EXTREMLY destructive, don't keep it running in background!** 6 | 7 | ## ToDo's/Limitations/...: 8 | - Implement Pruning / Repo Cleanup methods 9 | - Pre-/Post-Commands f.e. to dump a Database before Backup 10 | - Only "local"-Volumes supported 11 | - Restore whole containers only 12 | - also still needs a "prune before restore" method 13 | - ... (much more) 14 | 15 | ## Features 16 | - Full automated [BorgBackup](https://borgbackup.readthedocs.io/en/stable/) of all volumes of all running containers 17 | - Full automated restore of whole containers to a given backup-state 18 | - Flexible configuration using Environment-Variables & Labels 19 | 20 | ## Building / Installation 21 | ### Using Docker-Hub Image 22 | There are three requirements to run this container: 23 | - Environemnt-Variable BORG_REPO must be configured 24 | - All pathes to volumes, MUST be mounted to this container (f.e. /var/lib/docker/volumes/) 25 | - The Docker-Socket must be mounted to the container 26 | 27 | Since it's a lot to include in a single command, I will only show how to use this container in combination with docker-compose. 28 | 29 | ### Self-Building 30 | Clone git & build using docker/-compose: 31 | ``` 32 | git clone https://github.com/nold360/docker-borgbackup 33 | cd docker-borgbackup 34 | docker build . 35 | ``` 36 | 37 | ## Docker-Compose Configuration 38 | Example: 39 | ``` 40 | version: "3" 41 | services: 42 | backup: 43 | #build: . 44 | image: nold360/docker-borgbackup 45 | labels: 46 | # Don't backup your backup... 47 | one.gnu.docker.backup: "False" 48 | environment: 49 | #BORG_REPO MUST be set 50 | BORG_REPO: "/backup" 51 | 52 | # Those are default-values: 53 | BORG_INIT_OPTIONS: "--encryption=none" 54 | BORG_CREATE_OPTIONS: "-s --progress" 55 | BORG_SKIP_VOLUME_SOURCES: "/proc,/sys,/var/run,/var/cache,/var/tmp" 56 | BORG_BACKUP_ALL: "True" 57 | BORG_BREAK_LOCK: "True" 58 | volumes: 59 | # needed for SSH-BORG_REPO: 60 | - "./ssh:/ssh:ro" 61 | 62 | # needed for local BORG_REPO: 63 | - "./backup:/backup" 64 | 65 | # needed for connecting the Docker-API 66 | - "/var/run/docker.sock:/var/run/docker.sock" 67 | 68 | # Include every volume-Path you might want to backup! 69 | - "/srv:/srv" 70 | - "/var/lib/docker:/var/lib/docker" 71 | 72 | ``` 73 | 74 | ### Running the Container 75 | After setting up docker-compose.yml we can run our backup: 76 | ``` 77 | docker-compose run --rm backup 78 | ``` 79 | 80 | ## Global-/ Borg-Configuration (using Environment) 81 | #### BORG_REPO 82 | Path/Definition of the Borg-Repository. Can be a local path to a container volume (f.e. "/backup") or an SSH-Borg-Server (f.e. "borg@backup.domain:myRepo"). 83 | 84 | **Note:** When using SSH-Repo make sure to setup "/ssh" with ssh-key (and maybe known_hosts/config)! 85 | 86 | **Default:** None (**Must be configured!**) 87 | 88 | #### BORG_INIT_OPTIONS 89 | Options to pass to "borg init" when creating a new BORG_REPO 90 | **Default:** "--encryption=none" 91 | 92 | #### BORG_CREATE_OPTIONS 93 | Options to pass to "borg create" 94 | **Default:** "-s --progress" 95 | 96 | #### BORG_SKIP_VOLUME_SOURCES 97 | List of Volume-Pathes to exclude globaly from backup. 98 | **Default:** "/proc,/sys,/var/run,/var/cache,/var/tmp" 99 | 100 | #### BORG_BACKUP_ALL 101 | Backup every container? 102 | **Default:** "True" 103 | 104 | #### BORG_BREAK_LOCK 105 | Forces "borg break-lock" before backing up; Helps recovering from aborted backups 106 | **Warning:** Can be **ultimate destructive**, when more then one borg-process is using the same BORG_REPO! 107 | 108 | **Default:** "True" 109 | 110 | 111 | ## Client Configuration (using labels) 112 | ### one.gnu.docker.backup 113 | Backup this container? 114 | **Values:** "True|False" 115 | 116 | ### one.gnu.docker.backup.only 117 | Only backup volumes specified in this comma-separated list: 118 | **Example:** "/data/foo,/bar" 119 | 120 | ### one.gnu.docker.backup.skip 121 | Skip volumes specified in this comma-seperated list: 122 | **Example:** "/skip/me,/too" 123 | 124 | ### one.gnu.docker.backup.options 125 | Options/Parameters to pass to "borg create" 126 | **Example:** "-v --stats" 127 | 128 | 129 | ## Using this Container 130 | Included in this container is a Python-Wrapper script called just "backup". 131 | It implements the main backup/restore tasks. 132 | 133 | ### Overview 134 | ``` 135 | docker-compose run backup --help 136 | 137 | -------- Borg Docker Backup -------- 138 | usage: backup.py [-h] {restore,list,backup,info,borg} ... 139 | 140 | Awesome Borg-Backup for Docker made simple! 141 | 142 | positional arguments: 143 | {restore,list,backup,info,borg} 144 | Sub-Commands 145 | restore Restore Container from Backup 146 | list List archives / files in archive 147 | backup Backup all / a single container 148 | info Show infos about repo / archive 149 | borg Run every board-command you like (not yet working) 150 | 151 | optional arguments: 152 | -h, --help show this help message and exit 153 | 154 | ``` 155 | 156 | ### Backup 157 | The wrapper will collect all volumes mapped to a container and create a single backup archive for every container. 158 | The archives will be named like this: `my_container_name+2018-04-14_08:42` 159 | 160 | #### Help 161 | ``` 162 | # docker-compose run backup backup -h 163 | -------- Borg Docker Backup -------- 164 | usage: backup.py backup [-h] [container] 165 | 166 | positional arguments: 167 | container Name of the Container to backup (default: all) 168 | 169 | ``` 170 | 171 | #### Backing up all Containers 172 | Simply run the container without any arguments: 173 | ``` 174 | docker-compose run backup 175 | ``` 176 | 177 | #### Backing up single container 178 | ``` 179 | docker-compose run backup my_container_name 180 | ``` 181 | 182 | ### Restore 183 | Restoring a whole container is as simple as backing it up. You just need the name the the backup archive (see "list" subcommand). 184 | 185 | ``` 186 | docker-compose run backup restore my_container_name+2018-04-14_08:42 187 | ``` 188 | 189 | The wrapper will automatically: 190 | - Pause the running container 191 | - Restore the backup from BORG_REPO 192 | - Restart the container 193 | 194 | 195 | 196 | #### Help 197 | ``` 198 | # docker-compose run backup restore -h 199 | -------- Borg Docker Backup -------- 200 | usage: backup.py restore [-h] archive 201 | 202 | positional arguments: 203 | archive Archive to Restore container from 204 | 205 | optional arguments: 206 | -h, --help show this help message and exit 207 | ``` 208 | 209 | 210 | ### List Backup-Archives 211 | ``` 212 | docker-compose run backup list 213 | ``` 214 | 215 | -------------------------------------------------------------------------------- /backup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import docker 4 | import subprocess 5 | import argparse 6 | from sys import exit 7 | from os import environ 8 | 9 | print("-------- Borg Docker Backup --------") 10 | parser = argparse.ArgumentParser(description='Awesome Borg-Backup for Docker made simple!') 11 | 12 | subparsers = parser.add_subparsers(help='Sub-Commands', dest='action') 13 | restore_parser = subparsers.add_parser('restore', help='Restore Container from Backup') 14 | restore_parser.add_argument("archive", help='Archive to Restore container from') 15 | 16 | list_parser = subparsers.add_parser('list', help='List archives / files in archive') 17 | list_parser.add_argument("archive", nargs='?', default=None, help='Name of the archive') 18 | 19 | backup_parser = subparsers.add_parser('backup', help='Backup all / a single container') 20 | backup_parser.add_argument("container", nargs='?', default=None, help='Name of the Container to backup (default: all)') 21 | 22 | info_parser = subparsers.add_parser('info', help="Show infos about repo / archive") 23 | info_parser.add_argument("archive", nargs='?', default=None, help='Archive to show') 24 | 25 | borg_parser = subparsers.add_parser('borg', help="Run every board-command you like") 26 | borg_parser.add_argument("borg", nargs='+', default=None, help='Borg Parameters') 27 | 28 | args = parser.parse_args() 29 | 30 | ### 31 | # Global configuration class 32 | class Config: 33 | # Default values 34 | excludes = [] 35 | create_options = ['--stats'] 36 | init_options = ['--encryption=none'] 37 | backup_enabled = True 38 | break_lock = True 39 | hostname = "" 40 | 41 | # Connect to Docker Socket 42 | client = docker.from_env() 43 | 44 | # Read Environment Configuration 45 | def __init__(self): 46 | # This should never fail anyways... 47 | self.hostname = environ["HOSTNAME"] 48 | try: 49 | # $BORG_REPO will be parsed by borg directly 50 | if environ["BORG_REPO"]: 51 | pass 52 | except: 53 | print("ERROR: Environment Variable BORG_REPO must be configured!") 54 | exit(1) 55 | 56 | # Borg INIT options 57 | try: 58 | if environ["BORG_INIT_OPTIONS"]: 59 | self.init_options = environ["BORG_INIT_OPTIONS"].split(' ') 60 | except: 61 | print("Environment variable BORG_INIT_OPTIONS unconfigured.") 62 | print(" --> Borg will create repositories without encryption!") 63 | 64 | try: 65 | if environ["BORG_CREATE_OPTIONS"]: 66 | self.create_options = environ["BORG_CREATE_OPTIONS"].split(' ') 67 | print("Global BORG_CREATE_OPTIONS: %s" % environ["BORG_CREATE_OPTIONS"]) 68 | except: 69 | print("Environment variable BORG_CREATE_OPTIONS unconfigured.") 70 | print(" --> Borg will create backups with default settings.") 71 | 72 | try: 73 | if environ["BORG_EXCLUDE_SOURCE"]: 74 | self.excludes = environ["BORG_EXCLUDE_SOURCE"].split(',') 75 | print("Global BORG_EXCLUDE_SOURCE: %s" % environ["BORG_EXCLUDE_SOURCE"]) 76 | except: 77 | pass 78 | 79 | # Always exclude those 80 | self.excludes += [ '/proc', '/sys', '/var/run', '/var/cache', '/var/tmp' ] 81 | self.excludes.append('/var/run/docker.sock') 82 | 83 | # Will we backup every container? 84 | # If "False" only backup containers by Label-Config 85 | try: 86 | if "False" in environ["BORG_BACKUP_ALL"]: 87 | self.backup_enabled = False 88 | except: 89 | pass 90 | 91 | # Force Lock-break 92 | # Warning! Only use if BORG_REPO is exclusvily used by this container! 93 | try: 94 | if not "False" in environ["BORG_BREAK_LOCK"]: 95 | self.break_lock = False 96 | except: 97 | pass 98 | 99 | ### 100 | # Borg Command Class 101 | class borg: 102 | def cmd(params): 103 | command = ["borg"] + params 104 | proc = subprocess.Popen(command,stdout=subprocess.PIPE) 105 | for line in proc.stdout: 106 | print("> %s" % line.rstrip()) 107 | 108 | @staticmethod 109 | def create(options, name, volumes): 110 | borg.cmd(['create'] + options + ["::" + name + '+{now:%Y-%m-%d_%H:%M}' ] + volumes) 111 | 112 | @staticmethod 113 | def init(options): 114 | borg.cmd(['init'] + options) 115 | 116 | @staticmethod 117 | def list(archive=None): 118 | options = [] 119 | if archive != None: 120 | options = ["::" + archive] 121 | borg.cmd(['list'] + options) 122 | 123 | @staticmethod 124 | def restore(archive): 125 | borg.cmd(["extract", archive]) 126 | 127 | @staticmethod 128 | def break_lock(): 129 | borg.cmd(["break-lock"]) 130 | 131 | @staticmethod 132 | def info(archive=None): 133 | options = [] 134 | if archive != None: 135 | options = ["::" + archive] 136 | borg.cmd(['info'] + options) 137 | 138 | ### 139 | # This class implements all the important stuff 140 | class Action: 141 | @staticmethod 142 | def backup(config, container_name=None): 143 | if config.break_lock: 144 | borg.break_lock() 145 | 146 | borg.init(config.init_options) 147 | 148 | print("Starting backup of container volumes - this could take a while...") 149 | for container in config.client.containers.list(): 150 | if container_name != None and container_name != container.name: 151 | continue 152 | 153 | # FIXME: Skip myself 154 | # Will use label ATM 155 | try: 156 | if 'False' == container.labels["one.gnu.docker.backup"]: 157 | print("Skipping backup of '%s', because of label configuration!" % container.name) 158 | continue 159 | except: 160 | pass 161 | 162 | # Skip if backup is disabled by default 163 | try: 164 | if not config.backup_enabled and \ 165 | not "True" in container.labels["one.gnu.docker.backup"]: 166 | continue 167 | except: 168 | pass 169 | 170 | print("---------------------------------------") 171 | print("Backing up: '%s'..." % container.name) 172 | print("Volumes:") 173 | 174 | # Only backup specified volumes/files/...? 175 | try: 176 | if container.labels["one.gnu.docker.backup.only"]: 177 | volumes_only = container.labels["one.gnu.docker.backup.only"].split(',') 178 | except: 179 | volumes_only = False 180 | 181 | volumes=[] 182 | for volume in container.attrs["Mounts"]: 183 | volume_src = volume["Source"] 184 | volume_dest = volume["Destination"] 185 | volume_type = volume["Type"] 186 | 187 | # Skip containers on unsupported driver 188 | # Supported: 189 | # - local 190 | try: 191 | volume_driver = volume["Driver"] 192 | if not 'local' in volume_driver: 193 | print(" - '%s' [%s] (Skipped - Unsupported driver '%s')" % \ 194 | (volume_dest, volume_src, volume_driver)) 195 | continue 196 | except: 197 | pass 198 | 199 | # Skip volume by label 200 | # Syntax: one.gnu.docker.backup.skip: "/mountpoint1,/mountpoint2, ..." 201 | try: 202 | if container.labels["one.gnu.docker.backup.skip"]: 203 | skip = container.labels["one.gnu.docker.backup.skip"].split(",") 204 | except: 205 | skip = [] 206 | 207 | if volume_dest in skip: 208 | print(" - %s [%s] (skipped by label)" % (volume_dest, volume_src)) 209 | continue 210 | 211 | # Skip volume if label configuration tells us to only backup 212 | # specified volume destinations 213 | if volumes_only: 214 | if volume_dest not in volumes_only: 215 | continue 216 | 217 | volumes.append(volume_src) 218 | print(" - %s [%s]" % (volume_dest, volume_src)) 219 | 220 | # Borg-Parameters 221 | # - From Environment of this container 222 | # - Override by Container-Labels 223 | try: 224 | if container.labels["one.gnu.docker.backup.options"]: 225 | config.create_options = container.labels["one.gnu.docker.backup.options"] 226 | except: 227 | config.create_options = ['-s', '--progress'] 228 | 229 | # FIXME: Skip volume, if source is not mounted to this container 230 | # FIXME: Exclude Source & Destinations 231 | for exclude in config.excludes: 232 | config.create_options += ['--exclude', exclude] 233 | 234 | 235 | # Let's do it! 236 | if len(volumes) > 0: 237 | borg.create(config.create_options, container.name, volumes) 238 | else: 239 | print("Skipping Container - Nothing to do...") 240 | 241 | @staticmethod 242 | def list_backups(archive): 243 | borg.list(archive) 244 | 245 | @staticmethod 246 | def restore(config,archive): 247 | container_name = archive.split('+')[0] 248 | 249 | print(" ----> Starting Restore of Containter %s" % container_name) 250 | container = config.client.containers.get(container_name) 251 | 252 | print(" -> Pausing Container...") 253 | if "running" in container.status: 254 | container.pause() 255 | 256 | print(" -> Restoring Archive '%s'..." % archive) 257 | borg.restore("::" + archive) 258 | 259 | print(" -> Restarting Container...") 260 | container.restart() 261 | 262 | print("-> Restore Done!") 263 | 264 | @staticmethod 265 | def info(archive): 266 | borg.info(archive) 267 | 268 | config = Config() 269 | 270 | if args.action == "backup": 271 | Action.backup(config, args.container) 272 | elif args.action == "list": 273 | Action.list_backups(args.archive) 274 | elif args.action == "restore": 275 | Action.restore(config, args.archive) 276 | elif args.action == "info": 277 | Action.info(args.archive) 278 | elif args.action == "borg": 279 | borg.cmd(args.borg) 280 | 281 | exit(0) 282 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3" 2 | services: 3 | backup: 4 | # image: nold360/docker-borgbackup 5 | build: . 6 | labels: 7 | one.gnu.docker.backup: "False" 8 | # one.gnu.docker.backup.only: "/data,/more/data" 9 | # one.gnu.docker.backup.skip: "/more/tmp" 10 | # one.gnu.docker.backup.options: "-v --stats" 11 | environment: 12 | BORG_REPO: "/backup" 13 | # BORG_INIT_OPTIONS: "--encryption=none" 14 | # BORG_CREATE_OPTIONS: "-s --progress" 15 | # BORG_SKIP_VOLUME_SOURCES: "/proc,/sys,/var/run,/var/cache,/var/tmp" 16 | # BORG_BACKUP_ALL: "True" 17 | # BORG_BREAK_LOCK: "True" 18 | volumes: 19 | - "./borg:/borg" 20 | - "./backup:/backup" 21 | - "/var/run/docker.sock:/var/run/docker.sock" 22 | # - "/srv/:/srv/" 23 | # - "/var/lib/docker/:/var/lib/docker" 24 | --------------------------------------------------------------------------------