├── .gitignore ├── restic_snapshots.sh ├── restic_excludelist.txt ├── restic_key_add_password.sh ├── restic_key_list.sh ├── restic_key_change_password.sh ├── restic_key_remove_password.sh ├── restic_cache_cleanup.sh ├── restic_initialize_repo.sh ├── restic_unlock.sh ├── restic_prune.sh ├── restic_stats.sh ├── restic_rebuild_index.sh ├── restic_findtree_byid.sh ├── restic_forget_byid.sh ├── restic_mount.sh ├── restic_forgetandprune_2weeks.sh ├── restic_env.sh ├── restic_backup.sh ├── restic_check.sh ├── restic_forgetandprune_bypolicy.sh ├── LICENSE └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | cache/* 2 | logs/* 3 | mount/* 4 | sources/* 5 | 6 | -------------------------------------------------------------------------------- /restic_snapshots.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | restic snapshots 7 | -------------------------------------------------------------------------------- /restic_excludelist.txt: -------------------------------------------------------------------------------- 1 | # exclude testuser account in cloud (often used for temporary file transfers and not necessary to back up permanently) 2 | /home/user/restic/sources/cloud/testuser 3 | /home/user/restic/sources/cloud/anotheruser 4 | -------------------------------------------------------------------------------- /restic_key_add_password.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | . /home/edaly/restic_backup/restic_unlock.sh 4 | 5 | echo "-" 6 | echo "-" 7 | echo "Adding password" 8 | echo "---------------------------" 9 | 10 | restic key add 11 | 12 | echo "" 13 | -------------------------------------------------------------------------------- /restic_key_list.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | . /home/edaly/restic_backup/restic_unlock.sh 4 | 5 | echo "-" 6 | echo "-" 7 | echo "Listing current keys" 8 | echo "---------------------------" 9 | 10 | restic key list 11 | 12 | echo "" 13 | -------------------------------------------------------------------------------- /restic_key_change_password.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | . /home/edaly/restic_backup/restic_unlock.sh 4 | 5 | echo "-" 6 | echo "-" 7 | echo "Changing password" 8 | echo "---------------------------" 9 | 10 | restic key passwd 11 | 12 | echo "" 13 | -------------------------------------------------------------------------------- /restic_key_remove_password.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | . /home/edaly/restic_backup/restic_unlock.sh 4 | 5 | echo "-" 6 | echo "-" 7 | echo "Removing password $1" 8 | echo "---------------------------" 9 | 10 | restic key remove $1 11 | 12 | echo "" 13 | -------------------------------------------------------------------------------- /restic_cache_cleanup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo "-" 4 | echo "-" 5 | echo "Cleaning up restic cache ($(date))..." 6 | echo "---------------------------" 7 | 8 | restic cache --cleanup 9 | 10 | echo "" 11 | echo "Finished cleaning cache $(date)" 12 | echo "---------------------------" 13 | -------------------------------------------------------------------------------- /restic_initialize_repo.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | . /home/edaly/restic_backup/restic_env.sh 4 | 5 | echo "-" 6 | echo "-" 7 | echo "Initializing new repo" 8 | echo "---------------------------" 9 | 10 | restic init 11 | 12 | echo "" 13 | echo "Finished initializing $(date)" 14 | echo "---------------------------" 15 | -------------------------------------------------------------------------------- /restic_unlock.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | . /home/edaly/restic_backup/restic_env.sh 4 | 5 | echo "-" 6 | echo "-" 7 | echo "Attempting to unlock restic repo" 8 | echo "---------------------------" 9 | 10 | restic unlock --cleanup-cache 11 | 12 | echo "" 13 | echo "Finished unlocking $(date)" 14 | echo "---------------------------" 15 | -------------------------------------------------------------------------------- /restic_prune.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Pruning backup ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic prune 12 | 13 | echo "" 14 | echo "Finished pruning $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_stats.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Getting backup stats for latest snapshot ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic stats --mode restore-size latest 12 | restic stats --mode raw-data latest 13 | -------------------------------------------------------------------------------- /restic_rebuild_index.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Rebuilding index ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic rebuild-index 12 | 13 | echo "" 14 | echo "Finished rebuilding index $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_findtree_byid.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Finding tree with id: $1 ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic find --tree $1 12 | 13 | echo "" 14 | echo "Finished finding tree $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_forget_byid.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Forgetting snapshot with id: $1... ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic forget $1 12 | 13 | echo "" 14 | echo "Finished forgetting $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_mount.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Attempting to mount backup to $1 ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic mount ./mount 12 | 13 | echo "" 14 | echo "Finished mounting backup at $1 $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_forgetandprune_2weeks.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Pruning backup keeping only last 14 days ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic forget --keep-within 14d --prune 12 | 13 | echo "" 14 | echo "Finished pruning $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_env.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Backblaze B2 credentials 4 | export B2_ACCOUNT_ID= 5 | export B2_ACCOUNT_KEY= 6 | 7 | # restic variables 8 | export RESTIC_REPOSITORY= 9 | export RESTIC_PASSWORD= 10 | export RESTIC_INCLUDE_PATHS="/home/user/restic/sources/nas/ /home/user/restic/sources/cloud" 11 | export RESTIC_EXCLUDE_FILE_PATH="/home/user/restic/excludelist.txt" 12 | export RESTIC_CACHE_DIR="/home/user/restic/cache" 13 | export B2_CONNECTIONS=50 14 | -------------------------------------------------------------------------------- /restic_backup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Running scheduled backup ($(date))..." 9 | echo "---------------------------" 10 | 11 | restic -o b2.connections=$B2_CONNECTIONS backup --exclude-file=$RESTIC_EXCLUDE_FILE_PATH $RESTIC_INCLUDE_PATHS 12 | 13 | echo "" 14 | echo "Finished scheduled backup $(date)" 15 | echo "---------------------------" 16 | -------------------------------------------------------------------------------- /restic_check.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # make Go garbage collector very aggresive in order to try to prevent out of memory issues 4 | export GOGC=20 5 | 6 | # run unlock script first (includes setting env vars) 7 | . /home/edaly/restic_backup/restic_unlock.sh 8 | 9 | echo "-" 10 | echo "-" 11 | echo "Checking backup ($(date))..." 12 | echo "---------------------------" 13 | 14 | restic check --cache-dir "${RESTIC_CACHE_DIR}" --cleanup-cache 15 | 16 | echo "" 17 | echo "Finished checking backup $(date)" 18 | echo "---------------------------" 19 | df -h 20 | -------------------------------------------------------------------------------- /restic_forgetandprune_bypolicy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # run unlock script first (includes setting env vars) 4 | . /home/edaly/restic_backup/restic_unlock.sh 5 | 6 | echo "-" 7 | echo "-" 8 | echo "Policy: --keep-daily 30 --keep-weekly 8 --keep-monthly 12 --keep-yearly 100 --keep-last 4" 9 | echo "Forgetting and pruning snapshots according to retention policy ($(date))..." 10 | echo "---------------------------" 11 | 12 | restic forget --keep-daily 30 --keep-weekly 8 --keep-monthly 12 --keep-yearly 100 --keep-last 4 --prune 13 | 14 | echo "" 15 | echo "Finished forget and prune $(date)" 16 | echo "---------------------------" 17 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # restic-scripts 2 | shell scripts for restic to simplify backup/check/restore and other operations and simplify cron scheduling 3 | 4 | These are bash scripts designed to be run under Linux, so I offer no support for Windows or Mac based systems although feel free to modify these scripts for your own purposes! I am not opposed to anyone posting issues or pull requests or just comments or questions! 5 | 6 | I exclusively run restic with Backblaze B2 and use the B2 API as opposed to the S3 API because when I originally set these up Backblaze did not offer an S3 API. If I could do it again today I would most definitely use the S3 API (but I'm sure it would be fairly trivial to adapt). 7 | 8 | The main script is `restic_backup.sh` which you will probably want to schedule with cron. Most other scripts are to make maintenance/troubleshooting tasks easier. You can optionally schedule `restic_check.sh` and `restic_forgetandprune_bypolicy.sh`. 9 | 10 | # More Documentation 11 | [Check out my blog post about how I use these scripts and some of the use cases](https://blog.dalydays.com/post/2021-10-20-using-restic-with-backblaze-b2/) 12 | 13 | # Getting Started 14 | - Make sure your restic machine has a sufficient amount of RAM and CPU power depending on your backup size and number of files. Also make sure you have enough local filesystem storage for the restic cache (see `RESTIC_CACHE_DIR` below). 15 | - Install restic. I prefer to just download the latest stable binary from Github: [https://github.com/restic/restic/releases/](https://github.com/restic/restic/releases/) but you could also use your distro's package manager 16 | - Clone this repo to the machine you will be using to run restic 17 | - Mount any network directories if you want to back them up (for example a NAS share). I mount mine into the sources directory but they can be anywhere. 18 | - Update `/etc/fstab` to mount these directories automatically on reboot 19 | - Edit the restic_env.sh file: 20 | - Update `B2_ACCOUNT_ID` 21 | - Update `B2_ACCOUNT_KEY` 22 | - Update `RESTIC_REPOSITORY` (path will be in this format: `b2:your-unique-bucket-name:/`) 23 | - Update `RESTIC_PASSWORD` 24 | - Update `RESTIC_INCLUDE_PATHS` based on your system 25 | - Update `RESTIC_CACHE_DIR` based on where you want to store the cache on your system (I use the cache subdirectory) 26 | - Edit restic_excludefiles.txt and at a minimum remove the example directories that I provide 27 | - Manually run restic_backup.sh to make sure backup is working. I don't know what happens if you are starting a brand new restic repository since this script attempts to unlock the restic repo before running the backup, but if it fails then you would need to manually run restic backup for the first time to create the initial snapshot. 28 | 29 | # Scheduling Cron Jobs 30 | - After you verity restic_backup.sh works, you can schedule cron to run the backup periodically. 31 | - I also run restic_check.sh once a week, but due to memory issues I've had in the past, I have a root cron job that reboots the machine 10 minutes before restic_check.sh is scheduled to run. Feel free to modify this schedule based on your needs. 32 | ``` 33 | # Run restic backup at 6:30 and 18:30 every day 34 | 30 6,18 * * * sh /home/user/restic-scripts/restic_backup.sh >> /home/user/restic-scripts/logs/restic_backup.log 2>&1 35 | 36 | # Run restic check every Thursday at 14:00 (optionally schedule a system reboot in root's crontab 10 minutes before this runs to free up system resources) 37 | 0 14 * * 4 sh /home/user/restic-scripts/restic_check.sh >> /home/user/restic-scripts/logs/restic_backup.log 2>&1 38 | ``` 39 | 40 | Don't forget to monitor the logs periodically and take any necessary action. All the other scripts are meant to be run manually and simplify the maintenance tasks you might need to perform on your restic backup repository. 41 | --------------------------------------------------------------------------------