├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── TODO ├── bin ├── .gitignore ├── backup ├── check ├── common ├── cron-backup ├── cron-daily ├── cron-monthly ├── cron-weekly ├── crontab-install ├── crontab-remove ├── download-duplicacy ├── init-storage ├── logs ├── prune ├── restore └── update ├── etc ├── prune └── settings ├── lib └── crontab └── prefs ├── filters └── preferences /.gitignore: -------------------------------------------------------------------------------- 1 | # 2 | # Files to be ignored by Git 3 | # 4 | test 5 | *~ 6 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # Makefie for Duplicacy Scripts 3 | # 4 | 5 | # Configure the following to taste. 6 | 7 | # Suitable for general use: 8 | 9 | # Root of backup 10 | ROOT=/ 11 | 12 | # Where the scripts and data should be installed 13 | DEST=/opt/duplicacy 14 | 15 | # Suitable for experimentation in /tmp 16 | #ROOT=/tmp 17 | #DEST=/tmp/duplicacy-scripts 18 | 19 | # Where to link the binary for command-line use 20 | BINARY_LINK=/usr/local/bin 21 | 22 | # No user-serviceable parts below this point. 23 | 24 | GIT_URL=https://github.com/markfeit/duplicacy-scripts.git 25 | GIT_BRANCH=origin/master 26 | 27 | BIN=$(DEST)/bin 28 | ETC=$(DEST)/etc 29 | LIB=$(DEST)/lib 30 | PREFS=$(DEST)/prefs 31 | VAR=$(DEST)/var 32 | 33 | CACHE=$(VAR)/cache 34 | HOLE=$(VAR)/hole 35 | LOG=$(VAR)/log 36 | UPDATE=$(VAR)/update 37 | 38 | DUPLICACY_BINARY=$(BIN)/duplicacy 39 | DIST_CLEAN += $(DUPLICACY_BINARY) 40 | 41 | default: 42 | @echo Nothing to do. 43 | 44 | 45 | $(DEST): 46 | mkdir -p $@ 47 | 48 | 49 | ROOT_LINK=$(DEST)/root 50 | $(ROOT_LINK): $(DEST) 51 | rm -f $@ 52 | ln -s "$(shell cd $(ROOT) && pwd -P)" $@ 53 | 54 | $(DEST)/%: 55 | mkdir -p $@ 56 | 57 | # File that points Duplicacy at its local storage 58 | LOCATION_FILE=$(ROOT)/.duplicacy 59 | $(LOCATION_FILE): $(ROOT) 60 | rm -f $@ 61 | echo "$(shell cd $(DEST)/prefs && pwd)" > $@ 62 | 63 | 64 | # Duplicacy binary. 65 | 66 | $(DUPLICACY_BINARY): FORCE 67 | ./bin/download-duplicacy "$@" 68 | 69 | # Crontab 70 | CRONTAB=$(LIB)/crontab 71 | $(CRONTAB):: 72 | mkdir -p $(LIB) 73 | sed -e 's|__BIN__|$(BIN)|g' lib/crontab > $@ 74 | 75 | 76 | # Prime updating with a full copy of the sources unless we're already 77 | # doing an update from that directory. 78 | 79 | $(UPDATE):: 80 | ifeq ($(NO_GIT),) 81 | rm -rf $@ 82 | mkdir -p $@ 83 | cp -r $(shell ls -a | egrep -v -e '^(\.+|test)$$') $@ 84 | (cd "$@" && git remote set-url origin "$(GIT_URL)") 85 | else 86 | @true 87 | endif 88 | 89 | # If files in etc differ from what was installed, install them as 90 | # *-upgrade and let the user sort it out. 91 | 92 | install: clean \ 93 | $(BIN) $(DUPLICACY_BINARY) \ 94 | $(ETC) \ 95 | $(LIB) \ 96 | $(CRONTAB) \ 97 | $(PREFS) \ 98 | $(LOCATION_FILE) \ 99 | $(LINKED_BINARY) \ 100 | $(CACHE) $(HOLE) $(LOG) $(UPDATE) \ 101 | $(ROOT_LINK) 102 | cp -rfp bin/* $(BIN) 103 | for FILE in etc/* ; \ 104 | do \ 105 | BASE=$$(basename "$${FILE}") ; \ 106 | if [ ! -e "$(ETC)/$${BASE}" ] ; \ 107 | then \ 108 | cp -fp "$${FILE}" "$(ETC)" ; \ 109 | else \ 110 | [ -e "$(ETC)/$${BASE}" ] \ 111 | && diff "$${FILE}" "$(ETC)/$${BASE}" > /dev/null \ 112 | && continue ; \ 113 | [ -e "$(ETC)/$$BASE" ] \ 114 | && cp -fp "$${FILE}" "$(ETC)/$${BASE}-upgrade" \ 115 | || cp -fp "$${FILE}" "$(ETC)" ; \ 116 | fi ; \ 117 | done 118 | rm -rf "$(PREFS)/logs" 119 | ln -s "../var/hole" "$(PREFS)/logs" 120 | rm -rf "$(PREFS)/cache" 121 | ln -s "../var/cache" "$(PREFS)/cache" 122 | ifeq ($(TEST_BUILD),) 123 | crontab -l | $(BIN)/crontab-install | crontab - 124 | endif 125 | 126 | # $(LINKED_BINARY) is a special case that gets handled in the 127 | # uninstall target. 128 | TO_UNINSTALL += $(BIN) $(LIB) $(LOCATION_FILE) 129 | 130 | 131 | # The remake of $(DUPLICACY_BINARY) is done separately rather than as 132 | # a dependency so it happens post-fetch. 133 | update: 134 | git fetch 135 | if [ $$(git diff $(GIT_BRANCH) | wc -l) -gt 0 ]; \ 136 | then \ 137 | git merge $(GIT_BRANCH) \ 138 | && $(MAKE) NO_GIT=1 install ; \ 139 | fi 140 | $(MAKE) $(DUPLICACY_BINARY) 141 | 142 | 143 | uninstall: 144 | ifeq ($(TEST_BUILD),) 145 | crontab -l | $(BIN)/crontab-remove | crontab - 146 | endif 147 | rm -rf $(TO_UNINSTALL) 148 | if [ -w "$(BINARY_LINK)" ]; then rm -f "$(LINKED_BINARY)" ; fi 149 | @echo "NOTE: Configuration, cache and logs were left in place." 150 | 151 | 152 | # Install a test copy 153 | TEST_DIR=test 154 | TEST_DEST=$(TEST_DIR)/duplicacy 155 | TEST_ROOT=$(TEST_DIR)/root 156 | 157 | $(TEST_ROOT) $(TEST_DEST): 158 | rm -rf $@ 159 | mkdir -p $@ 160 | 161 | test: $(TEST_ROOT) $(TEST_DEST) 162 | $(MAKE) \ 163 | DEST=$(TEST_DEST) \ 164 | ROOT=$(TEST_ROOT) \ 165 | TEST_BUILD=1 \ 166 | install 167 | TO_CLEAN += $(TEST_DIR) $(TEST_ROOT) 168 | 169 | 170 | clean: 171 | rm -rf $(TO_CLEAN) 172 | find . -name "*~" | xargs rm -f 173 | 174 | distclean: clean 175 | rm -rf $(DIST_CLEAN) 176 | 177 | 178 | FORCE: 179 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # duplicacy-scripts 2 | 3 | This is a set of scripts that can be used to manage running 4 | [Duplicacy](https://duplicacy.com)([GitHub](https://github.com/gilbertchen/duplicacy)) 5 | on Unix systems. They were developed and tested under Linux but 6 | should work in any environment that complies with POSIX except Windows 7 | (see note below). 8 | 9 | ## Notes 10 | 11 | **Release 1.3:** The `duplicacy` binary that was manually installed in 12 | the top-level directory per the instructions from earlier versions 13 | is no longer used and may be removed. 14 | 15 | These scripts download and use the Duplicacy binary from [Gilbert 16 | Chen's release 17 | page](https://github.com/gilbertchen/duplicacy/releases) on GitHub. 18 | The license for Duplicacy imposes some restrictions on its use. 19 | Please abide by them. 20 | 21 | Limitations in the current implementation of Duplicacy and the fact 22 | that Windows does not handle symbolic linking in a POSIX-like way 23 | makes these scripts unsuitable for that environment. 24 | 25 | 26 | 27 | ## Prerequisites 28 | 29 | Your system must have the following installed: 30 | 31 | * A POSIX-compliant environment 32 | * cURL 33 | * GNU Make 34 | * jq 35 | 36 | 37 | ## Installation 38 | 39 | To install this package on your system: 40 | 41 | Clone this repository: `git clone https://github.com/markfeit/duplicacy-scripts.git` 42 | 43 | `cd duplicacy-scripts` 44 | 45 | 46 | Select a location where duplicacy-scripts, the Duplicacy 47 | configuration, its cache and log files will be kept. This location 48 | will be referred to as `$DEST`. The default is `/opt/duplicacy`. 49 | 50 | Select the location which will form the root of the volume(s) to be 51 | backed up. This will be referred to as `$ROOT`. The default is `/`, 52 | which is suitable for most systems. (Specific parts of the filesystem 53 | may be included or excluded using Duplicacy's filter mechanism.) 54 | 55 | Become `root` and execute: 56 | 57 | * `make install` to install using the defaults 58 | * `make DEST=$DEST ROOT=$ROOT install` to install using other directories. 59 | 60 | Installation may be done as any other user, but be aware that this 61 | will limit the set of files backed up to those the user can read. In 62 | addition to installing these scripts, a `.duplicacy` file will be 63 | placed in `$ROOT`. 64 | 65 | Set up Duplicacy by placing a `preferences` and optional `filter` file 66 | in `$DEST/prefs` Samples are provided in the `prefs` directory of the 67 | sources. These files are not installed by default. 68 | 69 | Set up the scripts by editing `$DEST/etc/settings`. Note that if 70 | `CONFIG_AUTO_UPDATE` is enabled: 71 | 72 | * Any changes in the original GitHub repository will be applied to 73 | `$DEST/etc/settings-update` rather than overwriting `settings`. 74 | 75 | * The Duplicacy binary will be updated as new releases happen. Any 76 | breaking changes in that will be breaking changes here as well. 77 | 78 | At this point, backups and maintenance will be done automatically by 79 | cron. 80 | 81 | 82 | ## Backups 83 | 84 | Backups are run at 00:45 local time each morning. If there is another 85 | backup running (common when there is a long initial backup running), 86 | the newer backup will be aborted. 87 | 88 | Logs of what happens during each backup and other matinenance 89 | activities are stored in `$DEST/var/log`. The latest logs or those 90 | for a specific date can be retrieved and read with `$DEST/bin/logs`. 91 | 92 | 93 | ## Restoring Files 94 | 95 | To restore files, execute `$DEST/bin/restore`. Detailed help may be 96 | obtained with the `--help` switch. 97 | 98 | 99 | ## Maintenance 100 | 101 | Daily (03:00): 102 | 103 | * Prune old snapshots according to the rules in `etc/prune`. 104 | * Remove old log files per `CONFIG_LOG_LIFE` in settings. 105 | * Remove old cache files per `CONFIG_CACHE_LIFE` in settings. 106 | * Pull and update the software (if enabled in `$DEST/etc/settings`). 107 | 108 | Weekly (Sunday at 03:15): 109 | 110 | * Fossilize and remove chunks that are no longer referenced. 111 | 112 | * Check integrity. This verifies that all chunks that should be 113 | present and attempts to resurrect missing chunks from fossils if 114 | possible. There is no attempt to download and verify the contents 115 | of the chunks. (Not implemented yet.) 116 | 117 | Monthly (First Sunday at 03:30): 118 | 119 | * Nothing yet. 120 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | List of things that need to be done: 2 | 3 | 4 | Make sure maintenance operations run quietly and failures produce 5 | output for mailing. 6 | 7 | Rearrange CONFIG_NO_MAINT to be CONFIG_MAINT. (Makes more sense that 8 | way.) 9 | 10 | Make maintenance activities avoid backups running on the local host. 11 | -------------------------------------------------------------------------------- /bin/.gitignore: -------------------------------------------------------------------------------- 1 | # This gets downloaded by the install and update process 2 | duplicacy 3 | -------------------------------------------------------------------------------- /bin/backup: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Script to run this backup 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | avoid_running_backup 9 | 10 | list_storage_names | ( \ 11 | while read NAME 12 | do 13 | printf "\n#\n# Backing up to ${NAME}\n#\n\n" 14 | duplicacy_cmd \ 15 | -v \ 16 | backup \ 17 | $(dry_run_arg) \ 18 | -stats \ 19 | -storage "${NAME}" \ 20 | $(limit_rate) \ 21 | "$@" \ 22 | || true 23 | done 24 | ) 25 | 26 | exit 0 27 | -------------------------------------------------------------------------------- /bin/check: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Do a full check and repair of all storage 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | avoid_running_backup 9 | 10 | list_storage_names | ( \ 11 | while read NAME 12 | do 13 | printf "#\n# Checking ${NAME}\n#\n\n" 14 | duplicacy_cmd -v check -all -stats -fossils -resurrect \ 15 | -tabular -storage "${NAME}" 16 | done 17 | ) 18 | 19 | exit 0 20 | -------------------------------------------------------------------------------- /bin/common: -------------------------------------------------------------------------------- 1 | # 2 | # Common functions and variables 3 | # 4 | 5 | # Configure these to taste. 6 | 7 | # ... 8 | 9 | # No user-serviceable parts below this point. 10 | 11 | NAME=duplicacy-scripts 12 | 13 | WHEREAMI=$(dirname $0) 14 | WHOAMI=$(basename $0) 15 | 16 | TOP=$(cd "${WHEREAMI}/.." && pwd -P) 17 | PREFS="${TOP}/prefs" 18 | PREFERENCES=${PREFS}/preferences 19 | BIN="${TOP}/bin" 20 | ETC="${TOP}/etc" 21 | LIB="${TOP}/lib" 22 | VAR="${TOP}/var" 23 | 24 | 25 | 26 | LOG="${VAR}/log" 27 | mkdir -p "${LOG}" 28 | 29 | HOLE="${VAR}/hole" 30 | mkdir -p "${HOLE}" 31 | 32 | CACHE="${VAR}/cache" 33 | mkdir -p "${CACHE}" 34 | 35 | PATH=${BIN}:${PATH} 36 | 37 | 38 | # Read the settings file if it exists 39 | [ -e "${ETC}/settings" ] && . "${ETC}/settings" 40 | 41 | # Exit with an error 42 | die() 43 | { 44 | echo "$@" 1>&2 45 | exit 1 46 | } 47 | 48 | 49 | # Determine if this host should do maintenance 50 | do_maint() 51 | { 52 | if [ -z "${CONFIG_NO_MAINT}" ] 53 | then 54 | return 0 55 | else 56 | return 1 57 | fi 58 | die "Not reached." 59 | } 60 | 61 | 62 | # Run a duplicacy command from the right place 63 | duplicacy_cmd() 64 | { 65 | [ -h "${TOP}/root" ] \ 66 | || die "Can't find backup root" 67 | 68 | (cd "${TOP}/root" && duplicacy "$@") 69 | } 70 | 71 | 72 | # Return the command-line switch for the bandwidth limit if one is 73 | # configured. 74 | limit_rate() 75 | { 76 | if echo "${CONFIG_BANDWIDTH}" | egrep -q -e '^([0-9]+)?$' 77 | then 78 | if [ -n "${CONFIG_BANDWIDTH}" ] 79 | then 80 | echo "-limit-rate" "$((${CONFIG_BANDWIDTH} * 1000 / 8))" 81 | fi 82 | else 83 | die "Invalid configured bandwidth '${CONFIG_BANDWIDTH}'" 84 | fi 85 | } 86 | 87 | 88 | # 89 | # Run a program and log the output 90 | # 91 | # Args: 92 | # 1 - Log file name tag (e.g. "backup" or "prune") 93 | # 2+- Program name and arguments 94 | # 95 | log() 96 | { 97 | TAG=$1 98 | shift 99 | 100 | TIMESTAMP=$(date +"%Y-%m-%dT%H:%M:%S") 101 | OUT_LOG="${LOG}/$(date +"%Y-%m-%dT%H:%M:%S")-${TAG}" 102 | 103 | echo "START ${TAG} at $(date)" > "${OUT_LOG}" 104 | echo >> "${OUT_LOG}" 105 | echo "Running $@" >> "${OUT_LOG}" 106 | echo >> "${OUT_LOG}" 107 | 108 | if [ -t 1 ] 109 | then 110 | 111 | # Build the output separately because the shell doesn't always flush. 112 | LOG_BUILD="${HOLE}/log-build.$$" 113 | 114 | # Exit status catcher from 115 | # https://unix.stackexchange.com/a/70675/15184 116 | (((("$@" 2>&1 ; echo $? >&3) >&4) 3>&1) | (read xs; exit $xs)) > "${LOG_BUILD}" 4>&1 117 | STATUS=$? 118 | PRODUCE_ON_STDERR=true 119 | 120 | cat "${LOG_BUILD}" >> "${OUT_LOG}" 121 | rm -f "${LOG_BUILD}" 122 | 123 | else 124 | 125 | "$@" 2>&1 >> "${OUT_LOG}" 126 | STATUS=$? 127 | PRODUCE_ON_STDERR=false 128 | 129 | fi 130 | 131 | echo >> "${OUT_LOG}" 132 | echo "Exited ${STATUS}" >> "${OUT_LOG}" 133 | echo >> "${OUT_LOG}" 134 | echo "END ${TAG} at $(date)" >> "${OUT_LOG}" 135 | 136 | ${PRODUCE_ON_STDERR} && cat "${OUT_LOG}" 1>&2 137 | } 138 | 139 | 140 | 141 | # 142 | # Utilities 143 | # 144 | 145 | # Exit nicely if a backup is already in progress 146 | avoid_running_backup() 147 | { 148 | RUNNING_PID=$( \ 149 | ps -e -o pid,args \ 150 | | egrep -e ' duplicacy .* backup ' \ 151 | | awk '$2 == "duplicacy" { print $1 }' 152 | ) 153 | 154 | if [ -n "${RUNNING_PID}" ] 155 | then 156 | echo "Backup is already running, PID ${RUNNING_PID}" 157 | exit 0 158 | fi 159 | } 160 | 161 | 162 | # Get a list of the storage names in the preferences. 163 | 164 | # TODO: This is highly dependent on the formatting of the file, which 165 | # is JSON and shouldn't matter. 166 | 167 | list_storage_names() 168 | { 169 | < "${PREFERENCES}" \ 170 | egrep -e '"name":\s*"[^"]+"' \ 171 | | sed -e 's/^.*"name":\s*"\([^"]*\)".*$/\1/' 172 | } 173 | 174 | 175 | 176 | # Get a list of the revision dates and numbers in a machine-readable 177 | # format. 178 | 179 | # Args: 180 | # 1 - Snapshot 181 | # 2 - Storage (Optional) 182 | 183 | list_revisions() 184 | { 185 | local SNAPSHOT="$1" 186 | local OPT= 187 | [ -n "$2" ] && OPT="--storage $2" 188 | duplicacy_cmd list --id "${SNAPSHOT}" ${OPT}\ 189 | | egrep -e '^Snapshot ' \ 190 | | awk '{ printf "%sT%s %s\n", $7, $8, $4 }' 191 | } 192 | 193 | 194 | # Return a dry-run argument if we're configured for that. 195 | dry_run_arg() 196 | { 197 | if [ -n "${CONFIG_DRY_RUN}" ] 198 | then 199 | echo '-dry-run' 200 | fi 201 | } 202 | 203 | # See if the variable named in $1 is enabled 204 | is_enabled() 205 | { 206 | VALUE=$(eval echo "\$$1") 207 | [ -n "${VALUE}" ] || VALUE=no 208 | 209 | case $(echo "${VALUE}" | tr A-Z a-z) in 210 | 0|false|no) 211 | return 1 212 | ;; 213 | 1|true|yes) 214 | return 0 215 | ;; 216 | *) 217 | die "Don't understand '$1' value of '${VALUE}'" 218 | ;; 219 | 220 | esac 221 | } 222 | 223 | 224 | # Exit if CONFIG_SKIP_CRON_JOBS is set 225 | exit_if_no_cron() 226 | { 227 | if is_enabled CONFIG_SKIP_CRON_JOBS 228 | then 229 | exit 0 230 | fi 231 | } 232 | -------------------------------------------------------------------------------- /bin/cron-backup: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Run a backup 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | exit_if_no_cron 9 | 10 | log backup "${BIN}/backup" 11 | 12 | exit 0 13 | -------------------------------------------------------------------------------- /bin/cron-daily: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Daily Maintenance 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | exit_if_no_cron 9 | 10 | # Remove old logs 11 | if [ -n "${CONFIG_LOG_LIFE}" ] 12 | then 13 | find "${LOG}" -depth -mtime "+${CONFIG_LOG_LIFE}" -print0 \ 14 | | xargs -0 rm -rf 15 | fi 16 | 17 | 18 | # Clear out the black hole. Prune writes empty logs here. 19 | find ${HOLE}/ -type f -print0 \ 20 | | xargs -0 rm -rf 21 | 22 | 23 | # Prune old snapshots and clean the cache. 24 | 25 | if do_maint 26 | then 27 | 28 | log prune "${BIN}/prune" 29 | 30 | elif [ -n "${CONFIG_CACHE_LIFE}" ] 31 | then 32 | 33 | # Not doing maintenance doesn't prune the cache, so do our own 34 | # version. 35 | 36 | OLD_SIZE=$(du -sh "${CACHE}/." | awk '{ print $1 }') 37 | find "${CACHE}/." -type f -atime "+${CONFIG_CACHE_LIFE}" -print0 \ 38 | | xargs -0 rm -f 39 | NEW_SIZE=$(du -sh "${CACHE}/." | awk '{ print $1 }') 40 | if [ "${NEW_SIZE}" != "${OLD_SIZE}" ] 41 | then 42 | log prune echo "Trimmed cache from ${OLD_SIZE} to ${NEW_SIZE}" 43 | fi 44 | 45 | fi 46 | 47 | exit 0 48 | -------------------------------------------------------------------------------- /bin/cron-monthly: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Monthly Maintenance 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | exit_if_no_cron 9 | 10 | # Nothing yet 11 | 12 | exit 0 13 | -------------------------------------------------------------------------------- /bin/cron-weekly: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Weekly Maintenance 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | exit_if_no_cron 9 | 10 | do_maint && log exhaustive "${BIN}/prune" -exhaustive 11 | 12 | do_maint && log check "${BIN}/check" 13 | 14 | 15 | exit 0 16 | -------------------------------------------------------------------------------- /bin/crontab-install: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Insert the crontab into the one provided on stdin and put the result 4 | # onto stdout. 5 | # 6 | 7 | . $(dirname $0)/common 8 | 9 | sed -e "/^#BEGIN-${NAME}/,/^#END-${NAME}/d" 10 | 11 | echo "#BEGIN-${NAME}" 12 | cat "${LIB}/crontab" 13 | echo "#END-${NAME}" 14 | -------------------------------------------------------------------------------- /bin/crontab-remove: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Remove the crontab from the current user's crontab provided via 4 | # stdin 5 | # 6 | 7 | . $(dirname $0)/common 8 | 9 | sed -e "/^#BEGIN-${NAME}/,/^#END-${NAME}/d" 10 | -------------------------------------------------------------------------------- /bin/download-duplicacy: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Download the latest version of Duplicacy from GitHub and store it in 4 | # a specified location. If the latest available is the same version as 5 | # the installed file, do nothing. 6 | # 7 | # Usage: download-duplicacy [ DEST ] 8 | # 9 | # Where DEST is where the downloaded file should reside. The default 10 | # is ./duplicacy. 11 | # 12 | 13 | NAME=duplicacy 14 | GITHUB_REPO="gilbertchen/${NAME}" 15 | 16 | die() 17 | { 18 | echo "$@" 1>&2 19 | exit 1 20 | } 21 | 22 | TMPBASE=$(mktemp -d) 23 | cleanup() 24 | { 25 | rm -rf "${TMPBASE}" 26 | } 27 | trap cleanup EXIT 28 | 29 | 30 | [ $# -le 1 ] \ 31 | || die "Usage: $(basename $0) [ DEST ]" 32 | 33 | [ -z "$1" ] \ 34 | && DEST="./duplicacy" \ 35 | || DEST="$1" 36 | 37 | 38 | RELEASE_INFO="${TMPBASE}/release-info" 39 | curl -s -o "${RELEASE_INFO}" https://api.github.com/repos/${GITHUB_REPO}/releases/latest 40 | 41 | [ "$(jq -r '.status' "${RELEASE_INFO}")" == 'null' ] \ 42 | || die "Failed to download release info: $(jq -r '.message' "${RELEASE_INFO}")" 43 | 44 | TAG_NAME="$(jq -r '.tag_name' "${RELEASE_INFO}")" 45 | VERSION=$(echo "${TAG_NAME}" | sed -e 's/^v//') 46 | 47 | # If the destination is an executable file, it was probably installed 48 | # by this program and the process can be short-circuited by a version 49 | # check. If anything else, it's probably a symlink maintained by 50 | # versions < 1.3 and should be replaced. 51 | 52 | if [ ! -L "${DEST}" -a -f "${DEST}" -a -x "${DEST}" ] 53 | then 54 | INSTALLED_VERSION=$("${DEST}" | sed -e '1,/^VERSION:/d; s/^\s*\([0-9.]\+\)\s.*$/\1/' | head -1) 55 | if [ "${INSTALLED_VERSION}" = "${VERSION}" ] 56 | then 57 | echo "${DEST} is already the latest version (${VERSION})" 58 | exit 0 59 | fi 60 | fi 61 | 62 | # These adapt the output of uname(1) to the conventions Gilbert uses 63 | # for download names. 64 | 65 | UNAME_S=$(uname -s) 66 | case "${UNAME_S}" in 67 | Darwin) 68 | SYSTEM=osx 69 | ;; 70 | FreeBSD) 71 | SYSTEM=freebsd 72 | ;; 73 | Linux) 74 | SYSTEM=linux 75 | ;; 76 | *) 77 | die "${UNAME_S} is not supported" 78 | ;; 79 | esac 80 | 81 | UNAME_M=$(uname -m) 82 | case "${UNAME_M}" in 83 | arm|i386) 84 | ARCH="${UNAME_M}" 85 | ;; 86 | aarch64) 87 | ARCH=arm64 88 | ;; 89 | x86_64) 90 | ARCH=x64 91 | ;; 92 | *) 93 | die "${UNAME_M} is not supported" 94 | ;; 95 | esac 96 | 97 | DOWNLOAD_URL="https://github.com/${GITHUB_REPO}/releases/download/${TAG_NAME}/${NAME}_${SYSTEM}_${ARCH}_${VERSION}" 98 | 99 | echo "Downloading ${NAME} ${VERSION} for ${SYSTEM} on ${ARCH}" 100 | DEST_DOWNLOADED="${TMPBASE}/download" 101 | curl -s -L -o "${DEST_DOWNLOADED}" "${DOWNLOAD_URL}" 102 | chmod +x "${DEST_DOWNLOADED}" 103 | rm -f "${DEST}" 104 | mv "${DEST_DOWNLOADED}" "${DEST}" 105 | -------------------------------------------------------------------------------- /bin/init-storage: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Initialize an empty local directory for storage 4 | # 5 | # Usage: init-storage ARGS 6 | # 7 | # ARGS is the same set of arguments that would be passed to 'duplicacy 8 | # init', normally -e snap-name /path/to/storage 9 | # 10 | 11 | . $(dirname $0)/common 12 | 13 | # Last argument is the directory. 14 | eval DIR=\${$#} 15 | 16 | [ -e "${DIR}" ] \ 17 | || mkdir -p "${DIR}" 18 | 19 | [ -d "${DIR}" ] \ 20 | || die "${DIR}: Not a directory." 21 | 22 | [ "$(ls -a "${DIR}" | wc -l)" -eq 2 ] \ 23 | || die "${DIR}: Not empty" 24 | 25 | 26 | # Duplicacy assumes $CWD is your root and leaves a .duplicacy 27 | # directory there. Let it be created in the storage directory and get 28 | # rid of it later. 29 | cd "${DIR}" 30 | 31 | duplicacy init "$@" 32 | 33 | exec rm -rf "${DIR}/.duplicacy" 34 | -------------------------------------------------------------------------------- /bin/logs: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Show logs for a specified or the most recent day. 4 | # 5 | # By default, backup logs are stripped of information about patterns, 6 | # packing, skipping and uploading. Use the --long switch to see 7 | # everything. 8 | # 9 | 10 | . $(dirname $0)/common 11 | 12 | 13 | LOG_FILTER=filter_log 14 | case "$1" in 15 | --long) 16 | LOG_FILTER=cat 17 | shift 18 | ;; 19 | esac 20 | 21 | 22 | case $# in 23 | 0) 24 | DATE=$(ls "${LOG}" | sort | tail -1 | sed -e 's/T.*$//') 25 | ;; 26 | 1) 27 | DATE="$1" 28 | ;; 29 | *) 30 | die "Usage: ${WHOAMI} [ YYYY-MM-DD ]" 31 | ;; 32 | esac 33 | 34 | # Do a rough approximation of date validation. 35 | echo "${DATE}" | egrep -qe '^2[0-9][0-9][0-9]-[01][0-9]-[0123][0-9]$' \ 36 | || die "Bad date." 37 | 38 | 39 | # Filter out crufty stuff from backup logs only. 40 | filter_log() 41 | { 42 | if (echo "$1" | egrep -q -e '-backup$') 43 | then 44 | echo "(Short form; use --long to see everything.)" 45 | echo 46 | egrep -v -e '^(Pattern:|Packing|Packed|Skipped|Uploaded) ' "$1" 47 | else 48 | cat "$1" 49 | fi 50 | } 51 | 52 | 53 | ( 54 | echo "Duplicacy logs for ${DATE}:" 55 | 56 | for FILE in $(ls "${LOG}" | egrep -e "^${DATE}" | sort) 57 | do 58 | printf "\n\n---------- %s\n\n" \ 59 | $(echo "${FILE}" \ 60 | | sed -e 's/^.*-\([^-]*\)$/\1/' \ 61 | | tr a-z A-Z) 62 | $LOG_FILTER "${LOG}/${FILE}" 63 | done 64 | ) | ${PAGER:-less} 65 | 66 | exit 0 67 | -------------------------------------------------------------------------------- /bin/prune: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Prune snapshots according to a list 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | avoid_running_backup 9 | 10 | PRUNES="${ETC}/prune" 11 | 12 | if [ ! -f "${PRUNES}" -o ! -r "${PRUNES}" ] 13 | then 14 | echo "No prune configuration found." 15 | exit 0 16 | fi 17 | 18 | PRUNESPEC=$( \ 19 | cat "${PRUNES}" \ 20 | | sed -e 's/\s*#.*$//g; /^\s*$/d' \ 21 | | awk '$0 ~ /^[0-9]+\s*[0-9]+$/ { print $1 ":" $2 }' \ 22 | | sort -r -n -t: -k 2 \ 23 | | sed -e 's/^/-keep /g' \ 24 | ) 25 | 26 | list_storage_names | ( \ 27 | while read NAME 28 | do 29 | printf "\n#\n# Pruning ${NAME}\n#\n\n" 30 | duplicacy_cmd -v prune $(dry_run_arg) -storage "${NAME}" ${PRUNESPEC} "$@" 31 | done 32 | ) 33 | 34 | 35 | exit 0 36 | -------------------------------------------------------------------------------- /bin/restore: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | # 3 | # Restore files from a backup 4 | # 5 | #BEGIN-HELP 6 | # 7 | # Usage: 8 | # 9 | # restore [ OPTIONS ] PATH [ PATH ... ] 10 | # 11 | # Where PATH is a path to a directory or file relative to the root of 12 | # the backup set. If a PATH exists in the file system, it will be 13 | # restored as its type (file or directory). If it does not exist, 14 | # append a slash to force it to be treated as a directory. PATHs 15 | # beginning with "+", "-", "i:" or "e:" will be passed directly to 16 | # Duplicacy and used as filters. 17 | # 18 | # Options: 19 | # 20 | # --dest DIR Restore to DIR instead of their original location. 21 | # DIR must be nonexistant or empty. 22 | # 23 | # --list List the times of all available snapshots in ISO 24 | # 8601 format an exit without restoring anything and 25 | # ignoring all other options except --storage. 26 | # 27 | # --snapshot S Operate on snapshot S 28 | # 29 | # --storage ID Restore from the storage named ID. If not provided, 30 | # files will be restored from "default". 31 | # 32 | # --time TIME Restore from the most-recent snapshot in the backup 33 | # set that occurred before the ISO 8601 timestame TIME 34 | # (e.g., 2021-07-03T08:30:00). 35 | # 36 | #END-HELP 37 | # 38 | . $(dirname $0)/common 39 | 40 | 41 | # Gargle the arguments 42 | 43 | DEST="" 44 | LIST=false 45 | RESTORE_TIME=$(date +"%Y-%m-%dT%H:%M") 46 | STORAGE="default" 47 | SNAPSHOT="default" 48 | 49 | while echo "$1" | egrep -q -e '^--' 50 | do 51 | ARG="$1" 52 | shift 53 | case "${ARG}" in 54 | --help) 55 | sed -n -e '/^#BEGIN-HELP/,${/^#BEGIN-HELP/d; /^#END-HELP/q; s/^#//; s/^ //; p}' "$0" 56 | exit 0 57 | ;; 58 | --dest) 59 | DEST=$1 60 | shift 61 | ;; 62 | --list) 63 | LIST=true 64 | ;; 65 | --snapshot) 66 | SNAPSHOT="$1" 67 | shift 68 | ;; 69 | --storage) 70 | STORAGE="$1" 71 | shift 72 | ;; 73 | --time) 74 | RESTORE_TIME="$1" 75 | shift 76 | ;; 77 | *) 78 | die "Unknown option ${ARG}" 79 | ;; 80 | esac 81 | done 82 | 83 | 84 | # Handle the easy-out case 85 | if ${LIST} 86 | then 87 | list_revisions "${SNAPSHOT}" "${STORAGE}" 88 | #### | awk '{ print $1 }' 89 | exit 0 90 | fi 91 | 92 | 93 | # Validate everything 94 | 95 | [ "$#" -gt 0 ] \ 96 | || die "Usage: ${WHOAMI} OPTIONS PATH [ PATH ... ]" 97 | 98 | if [ -d "${DEST}" ] 99 | then 100 | [ "$(ls -a "${DEST}" | wc -l)" -gt 2 ] \ 101 | && die "${DEST}: Must be empty." 102 | fi 103 | 104 | # This validates ISO8601 down to seconds. 105 | REGEX='^(-?(?:[1-9][0-9]*)?[0-9]{4})-(1[0-2]|0[1-9])-(3[01]|0[1-9]|[12][0-9])T(2[0-3]|[01][0-9]):[0-5][0-9]$' 106 | echo "${RESTORE_TIME}" | egrep -q -e "${REGEX}" \ 107 | || die "Invalid time ${RESTORE_TIME}" 108 | 109 | 110 | 111 | 112 | 113 | # Translate all of the arguments into Duplicacy filters 114 | 115 | analyze_paths() 116 | { 117 | for FILE in "$@" 118 | do 119 | 120 | # Try to divine the physical path of the file 121 | if [ -d "${FILE}" ] 122 | then 123 | FILE=$(cd "${FILE}" && pwd -P) 124 | elif [ -f "${FILE}" ] 125 | then 126 | FILE=$(cd $(dirname "${FILE}") && pwd -P)/$(basename "${FILE}") 127 | else 128 | true 129 | fi 130 | 131 | FILE=$(echo "${FILE}" | sed -e 's|^/||') 132 | case "${FILE}" in 133 | -*|+*|i:*|e:*) 134 | # Straight-through filters 135 | ;; 136 | */) 137 | # Explicit directories 138 | FILE="+${FILE}/*" 139 | ;; 140 | *) 141 | # Educated guesses 142 | GUESS="${TOP}/root/${FILE}" 143 | if [ -d "${GUESS}" ] 144 | then 145 | FILE="+${FILE}/*" 146 | elif [ -f "${GUESS}" ] 147 | then 148 | FILE="+${FILE}" 149 | else 150 | FILE="+${FILE}" 151 | fi 152 | ;; 153 | esac 154 | echo "${FILE}" 155 | done 156 | } 157 | 158 | set -- $(analyze_paths "$@") 159 | 160 | 161 | 162 | 163 | 164 | # Sort the desired time into the list of snapshots and pick out the 165 | # one just before it. 166 | 167 | SNAPSHOT_LINE=$( \ 168 | ( list_revisions "${SNAPSHOT}" "${STORAGE}" | sed -e 's/$/ S/g' && echo "${RESTORE_TIME} XX R" ) \ 169 | | sort \ 170 | | sed '/ R$/,//d' \ 171 | | tail -1 \ 172 | ) 173 | 174 | [ -n "${SNAPSHOT_LINE}" ] \ 175 | || die "No backups available for ${RESTORE_TIME}." 176 | 177 | REVISION=$(echo "${SNAPSHOT_LINE}" | awk '{ print $2 }') 178 | REVISION_TIME=$(echo "${SNAPSHOT_LINE}" | awk '{ print $1 }') 179 | 180 | 181 | # TODO: Make output quiet if stdout is not a tty. 182 | 183 | echo "Restoring from revision ${REVISION} as of ${REVISION_TIME} from ${STORAGE}" 184 | 185 | if [ -z "${DEST}" ] 186 | then 187 | DEST="${TOP}/root" 188 | else 189 | mkdir -p "${DEST}" 190 | echo "${PREFS}" > "${DEST}/.duplicacy" 191 | fi 192 | 193 | 194 | echo "Restoring to ${DEST}" 195 | 196 | ( cd "${DEST}" \ 197 | && duplicacy \ 198 | restore \ 199 | -r "${REVISION}" \ 200 | -stats \ 201 | -storage "${STORAGE}" \ 202 | "$@" \ 203 | ) 204 | 205 | 206 | # Clean up any leftovers 207 | 208 | rm -rf "${DEST}/.duplicacy" 209 | 210 | exit 0 211 | -------------------------------------------------------------------------------- /bin/update: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Do a self-update 4 | # 5 | 6 | . $(dirname $0)/common 7 | 8 | # If not doing updates, exit quietly. 9 | [ -n "${CONFIG_AUTO_UPDATE}" ] \ 10 | || exit 0 11 | 12 | # Before doing an update, make and execute a copy of this script so 13 | # any updates don't self-clobber. 14 | if [ "$1" != "--copied" ] 15 | then 16 | NEW_RUN="${WHEREAMI}/${WHOAMI}.copy" 17 | rm -f "${NEW_RUN}" 18 | cp "$0" "${NEW_RUN}" 19 | exec "${NEW_RUN}" --copied 20 | fi 21 | 22 | UPDATE_DIR="${VAR}/update" 23 | 24 | mkdir -p "${UPDATE_DIR}" 25 | 26 | [ -d "${UPDATE_DIR}/.git" ] \ 27 | || die "Unable to find sources in ${UPDATE_DIR}" 28 | 29 | log update make -C "${UPDATE_DIR}" DEST="${TOP}" ROOT="${TOP}/root" update 30 | 31 | # Get rid of the temporary script 32 | [ "$1" = "--copied" ] && exec rm -f "$0" 33 | 34 | exit 0 35 | -------------------------------------------------------------------------------- /etc/prune: -------------------------------------------------------------------------------- 1 | # 2 | # Prune Policy for Duplicacy 3 | # 4 | 5 | # This file will be sorted into the proper order so pruning of older 6 | # snapshots happens first. 7 | 8 | 9 | # General-use policy that keeps overall storage use down. 10 | 11 | # Prior to seven days, keep all snapshots. 12 | 1 7 # One per day for snapshots older than 7 days 13 | 7 30 # One per week for snapshots older than a month 14 | 30 180 # One per 30 days for snapshots older than 180 days 15 | 0 360 # None older than 360 days 16 | 17 | 18 | 19 | # This policy is better for services like Wasabi that charge for 90 20 | # days of storage even if an object has a shorter life than that. 21 | 22 | # # Prior to 89 days, keep all snapshots. 23 | # 30 89 # One per 30 days for snapshots older than the limit 24 | # 0 360 # None older than 360 days 25 | -------------------------------------------------------------------------------- /etc/settings: -------------------------------------------------------------------------------- 1 | # 2 | # Settings for Duplicacy scripts 3 | # 4 | 5 | # NOTE: If this file exists during an upgrade, it will be left alone 6 | # and the new version will be placed in settings-upgrade. 7 | 8 | 9 | # If this is set, do an automatic update of the scripts by pulling 10 | # the sources from GitHub and doing a 'make update'. 11 | # CONFIG_AUTO_UPDATE=yes 12 | 13 | 14 | # If defined, limit bandwidth use to this value in megabits per 15 | # second. (This contrasts with Duplicacy's parameter, which is 16 | # expressed in kilobytes per second.) 17 | # CONFIG_BANDWIDTH=20 18 | 19 | 20 | # If this is not set, skip doing storage maintenance. Setting this is 21 | # useful when multiple machines share the same storage and saves on 22 | # download traffic if the storage is a paid service. 23 | # 24 | # IMPORTANT NOTE: Maintenance should be disabled during long-running 25 | # initial backups. This will prevent having unrefernced chunks from 26 | # the incomplete backup fossilized, resulting in additional stored 27 | # data and lower deduplication rates. 28 | 29 | # CONFIG_NO_MAINT=yes 30 | 31 | 32 | # If CONFIG_NO_MAINT is not set, remove files from the cache that have 33 | # not been accesses in more than this number of days. 34 | CONFIG_CACHE_LIFE=90 35 | 36 | 37 | # Number of days to keep logs 38 | CONFIG_LOG_LIFE=90 39 | 40 | # Un-comment to make backup and prune operations inert 41 | # CONFIG_DRY_RUN=yes 42 | 43 | # Un-comment to skip cron jobs 44 | # CONFIG_SKIP_CRON_JOBS=yes 45 | -------------------------------------------------------------------------------- /lib/crontab: -------------------------------------------------------------------------------- 1 | # 2 | # Crontab addition for Duplicacy Scripts 3 | # 4 | 5 | # Minute, Hour, DoM, Month, DoW 6 | 7 | 8 | # Software update. Do this well before anything else. 9 | #Min Hour DoM Mon DoW Command 10 | 0 0 * * * __BIN__/update 11 | 12 | # Backups 13 | #Min Hour DoM Mon DoW Command 14 | 45 0 * * * __BIN__/cron-backup 15 | 16 | # Routine Maintenance 17 | #Min Hour DoM Mon DoW Command 18 | 0 3 * * * __BIN__/cron-daily 19 | 15 3 * * Sun __BIN__/cron-weekly 20 | 30 3 1-7 * Sun __BIN__/cron-monthly 21 | -------------------------------------------------------------------------------- /prefs/filters: -------------------------------------------------------------------------------- 1 | # 2 | # Filters for Duplicacy 3 | # 4 | # This is just a sample. Edit and install for your own installation. 5 | # 6 | 7 | # Don't back up anything in directories named "nobackup" or anywhere 8 | # below them. 9 | -nobackup/* 10 | -*/nobackup/* 11 | 12 | # Keep the duplicacy configuration except the cache. 13 | -local/duplicacy/prefs/cache/ 14 | +local/duplicacy/* 15 | 16 | # Paths relative to the root to include 17 | +local0/* 18 | 19 | # This works, but isn't pretty. 20 | +local/export/home/* 21 | +local/export/home/ 22 | +local/export/ 23 | +local/ 24 | 25 | # Ignore anything else. 26 | -* 27 | -------------------------------------------------------------------------------- /prefs/preferences: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "name": "default", 4 | "id": "myhost", 5 | "storage": "s3://...", 6 | "encrypted": true, 7 | "no_backup": false, 8 | "no_restore": false, 9 | "no_save_password": false, 10 | "keys": { 11 | "password": "mumble-mumble-mumble", 12 | "s3_id": "281apple", 13 | "s3_secret": "OU812" 14 | } 15 | } 16 | ] 17 | --------------------------------------------------------------------------------