├── .github ├── CODEOWNERS ├── FUNDING.yml ├── workflows │ └── ci.yml ├── CONTRIBUTING.md └── CODE_OF_CONDUCT.md ├── filters ├── linux ├── osx └── windows ├── vbackup.ps1 ├── vbackup.sh ├── LICENSE └── README.md /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # Owners 2 | * @vladgh 3 | -------------------------------------------------------------------------------- /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: [vladgh] 4 | -------------------------------------------------------------------------------- /filters/linux: -------------------------------------------------------------------------------- 1 | # Exclude any cache files/directories with cache in the name (case insensitive) 2 | e:(?i).*cache.* 3 | 4 | # exclude temporary file names 5 | e:.*/?~.*$ 6 | 7 | # Misc 8 | e:(?i)\.Trash/.*$ 9 | e:(?i)\.dropbox/.*$ 10 | e:(?i).*\.rnd$ 11 | 12 | # Dev 13 | e:(?i).*\.(sock|socket)$ 14 | e:(?i)/\.bundle/.*$ 15 | e:(?i)/\.rvm/.*$ 16 | e:(?i)/\.serverless/.*$ 17 | e:(?i)/\.terraform/.*$ 18 | e:(?i)/node_modules/.*$ 19 | e:(?i)\.(git|svn|hg)/.*$ 20 | e:(?i)\.gnupg/S\..*$ 21 | e:(?i)\.sqlite-.*$ 22 | e:(?i)\.vsliveshare/vsls-launcher$ 23 | e:(?i)\.vscode-server/.*$ 24 | -------------------------------------------------------------------------------- /vbackup.ps1: -------------------------------------------------------------------------------- 1 | # VARs 2 | # Repository location 3 | $backupLocation = "C:\backup" 4 | # Logs 5 | $logPath = "$backupLocation\.duplicacy\logs" 6 | $logFile = "$logPath\backup.log" 7 | # Size to roll over at in megabytes 8 | $logMaxSize = 5mb 9 | 10 | # Ensure log file exists 11 | If(!(test-path $logPath)) { 12 | New-Item -ItemType Directory -Force -Path $logPath 13 | New-Item -ItemType File $logFile 14 | } 15 | 16 | # Roll over logs when they get too big. 17 | if ((Get-Item $logFile).Length -gt $logMaxSize) { 18 | $currentDate = (Get-Date -UFormat %Y%m%d%H%M%S%Z) 19 | Rename-Item $logFile ($logFile + "." + $currentDate) 20 | Write-Output "Rolled over logs at $currentDate" *>> $logFile 21 | } 22 | 23 | # Run backup 24 | Set-Location $backupLocation 25 | duplicacy -log backup -stats -vss *>> $logFile 26 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: CI 2 | 3 | on: push 4 | 5 | jobs: 6 | build: 7 | name: Build 8 | runs-on: ubuntu-latest 9 | env: 10 | SC_VERSION: stable 11 | steps: 12 | - uses: actions/checkout@v2 13 | - name: Install Shellcheck 14 | run: | 15 | wget "https://github.com/koalaman/shellcheck/releases/download/${SC_VERSION?}/shellcheck-${SC_VERSION?}.linux.x86_64.tar.xz" 16 | wget "https://storage.googleapis.com/shellcheck/shellcheck-${SC_VERSION}.linux.x86_64.tar.xz" 17 | tar --xz -xvf "shellcheck-${SC_VERSION}.linux.x86_64.tar.xz" 18 | shellcheck() { "shellcheck-${SC_VERSION}/shellcheck" "$@"; } 19 | shellcheck --version 20 | - name: Check shell scripts 21 | run: find . -type f -name '*.sh' -exec shellcheck {} + 22 | -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contribute 2 | 3 | [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com) 4 | 5 | Bug reports and pull requests are welcome. When contributing to this repository, please first discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change. 6 | 7 | Please note we have a code of conduct, please follow it in all your interactions with the project. 8 | 9 | 1. Open an issue to discuss proposed changes 10 | 2. Fork the repository 11 | 3. Create your feature branch: `git checkout -b my-new-feature` 12 | 4. Commit your changes: `git commit -am 'Add some feature'` 13 | 5. Push to the branch: `git push origin my-new-feature` 14 | 6. Submit a pull request :D 15 | 16 | **Working on your first Pull Request?** You can learn how from this *free* series [How to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github) 17 | -------------------------------------------------------------------------------- /filters/osx: -------------------------------------------------------------------------------- 1 | # Exclude any cache files/directories with cache in the name (case insensitive) 2 | e:(?i).*cache.* 3 | 4 | # Exclude temporary file names 5 | e:.*/?~.*$ 6 | 7 | # Misc 8 | e:(?i)\.Trash/.*$ 9 | e:(?i).*$RECYCLE.BIN/ 10 | e:(?i)\.dropbox/.*$ 11 | e:(?i).*\.rnd$ 12 | e:(?i).*/Google/Chrome/.*cache.* 13 | e:(?i).*/Google/Chrome/Safe Browsing.* 14 | e:(?i).*/Mozilla/Firefox/.*cache.* 15 | e:(?i).*\.(gsheet|gdoc|gform|gdraw)$ 16 | 17 | # Libraries 18 | e:(?i)iTunes/Album Artwork/Cache/ 19 | e:(?i)Library/Application Support/Adobe/Acrobat/DC/Acrobat/Synchronizer/(Commands|Notification)$ 20 | e:(?i)Library/Application Support/AvastHUB/run/ 21 | e:(?i)Library/Application Support/SyncServices/ 22 | e:(?i)Library/Caches/ 23 | e:(?i)Library/Containers/ 24 | e:(?i)Library/Google/GoogleSoftwareUpdate/Stats/ 25 | e:(?i)Library/Group Containers/ 26 | e:(?i)Library/Logs/ 27 | e:(?i)Library/Mail/AvailableFeeds/ 28 | e:(?i)Library/Mail/Envelope Index/ 29 | e:(?i)Library/Mirrors/ 30 | e:(?i)Library/Mobile Documents/* 31 | e:(?i)Library/PubSub/Database/ 32 | e:(?i)Library/PubSub/Downloads/ 33 | e:(?i)Library/PubSub/Feeds/ 34 | e:(?i)Library/Safari/HistoryIndex.sk 35 | e:(?i)Library/Safari/Icons.db 36 | e:(?i)Library/Safari/WebpageIcons.db 37 | e:(?i)Library/Saved Application State/ 38 | 39 | # Dev 40 | e:(?i).*\.sock$ 41 | e:(?i)/\.bundle/ 42 | e:(?i)/\.rvm/ 43 | e:(?i)/\.serverless/ 44 | e:(?i)/\.terraform/ 45 | e:(?i)/node_modules/ 46 | e:(?i)\.(git|svn|hg)/.*$ 47 | e:(?i)\.gnupg/S\..*$ 48 | e:(?i)\.sqlite-.*$ 49 | -------------------------------------------------------------------------------- /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | 2 | # Contributor Covenant Code of Conduct 3 | 4 | ## Our Pledge 5 | 6 | We as members, contributors, and leaders pledge to make participation in our 7 | community a harassment-free experience for everyone, regardless of age, body 8 | size, visible or invisible disability, ethnicity, sex characteristics, gender 9 | identity and expression, level of experience, education, socio-economic status, 10 | nationality, personal appearance, race, religion, or sexual identity 11 | and orientation. 12 | 13 | We pledge to act and interact in ways that contribute to an open, welcoming, 14 | diverse, inclusive, and healthy community. 15 | 16 | ## Our Standards 17 | 18 | Examples of behavior that contributes to a positive environment for our 19 | community include: 20 | 21 | * Demonstrating empathy and kindness toward other people 22 | * Being respectful of differing opinions, viewpoints, and experiences 23 | * Giving and gracefully accepting constructive feedback 24 | * Accepting responsibility and apologizing to those affected by our mistakes, 25 | and learning from the experience 26 | * Focusing on what is best not just for us as individuals, but for the 27 | overall community 28 | 29 | Examples of unacceptable behavior include: 30 | 31 | * The use of sexualized language or imagery, and sexual attention or 32 | advances of any kind 33 | * Trolling, insulting or derogatory comments, and personal or political attacks 34 | * Public or private harassment 35 | * Publishing others' private information, such as a physical or email 36 | address, without their explicit permission 37 | * Other conduct which could reasonably be considered inappropriate in a 38 | professional setting 39 | 40 | ## Enforcement Responsibilities 41 | 42 | Community leaders are responsible for clarifying and enforcing our standards of 43 | acceptable behavior and will take appropriate and fair corrective action in 44 | response to any behavior that they deem inappropriate, threatening, offensive, 45 | or harmful. 46 | 47 | Community leaders have the right and responsibility to remove, edit, or reject 48 | comments, commits, code, wiki edits, issues, and other contributions that are 49 | not aligned to this Code of Conduct, and will communicate reasons for moderation 50 | decisions when appropriate. 51 | 52 | ## Scope 53 | 54 | This Code of Conduct applies within all community spaces, and also applies when 55 | an individual is officially representing the community in public spaces. 56 | Examples of representing our community include using an official e-mail address, 57 | posting via an official social media account, or acting as an appointed 58 | representative at an online or offline event. 59 | 60 | ## Enforcement 61 | 62 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 63 | reported to the community leaders responsible for enforcement at 64 | [INSERT CONTACT METHOD]. 65 | All complaints will be reviewed and investigated promptly and fairly. 66 | 67 | All community leaders are obligated to respect the privacy and security of the 68 | reporter of any incident. 69 | 70 | ## Enforcement Guidelines 71 | 72 | Community leaders will follow these Community Impact Guidelines in determining 73 | the consequences for any action they deem in violation of this Code of Conduct: 74 | 75 | ### 1. Correction 76 | 77 | **Community Impact**: Use of inappropriate language or other behavior deemed 78 | unprofessional or unwelcome in the community. 79 | 80 | **Consequence**: A private, written warning from community leaders, providing 81 | clarity around the nature of the violation and an explanation of why the 82 | behavior was inappropriate. A public apology may be requested. 83 | 84 | ### 2. Warning 85 | 86 | **Community Impact**: A violation through a single incident or series 87 | of actions. 88 | 89 | **Consequence**: A warning with consequences for continued behavior. No 90 | interaction with the people involved, including unsolicited interaction with 91 | those enforcing the Code of Conduct, for a specified period of time. This 92 | includes avoiding interactions in community spaces as well as external channels 93 | like social media. Violating these terms may lead to a temporary or 94 | permanent ban. 95 | 96 | ### 3. Temporary Ban 97 | 98 | **Community Impact**: A serious violation of community standards, including 99 | sustained inappropriate behavior. 100 | 101 | **Consequence**: A temporary ban from any sort of interaction or public 102 | communication with the community for a specified period of time. No public or 103 | private interaction with the people involved, including unsolicited interaction 104 | with those enforcing the Code of Conduct, is allowed during this period. 105 | Violating these terms may lead to a permanent ban. 106 | 107 | ### 4. Permanent Ban 108 | 109 | **Community Impact**: Demonstrating a pattern of violation of community 110 | standards, including sustained inappropriate behavior, harassment of an 111 | individual, or aggression toward or disparagement of classes of individuals. 112 | 113 | **Consequence**: A permanent ban from any sort of public interaction within 114 | the community. 115 | 116 | ## Attribution 117 | 118 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 119 | version 2.0, available at 120 | https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. 121 | 122 | Community Impact Guidelines were inspired by [Mozilla's code of conduct 123 | enforcement ladder](https://github.com/mozilla/diversity). 124 | 125 | [homepage]: https://www.contributor-covenant.org 126 | 127 | For answers to common questions about this code of conduct, see the FAQ at 128 | https://www.contributor-covenant.org/faq. Translations are available at 129 | https://www.contributor-covenant.org/translations. 130 | -------------------------------------------------------------------------------- /filters/windows: -------------------------------------------------------------------------------- 1 | # From https://github.com/mattjm/duplicacy-script 2 | e:.*\$RECYCLE\.BIN/.* 3 | e:.*/System\sVolume\sInformation/.* 4 | e:.*/RECYCLER/.* 5 | e:.*/I386.* 6 | e:.*/pagefile.sys 7 | e:.*/MSOCache.* 8 | e:.*UsrClass\.dat\.LOG 9 | e:.*UsrClass\.dat 10 | e:.*/Temporary\sInternet\sFiles/.* 11 | e:(?i).*ntuser.dat.* 12 | e:.*Local\sSettings/Temp.* 13 | e:.*AppData/Local/Temp.* 14 | e:.*AppData/Temp.* 15 | e:.*/Windows/Temp.* 16 | e:(?i).*/Microsoft.*/Windows/.*\.log 17 | e:.*/Microsoft.*/Windows/Cookies.* 18 | e:.*/Microsoft.*/RecoveryStore.* 19 | e:(?i).:/Config\\.Msi.* 20 | e:(?i).*\\.rbf 21 | e:.*/Windows/Installer.* 22 | e:.*/Application\sData/Application\sData.* 23 | e:(?i).:/Config\.Msi.* 24 | e:(?i).*\.rbf 25 | e:(?i).*/Microsoft.*/Windows/.*\.edb 26 | e:(?i).*/Google/Chrome/User\sData/Default/Cookies(-journal)?", "(?i).*/Safari/Library/Caches/.* 27 | e:.*\.tmp 28 | e:.*\.tmp/.* 29 | e:.*\.part 30 | e:.*/iPhoto Library/iPod\sPhoto\sCache/.* 31 | e:.*/Music/Subscription/.* 32 | e:(?i).*/Google/Chrome/.*cache.* 33 | e:(?i).*/Mozilla/Firefox/.*cache.* 34 | e:.*/Google/Chrome/Safe\sBrowsing.* 35 | e:.*/Google/Chrome/User\sData/Default/File\sSystem.* 36 | e:.*/Google/Chrome/User\sData/Default/IndexedDB.* 37 | e:.*/Google/Chrome/User\sData/Default/Local\sStorage.* 38 | e:.*/Google/Chrome/User\sData/Safe\sBrowsing.* 39 | e:.*/(cookies|permissions).sqllite(-.{3})? 40 | e:.*Downloads 41 | e:.*AppData/Local/Akamai.* 42 | e:.*AppData/Roaming/Amazon\sCloud\sDrive.* 43 | e:.*Dashlane/.* 44 | e:.*Spotify/.* 45 | e:.*AppData/Local/Packages.* 46 | e:.*AppData/Roaming/Slack.* 47 | e:.*go/.* 48 | e:.*backup_logs/duplicacy.log 49 | 50 | # From on https://github.com/TheBestPessimist/duplicacy-utils 51 | 52 | # .VCS subfolder 53 | e:(?i)(^|/)\.git/ 54 | e:(?i)(^|/)\.hg/ 55 | e:(?i)(^|/)\.svn/ 56 | 57 | # temp/cache files/folders everywhere 58 | e:(?i)\.(tmp|temp)$ 59 | e:(?i)(^|/)temp/ 60 | e:(?i)(^|/)tmp/ 61 | e:(?i)(^|/)cache/ 62 | e:(?i)/Package Cache/ 63 | 64 | # npm/node related 65 | e:(?i)/compile-cache/ 66 | e:(?i)/node_modules/ 67 | e:(?i)/npm-cache/ 68 | 69 | # Google Drive Cache 70 | e:(?i)/\.tmp\.drivedownload/ 71 | 72 | # Onedrive Cache 73 | e:(?i)/OneDriveTemp/ 74 | e:(?i)/OneDrive(.*)?/\..*-.*-.*-.*-.*$ 75 | 76 | # Dropbox Cache 77 | e:(?i)/\.dropbox\.cache/ 78 | 79 | # system files anywhere 80 | e:(?i)ntuser\.dat 81 | e:(?i)(^|/)thumbs\.db$ 82 | e:(?i)(^|/)IconCache\.db 83 | 84 | # unneeded folders in Users profiles: systems, junctions 85 | e:(?i)^[^/]+/Users/[^/]+/\.oracle_jre_usage/ 86 | e:(?i)^[^/]+/Users/[^/]+/Application Data 87 | e:(?i)^[^/]+/Users/[^/]+/Cookies 88 | e:(?i)^[^/]+/Users/[^/]+/IntelGraphicsProfiles 89 | e:(?i)^[^/]+/Users/[^/]+/Local Settings 90 | e:(?i)^[^/]+/Users/[^/]+/MicrosoftEdgeBackups/ 91 | e:(?i)^[^/]+/Users/[^/]+/NetHood 92 | e:(?i)^[^/]+/Users/[^/]+/PrintHood 93 | e:(?i)^[^/]+/Users/[^/]+/Recent 94 | e:(?i)^[^/]+/Users/[^/]+/Searches/ 95 | e:(?i)^[^/]+/Users/[^/]+/SendTo 96 | e:(?i)^[^/]+/Users/[^/]+/Start Menu 97 | e:(?i)^[^/]+/Users/[^/]+/Templates 98 | e:(?i)^[^/]+/Users/[^/]+/Tracing/ 99 | 100 | # unneeded folders in Users profiles: runtimes, dev, others 101 | e:(?i)^[^/]+/Users/[^/]+/\.?anaconda/ 102 | e:(?i)^[^/]+/Users/[^/]+/\.?android[^/]*/ 103 | e:(?i)^[^/]+/Users/[^/]+/\.?astropy/ 104 | e:(?i)^[^/]+/Users/[^/]+/\.?conda/ 105 | e:(?i)^[^/]+/Users/[^/]+/\.?dia/ 106 | e:(?i)^[^/]+/Users/[^/]+/\.?eclipse/ 107 | e:(?i)^[^/]+/Users/[^/]+/\.?gradle/ 108 | e:(?i)^[^/]+/Users/[^/]+/\.?kivy/ 109 | e:(?i)^[^/]+/Users/[^/]+/\.?matplotlib/ 110 | e:(?i)^[^/]+/Users/[^/]+/\.?Mendeley Desktop/ 111 | e:(?i)^[^/]+/Users/[^/]+/\.?p2/ 112 | e:(?i)^[^/]+/Users/[^/]+/\.?pgadmin./ 113 | e:(?i)^[^/]+/Users/[^/]+/\.?QtWeb/ 114 | 115 | # Global excludes 116 | # useless files anywhere, by extension 117 | e:(?i).*.msf*$ 118 | e:(?i)\.bac$ 119 | e:(?i)\.back$ 120 | e:(?i)\.bak$ 121 | e:(?i)\.bkp$ 122 | e:(?i)\.obj$ 123 | e:(?i)\.old$ 124 | e:(?i)\.part$ 125 | e:(?i)\.!ut$ 126 | e:(?i)\.bup$ 127 | e:(?i)\.cache$ 128 | e:(?i)\.crdownload$ 129 | e:(?i)\.dmp$ 130 | e:(?i)\.dov$ 131 | e:(?i)\.dump$ 132 | e:(?i)\.err$ 133 | e:(?i)\.ffs_lock$ 134 | e:(?i)\.ffs_tmp$ 135 | e:(?i)\.lck$ 136 | e:(?i)\.prv$ 137 | e:(?i)\.rbf$ 138 | 139 | # AppData folders with known useless-but-heavy content 140 | ## C:/Users//AppData/ holds app's configurations specific for each user 141 | e:(?i)/AppData/LocalLow/ 142 | e:(?i)/AppData/[^/]+/Acronis/ 143 | e:(?i)/AppData/[^/]+/Adobe/ 144 | e:(?i)/AppData/[^/]+/Atlassian/SourceTree/[^/]*local/ 145 | e:(?i)/AppData/[^/]+/Comms/ 146 | e:(?i)/AppData/[^/]+/Code/CachedData/ 147 | e:(?i)/AppData/[^/]+/ConnectedDevicesPlatform/ 148 | e:(?i)/AppData/[^/]+/CrashDumps/ 149 | e:(?i)/AppData/[^/]+/CrashPlan/ 150 | e:(?i)/AppData/[^/]+/DBG/ 151 | e:(?i)/AppData/[^/]+/Diagnostics/ 152 | e:(?i)/AppData/[^/]+/Downloaded Installations/ 153 | e:(?i)/AppData/[^/]+/Dropbox/ 154 | e:(?i)/AppData/[^/]+/Duplicati/ 155 | e:(?i)/AppData/[^/]+/ElevatedDiagnostics/ 156 | e:(?i)/AppData/[^/]+/JetBrains/ 157 | e:(?i)/AppData/[^/]+/npm/ 158 | e:(?i)/AppData/[^/]+/NVIDIA[^/]*/ 159 | e:(?i)/AppData/[^/]+/Oracle/ 160 | e:(?i)/AppData/[^/]+/PackageStaging/ 161 | e:(?i)/AppData/[^/]+/PeerDistRepub/ 162 | e:(?i)/AppData/[^/]+/Publishers/ 163 | e:(?i)/AppData/[^/]+/Sun/ 164 | e:(?i)/AppData/[^/]+/TileDataLayer/ 165 | e:(?i)/AppData/[^/]+/Veeam/ 166 | e:(?i)/AppData/[^/]+/VirtualStore/ 167 | e:(?i)/AppData/[^/]+/[^/]+/GPUCache/ 168 | ## C:/ProgramData/ holds app's configurations global to all users (every time you select "Install to all users") 169 | e:(?i)^[^/]+/ProgramData/Acronis/ 170 | e:(?i)^[^/]+/ProgramData/Adobe/ 171 | e:(?i)^[^/]+/ProgramData/Apple/Installer Cache/ 172 | e:(?i)^[^/]+/ProgramData/AVAST[^/]*/ 173 | e:(?i)^[^/]+/ProgramData/CrashPlan/ 174 | e:(?i)^[^/]+/ProgramData/Dell/drivers 175 | e:(?i)^[^/]+/ProgramData/Dropbox/ 176 | e:(?i)^[^/]+/ProgramData/Duplicati/ 177 | e:(?i)^[^/]+/ProgramData/Epson/ 178 | e:(?i)^[^/]+/ProgramData/Eset/ 179 | e:(?i)^[^/]+/ProgramData/GetSupportService/ 180 | e:(?i)^[^/]+/ProgramData/Hp/ 181 | e:(?i)^[^/]+/ProgramData/Intel[^/]*/ 182 | e:(?i)^[^/]+/ProgramData/Malwarebytes/ 183 | e:(?i)^[^/]+/ProgramData/Microsoft[^/]*/ 184 | e:(?i)^[^/]+/ProgramData/MiKTeX/ 185 | e:(?i)^[^/]+/ProgramData/Mozilla/ 186 | e:(?i)^[^/]+/ProgramData/NVIDIA[^/]*/ 187 | e:(?i)^[^/]+/ProgramData/Oracle/ 188 | e:(?i)^[^/]+/ProgramData/Packages/ 189 | e:(?i)^[^/]+/ProgramData/Skype/ 190 | e:(?i)^[^/]+/ProgramData/SoftwareDistribution/ 191 | e:(?i)^[^/]+/ProgramData/Soluto/ 192 | e:(?i)^[^/]+/ProgramData/SteelSeries/ 193 | e:(?i)^[^/]+/ProgramData/SupremoRemoteDesktop/ 194 | e:(?i)^[^/]+/ProgramData/SystemAcCrux/ 195 | e:(?i)^[^/]+/ProgramData/USOPrivate/ 196 | e:(?i)^[^/]+/ProgramData/USOShared/ 197 | e:(?i)^[^/]+/ProgramData/Veeam/ 198 | e:(?i)^[^/]+/ProgramData/WindowsHolographicDevices/ 199 | 200 | # Include microsoft email clients but nothing else from microsoft 201 | # There is an error with .nst/.ost files so these need to be excluded first 202 | # ERROR CHUNK_MAKER Failed to read 0 bytes: read C:\duplicacy/.duplicacy\shadow\\duplicacy/vlad/AppData/Local/Microsoft/Outlook/xxx@yyy.edu - zzz.nst: The process cannot access the file because another process has locked a portion of the file. 203 | e:(?i)/AppData/[^/]+/Microsoft/Outlook/.*\.(n|o)st$ 204 | i:(?i)/AppData/[^/]+/Microsoft/$ 205 | i:(?i)/AppData/[^/]+/Microsoft/(Outlook|Windows Mail|Windows Live Mail)/ 206 | e:(?i)/AppData/[^/]+/Microsoft[^/]*/ 207 | 208 | # Include Firefox profiles but nothing else from Mozilla 209 | # note that we include the whole profile, because we are unsure how many "users" are added beside the "Default" profile 210 | i:(?i)/AppData/[^/]+/Mozilla/$ 211 | i:(?i)/AppData/[^/]+/Mozilla/Firefox/ 212 | e:(?i)/AppData/[^/]+/Mozilla/ 213 | 214 | # Include Chrome profile and Picasa data, but nothing else from Google 215 | # note that we include the whole profile, because we are unsure how many "users" are added beside the "Default" profile 216 | i:(?i)/AppData/[^/]+/Google/$ 217 | i:(?i)/AppData/[^/]+/Google/Chrome/$ 218 | e:(?i)/AppData/[^/]+/Google/Chrome/.*Safe Browsing 219 | e:(?i)/AppData/[^/]+/Google/Chrome/User Data/Default/Current.* 220 | i:(?i)/AppData/[^/]+/Google/Chrome/User Data/ 221 | i:(?i)/AppData/[^/]+/Google/Picasa[^/]*/ 222 | e:(?i)/AppData/[^/]+/Google/ 223 | -------------------------------------------------------------------------------- /vbackup.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Bash strict mode 4 | set -euo pipefail 5 | IFS=$'\n\t' 6 | 7 | # DEFAULTS 8 | # The path to the source files 9 | export DUPLICACY_REPOSITORY_PATH="${DUPLICACY_REPOSITORY_PATH:-$(pwd -P)}" 10 | # The path to the dotenv file for this script 11 | export DUPLICACY_ENV_FILE="${DUPLICACY_ENV_FILE:-${DUPLICACY_REPOSITORY_PATH}/.duplicacy/.env}" 12 | # Set to 'true' to enable the Volume Shadow Copy service (Windows and macOS using APFS only) 13 | export DUPLICACY_VSS="${DUPLICACY_VSS:-false}" 14 | # Execute the script only when connected to the specified Wi-Fi SSID 15 | export DUPLICACY_SSID="${DUPLICACY_SSID:-}" 16 | # Number of seconds to wait 17 | export DUPLICACY_TIMEOUT="${DUPLICACY_TIMEOUT:-300}" 18 | # Number of uploading threads 19 | export DUPLICACY_THREADS="${DUPLICACY_THREADS:-4}" 20 | # Extra storage (ex: B2) 21 | export DUPLICACY_EXTRA_STORAGE="${DUPLICACY_EXTRA_STORAGE:-}" 22 | # The Slack webhook used for posting messages 23 | export SLACK_ALERTS_WEBHOOK="${SLACK_ALERTS_WEBHOOK:-}" 24 | # The webhook used for healthchecks (HealthChecks.io) 25 | export HEALTHCHECKS_URL="${HEALTHCHECKS_URL:-}" 26 | # Export PATH if needed by scripts that do not load the entire environment 27 | export PATH="${PATH}:/bin:/sbin:/usr/bin:/usr/local/bin:/usr/local/opt/coreutils/libexec/gnubin" 28 | 29 | # Log format (ex: log INFO Message) 30 | # Levels: DEBUG, INFO, WARN, ERROR 31 | log(){ 32 | local type=${1:?Must specify the type first}; shift 33 | echo "$(date '+%Y-%m-%d %H:%M:%S.%3N') ${type} DUPLICACY_SCRIPT ${*:-}" 34 | } 35 | 36 | # Check if command exists 37 | is_cmd(){ 38 | command -v "$@" >/dev/null 2>&1 39 | } 40 | 41 | # Load .env file 42 | load_dotenv(){ 43 | if [[ -s "$DUPLICACY_ENV_FILE" ]]; then 44 | # shellcheck disable=1090 45 | . "$DUPLICACY_ENV_FILE" 46 | fi 47 | } 48 | 49 | # Post message to Slack webhook 50 | notify(){ 51 | if [[ -n "$SLACK_ALERTS_WEBHOOK" ]]; then 52 | log INFO 'Notify Slack' 53 | /usr/bin/curl --silent --output /dev/null --show-error --fail --request POST \ 54 | --header 'Content-type: application/json' \ 55 | --data "{\"text\":\"${1:?Must specify the message}\"}" \ 56 | "$SLACK_ALERTS_WEBHOOK" 57 | fi 58 | } 59 | 60 | # Ensure that the repository is initialized 61 | check_repository_initialized(){ 62 | if [[ ! -s "${DUPLICACY_REPOSITORY_PATH}/.duplicacy/preferences" ]]; then 63 | log ERROR 'The repository is not initialized'; exit 1 64 | fi 65 | } 66 | 67 | # Check if the process is already running 68 | check_process_running(){ 69 | if [ -f "$DUPLICACY_PID_FILE" ]; then 70 | PID=$(cat "$DUPLICACY_PID_FILE") 71 | if ps -p "$PID" >/dev/null 2>&1; then 72 | log WARN 'Process already running'; exit 3 73 | else 74 | if ! echo $$ > "$DUPLICACY_PID_FILE"; then 75 | log ERROR 'Could not create PID file'; exit 1 76 | fi 77 | fi 78 | else 79 | if ! echo $$ > "$DUPLICACY_PID_FILE"; then 80 | log ERROR 'Could not create PID file'; exit 1 81 | fi 82 | fi 83 | } 84 | 85 | # Execute the script only when connected to the specified Wi-Fi SSID 86 | check_ssid(){ 87 | if [[ -z "$DUPLICACY_SSID" ]]; then return; fi 88 | if [[ ! -x /System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport ]]; then return; fi 89 | 90 | if [[ "$(/System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F': ' '/AirPort/{print $2}')" == 'Off' ]]; then 91 | log INFO 'The backup will be skipped because Wi-Fi is Off (probably sleeping)'; exit 2 92 | elif [[ "$(/System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F': ' '/ SSID/{print $2}')" != "$DUPLICACY_SSID" ]]; then 93 | log INFO 'The backup will be skipped for the current Wi-Fi SSID'; exit 2 94 | fi 95 | } 96 | 97 | # This is here because of tmutil timeout errors (`tmutil localsnapshot` is used for VSS - Shadow Copy) 98 | wait_for_tmutil(){ 99 | if ! is_cmd tmutil; then return; fi 100 | 101 | until tmutil listlocalsnapshots / >/dev/null 2>&1 || [[ $((DUPLICACY_TIMEOUT--)) == 0 ]]; do 102 | log INFO 'Waiting for tmutil...'; sleep 5 103 | done 104 | 105 | if ! tmutil listlocalsnapshots / >/dev/null 2>&1; then 106 | log ERROR 'Tmutil did not respond in a timely manner'; exit 1 107 | fi 108 | } 109 | 110 | # Initialize script 111 | do_initialize(){ 112 | # Initialize trap 113 | trap 'clean_up $?' EXIT HUP INT QUIT TERM 114 | 115 | # Load settings 116 | load_dotenv 117 | 118 | # Log everything to file 119 | mkdir -p "$(dirname "$DUPLICACY_LOG_FILE")" 120 | exec > >(tee -a "$DUPLICACY_LOG_FILE") 2>&1 121 | 122 | # Sanity checks 123 | check_repository_initialized 124 | check_process_running 125 | check_ssid 126 | 127 | # Wait for other processes 128 | wait_for_tmutil 129 | } 130 | 131 | # Concatenate command 132 | concatenate_duplicacy_cmd(){ 133 | duplicacy_cmd='duplicacy -log backup -stats' 134 | 135 | # Use `caffeinate` command if available 136 | if is_cmd caffeinate; then 137 | duplicacy_cmd="caffeinate -s ${duplicacy_cmd}" 138 | fi 139 | 140 | # Enable the Volume Shadow Copy service 141 | if [[ "$DUPLICACY_VSS" == 'true' ]]; then 142 | duplicacy_cmd="${duplicacy_cmd} -vss" 143 | fi 144 | 145 | # Use the specified extra storage 146 | if [[ -n "$DUPLICACY_EXTRA_STORAGE" ]]; then 147 | duplicacy_cmd="${duplicacy_cmd} -storage ${DUPLICACY_EXTRA_STORAGE}" 148 | fi 149 | } 150 | 151 | # Run backup (use caffeinate command if it exists to prevent sleeping on MacOS) 152 | do_backup(){ 153 | # The path to the log file for this script 154 | export DUPLICACY_LOG_FILE="${DUPLICACY_REPOSITORY_PATH}/.duplicacy/logs/backup.log" 155 | # The path to the file containing the pid of the running process 156 | export DUPLICACY_PID_FILE="${DUPLICACY_REPOSITORY_PATH}/.duplicacy/running.pid" 157 | 158 | # Initialize script 159 | do_initialize 160 | 161 | # Concatenate command 162 | concatenate_duplicacy_cmd 163 | 164 | # Run 165 | log INFO 'Start backup' 166 | eval "${duplicacy_cmd:-}" 167 | } 168 | 169 | # Prune local storage 170 | prune_local_snapshots(){ 171 | # -keep [+] keep 1 snapshot every n days for snapshots older than m days 172 | # Keep no snapshots older than 1825 days 173 | # Keep 1 snapshot every 30 days if older than 180 days 174 | # Keep 1 snapshot every 7 days if older than 30 days 175 | # Keep 1 snapshot every 1 day if older than 7 days 176 | log INFO 'Prune local snapshots' 177 | duplicacy -log prune -all -keep 0:1825 -keep 30:180 -keep 7:30 -keep 1:7 || notify "Prune local snapshots failed with exit code '$?'. Skipping..." 178 | } 179 | 180 | # Prune remote storage 181 | prune_remote_snapshots(){ 182 | if [[ -n "$DUPLICACY_EXTRA_STORAGE" ]]; then 183 | log INFO 'Prune remote snapshots' 184 | duplicacy -log prune -all -keep 0:1825 -keep 30:180 -keep 7:30 -keep 1:7 -storage "$DUPLICACY_EXTRA_STORAGE" || notify "Prune remote snapshots failed with exit code '$?'. Skipping..." 185 | fi 186 | } 187 | 188 | # Copy to external storage 189 | copy_snapshots(){ 190 | if [[ -n "$DUPLICACY_EXTRA_STORAGE" ]]; then 191 | log INFO 'Copy snapshots' 192 | duplicacy -log copy -to "$DUPLICACY_EXTRA_STORAGE" -threads "$DUPLICACY_THREADS" || notify "Copy snapshots failed with exit code '$?'. Skipping..." 193 | fi 194 | } 195 | 196 | # Run maintenance 197 | do_maintenance(){ 198 | # The path to the log file for this script 199 | export DUPLICACY_LOG_FILE="${DUPLICACY_REPOSITORY_PATH}/.duplicacy/logs/maintenance.log" 200 | # The path to the file containing the pid of the running process 201 | export DUPLICACY_PID_FILE="${DUPLICACY_REPOSITORY_PATH}/.duplicacy/running-maintenance.pid" 202 | 203 | # Initialize script 204 | do_initialize 205 | 206 | # Copy to external storage 207 | copy_snapshots 208 | 209 | # Prune local storage 210 | prune_local_snapshots 211 | 212 | # Prune remote storage 213 | prune_remote_snapshots 214 | 215 | # Notify HealthChecks.io 216 | if [[ -n "$HEALTHCHECKS_URL" ]]; then 217 | curl --silent --output /dev/null --show-error --fail --retry 3 "$HEALTHCHECKS_URL" 218 | fi 219 | 220 | log INFO 'MAINTENANCE_END' 221 | } 222 | 223 | # Clean-up and notify 224 | clean_up(){ 225 | # Exit codes: 1 - Errors; 2 - Skipped; 3 - Already running 226 | # If it's already running, just clean exit directly 227 | if [[ ${1:-0} == 3 ]]; then return; fi 228 | 229 | # Remove the PID file 230 | if [[ -s "$DUPLICACY_PID_FILE" ]]; then rm -f "$DUPLICACY_PID_FILE"; fi 231 | 232 | # If it's skipped, just clean exit (after cleaning the PID file) 233 | if [[ ${1:-0} == 2 ]]; then return; fi 234 | 235 | # For all other cases notify failure (after cleaning the PID file) 236 | if [[ ${1:-0} != 0 ]]; then 237 | log ERROR "Backup Failed with exit code '${1}'" 238 | notify "Backup Failed on $(hostname) ($(date))" 239 | fi 240 | } 241 | 242 | # Script 243 | main(){ 244 | # Process command line arguments 245 | local -r cmd="${1:-backup}"; shift 246 | 247 | # Go to the repository path 248 | cd "$DUPLICACY_REPOSITORY_PATH" || exit 1 249 | 250 | case "$cmd" in 251 | backup) 252 | # Backup 253 | do_backup 254 | ;; 255 | maintenance) 256 | # Maintenance 257 | do_maintenance 258 | ;; 259 | *) 260 | # Default 261 | log ERROR "'${cmd}' is not recognized as a valid command"; exit 1 262 | ;; 263 | esac 264 | } 265 | 266 | # Run 267 | main "${@:-}" 268 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Vlad's Duplicacy Backup Scripts 2 | 3 | ![Build Status](https://github.com/vladgh/backup/workflows/CI/badge.svg) 4 | 5 | ## Install Duplicacy 6 | 7 | Download latest release from and add it to PATH 8 | 9 | Linux 10 | 11 | ```sh 12 | sudo wget -O /usr/local/bin/duplicacy https://github.com/gilbertchen/duplicacy/releases/download/v2.5.2/duplicacy_linux_x64_2.5.2 && sudo chmod 755 /usr/local/bin/duplicacy 13 | ``` 14 | 15 | MacOS 16 | 17 | ```sh 18 | wget -O /usr/local/bin/duplicacy https://github.com/gilbertchen/duplicacy/releases/download/v2.5.2/duplicacy_osx_x64_2.5.2 && chmod 755 /usr/local/bin/duplicacy 19 | ``` 20 | 21 | Windows 22 | 23 | ```powershell 24 | # Requires Administrator rights 25 | Invoke-WebRequest -Uri "https://github.com/gilbertchen/duplicacy/releases/download/v2.5.2/duplicacy_win_x64_2.5.2.exe" -OutFile "C:\Windows\System32\duplicacy.exe" 26 | ``` 27 | 28 | ## Initialize the repository 29 | 30 | Pick the desired location for the .duplicacy folder, this will be your repository root. Defaults to your user's home. 31 | When you initialize the repository, it will ask for the encryption password and the path to the RSA key used by SFTP. 32 | 33 | ```sh 34 | duplicacy init -encrypt MyID sftp://user@192.168.1.2//path/to/backup/storage 35 | ``` 36 | 37 | By default Duplicacy will follow the first-level symlinks (those under the root of the repository). 38 | Symlinks located under any subdirectories of the repository will be backed up as symlinks and will not be followed. 39 | 40 | For example, on Windows: 41 | 42 | ```powershell 43 | mkdir C:\backup 44 | cd C:\backup 45 | cmd /c mklink /D myname C:\Users\myname 46 | cmd /c mklink /D docs D:\DOCs 47 | ``` 48 | 49 | ## Configure the repository 50 | 51 | ```sh 52 | duplicacy set -storage default -key password -value 'MyPassword' 53 | duplicacy set -storage default -key ssh_key_file -value '/path/to/.ssh/duplicacy_rsa' 54 | ``` 55 | 56 | Place the appropriate filters file inside the `.duplicacy` folder 57 | 58 | ```sh 59 | wget -O ~/.duplicacy/filters https://github.com/vladgh/backup/raw/master/filters/osx 60 | ``` 61 | 62 | For Windows 63 | 64 | ```powershell 65 | Invoke-WebRequest -Uri "https://github.com/vladgh/backup/raw/master/filters/windows" -OutFile "C:\backup\.duplicacy\filters" 66 | ``` 67 | 68 | ## Backup 69 | 70 | ```sh 71 | duplicacy -log backup -stats -vss 72 | ``` 73 | 74 | ## VBackup Script OSX 75 | 76 | Download the OSX/Linux backup script `vbackup.sh`, add it to PATH and make it executable 77 | 78 | ```sh 79 | wget -O /usr/local/bin/vbackup https://github.com/vladgh/backup/raw/master/vbackup.sh 80 | chmod a+x /usr/local/bin/vbackup 81 | ``` 82 | 83 | Use a dotenv file for secrets and other settings 84 | 85 | ```sh 86 | tee ~/.duplicacy/.env << CONFIG 87 | SLACK_ALERTS_WEBHOOK='https://hooks.slack.com/services/xxxxxxxxxx' 88 | CONFIG 89 | chmod 600 ~/.duplicacy/.env 90 | ``` 91 | 92 | Update the paths and settings in the sample .plist launch script below, copy it to ~/Library/LaunchAgents and load it (-w enables it at the next boot) 93 | 94 | ```sh 95 | # Install launch agent 96 | tee ~/Library/LaunchAgents/duplicacy.plist << 'EOF' 97 | 98 | 99 | 100 | 101 | Label 102 | duplicacy.startup 103 | ProgramArguments 104 | 105 | /usr/local/bin/bash 106 | /usr/local/bin/vbackup 107 | 108 | RunAtLoad 109 | 110 | UserName 111 | vlad 112 | GroupName 113 | staff 114 | WorkingDirectory 115 | /Users/vlad 116 | StandardErrorPath 117 | /dev/null 118 | StandardOutPath 119 | /dev/null 120 | StartInterval 121 | 3600 122 | 123 | 124 | EOF 125 | 126 | # Load/Unload agent 127 | launchctl load -w ~/Library/LaunchAgents/duplicacy.plist 128 | ``` 129 | 130 | Restart with 131 | 132 | ```sh 133 | launchctl unload -w ~/Library/LaunchAgents/duplicacy.plist && launchctl load -w ~/Library/LaunchAgents/duplicacy.plist 134 | ``` 135 | 136 | ## VBackup Script Linux 137 | 138 | Download the OSX/Linux backup script `vbackup.sh`, add it to PATH and make it executable 139 | 140 | ```sh 141 | sudo wget -O /usr/local/bin/vbackup https://github.com/vladgh/backup/raw/master/vbackup.sh 142 | sudo chmod a+x /usr/local/bin/vbackup 143 | ``` 144 | 145 | Use a dotenv file for secrets and other settings 146 | 147 | ```sh 148 | tee ~/.duplicacy/.env << CONFIG 149 | SLACK_ALERTS_WEBHOOK='https://hooks.slack.com/services/xxxxxxxxxx' 150 | CONFIG 151 | chmod 600 ~/.duplicacy/.env 152 | ``` 153 | 154 | Use SystemD timers to trigger the backup service 155 | 156 | ```sh 157 | tee ~/.config/systemd/user/vbackup.service << 'EOF' 158 | [Unit] 159 | Description=Run VBackup 160 | After=network-online.target 161 | Wants=network-online.target 162 | 163 | [Service] 164 | Type=oneshot 165 | WorkingDirectory=%h 166 | ExecStart=/usr/local/bin/vbackup 167 | EOF 168 | 169 | tee /home/vlad/.config/systemd/user/vbackup.timer << 'EOF' 170 | [Unit] 171 | Description=Run VBackup 172 | 173 | [Timer] 174 | OnBootSec=15min 175 | OnUnitActiveSec=60min 176 | 177 | [Install] 178 | WantedBy=timers.target 179 | EOF 180 | 181 | systemctl --user --now enable vbackup.timer 182 | systemctl --user list-timers --all 183 | ``` 184 | 185 | Alternatively, create a cronjob to run the script as your user, at some interval 186 | 187 | ```sh 188 | sudo tee /etc/cron.d/vbackup << CONFIG 189 | 10 * * * * vlad /usr/local/bin/vbackup.sh >/dev/null 2>&1 190 | CONFIG 191 | ``` 192 | 193 | ## VBackup Script Windows 194 | 195 | Download the Windows backup script `vbackup.ps1` 196 | 197 | ```powershell 198 | Invoke-WebRequest -Uri "https://github.com/vladgh/backup/raw/master/vbackup.ps1" -OutFile "C:\backup\vbackup.ps1" 199 | ``` 200 | 201 | Import the sample xml below into the Task Scheduler to run the powershell script at some interval with elevated privileges. Modify as needed! 202 | 203 | ```xml 204 | 205 | 206 | 207 | MyPC\MyName 208 | \DuplicacyBackup 209 | 210 | 211 | 212 | 213 | PT1H 214 | P10000D 215 | true 216 | 217 | 2019-05-02T06:00:00+02:00 218 | true 219 | PT5M 220 | 221 | 222 | 223 | 224 | MyPC\MyName 225 | Password 226 | HighestAvailable 227 | 228 | 229 | 230 | IgnoreNew 231 | true 232 | false 233 | true 234 | true 235 | false 236 | 237 | true 238 | false 239 | 240 | true 241 | true 242 | false 243 | false 244 | false 245 | true 246 | false 247 | PT72H 248 | 7 249 | 250 | 251 | 252 | powershell.exe 253 | -NoProfile -ExecutionPolicy Bypass -File "C:\backup\vbackup.ps1" -Verb RunAs; quit 254 | 255 | 256 | 257 | ``` 258 | 259 | Manually trigger scheduled task 260 | 261 | ```powershell 262 | Start-ScheduledTask -TaskName "DuplicacyBackup" 263 | ``` 264 | 265 | --- 266 | 267 | ## Recovery 268 | 269 | Create a folder to restore files 270 | 271 | ```sh 272 | mkdir -p /tmp/restore 273 | cd /tmp/restore 274 | ``` 275 | 276 | Initialize with THE SAME ID AND PASSWORD as the one in the storage 277 | 278 | ```sh 279 | duplicacy init -encrypt MyID sftp://user@192.168.1.2//path/to/backup/storage 280 | ``` 281 | 282 | Restore the wanted path from the wanted revision 283 | 284 | ```sh 285 | duplicacy -log restore -r 604 -stats 'Dropbox/Projects/*' 286 | ``` 287 | 288 | Look for files, use the list command and grep 289 | 290 | ```sh 291 | duplicacy list -r 12 -files | grep pattern 292 | duplicacy list -r 1-100 -files | grep pattern 293 | ``` 294 | 295 | For Windows 296 | 297 | ```powershell 298 | duplicacy list -r 12 -files | Select-String -Pattern myFile 299 | duplicacy list -r 1-100 -files | Select-String -Pattern myFile 300 | ``` 301 | 302 | ## Restore local default storage from remote 303 | 304 | ```sh 305 | # With B2 configured in preferences, copy from B2 to default, with bit-identical and set the id and path 306 | duplicacy add -e -copy B2 -bit-identical default MyID /path/to/backup/storage 307 | ``` 308 | 309 | ## Fix corrupted snapshot 310 | 311 | Make sure no other backups are running and delete the corrupted snapshot 312 | 313 | ```sh 314 | duplicacy check -a -tabular # Check all snapshots (this is better than individual id checks, because of deduplication, when the corrupted chunks could belong to different IDs) 315 | duplicacy check -tabular -id VDev -r 1510-1520 # OR check a range of snapshots 316 | duplicacy check -tabular -storage B2 -id VDev -r 1510-1520 # OR check a range of snapshots for a single ID on a remote storage 317 | 318 | duplicacy prune -exhaustive -exclusive -id VDev -r 1515 # Remove the corrupted snapshot 319 | 320 | # IF `duplicacy check -a -tabular` fails with: 321 | # All chunks referenced by snapshot VDev at revision 5114 exist 322 | # Chunk cc86bc8351524fa2d17f65abd38b8609489a20410370965291de881b21b15226 can't be found 323 | # Go and delete the `snapshots/VDev/5115` file 324 | # Then run `duplicacy prune -exhaustive -exclusive` 325 | ``` 326 | 327 | --- 328 | 329 | ## Miscellaneous 330 | 331 | ### Create log rotation script on Mac OS 332 | 333 | ```sh 334 | sudo tee /etc/newsyslog.d/duplicacy.conf << CONFIG 335 | # logfilename [owner:group] mode count size when flags [/pid_file] [sig_num] 336 | /Users/vlad/.duplicacy/logs/backup.log vlad:staff 644 10 1000 * NJ 337 | CONFIG 338 | ``` 339 | 340 | ### Log rotation in Linux 341 | 342 | ```sh 343 | sudo tee /etc/logrotate.d/duplicacy << CONFIG 344 | /home/vlad/.duplicacy/logs/backup.log { 345 | daily 346 | rotate 14 347 | compress 348 | delaycompress 349 | missingok 350 | notifempty 351 | create 644 vlad vlad 352 | } 353 | CONFIG 354 | ``` 355 | 356 | ### Full disk access on Mac OS 357 | 358 | This script needs Full Disk Access so we need to add `/usr/local/bin/bash` to Security & Privacy - Privacy - Full Disk Access (Use CMD+SHIFT+. to show hidden files in the Open file dialog). 359 | The .plist file should run with ProgramArguments, `bash` being the first one (or environment variables). 360 | 361 | ### Linux tail Logs 362 | 363 | ```sh 364 | tail -f ~/.duplicacy/logs/backup.log 365 | grep -A 1 -e 'INFO BACKUP_END' ~/.duplicacy/logs/backup.log 366 | ``` 367 | 368 | ### PowerShell tail logs 369 | 370 | ```powershell 371 | Get-Content C:\backup\.duplicacy\logs\backup.log -Tail 20 -Wait 372 | ``` 373 | 374 | ### PowerShell pattern in logs 375 | 376 | ```powershell 377 | # All 378 | Get-Content C:\backup\.duplicacy\logs\backup.log | Select-String -Pattern 'INFO BACKUP_END' -Context 0,1 379 | # In the last 1000 lines 380 | Get-Content C:\backup\.duplicacy\logs\backup.log -Tail 1000 | Select-String -Pattern 'INFO BACKUP_END' -Context 0,1 381 | ``` 382 | 383 | ### Mount backup drive, and save credentials on Windows 384 | 385 | In Windows 10, the credentials need to include the workgroup (ex: WORKGROUP\myname) 386 | 387 | ```powershell 388 | net use Z: \\192.168.1.2\PathToBackup /savecred /persistent:yes 389 | ``` 390 | 391 | ### Clone storage on Backblaze B2 392 | 393 | Create a duplicate encrypted storage on Backblaze B2 394 | 395 | ```sh 396 | duplicacy add -encrypt -copy default -bit-identical B2 MyID b2://vladgh 397 | ``` 398 | 399 | Add keys and passwords to preferences 400 | 401 | ```sh 402 | duplicacy set -storage default -key password -value 'mypwd' 403 | duplicacy set -storage B2 -key b2_id -value 'xxxxx' 404 | duplicacy set -storage B2 -key b2_key -value 'xxxxx' 405 | duplicacy set -storage B2 -key password -value 'mypwd' 406 | ``` 407 | 408 | Copy all snapshots to the new storage 409 | 410 | ```sh 411 | duplicacy -log copy -to B2 -threads 10 412 | ``` 413 | 414 | Prune (run only from the NAS for both storages) 415 | 416 | ```sh 417 | duplicacy -log prune -all -keep 0:1825 -keep 30:180 -keep 7:30 -keep 1:7 418 | duplicacy -log prune -all -keep 0:1825 -keep 30:180 -keep 7:30 -keep 1:7 -storage B2 419 | ``` 420 | 421 | ## Contribute 422 | 423 | [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-007ba7.svg)](https://www.contributor-covenant.org/version/2/0/code_of_conduct.html) 424 | 425 | Contributions are always welcome! Please read the [contribution guidelines](.github/CONTRIBUTING.md) and the [code of conduct](.github/CODE_OF_CONDUCT.md). 426 | 427 | ## License 428 | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) 429 | 430 | Licensed under the Apache License, Version 2.0. 431 | See [LICENSE](LICENSE) file. 432 | --------------------------------------------------------------------------------