├── .gitignore ├── .scripts ├── dev-support │ ├── Experiments │ │ ├── _error-log-test-sub2 │ │ ├── _error-log-test-sub1 │ │ ├── error-log-test.sh │ │ ├── boxcryptor-timestamp-test-lib.ps1 │ │ ├── timestamp-test-move.sh │ │ ├── boxcryptor-timestamp-test.sh │ │ └── timestamp-test-copy.sh │ └── Notes │ │ └── Developer Notes.md ├── utils │ └── show-error-log ├── lib │ ├── get-keepass-db-dir │ ├── has-new-errors │ ├── get-cloud-sync-dir │ ├── process-new-errors │ ├── is-included-db │ ├── get-support-dir │ ├── log-errors │ ├── windows-user-profile-path │ ├── safe-filecopy │ ├── safe-is-binary-same │ ├── get-error-log │ ├── safe-file-exists │ ├── get-config-setting │ └── wsl-windows-path ├── has-new-errors.sh ├── get-cloud-sync-dir.sh ├── process-new-errors.sh ├── is-included-db.sh ├── get-import-dir.sh ├── sync-outgoing-changes--post-sync.sh ├── sync-incoming-changes--post-sync.sh ├── has-cloud-copy-changed.sh ├── sync-outgoing-changes--pre-sync.sh └── sync-incoming-changes--pre-sync.sh ├── Config ├── sample.sync.conf └── sync.defaults.conf ├── License ├── Trigger Definitions └── sync-triggers.xml └── Readme.md /.gitignore: -------------------------------------------------------------------------------- 1 | # Specific exclusions 2 | /Config/sync.conf 3 | Backup/ 4 | Logs/ 5 | Sync/ 6 | .sync/ 7 | /.scripts/dev-support/Experiments/ERROR_LOG_TEST.log 8 | /.scripts/dev-support/Experiments/test_data.dat 9 | 10 | # Jetbrains 11 | .idea/ 12 | 13 | # Generic exclusions 14 | tmp/ 15 | *.orig 16 | .DS_Store 17 | Thumbs.db 18 | 19 | # Local 20 | _ dev/ 21 | _ originals/ 22 | todo.md 23 | _ignore-tmp/ 24 | .scripts - old/ 25 | -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/_error-log-test-sub2: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -u 3 | 4 | PROGNAME="$(basename "$BASH_SOURCE")" 5 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 6 | 7 | export PATH="$progdir:$PATH" 8 | 9 | LOG="$progdir/ERROR_LOG_TEST.log" 10 | 11 | # See sub1 for the rationale behind the `sleep` 12 | exec 2> >(sleep 0.1; tee -a "$LOG" >&2) 13 | 14 | echo "This is an error message from $PROGNAME" >&2 -------------------------------------------------------------------------------- /Config/sample.sync.conf: -------------------------------------------------------------------------------- 1 | # User-defined sync configuration 2 | # 3 | # The sync settings are defined in sync.defaults.conf. If you need to tweak one 4 | # of them, copy the setting and modify it here. 5 | 6 | # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 7 | # 8 | # NO BACKSLASH AT THE END of a path to a directory! 9 | # 10 | # Example: 11 | # - CLOUD_SYNC_DIR="D:\Dropbox\Secrets" OK 12 | # - CLOUD_SYNC_DIR="D:\Dropbox\Secrets\" WRONG! 13 | # 14 | # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 15 | 16 | -------------------------------------------------------------------------------- /.scripts/utils/show-error-log: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the content of the error log. 4 | # 5 | # If the error log does not exist or is empty, nothing is returned. 6 | 7 | set -eu 8 | 9 | # Absolute paths to the script and the library directory. 10 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 11 | libdir="$(realpath "$progdir/../lib")" 12 | 13 | export PATH="$progdir:$libdir:$PATH" 14 | 15 | ERROR_LOG="$(get-error-log)" || exit $? 16 | cat "$ERROR_LOG" -------------------------------------------------------------------------------- /.scripts/lib/get-keepass-db-dir: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the path of the Keepass database directory, where the local Keepass KDBX files are stored. 4 | # The path is returned in Linux format. 5 | # 6 | # The directory is known to exist and is accessible. It is the root directory of the application and 7 | # contains the .admin directory. 8 | 9 | set -eu 10 | 11 | # Absolute path to the script. 12 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 13 | 14 | realpath "$progdir/../../.." 15 | -------------------------------------------------------------------------------- /.scripts/has-new-errors.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Top-level wrapper script for lib/has-new-errors. 4 | 5 | set -eu 6 | 7 | # Absolute path to the script. 8 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 9 | 10 | export PATH="$progdir/lib:$PATH" 11 | 12 | # Set up error reporting 13 | close_log() { exec 2>&-; wait $log_process_id; } 14 | end_script_with_status() { close_log; exit "${1:-0}"; } 15 | 16 | exec 2> >(log-errors) 17 | log_process_id=$! 18 | 19 | # Task 20 | has-new-errors 21 | 22 | end_script_with_status $? -------------------------------------------------------------------------------- /.scripts/get-cloud-sync-dir.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Top-level wrapper script for lib/get-cloud-sync-dir. 4 | 5 | set -eu 6 | 7 | # Absolute path to the script. 8 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 9 | 10 | export PATH="$progdir/lib:$PATH" 11 | 12 | # Set up error reporting 13 | close_log() { exec 2>&-; wait $log_process_id; } 14 | end_script_with_status() { close_log; exit "${1:-0}"; } 15 | 16 | exec 2> >(log-errors) 17 | log_process_id=$! 18 | 19 | # Task 20 | get-cloud-sync-dir 21 | 22 | end_script_with_status $? 23 | -------------------------------------------------------------------------------- /.scripts/process-new-errors.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Top-level wrapper script for lib/process-new-errors. 4 | 5 | set -eu 6 | 7 | # Absolute path to the script. 8 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 9 | 10 | export PATH="$progdir/lib:$PATH" 11 | 12 | # Set up error reporting 13 | close_log() { exec 2>&-; wait $log_process_id; } 14 | end_script_with_status() { close_log; exit "${1:-0}"; } 15 | 16 | exec 2> >(log-errors) 17 | log_process_id=$! 18 | 19 | # Task 20 | process-new-errors 21 | 22 | end_script_with_status $? 23 | -------------------------------------------------------------------------------- /.scripts/is-included-db.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Top-level wrapper script for lib/is-included-db. 4 | # 5 | # For exit codes, see there. 6 | 7 | set -eu 8 | 9 | # Absolute path to the script. 10 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 11 | 12 | export PATH="$progdir/lib:$PATH" 13 | 14 | # Set up error reporting 15 | close_log() { exec 2>&-; wait $log_process_id; } 16 | end_script_with_status() { close_log; exit "${1:-0}"; } 17 | 18 | exec 2> >(log-errors) 19 | log_process_id=$! 20 | 21 | # Task 22 | is-included-db "$@" 23 | 24 | end_script_with_status $? -------------------------------------------------------------------------------- /.scripts/lib/has-new-errors: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Checks if the log for new errors exists and is not empty. Sets exit code 0 (truthy) if that is the 4 | # case, or if an error occurred during execution (!). Sets exit code 1 (falsy) if the log is empty 5 | # or nonexistent. Does not create output. 6 | 7 | set -u 8 | 9 | # Absolute path to the script. 10 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 11 | 12 | export PATH="$progdir:$PATH" 13 | 14 | NEW_ERRORS_LOG="$(get-error-log --new-errors)" || exit 0 # Runtime error converted to exit status 0! 15 | 16 | [ -s "$NEW_ERRORS_LOG" ] -------------------------------------------------------------------------------- /.scripts/get-import-dir.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the path of the import directory (which is the temp-import directory) in Windows format. 4 | # Throws an error if the directory does not exist or is not accessible. 5 | 6 | set -eu 7 | 8 | # Script name 9 | PROGNAME=$(basename "$0") 10 | 11 | # Absolute path to the script. 12 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 13 | 14 | export PATH="$progdir/lib:$PATH" 15 | 16 | # Set up error reporting 17 | close_log() { exec 2>&-; wait $log_process_id; } 18 | end_script_with_status() { close_log; exit "${1:-0}"; } 19 | fatal_error() { echo -e "$PROGNAME: $1" >&2; end_script_with_status 1; } 20 | 21 | exec 2> >(log-errors) 22 | log_process_id=$! 23 | 24 | # Task 25 | get-support-dir --windows-format temp-import || fatal_error "Failed to determine the path to the import directory." 26 | 27 | end_script_with_status 0 -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/_error-log-test-sub1: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -u 3 | 4 | PROGNAME="$(basename "$BASH_SOURCE")" 5 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 6 | 7 | export PATH="$progdir:$PATH" 8 | 9 | LOG="$progdir/ERROR_LOG_TEST.log" 10 | 11 | exec 2> >(tee -a "$LOG" >&2) 12 | 13 | _error-log-test-sub2 14 | # Race #2: 15 | # Error message of sub2 (called above) vs error message of sub1 (echo'ed below). 16 | # 17 | # The error message below may appear BEFORE the error message of sub2. It's asynchronous and a race. 18 | # To ensure proper order, reflecting the order of calls, some amount of time needs to pass, 19 | # otherwise the order of log entries is completely random. 20 | # 21 | # In order to illustrate the race condition and show the reverse ordering, the `tee` command is 22 | # artificially delayed by 0.1s in sub2. (Otherwise, in WSL, the race is tight and results vary.) 23 | echo "This is an error message from $PROGNAME" >&2 24 | 25 | -------------------------------------------------------------------------------- /License: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Michael Heim 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/error-log-test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -u 3 | 4 | PROGNAME="$(basename "$BASH_SOURCE")" 5 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 6 | 7 | export PATH="$progdir:$PATH" 8 | 9 | LOG="$progdir/ERROR_LOG_TEST.log" 10 | 11 | [ -f "$LOG" ] && rm "$LOG" 12 | 13 | # Race #1: 14 | # The exit of the main script (this one), as the last one on the call stack, competes against the 15 | # asynchronous logging, which ends in (re-)directing the error messages to stderr, after which they 16 | # will be written to the terminal. If the main script exits first, it returns control to the command 17 | # prompt. Then, the substituted logging process resurfaces, printing error messages to the terminal 18 | # and leaving it without a prompt, which must be regained with Ctrl-C (SIGINT). 19 | # 20 | # In order to demonstrate the outcome, it is no longer left to chance: an artificial `sleep` 21 | # simulates a delay inside the substituted process. Without it, the race is tight (in WSL) and the 22 | # result varies. 23 | exec 2> >(sleep 0.1; tee -a "$LOG" >&2) 24 | 25 | # Race #2: See sub1 26 | _error-log-test-sub1 27 | echo "This is an error message from $PROGNAME" >&2 -------------------------------------------------------------------------------- /.scripts/lib/get-cloud-sync-dir: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the path of the cloud sync directory (aka Dropbox/Boxcryptor/etc directory) in Windows 4 | # format. Throws an error if the directory does not exist or is not accessible. 5 | 6 | set -eu 7 | 8 | # Script name 9 | PROGNAME="$(basename "$BASH_SOURCE")" 10 | 11 | # Absolute path to the script. 12 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 13 | 14 | export PATH="$progdir:$PATH" 15 | 16 | # Functions 17 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 1; } 18 | 19 | # Read the configuration 20 | CLOUD_SYNC_DIR="$(get-config-setting CLOUD_SYNC_DIR)" || fatal_error "The cloud sync directory could not be retrieved from the configuration. Its location has to be defined by the CLOUD_SYNC_DIR variable in the user or default configuration file." 21 | 22 | # Verify that the directory exists and return it 23 | safe-file-exists -d "$CLOUD_SYNC_DIR" || fatal_error "The cloud sync directory does not exist or is not accessible.\n The path to the directory is defined as $CLOUD_SYNC_DIR\n Please check your configuration file: $(wsl-windows-path -e "$(get-config-setting --user-config-path)")\n If you don't find the setting there, check the configuration defaults at $(wsl-windows-path -e "$(get-config-setting --default-config-path)")" 24 | echo "$CLOUD_SYNC_DIR" 25 | -------------------------------------------------------------------------------- /.scripts/lib/process-new-errors: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the content of the log for new errors, embedded in an explanatory message, and deletes the 4 | # content (but not the file). 5 | # 6 | # If the log for new errors does not exist or is empty, a corresponding message is displayed 7 | # instead. 8 | 9 | set -eu 10 | 11 | # Absolute path to the script. 12 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 13 | 14 | export PATH="$progdir:$PATH" 15 | 16 | NEW_ERRORS_LOG="$(get-error-log --new-errors)" || exit $? 17 | 18 | # Allow a bit of extra time for other processes to finish logging 19 | sleep 0.1 20 | 21 | error_message_postfix="\n\nPlease examine the log files in the following directory:\n'$(dirname "$NEW_ERRORS_LOG")'" 22 | 23 | if [ -s "$NEW_ERRORS_LOG" ]; then 24 | logged="$(<"$NEW_ERRORS_LOG")" || "Something went wrong here. Additional errors have occurred while reading the error log file.$error_message_postfix" 25 | : > "$NEW_ERRORS_LOG" 26 | else 27 | logged="Something went wrong here. No new errors have been found in the log file.$error_message_postfix" 28 | fi 29 | 30 | cat < 0 => exclude from processing) is returned. 11 | # 12 | # The list of excluded files is defined in the config files. 13 | 14 | set -eu 15 | 16 | # Script name 17 | PROGNAME="$(basename "$BASH_SOURCE")" 18 | 19 | # Absolute path to the script. 20 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 21 | 22 | export PATH="$progdir:$PATH" 23 | 24 | # Functions 25 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 2; } # Exit status 2! 26 | 27 | # Arguments 28 | (( $# != 1 )) && fatal_error "Incorrect number of arguments. $PROGNAME expects one argument, but $# were provided." 29 | db_name="$1" 30 | 31 | # Read the configuration 32 | EXCLUDE_FROM_SYNC="$(get-config-setting EXCLUDE_FROM_SYNC)" || fatal_error "The list of Keepass database files which are excluded from syncing could not be retrieved from the configuration.\n The list has to be defined by the EXCLUDE_FROM_SYNC variable in the user or default configuration file.\n The list can be empty (as an empty array), but the variable must exist." 33 | 34 | ! (grep -Fxq "$db_name" <<<"$EXCLUDE_FROM_SYNC") -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/boxcryptor-timestamp-test-lib.ps1: -------------------------------------------------------------------------------- 1 | 2 | function Create-TestFixture { 3 | param( 4 | [Parameter(Mandatory=$True, Position=0)] 5 | [string]$encryptedDirPath, 6 | 7 | [Parameter(Mandatory=$True, Position=1)] 8 | [string]$testDirName 9 | ) 10 | 11 | return [string](New-Item -ItemType Directory -Path "$encryptedDirPath\$testDirName") 12 | } 13 | 14 | function Get-FirstTimestamp { 15 | param( 16 | [Parameter(Mandatory=$True, Position=0)] 17 | [string]$testFileName, 18 | 19 | [Parameter(Mandatory=$True, Position=1)] 20 | [string]$testFixture, 21 | 22 | [Parameter(Position=2)] 23 | [string]$sourceFilePath="" 24 | ) 25 | 26 | if ( $sourceFilePath -eq "" ) { 27 | # Create a test file with arbitrary data. 28 | 29 | # Create content 30 | $randomSnippet="$([System.Guid]::NewGuid())" 31 | $sb = [System.Text.StringBuilder]::new() 32 | 33 | foreach( $i in 1..100000) 34 | { 35 | [void]$sb.Append( $randomSnippet ) 36 | } 37 | 38 | $content=$sb.ToString() 39 | 40 | # Create file, return timestamp 41 | return (New-Item -Path "$testFixture" -Name "$testFileName" -ItemType File -Value "$content").LastWriteTime.ToString('yyyy-MM-dd HH:mm:ss.fffffff') 42 | } else { 43 | # Copy the provided file, return timestamp 44 | return (Copy-Item -PassThru -LiteralPath "$sourceFilePath" -Destination "$testFixture\$testFileName").LastWriteTime.ToString('yyyy-MM-dd HH:mm:ss.fffffff') 45 | } 46 | } 47 | 48 | function Get-FollowUpTimestamp { 49 | param( 50 | [Parameter(Mandatory=$True, Position=0)] 51 | [string]$testFileName, 52 | 53 | [Parameter(Mandatory=$True, Position=1)] 54 | [string]$testFixture 55 | ) 56 | 57 | return (Get-Item -LiteralPath "$testFixture\$testFileName").LastWriteTime.ToString('yyyy-MM-dd HH:mm:ss.fffffff') 58 | } 59 | -------------------------------------------------------------------------------- /.scripts/lib/get-support-dir: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the absolute path of the requested sync support directory, in Linux format (by default). 4 | # Expects the type of support directory as argument. 5 | # 6 | # Creates the directory if necessary. Throws an error if it can't be created or is not accessible. 7 | # 8 | # Valid directory type arguments: 9 | # 10 | # - last-synced 11 | # - last-known-good 12 | # - temp-import 13 | # 14 | # Options: 15 | # 16 | # -w, --windows-format: returns the path in Windows format (Linux-specific path: as UNC path) 17 | 18 | set -eu 19 | 20 | # Script name 21 | PROGNAME="$(basename "$BASH_SOURCE")" 22 | 23 | # Absolute path to the script. 24 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 25 | 26 | export PATH="$progdir:$PATH" 27 | 28 | # Functions 29 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 1; } 30 | 31 | # Preconfigured paths 32 | admin_dir="$(realpath $progdir/../..)" 33 | 34 | # Arguments 35 | (( $# == 0 )) && fatal_error "The directory type argument is missing." 36 | (( $# > 2 )) && fatal_error "Incorrect number of arguments. $PROGNAME expects up to two arguments, but $# were provided." 37 | 38 | windows_format=false 39 | if [[ "$1" == "-w" || "$1" == "--windows-format" ]]; then 40 | windows_format=true 41 | shift 1 42 | (( $# == 0 )) && fatal_error "The directory type argument is missing." 43 | fi 44 | 45 | case "$1" in 46 | last-synced) 47 | support_dir="$admin_dir/.sync/last-synced" 48 | ;; 49 | last-known-good) 50 | support_dir="$admin_dir/.sync/last-known-good" 51 | ;; 52 | temp-import) 53 | support_dir="$admin_dir/.sync/temp-import" 54 | ;; 55 | *) 56 | fatal_error "Unknown directory type argument '$1'" 57 | ;; 58 | esac 59 | 60 | # Make sure the support directory exists, create it if missing 61 | [ ! -d "$support_dir" ] && { mkdir -p "$support_dir" || fatal_error "Failed to create sync support directory at: $(wsl-windows-path -e "$support_dir")"; } 62 | 63 | if $windows_format; then 64 | wsl-windows-path -f "$support_dir" || fatal_error "Failed to convert the path of the sync support directory '$1' to Windows format.\n Input path (Linux format): $support_dir" 65 | else 66 | echo "$support_dir" 67 | fi 68 | -------------------------------------------------------------------------------- /.scripts/lib/log-errors: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Reads error messages **from stdin**, writes (appends) them to the permanent error log and to the 4 | # log for new errors, and sends the error messages to stderr. 5 | # 6 | # As a result, it is easier to set up error handling in top-level scripts. This line is sufficient: 7 | # 8 | # exec 2> >(log-errors) 9 | # 10 | # It replaces the following incantation: 11 | # 12 | # ERROR_LOG="$(get-error-log)" || exit $? 13 | # NEW_ERRORS_LOG="$(get-error-log --new-errors)" || exit $? 14 | # exec 2> >(tee -a "$ERROR_LOG" "$NEW_ERRORS_LOG" >&2) 15 | # 16 | # which is no longer necessary now. 17 | # 18 | # IMPORTANT! Avoiding duplicate log entries: 19 | # 20 | # The redirection to the logs (`exec 2> >(log-errors)`) **must only happen in the top-level script**, 21 | # not in any utility scripts which are invoked by it. Utility scripts must write their errors to 22 | # stderr only, just as any other command line program does it. 23 | # 24 | # If utility scripts contained the `exec` redirection command as well, the entries would double, 25 | # triple or multiply even more, depending on the depth of the call stack at the time of the error. 26 | # Also, entries might not appear in the right order. See the developer notes for details. 27 | # 28 | # A note on the logs: 29 | # 30 | # ERROR_LOG is the permanent error log. NEW_ERRORS_LOG is a log for new errors. 31 | # 32 | # If the log for new errors is not empty, it indicates that new, unprocessed errors have occurred. 33 | # The NEW_ERRORS_LOG contains a duplicate of the log records in ERROR_LOG for these new errors. For 34 | # better readability in the Keepass message box, timestamps are omitted in NEW_ERRORS_LOG. The 35 | # NEW_ERRORS_LOG file must be emptied or deleted once the errors have been displayed to the user or 36 | # processed in another way. By contrast, the ERROR_LOG stays around as a permanent record. 37 | 38 | set -u 39 | 40 | # Absolute path to the script. 41 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 42 | 43 | export PATH="$progdir:$PATH" 44 | 45 | ERROR_LOG="$(get-error-log)" || exit $? 46 | NEW_ERRORS_LOG="$(get-error-log --new-errors)" || exit $? 47 | 48 | # Processing stdin 49 | timestamp="$(date +"%F %T.%2N")" 50 | sed "s/^/$timestamp: /" | tee -a "$ERROR_LOG" | sed "s/^$timestamp: //" | tee -a "$NEW_ERRORS_LOG" >&2 -------------------------------------------------------------------------------- /.scripts/sync-outgoing-changes--post-sync.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Must be executed after a local KDBX file is saved and the changes in the local file have already 4 | # been synced to the cloud copy. Expects the name of the KDBX file (including extension) as 5 | # argument. 6 | # 7 | # - Creates a duplicate of the cloud copy in the support directory 'last-synced', for future 8 | # reference. (The duplicate is used to check if the cloud copy has been changed externally, ie on 9 | # another machine.) 10 | # - Respects the EXCLUDE_FROM_SYNC config setting and skips KDBX files listed there. 11 | 12 | set -eu 13 | 14 | # Script name 15 | PROGNAME=$(basename "$0") 16 | 17 | # Absolute path to the script. 18 | progdir=$([[ $0 == /* ]] && dirname "$0" || { _dir="$( realpath -e "$0")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; }) 19 | 20 | export PATH="$progdir/lib:$PATH" 21 | 22 | # Set up error reporting 23 | close_log() { exec 2>&-; wait $log_process_id; } 24 | end_script_with_status() { close_log; exit "${1:-0}"; } 25 | fatal_error() { echo -e "$PROGNAME: $1" >&2; end_script_with_status 1; } 26 | 27 | exec 2> >(log-errors) 28 | log_process_id=$! 29 | 30 | # Argument 31 | (( $# == 0 )) && fatal_error "Missing argument. KDBX database filename not provided." 32 | pwd_filename="$1" 33 | 34 | # Check if the local KDBX file is excluded from cloud sync. Exit quietly if excluded, or log an 35 | # error if one occurred (exit code of is-included-db > 1). 36 | is-included-db "$pwd_filename" || { (($?==1)) && end_script_with_status 0 || fatal_error "Failed to establish if the database is excluded from synchronization."; } 37 | 38 | # Verify cloud database path 39 | cloud_sync_file_win="$(get-cloud-sync-dir)\\$pwd_filename" || fatal_error "Can't establish the path to the password file in the cloud sync directory." 40 | safe-file-exists "$cloud_sync_file_win" || fatal_error "The cloud-synced copy is missing or can't be accessed.\n Path: ${cloud_sync_file_win//\\/\\\\}" 41 | 42 | # Create local duplicate of cloud database, in the last-synced directory 43 | last_synced_file="$(get-support-dir last-synced)/$pwd_filename" || fatal_error "Failed to retrieve the path to the 'last-synced' directory." 44 | safe-filecopy "$cloud_sync_file_win" "$last_synced_file" || fatal_error "Failed to copy the cloud-synced file to a reference location (\"last synced\").\n Copy source: ${cloud_sync_file_win//\\/\\\\}\n Copy target: $(wsl-windows-path -e "$last_synced_file")" 45 | 46 | end_script_with_status 0 -------------------------------------------------------------------------------- /.scripts/lib/windows-user-profile-path: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the path to the Windows user profile of the current user, in Windows 4 | # format (e.g. C:\Users\foo). 5 | # 6 | # NB If a WSL path (e.g. /mnt/c/Users/foo) is required, convert the result with 7 | # `wslpath`. 8 | 9 | # NB Previously, `wslvar` has been used for this task. That has been much 10 | # cleaner than the current implementation. The script consisted of a single 11 | # call: `wslvar USERPROFILE`. 12 | # 13 | # However, the `wslvar` call has been **much** slower than the current 14 | # implementation. Calling the script led to a visible delay. 15 | # 16 | # In addition, `wslvar` is no longer bundled with Ubuntu on WSL. It is part of 17 | # the `wslu` package (WSL Utilities) which has been removed in Ubuntu 22.04 LTS. 18 | # 19 | # See 20 | # - https://github.com/wslutilities/wslu 21 | # - https://wslutiliti.es/wslu/install.html 22 | # - https://superuser.com/a/1568668/315475 23 | 24 | # The Windows environment variable %UserProfile% is queried with cmd.exe. The 25 | # parts of the solution are: 26 | # 27 | # - `set /p variable=[promptString]` displays a prompt string, then sets a 28 | # variable from user input. Unlike `echo`, the prompt string is printed 29 | # without a trailing CrLf (which is what we are aiming for here). 30 | # 31 | # - `set /p=[promptString]` omits the variable. According to ss64.com, "the 32 | # variable name can also be left empty but this is undocumented". 33 | # 34 | # - `/dev/null` suppresses any error messages cmd.exe might generate. 38 | # 39 | # - The entire command line must be wrapped in quotes in order to be treated as 40 | # a single argument, containing the entire set of instructions, which is then 41 | # processed by cmd.exe. 42 | # 43 | # - The single escaped quote before `=%UserProfile%`, in `set /p\"=%UserProfile%`, 44 | # ensures that cmd.exe doesn't choke on problematic characters, e.g. an " & " 45 | # in the username or elsewhere in the path. Odd as it may be, a closing 46 | # escaped quote `\"` behind %UserProfile%, added for symmetry, would in fact 47 | # be appended the return value (including the escaping backslash). Unescaped, 48 | # the closing quote would even cause an error. So a single escaped quote 49 | # character it is. 50 | # 51 | # See 52 | # - https://ss64.com/nt/set.html 53 | # - https://stackoverflow.com/a/44980824/508355 54 | 55 | USERPROFILE_PATH="$(cmd.exe /c "/dev/null)" 56 | echo "$USERPROFILE_PATH" 57 | -------------------------------------------------------------------------------- /.scripts/sync-incoming-changes--post-sync.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Must be executed after a local KDBX file is opened and changes in the cloud copy are synced to the 4 | # local file. Expects the name of the KDBX file (including extension) as argument. 5 | # 6 | # - Moves the last-synced file from its temporary location to the last-synced directory (and renames 7 | # the file). 8 | # This file is the new reference for checking if the database in the cloud directory has changed 9 | # (indicating modifications on another machine). 10 | # - Deletes the temporary database copy which was used for the import. 11 | 12 | set -eu 13 | 14 | # Script name 15 | PROGNAME=$(basename "$0") 16 | 17 | # Absolute path to the script. 18 | progdir=$([[ $0 == /* ]] && dirname "$0" || { _dir="$( realpath -e "$0")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; }) 19 | 20 | export PATH="$progdir/lib:$PATH" 21 | 22 | # Set up error reporting 23 | close_log() { exec 2>&-; wait $log_process_id; } 24 | end_script_with_status() { close_log; exit "${1:-0}"; } 25 | fatal_error() { echo -e "$PROGNAME: $1" >&2; end_script_with_status 1; } 26 | 27 | exec 2> >(log-errors) 28 | log_process_id=$! 29 | 30 | # Argument 31 | (( $# == 0 )) && fatal_error "Missing argument. KDBX database filename not provided." 32 | pwd_filename="$1" 33 | 34 | # Check if the local KDBX file is excluded from cloud sync. Exit quietly if excluded, or log an 35 | # error if one occurred (exit code of is-included-db > 1). 36 | is-included-db "$pwd_filename" || { (($?==1)) && end_script_with_status 0 || fatal_error "Failed to establish if the database is excluded from synchronization."; } 37 | 38 | # File paths 39 | temp_import_dir="$(get-support-dir temp-import)" || fatal_error "Failed to retrieve the path to the 'temp-import' directory." 40 | temp_import_file="$temp_import_dir/$pwd_filename" 41 | temp_last_synced_file="$temp_import_dir/$pwd_filename--in-progress" 42 | last_synced_file="$(get-support-dir last-synced)/$pwd_filename" || fatal_error "Failed to retrieve the path to the 'last-synced' directory." 43 | 44 | # Move the last-synced file to its final destination (and rename it). 45 | # NB mv is safe to use. It preserves the timestamp even on mounted Windows drives. 46 | [ -f "$temp_last_synced_file" ] || fatal_error "The temporary local copy of the cloud file (the future \"last-synced\" file) is missing or can't be accessed.\n File path: $(wsl-windows-path -e "$temp_last_synced_file")" 47 | mv -f "$temp_last_synced_file" "$last_synced_file" || fatal_error "Failed to move the local copy of the cloud file (the \"last-synced\" file) to its final location.\n Source: $(wsl-windows-path -e "$temp_last_synced_file")\n Destination: $(wsl-windows-path -e "$last_synced_file")" 48 | 49 | # Delete the temporary import file. 50 | rm "$temp_import_file" 51 | 52 | end_script_with_status $? 53 | -------------------------------------------------------------------------------- /.scripts/lib/safe-filecopy: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copies a file, preserving the timestamp. Requires absolute paths to the source and the target. 4 | # 5 | # ######################################## When to use: ######################################### 6 | # 7 | # Use it for EVERY COPY OPERATION. 8 | # 9 | # REQUIRED on virtual drives, e.g. the Boxcryptor drive. 10 | # REQUIRED on ordinary Windows drives. 11 | # 12 | # Not required on Linux drives (`cp` could be used instead), but the performance penalty is 13 | # negligible. Thus, it is best to use the safe, universal command everywhere. 14 | # 15 | # ################################################################################################# 16 | # 17 | # Features: 18 | # 19 | # - Copying works even on drives which have not been mounted in WSL (e.g. Boxcryptor). 20 | # - The copy is guaranteed to have an identical timestamp, even considering the quirks of Windows 21 | # drives in WSL. 22 | # 23 | # (See the developer notes in dev-support for these issues.) 24 | 25 | set -eu 26 | 27 | # Script name 28 | PROGNAME="$(basename "$BASH_SOURCE")" 29 | 30 | # Absolute path to the script. 31 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 32 | 33 | export PATH="$progdir:$PATH" 34 | 35 | # Functions 36 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 1; } 37 | 38 | # Checks if a path is in Windows format. Ie, it begins with [Drive letter]:\ 39 | # or \\[UNC host name]. A path consisting solely of [Drive letter]: is also 40 | # accepted. 41 | is_in_windows_format() { [[ "$1" =~ ^[a-zA-Z]:(\\|$)|^\\\\[a-zA-Z] ]]; } 42 | 43 | # Checks if a WSL path points to a location in the Windows file system. Ie, it 44 | # begins with /mnt/[drive letter]. Expects an absolute path. If necessary, 45 | # resolve it with `realpath -m` first. 46 | is_in_windows_filesystem() { [[ "$1" =~ ^/mnt/[a-zA-Z]($|/) ]]; } 47 | 48 | # Arguments 49 | (( $# != 2 )) && fatal_error "$PROGNAME expects two arguments, but $# were provided." 50 | source="$1" 51 | target="$2" 52 | 53 | # Safe copy 54 | if is_in_windows_format "$source" || is_in_windows_format "$target" || is_in_windows_filesystem "$source" || is_in_windows_filesystem "$target"; then 55 | source_win="$(wsl-windows-path -f "$source")" || fatal_error "Failed to normalize the source path to Windows format.\n Source: ${source//\\/\\\\}" 56 | target_win="$(wsl-windows-path -f "$target")" || fatal_error "Failed to normalize the target path to Windows format.\n Target: ${target//\\/\\\\}" 57 | 58 | Powershell.exe -command "Copy-Item -LiteralPath '$source_win' -Destination '$target_win'" || fatal_error "Failed to copy file (using Powershell Copy-Item). Please note that file paths must be absolute.\n Copy source: ${source//\\/\\\\}\n Copy target: ${target//\\/\\\\}" 59 | else 60 | cp --preserve=timestamps "$source" "$target" || fatal_error "Failed to copy file (using Linux cp). Please note that file paths must be absolute.\n Copy source: ${source//\\/\\\\}\n Copy target: ${target//\\/\\\\}" 61 | fi 62 | 63 | -------------------------------------------------------------------------------- /.scripts/lib/safe-is-binary-same: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Tests if two files are identical (binary comparison). Requires absolute paths to both files. Sets 4 | # exit status 0 (truthy) if the files are the same, 1 (falsy) if they are different, and 2 in case 5 | # of an error. 6 | # 7 | # ######################################## When to use: ######################################### 8 | # 9 | # Use it for CLOUD LOCATIONS only. 10 | # 11 | # REQUIRED on virtual drives, e.g. the Boxcryptor drive. 12 | # NOT REQUIRED, NOT RECOMMENDED on ordinary Windows drives (just use cmp). 13 | # NOT REQUIRED on Linux drives. 14 | # 15 | # The command works everywhere, but the performance penalty on ordinary Windows drives is 16 | # significant. On the other hand, the command is essential for virtual drives. Paths in the cloud 17 | # directory might well be on such a drive (Boxcryptor). Thus, it is best to limit use of the 18 | # command to cloud directories. 19 | # 20 | # ################################################################################################# 21 | # 22 | # Features: 23 | # 24 | # - Works even on drives which have not been mounted in WSL (e.g. Boxcryptor). 25 | # 26 | # (See the developer notes in dev-support for more on that issue.) 27 | 28 | set -eu 29 | 30 | # Script name 31 | PROGNAME="$(basename "$BASH_SOURCE")" 32 | 33 | # Absolute path to the script. 34 | progdir=$([[ $0 == /* ]] && dirname "$0" || { _dir="$( realpath -e "$0")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; }) 35 | 36 | export PATH="$progdir:$PATH" 37 | 38 | # Functions 39 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 2; } # Exit status 2! 40 | 41 | # Checks if a path is in Windows format. Ie, it begins with [Drive letter]:\ 42 | # or \\[UNC host name]. A path consisting solely of [Drive letter]: is also 43 | # accepted. 44 | is_in_windows_format() { [[ "$1" =~ ^[a-zA-Z]:(\\|$)|^\\\\[a-zA-Z] ]]; } 45 | 46 | # Checks if a WSL path points to a location in the Windows file system. Ie, it 47 | # begins with /mnt/[drive letter]. Expects an absolute path. If necessary, 48 | # resolve it with `realpath -m` first. 49 | is_in_windows_filesystem() { [[ "$1" =~ ^/mnt/[a-zA-Z]($|/) ]]; } 50 | 51 | # Arguments 52 | (($# != 2)) && fatal_error "$PROGNAME expects two arguments, but $# were provided." 53 | file_1="$1" 54 | file_2="$2" 55 | 56 | # Safe copy 57 | if is_in_windows_format "$file_1" || is_in_windows_format "$file_2" || is_in_windows_filesystem "$file_1" || is_in_windows_filesystem "$file_2"; then 58 | file_1_win="$(wsl-windows-path -f "$file_1")" || fatal_error "Failed to normalize the file path to Windows format (file #1).\n File Path: ${file_1//\\/\\\\}" 59 | file_2_win="$(wsl-windows-path -f "$file_2")" || fatal_error "Failed to normalize the file path to Windows format (file #2).\n File Path: ${file_2//\\/\\\\}" 60 | 61 | ps_result="$(Powershell.exe -command "(Get-FileHash -Algorithm MD5 -LiteralPath '$file_1_win').Hash -eq (Get-FileHash -Algorithm MD5 -LiteralPath '$file_2_win').Hash")" || fatal_error "Failed to test if the files are identical (using Powershell Get-FileHash). Please note that the file paths must be absolute.\n File path #1: ${file_1//\\/\\\\}\n File path #2: ${file_2//\\/\\\\}" 62 | [[ $(tr -d '\r' <<<"$ps_result") == "True" ]] && exit 0 || exit $? 63 | else 64 | cmp -s "$file_1" "$file_2" && exit 0 || { (($? == 1)) && exit 1 || fatal_error "Failed to test if the files are identical (using cmp). Please note that the file paths must be absolute.\n File path #1: ${file_1//\\/\\\\}\n File path #2: ${file_2//\\/\\\\}"; } 65 | fi 66 | -------------------------------------------------------------------------------- /.scripts/has-cloud-copy-changed.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Checks if the file in the cloud sync directory (e.g. the Dropbox directory) contains changes which 4 | # have not been imported yet. Expects the name of the KDBX file (including extension) as argument. 5 | # Must be executed as soon as a local KDBX file is opened, but before potential changes in the cloud 6 | # copy are synced to the local file. 7 | # 8 | # Sets exit code 0 (truthy, import required) if 9 | # 10 | # - the cloud file has been changed externally, ie on another machine 11 | # - the cloud file exists, but the last-synced copy does not. 12 | # Ie, the cloud file has not been imported previously, and the local database has not been synced 13 | # to the cloud copy on save, either. 14 | # 15 | # Sets exit code 1 (falsy = skip import) if 16 | # 17 | # - the cloud file has stayed the same and does not contain external changes 18 | # - the file is excluded from cloud sync (listed in then EXCLUDE_FROM_SYNC config setting) 19 | # - the cloud file does not exist 20 | # 21 | # Sets exit code 2 (also falsy = skip import) if 22 | # 23 | # - an error has occurred. 24 | # This behaviour prevents an import from going ahead in the presence of errors, ie in a situation 25 | # of uncertainty. 26 | # 27 | # For the check, the cloud file is compared against a local duplicate: the "last-synced" copy of the 28 | # cloud file, which has been created at the time of the last import or sync. Absent external 29 | # changes, the cloud file matches the local duplicate. 30 | 31 | set -eu 32 | 33 | # Script name 34 | PROGNAME=$(basename "$0") 35 | 36 | # Absolute path to the script. 37 | progdir=$([[ $0 == /* ]] && dirname "$0" || { _dir="$( realpath -e "$0")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; }) 38 | 39 | export PATH="$progdir/lib:$PATH" 40 | 41 | # Set up error reporting 42 | close_log() { exec 2>&-; wait $log_process_id; } 43 | end_script_with_status() { close_log; exit "${1:-0}"; } 44 | fatal_error() { echo -e "$PROGNAME: $1" >&2; end_script_with_status 2; } # Exit status 2! 45 | 46 | exec 2> >(log-errors) 47 | log_process_id=$! 48 | 49 | # Argument 50 | (($# == 0)) && fatal_error "Missing argument. KDBX database filename not provided." 51 | pwd_filename="$1" 52 | 53 | # Check if the local KDBX file is excluded from cloud sync. Exit quietly if excluded (but preserve 54 | # the exit status, passing on an error exit code > 1). 55 | is-included-db "$pwd_filename" || end_script_with_status $? 56 | 57 | # Verify cloud database path. Exit quietly if the cloud database doesn't exist (but preserve the 58 | # exit status, passing on an error exit code > 1). 59 | cloud_sync_file_win="$(get-cloud-sync-dir)\\$pwd_filename" || fatal_error "Can't establish the path to the password file in the cloud sync directory." 60 | safe-file-exists "$cloud_sync_file_win" || end_script_with_status $? 61 | 62 | # Verify last-synced database path. Exit quietly, with truthy exit status, if the file doesn't exist. 63 | last_synced_file="$(get-support-dir last-synced)/$pwd_filename" || fatal_error "Failed to retrieve the path to the 'last-synced' directory." 64 | [ -f "$last_synced_file" ] || end_script_with_status 0 65 | 66 | # Compare the current cloud database file to the "last-synced" reference file 67 | safe-is-binary-same "$cloud_sync_file_win" "$last_synced_file" && end_script_with_status 1 || { (($?==1)) && end_script_with_status 0 || fatal_error "Failed to compare the cloud-synced file to the reference file (the \"last-synced\" file).\n Cloud-synced file: ${cloud_sync_file_win//\\/\\\\}\n Reference file: $(wsl-windows-path -e "$last_synced_file")"; } 68 | -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/timestamp-test-move.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | is_wsl() { 4 | [ -n "${WSL_DISTRO_NAME}" ] 5 | } 6 | 7 | is_windows_dir() { 8 | is_wsl && [[ "$(wslpath -w ".")" =~ [a-zA-Z]:\\ ]] 9 | } 10 | 11 | # delay=0.01 matches ext fs timestamp resolution 12 | delay() { 13 | delay=${1:-0.01}; 14 | sleep $delay; 15 | echo "Moving the file delayed by: ${delay}s"; 16 | } 17 | 18 | long_delay() { 19 | delay 1 20 | } 21 | 22 | delay_unless_win_fs() { 23 | ! is_windows_dir && delay 24 | } 25 | 26 | get_timestamp() { 27 | stat -c "%.Y" "$1" 28 | } 29 | 30 | compare_to_prev_timestamp() { 31 | echo "Before: $2" 32 | echo "After: $(get_timestamp "$1")" 33 | 34 | [ "$2" != "$(get_timestamp "$1")" ] && echo "stat: Timestamp different" || echo "stat: Timestamp equal" 35 | echo 36 | } 37 | 38 | test_run() { 39 | cwd="$(pwd)" 40 | 41 | test_basedir="$(realpath -e "${1:-"$cwd"}")" || { echo "Test basedir does not exist." >&2; exit 1; } 42 | temp_dir_name="$test_basedir/timestamp_test_$(date +%s%N)" 43 | mkdir "$temp_dir_name" || { echo "Test subdir can't be created." >&2; exit 1; } 44 | 45 | cd "$temp_dir_name" 46 | 47 | fs_type="$(stat -f -c %T .)" 48 | 49 | echo "--- $(pwd) ---" 50 | echo 51 | echo "Filesystem Type: $fs_type" 52 | echo 53 | 54 | echo "Test mv" 55 | echo "- WSL ext3: OK Timestamps are equal." 56 | echo "- WSL 9p: OK Timestamps are equal." 57 | echo "test" > file1.txt 58 | long_delay 59 | original_timestamp="$(get_timestamp file1.txt)" 60 | mv file1.txt file2.txt 61 | compare_to_prev_timestamp file2.txt "$original_timestamp" 62 | 63 | echo "Test hardlink to identical file" 64 | echo "- WSL ext3: OK Timestamps are equal." 65 | echo "- WSL 9p: OK Timestamps are equal." 66 | echo "test" > file3.txt 67 | long_delay 68 | original_timestamp="$(get_timestamp file3.txt)" 69 | cp -l file3.txt file4.txt 70 | rm file3.txt 71 | compare_to_prev_timestamp file4.txt "$original_timestamp" 72 | 73 | if is_wsl; then 74 | echo "WSL: Testing Windows utilities" 75 | echo "" 76 | 77 | echo "Test DOS move" 78 | if is_windows_dir; then 79 | echo "- WSL ext3: n/a DOS move only works on Windows drives, not for \\\\wsl$ paths." 80 | echo "- WSL 9p: OK Timestamps are equal." 81 | echo "test" > file5.txt 82 | long_delay 83 | original_timestamp="$(get_timestamp file5.txt)" 84 | cmd.exe /c move /Y file5.txt file6.txt 85 | compare_to_prev_timestamp file6.txt "$original_timestamp" 86 | else 87 | echo "Test skipped. DOS copy only works on Windows drives, not for \\\\wsl$ paths." 88 | echo 89 | fi 90 | 91 | echo "Test Powershell Move-Item" 92 | echo "- WSL ext3: OK Timestamps are equal." 93 | echo "- WSL 9p: OK Timestamps are equal." 94 | echo "test" > file7.txt 95 | long_delay 96 | original_timestamp="$(get_timestamp file7.txt)" 97 | Powershell.exe -command "Move-Item file7.txt -Destination file8.txt" 98 | compare_to_prev_timestamp file8.txt "$original_timestamp" 99 | fi 100 | 101 | cd "$cwd" 102 | rm -rf "$temp_dir_name" 103 | } 104 | 105 | test_run ~ 106 | 107 | if is_wsl; then 108 | USERPROFILE_PATH="$(cmd.exe /c "/dev/null)" 109 | test_run "$(wslpath "$USERPROFILE_PATH")" 110 | fi 111 | -------------------------------------------------------------------------------- /.scripts/sync-outgoing-changes--pre-sync.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Must be executed after a local KDBX file is saved, but before the changes in the local file are 4 | # synced to the cloud copy. Expects the name of the KDBX file (including extension) as argument. 5 | # 6 | # - Creates a duplicate of the local KDBX file in the support directory 'last-known-good', as a 7 | # pre-operation backup, in case the sync corrupts the local file. 8 | # - Copies the local KDBX file to the cloud sync directory (e.g. the Dropbox directory) if it 9 | # doesn't exist there yet. 10 | # That matters when a new KDBX file has been created locally. The sync to the cloud is initiated 11 | # automatically then, and the most current version of the file is available to other machines. 12 | # However, remote machines do NOT scan the cloud directory for new files, so they don't pick up 13 | # the new file automatically. It needs to be copied from the cloud directory to the local Keepass 14 | # directory by hand. 15 | # - Respects the EXCLUDE_FROM_SYNC config setting and skips KDBX files listed there. 16 | 17 | set -eu 18 | 19 | # Script name 20 | PROGNAME=$(basename "$0") 21 | 22 | # Absolute path to the script. 23 | progdir=$([[ $0 == /* ]] && dirname "$0" || { _dir="$( realpath -e "$0")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; }) 24 | 25 | export PATH="$progdir/lib:$PATH" 26 | 27 | # Set up error reporting 28 | close_log() { exec 2>&-; wait $log_process_id; } 29 | end_script_with_status() { close_log; exit "${1:-0}"; } 30 | fatal_error() { echo -e "$PROGNAME: $1" >&2; end_script_with_status 1; } 31 | 32 | exec 2> >(log-errors) 33 | log_process_id=$! 34 | 35 | # Argument 36 | (( $# == 0 )) && fatal_error "Missing argument. KDBX database filename not provided. Script aborted." 37 | pwd_filename="$1" 38 | 39 | # Verify local database path 40 | local_master_file="$(get-keepass-db-dir)/$pwd_filename" || fatal_error "Failed to retrieve the path to the Keepass database directory." 41 | [ ! -f "$local_master_file" ] && fatal_error "Cannot find the local KDBX database provided by the filename argument.\n Filename argument: $pwd_filename\n Expected location: $local_master_file\n (in Windows notation: $(wsl-windows-path -e "$local_master_file"))" 42 | 43 | # Check if the local KDBX file is excluded from cloud sync. Exit quietly if excluded, or log an 44 | # error if one occurred (exit code of is-included-db > 1). 45 | is-included-db "$pwd_filename" || { (($?==1)) && end_script_with_status 0 || fatal_error "Failed to establish if the database is excluded from synchronization."; } 46 | 47 | # Create backup of local database, in the last-known-good directory 48 | last_known_good="$(get-support-dir last-known-good)/$pwd_filename" || fatal_error "Failed to retrieve the path to the 'last-known-good' directory." 49 | safe-filecopy "$local_master_file" "$last_known_good" || fatal_error "Failed to copy the local KDBX database to a short-term backup location (\"last known good\").\n Copy source: $(wsl-windows-path -e "$local_master_file")\n Copy target: $(wsl-windows-path -e "$last_known_good")" 50 | 51 | # Verify cloud database path. Create a copy of the local file there if it doesn't exist yet. 52 | cloud_sync_file_win="$(get-cloud-sync-dir)\\$pwd_filename" || fatal_error "Can't establish the path to the password file in the cloud sync directory." 53 | if ! safe-file-exists "$cloud_sync_file_win"; then 54 | safe-filecopy "$local_master_file" "$cloud_sync_file_win" || fatal_error "Failed to copy the local KDBX database to the cloud sync directory.\n NB The local database, '$pwd_filename', has not yet been present in the cloud sync directory.\n Copy source: $(wsl-windows-path -e "$local_master_file")\n Copy target: ${cloud_sync_file_win//\\/\\\\}" 55 | fi 56 | 57 | end_script_with_status 0 58 | -------------------------------------------------------------------------------- /.scripts/lib/get-error-log: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Returns the absolute path to the error log. Creates the log as well as its parent directories if 4 | # necessary. 5 | # 6 | # Throws an error if the log (or its parent directories) can't be created. 7 | # 8 | # Returns the path to another, special log, the log for new errors, when called with the -n flag 9 | # (see below). Creates that log file and its parent directories if necessary. 10 | # 11 | # The log for new errors vs the ordinary error log 12 | # ------------------------------------------------ 13 | # 14 | # The log for new errors is a separate log file. Any new errors are logged there as well as in the 15 | # ordinary error log, and are kept there until they have been displayed to the user (or been 16 | # processed in another way). Once that has happened, the entries are purged from the log for new 17 | # errors, but they remain in the ordinary log file as a permanent record. 18 | # 19 | # In practical terms: If the log for new errors is not empty, it means that new errors have 20 | # occurred. After they have been processed, the log can either be emptied or the file can be 21 | # deleted. It will be recreated the next time this command is called. 22 | # 23 | # Location in the Linux filesystem 24 | # -------------------------------- 25 | # 26 | # Earlier versions of the script had placed the logs into the .admin\Logs directory in the Windows 27 | # filesystem. But access was too slow. Even without writing to the logs, closing the output stream 28 | # at the end of every Cloudypass call took so long that it introduced intolerable delays. Moving the 29 | # logs to the Linux filesystem solved the problem. 30 | # 31 | # Location ID 32 | # ----------- 33 | # 34 | # In theory, multiple Cloudypass setups can coexist, even though that is unlikely in practice. Logs 35 | # should be location-specific and keep multiple installations apart. The path to the .admin 36 | # directory of a particular Cloudypass setup is hashed and becomes the unique identifier for that 37 | # setup. The corresponding logs are kept in a directory which is derived from the location ID. 38 | # 39 | # Option: 40 | # 41 | # -n, --new-errors: returns the path to the log for new errors 42 | 43 | set -u 44 | 45 | # Script name 46 | PROGNAME="$(basename "$BASH_SOURCE")" 47 | 48 | # Absolute path to the script. 49 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 50 | 51 | export PATH="$progdir:$PATH" 52 | 53 | # Functions 54 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 1; } 55 | 56 | # Preconfigured paths and location ID 57 | admin_dir="$(realpath "$progdir/../..")" 58 | location_id="$( 59 | set -o pipefail # See https://stackoverflow.com/a/19804002/508355 60 | 61 | echo -n "$admin_dir" | md5sum | cut -d ' ' -f 1 # See https://askubuntu.com/q/53846/1027405 62 | 63 | [ $? -ne 0 ] && exit 1; set +o pipefail 64 | )" || fatal_error "Error while determining the MD5 checksum of the path \"$admin_dir\". The location ID cannot be created." 65 | 66 | LOG_DIR="$HOME/.local/state/cloudypass/logs/location-id-$location_id" 67 | ERROR_LOG="$LOG_DIR/sync.error.log" 68 | NEW_ERRORS_LOG="$LOG_DIR/new.sync.errors.log" 69 | 70 | # Set up the error logs 71 | if [ ! -d "$LOG_DIR" ]; then 72 | mkdir -p "$LOG_DIR" || fatal_error "Failed to create the log directory at: $(wsl-windows-path -ef "$LOG_DIR")" 73 | fi 74 | 75 | touch "$ERROR_LOG" || fatal_error "Failed to create or access the error log file at: $(wsl-windows-path -ef "$ERROR_LOG")" 76 | touch "$NEW_ERRORS_LOG" || fatal_error "Failed to create or access the log file for new errors at: $(wsl-windows-path -ef "$NEW_ERRORS_LOG")" 77 | 78 | # Arguments 79 | (( $# > 1 )) && fatal_error "Incorrect number of arguments. $PROGNAME expects one argument at most, but $# were provided." 80 | 81 | # Output 82 | if (( $# == 0 )); then 83 | echo "$ERROR_LOG" 84 | elif [[ "$1" == "-n" || "$1" == "--new-errors" ]]; then 85 | echo "$NEW_ERRORS_LOG" 86 | else 87 | fatal_error "Unknown argument or option '$1'" 88 | fi 89 | -------------------------------------------------------------------------------- /.scripts/lib/safe-file-exists: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Tests if a file exists (default), or if a directory exists (with the -d option). Requires an 4 | # absolute path to the file. Sets exit status 0 (truthy) if the test succeeds, 1 (falsy) if it 5 | # fails, and 2 in case of an error. 6 | # 7 | # ######################################## When to use: ######################################### 8 | # 9 | # Use it for CLOUD LOCATIONS only. 10 | # 11 | # REQUIRED on virtual drives, e.g. the Boxcryptor drive. 12 | # NOT REQUIRED, NOT RECOMMENDED on ordinary Windows drives (just use [ -f ... ], [ -d ... ]). 13 | # NOT REQUIRED on Linux drives. 14 | # 15 | # The command works everywhere, but the performance penalty on ordinary Windows drives is 16 | # significant. On the other hand, the command is essential for virtual drives. Paths in the cloud 17 | # directory might well be on such a drive (Boxcryptor). Thus, it is best to limit use of the 18 | # command to cloud directories. 19 | # 20 | # ################################################################################################# 21 | # 22 | # Options: 23 | # 24 | # -f, --file tests if a file exists (default) 25 | # -d, --directory tests if a directory exists 26 | # 27 | # Features: 28 | # 29 | # - Works even on drives which have not been mounted in WSL (e.g. Boxcryptor). 30 | # 31 | # (See the developer notes in dev-support for more on that issue.) 32 | 33 | set -eu 34 | 35 | # Script name 36 | PROGNAME="$(basename "$BASH_SOURCE")" 37 | 38 | # Absolute path to the script. 39 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 40 | 41 | export PATH="$progdir:$PATH" 42 | 43 | # Functions 44 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 2; } # Exit status 2! 45 | 46 | # Checks if a path is in Windows format. Ie, it begins with [Drive letter]:\ 47 | # or \\[UNC host name]. A path consisting solely of [Drive letter]: is also 48 | # accepted. 49 | is_in_windows_format() { [[ "$1" =~ ^[a-zA-Z]:(\\|$)|^\\\\[a-zA-Z] ]]; } 50 | 51 | # Checks if a WSL path points to a location in the Windows file system. Ie, it 52 | # begins with /mnt/[drive letter]. Expects an absolute path. If necessary, 53 | # resolve it with `realpath -m` first. 54 | is_in_windows_filesystem() { [[ "$1" =~ ^/mnt/[a-zA-Z]($|/) ]]; } 55 | 56 | # Arguments 57 | (( $# == 0 )) && fatal_error "The file path argument is missing." 58 | (( $# > 2 )) && fatal_error "Incorrect number of arguments. $PROGNAME expects up to two arguments, but $# were provided." 59 | 60 | type="file" 61 | ps_path_type="Leaf" 62 | 63 | if (( $# == 2 )); then 64 | case "$1" in 65 | -f|--file) 66 | shift 1 67 | ;; 68 | -d|--directory) 69 | type="directory" 70 | ps_path_type="Container" 71 | shift 1 72 | ;; 73 | *) 74 | fatal_error "Unknown option '$1'" 75 | ;; 76 | esac 77 | fi 78 | 79 | filepath="$1" 80 | 81 | # Test 82 | if is_in_windows_format "$filepath" || is_in_windows_filesystem "$filepath"; then 83 | path_win="$(wsl-windows-path -f "$filepath")" || fatal_error "Failed to normalize the $type path to Windows format.\n ${type@u} path: ${filepath//\\/\\\\}" 84 | 85 | # In a single line, the test looks like this (for a file): 86 | # 87 | # [[ "$(Powershell.exe -command "Test-Path -PathType Leaf -LiteralPath '$path_win'" | tr -d '\r')" == "True" ]] 88 | # 89 | # Below, however, the line is split up, so errors can be captured and logged with an appropriate 90 | # message. 91 | # 92 | # NB Windows newlines (\r\n) in Powershell output must be fixed by removing \r. Otherwise, the 93 | # string test (== "True") would fail. 94 | ps_result="$(Powershell.exe -command "Test-Path -PathType $ps_path_type -LiteralPath '$path_win'")" || fatal_error "Failed to test if the $type exists (using Powershell Test-Path). Please note that the $type path must be absolute.\n ${type@u} path: ${filepath//\\/\\\\}" 95 | [[ $(tr -d '\r' <<<"$ps_result") == "True" ]] && exit 0 || exit $? 96 | elif [[ "$type" == "directory" ]]; then 97 | [ -d "$filepath" ] && exit 0 || exit $? 98 | else 99 | [ -f "$filepath" ] && exit 0 || exit $? 100 | fi -------------------------------------------------------------------------------- /.scripts/lib/get-config-setting: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Expects the name of a config setting as an argument and returns its value. 4 | # 5 | # Evaluates the user and default config files. Throws an error if the config setting does not exist 6 | # in any of these files. 7 | # 8 | # If the config variable contains an array, the array itself is discarded and its elements are 9 | # returned as text separated by newlines, ie as a multi-line stream. 10 | # 11 | # If the command is called with one of the following options **instead** of a setting name, the 12 | # full path to a config file is returned: 13 | # 14 | # -u, --user-config-path: returns the full path to the user config file (Linux format) 15 | # -d, --default-config-path: returns the full path to the default config file (Linux format) 16 | 17 | set -eu 18 | 19 | # Script name 20 | PROGNAME="$(basename "$BASH_SOURCE")" 21 | 22 | # Absolute path to the script. 23 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 24 | 25 | export PATH="$progdir:$PATH" 26 | 27 | # Preconfigured paths 28 | admin_dir="$(realpath $progdir/../..)" 29 | 30 | CONFIG_FILE="$admin_dir/Config/sync.conf" 31 | CONFIG_DEFAULTS_FILE="$admin_dir/Config/sync.defaults.conf" 32 | 33 | # Functions 34 | fatal_error() { echo -e "$PROGNAME: $1" >&2; exit 1; } 35 | 36 | # Arguments 37 | (( $# != 1 )) && fatal_error "Incorrect number of arguments. $PROGNAME expects one argument, but $# were provided." 38 | config_varname="$1" 39 | 40 | # Handle special options 41 | if [[ "$config_varname" == "-u" || "$config_varname" == "--user-config-path" ]]; then 42 | echo "$CONFIG_FILE" 43 | elif [[ "$config_varname" == "-d" || "$config_varname" == "--default-config-path" ]]; then 44 | echo "$CONFIG_DEFAULTS_FILE" 45 | else 46 | # Read the configuration 47 | [ ! -f "$CONFIG_FILE" ] && fatal_error "Cannot find the sync config file. Expected location: $(wsl-windows-path -e "$CONFIG_FILE")" 48 | [ ! -f "$CONFIG_DEFAULTS_FILE" ] && fatal_error "Cannot find the file containing the sync config defaults. Expected location: $(wsl-windows-path -e "$CONFIG_DEFAULTS_FILE")" 49 | 50 | # NB Remove any \r chars in linebreaks before sourcing the config. Windows line endings and 51 | # Windows tools can create them. 52 | . <(tr -d '\r' <"$CONFIG_DEFAULTS_FILE") 53 | . <(tr -d '\r' <"$CONFIG_FILE") 54 | 55 | # Check if the variable exists, in a way that also works for variables set to an empty array. 56 | # See https://stackoverflow.com/a/35412000/508355 57 | declare -p "$config_varname" &>/dev/null || fatal_error "The requested configuration setting '$config_varname' does not exist.\n Please check your configuration file: $(wsl-windows-path -e "$CONFIG_FILE")\n If you don't find the setting there, check the configuration defaults at $(wsl-windows-path -e "$CONFIG_DEFAULTS_FILE")" 58 | 59 | # Create a reference (nameref) to the value of config-varname. 60 | # NB The requested variable could also be accessed without creating an additional reference, 61 | # using shell parameter expansion: ${!config_varname}. But if the variable is an array, that 62 | # notation does not support iterating over the array. 63 | # See https://unix.stackexchange.com/a/412812/297737 64 | declare -n config_var_value="$config_varname" 65 | 66 | # Test if the variable is an array. See https://stackoverflow.com/a/66897754/508355 67 | # 68 | # NB Depending on the Bash version, an empty array makes the test throw the error 69 | # "config_var_value: unbound variable" if the script is configured with `set -u` to treat 70 | # undefined variables as an error. 71 | # 72 | # - In Bash 5.1 (Ubuntu 22.04 LTS), the script chokes on the expression `"${config_var_value@a}"` 73 | # even though an empty array shouldn't be treated as unset (and it isn't in other types of 74 | # expressions). 75 | # - In Bash 5.0 (Ubuntu 20.04 LTS), by contrast, the expression did not cause an error. 76 | # 77 | # In order to prevent the error and enable proper handling of empty arrays, the -u option is 78 | # turned off with `set +u` before proceeding. 79 | set +u 80 | if [[ "${config_var_value@a}" == a ]]; then 81 | # Variable is an array. 82 | for i in "${config_var_value[@]}"; do echo "$i"; done 83 | else 84 | echo "$config_var_value" 85 | fi 86 | fi 87 | -------------------------------------------------------------------------------- /.scripts/sync-incoming-changes--pre-sync.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Must be executed as soon as a local KDBX file is opened, but before changes in the cloud copy are 4 | # synced to the local file. Expects the name of the KDBX file (including extension) as argument. 5 | # 6 | # - Creates a duplicate of the local KDBX file in the support directory 'last-known-good', as a 7 | # pre-operation backup, in case the sync corrupts the local file. 8 | # - Copies the file from the cloud sync directory to a temp folder. This file will be used for the 9 | # import. 10 | # + As a local duplicate, the temp file is protected against any sudden changes initiated on 11 | # another machine while the import is in progress. 12 | # + The temp file is modified by the import (which is in fact a merge, altering both files even if 13 | # the content of the import source remains unchanged). The cloud file itself remains unaffected 14 | # by the import, as it should be. 15 | # + The temporary file can be discarded later on, in the post-sync script. 16 | # - Creates a duplicate of the temp file (ie, a duplicate of the duplicate) before the import 17 | # begins. 18 | # + That file, too, remains unaffected by the import and is, at least for now, identical to the 19 | # cloud file. 20 | # + Eventually, when the import has been successful, the file will be moved to the last-synced 21 | # directory and kept there as the future reference, in order to detect changes made on another 22 | # machine (see has-cloud-copy-changed.sh). The file will be moved in the post-sync script. 23 | 24 | set -eu 25 | 26 | # Script name 27 | PROGNAME=$(basename "$0") 28 | 29 | # Absolute path to the script. 30 | progdir=$([[ $0 == /* ]] && dirname "$0" || { _dir="$( realpath -e "$0")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; }) 31 | 32 | export PATH="$progdir/lib:$PATH" 33 | 34 | # Set up error reporting 35 | close_log() { exec 2>&-; wait $log_process_id; } 36 | end_script_with_status() { close_log; exit "${1:-0}"; } 37 | fatal_error() { echo -e "$PROGNAME: $1" >&2; end_script_with_status 1; } 38 | 39 | exec 2> >(log-errors) 40 | log_process_id=$! 41 | 42 | # Argument 43 | (( $# == 0 )) && fatal_error "Missing argument. KDBX database filename not provided." 44 | pwd_filename="$1" 45 | 46 | # Verify local database path 47 | local_master_file="$(get-keepass-db-dir)/$pwd_filename" || fatal_error "Failed to retrieve the path to the Keepass database directory." 48 | [ ! -f "$local_master_file" ] && fatal_error "Cannot find the local KDBX database provided by the filename argument.\n Filename argument: $pwd_filename\n Expected location: $local_master_file\n (in Windows notation: $(wsl-windows-path -e "$local_master_file"))" 49 | 50 | # Check if the local KDBX file is excluded from cloud sync. Exit quietly if excluded, or log an 51 | # error if one occurred (exit code of is-included-db > 1). 52 | is-included-db "$pwd_filename" || { (($?==1)) && end_script_with_status 0 || fatal_error "Failed to establish if the database is excluded from synchronization."; } 53 | 54 | # Create backup of local database, in the last-known-good directory 55 | last_known_good="$(get-support-dir last-known-good)/$pwd_filename" || fatal_error "Failed to retrieve the path to the 'last-known-good' directory." 56 | safe-filecopy "$local_master_file" "$last_known_good" || fatal_error "Failed to copy the local KDBX database to a short-term backup location (\"last known good\").\n Copy source: $(wsl-windows-path -e "$local_master_file")\n Copy target: $(wsl-windows-path -e "$last_known_good")" 57 | 58 | # File paths 59 | temp_import_dir="$(get-support-dir temp-import)" || fatal_error "Failed to retrieve the path to the 'temp-import' directory." 60 | temp_import_file="$temp_import_dir/$pwd_filename" 61 | temp_last_synced_file="$temp_import_dir/$pwd_filename--in-progress" 62 | 63 | # Remove old files from the temp directory 64 | [ -f "$temp_import_file" ] && rm "$temp_import_file" 65 | [ -f "$temp_last_synced_file" ] && rm "$temp_last_synced_file" 66 | 67 | # Verify cloud database path 68 | cloud_sync_file_win="$(get-cloud-sync-dir)\\$pwd_filename" || fatal_error "Can't establish the path to the password file in the cloud sync directory." 69 | safe-file-exists "$cloud_sync_file_win" || fatal_error "The cloud-synced copy is missing or can't be accessed.\n Path: ${cloud_sync_file_win//\\/\\\\}" 70 | 71 | # Create local duplicate of cloud database, in the temp-import directory, and create yet another 72 | # copy of that file (the future last-synced file). 73 | safe-filecopy "$cloud_sync_file_win" "$temp_import_file" || fatal_error "Failed to copy the cloud-synced file to a temporary location.\n Copy source: ${cloud_sync_file_win//\\/\\\\}\n Copy target: $(wsl-windows-path -e "$temp_import_file")" 74 | safe-filecopy "$temp_import_file" "$temp_last_synced_file" || fatal_error "Failed to duplicate the temporary file.\n Copy source: $(wsl-windows-path -e "$temp_import_file")\n Copy target: $(wsl-windows-path -e "$temp_last_synced_file")" 75 | 76 | end_script_with_status 0 77 | -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/boxcryptor-timestamp-test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -eu 4 | 5 | # Script name 6 | PROGNAME="$(basename "$BASH_SOURCE")" 7 | 8 | # Absolute path to the script. 9 | progdir="$([[ "$BASH_SOURCE" == /* ]] && dirname "$BASH_SOURCE" || { _dir="$( realpath -e "$BASH_SOURCE")"; [[ "$_dir" == /* ]] && dirname "$_dir" || pwd; })" 10 | 11 | export PATH="$(realpath "$progdir/../../lib"):$PATH" 12 | 13 | fatal_error() { 14 | echo -e "$PROGNAME: $1" >&2; 15 | exit 1; 16 | } 17 | 18 | is_wsl() { 19 | [ -n "${WSL_DISTRO_NAME}" ] 20 | } 21 | 22 | # Argument is a path in Linux format. 23 | get_wsl_drive_letter() { 24 | sed -rn 's_^/mnt/([a-zA-Z])($|/.*)_\1_p' <<<"$1" 25 | } 26 | 27 | # Argument is the drive letter. Case-insensitive. 28 | is_mounted_in_wsl() { 29 | [[ "$(findmnt -lfno TARGET -T "/mnt/${1,}")" =~ ^/mnt/${1,}$ ]] 30 | } 31 | 32 | is_windows_dir() { 33 | is_wsl && [[ "$(wslpath -w ".")" =~ [a-zA-Z]:\\ ]] 34 | } 35 | 36 | # Converts a Windows path to Linux format. Does the same as `wslpath`, but `wslpath` throws an error 37 | # if the drive is not mounted in WSL. Ie, `wslpath` doesn't work for the Boxcryptor drive. 38 | # 39 | # The path does not have to exist. If the path does not conform to an absolute Windows path pattern, 40 | # any backslashes are converted to forward slashes, but otherwise the path is returned as it was 41 | # passed in. Ie, a Linux path is returned unchanged. 42 | to_linux_path() { 43 | local path="${1:-$("$path" 96 | echo "$path" 97 | } 98 | 99 | test_run() { 100 | # Check if .ps1 files are allowed to execute 101 | can-execute-powershell-scripts || fatal_error "Powershell script execution is not permitted. Enable it in order to run this test.\n\nScript execution can be allowed for the current user with the following command:\n\n Set-ExecutionPolicy -Scope CurrentUser RemoteSigned\n" 102 | 103 | local encrypted_parent_dir='J:\Dropbox\Encryption Tests\Encrypted dir with name encryption' 104 | 105 | local test_dir_name="timestamp_test_$(date +%s%N)" 106 | local test_file_name="testfile.dat" 107 | 108 | local test_fixture_path="$(create_test_fixture "$encrypted_parent_dir" "$test_dir_name")" 109 | echo "Test dir path: $test_fixture_path" 110 | 111 | local test_data_path="$(create_local_test_file "test_data.dat" "10M")" 112 | 113 | # For generating a new file in place, rather than copying one from another location, leave out 114 | # the source file path (3rd arg): 115 | # local original_timestamp="$(get_first_timestamp "$test_file_name" "$test_fixture_path")" 116 | local original_timestamp="$(get_first_timestamp "$test_file_name" "$test_fixture_path" "$(wsl-windows-path -f "$test_data_path")")" 117 | echo "Timestamp #1: $original_timestamp (original timestamp)" 118 | 119 | local timestamp 120 | for i in {2..5}; do 121 | sleep 2 122 | timestamp="$(get_followup_timestamp "$test_file_name" "$test_fixture_path")" 123 | echo "Timestamp #$i: $timestamp $([[ "$timestamp" == "$original_timestamp" ]] && echo OK || echo CHANGED)" 124 | done 125 | } 126 | 127 | test_run 128 | -------------------------------------------------------------------------------- /.scripts/dev-support/Experiments/timestamp-test-copy.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | is_wsl() { 4 | [ -n "${WSL_DISTRO_NAME}" ] 5 | } 6 | 7 | is_windows_dir() { 8 | is_wsl && [[ "$(wslpath -w ".")" =~ [a-zA-Z]:\\ ]] 9 | } 10 | 11 | # delay=0.01 matches ext fs timestamp resolution 12 | delay() { 13 | delay=${1:-0.01}; 14 | sleep $delay; 15 | echo "Creation of second file delayed by: ${delay}s"; 16 | } 17 | 18 | long_delay() { 19 | delay 1 20 | } 21 | 22 | delay_unless_win_fs() { 23 | ! is_windows_dir && delay 24 | } 25 | 26 | test_comparison_methods() { 27 | echo "$1: $(stat -c "%.Y" "$1")" 28 | echo "$2: $(stat -c "%.Y" "$2")" 29 | 30 | [ "$(stat -c "%.Y" "$1")" != "$(stat -c "%.Y" "$2")" ] && echo "stat: Timestamp different" || echo "stat: Timestamp equal" 31 | [ "$1" -ot ""$2 -o "$1" -nt "$2" ] && echo "ot/nt: Timestamp different" || echo "ot/nt: Timestamp equal" 32 | 33 | echo 34 | } 35 | 36 | test_run() { 37 | cwd="$(pwd)" 38 | 39 | test_basedir="$(realpath -e "${1:-"$cwd"}")" || { echo "Test basedir does not exist." >&2; exit 1; } 40 | temp_dir_name="$test_basedir/timestamp_test_$(date +%s%N)" 41 | mkdir "$temp_dir_name" || { echo "Test subdir can't be created." >&2; exit 1; } 42 | 43 | cd "$temp_dir_name" 44 | 45 | fs_type="$(stat -f -c %T .)" 46 | 47 | echo "--- $(pwd) ---" 48 | echo 49 | echo "Filesystem Type: $fs_type" 50 | echo 51 | 52 | echo "Test timestamps with minimal difference (files created independently)" 53 | # ext3/4 accuracy: see 54 | # - https://stackoverflow.com/a/14393315/508355 55 | # - https://stackoverflow.com/a/60846117/508355 56 | echo "- WSL ext3: OK Timestamps are different if true difference > timestamp accuracy (Accuracy" 57 | echo " is dependent on the system. Safe bet on modern systems: 10ms resolution." 58 | echo " Super safe bet: 1s.)." 59 | echo "- WSL 9p: OK Timestamps are different even without sleep (minimal difference)." 60 | echo "test" > file1.txt 61 | delay_unless_win_fs 62 | echo "test" > file2.txt 63 | 64 | test_comparison_methods file1.txt file2.txt 65 | 66 | echo "Test simple file copy" 67 | echo "- WSL ext3: OK cp **modifies the timestamps** as expected." 68 | echo "- WSL 9p: OK cp **modifies the timestamps** as expected." 69 | echo "test" > file3.txt 70 | long_delay 71 | cp file3.txt file4.txt 72 | test_comparison_methods file3.txt file4.txt 73 | 74 | echo "Test file copy with --preserve=timestamps option" 75 | echo "- WSL ext3: OK Timestamps are equal." 76 | echo "- WSL 9p: FAIL Timestamps should be strictly equal, but they are not. Copied file loses" 77 | echo " sub-second precision (discarded, not rounded)." 78 | echo "test" > file5.txt 79 | long_delay 80 | cp --preserve=timestamps file5.txt file6.txt 81 | test_comparison_methods file5.txt file6.txt 82 | 83 | echo "Test hardlink to identical file" 84 | echo "- WSL ext3: OK Timestamps are equal." 85 | echo "- WSL 9p: OK Timestamps are equal." 86 | echo "test" > file7.txt 87 | long_delay 88 | cp -l file7.txt file8.txt 89 | test_comparison_methods file7.txt file8.txt 90 | 91 | echo "Test scp copy (locally)" 92 | echo "- Behaviour identical to cp (without --preserve=timestamps option)." 93 | echo "- WSL ext3: OK scp **modifies the timestamps** as expected." 94 | echo "- WSL 9p: OK scp **modifies the timestamps** as expected." 95 | echo "test" > file09.txt 96 | long_delay 97 | scp file09.txt file10.txt 98 | test_comparison_methods file09.txt file10.txt 99 | 100 | echo "Test rsync copy (locally)" 101 | echo "- Behaviour identical to cp (without --preserve=timestamps option)." 102 | echo "- WSL ext3: OK rsync **modifies the timestamps** as expected." 103 | echo "- WSL 9p: OK rsync **modifies the timestamps** as expected." 104 | echo "test" > file11.txt 105 | long_delay 106 | rsync file11.txt file12.txt 107 | test_comparison_methods file11.txt file12.txt 108 | 109 | if is_wsl; then 110 | echo "WSL: Testing Windows utilities" 111 | echo "" 112 | 113 | echo "Test DOS copy" 114 | if is_windows_dir; then 115 | echo "- WSL ext3: n/a DOS copy only works on Windows drives, not for \\\\wsl$ paths." 116 | echo "- WSL 9p: OK Timestamps are equal." 117 | echo "test" > file13.txt 118 | long_delay 119 | cmd.exe /c copy /B file13.txt file14.txt 120 | test_comparison_methods file13.txt file14.txt 121 | else 122 | echo "Test skipped. DOS copy only works on Windows drives, not for \\\\wsl$ paths." 123 | echo 124 | fi 125 | 126 | echo "Test Powershell Copy-Item" 127 | echo "- WSL ext3: OK Timestamps are equal." 128 | echo "- WSL 9p: OK Timestamps are equal." 129 | echo "test" > file15.txt 130 | long_delay 131 | Powershell.exe -command "Copy-Item file15.txt -Destination file16.txt" 132 | test_comparison_methods file15.txt file16.txt 133 | fi 134 | 135 | cd "$cwd" 136 | rm -rf "$temp_dir_name" 137 | } 138 | 139 | test_run ~ 140 | 141 | if is_wsl; then 142 | USERPROFILE_PATH="$(cmd.exe /c "/dev/null)" 143 | test_run "$(wslpath "$USERPROFILE_PATH")" 144 | fi 145 | -------------------------------------------------------------------------------- /.scripts/lib/wsl-windows-path: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Script name 4 | PROGNAME="$(basename "$BASH_SOURCE")" 5 | 6 | if [[ "$1" == '--version' || "$1" == '-v' ]]; then 7 | fmt -s <<- VERSION_TEXT 8 | $PROGNAME 1.1.2 9 | (c) 2022-2025 Michael Heim 10 | License: MIT 11 | 12 | VERSION_TEXT 13 | exit 0 14 | elif [[ "$1" == '--help' || "$1" == '-h' ]]; then 15 | fmt -s <<- HELP_TEXT 16 | 17 | Safely converts a WSL (Linux) path to a Windows path. 18 | 19 | If a path argument is not provided, input is read from standard input instead (so the command can be used in a pipe). 20 | 21 | Converts a path if it points to a location in the Windows file system (/mnt/[drive letter]). If the path is WSL-specific, ie within the Linux file system, it is returned unchanged by default. To force conversion to a UNC path (\\\\wsl$\\Ubuntu\\...), use the -f flag. 22 | 23 | A Windows path (e.g. C:\\Users\\foo) is returned unchanged. However, accidental forward slashes in the path are corrected to backslashes. The -f option converts a relative path into an absolute path. The -e option (escape backslashes) is honoured as well. 24 | 25 | The conversion is entirely string-based, the path does not have to exist. 26 | 27 | During conversion from Linux to Windows format, trailing backslashes are removed. They are also removed during conversion from a relative to an absolute path (-f option). Trailing slashes in unconverted Linux or Windows paths are returned as they are passed in, never added or removed. 28 | 29 | Usage: 30 | $PROGNAME [options] path 31 | ... | $PROGNAME [options] 32 | 33 | Options: 34 | 35 | -e Escape backslashes. 36 | -f Convert Linux-specific paths to UNC paths, and relative 37 | paths to absolute paths. 38 | -v, --version Show version and license. 39 | -h, --help Show help. 40 | 41 | Conversion examples: 42 | 43 | /mnt/d/Foo/Bar Baz/.quux/file.txt => D:\\Foo\\Bar Baz\\.quux\\file.txt 44 | /mnt/d/Foo/Bar Baz/.quux/ => D:\\Foo\\Bar Baz\\.quux 45 | /mnt/d/Foo/Bar Baz/.quux => D:\\Foo\\Bar Baz\\.quux 46 | /usr/local/bin/ => /usr/local/bin/ 47 | /usr/local/bin/command.sh => /usr/local/bin/command.sh 48 | ~ => /home/[user] (*) 49 | ./bar (assuming cwd /home/m/foo) => ./bar 50 | ./bar (assuming cwd /mnt/d/Foo) => D:\\Foo\\bar 51 | 52 | (*) as a result of shell expansion 53 | 54 | With the -f flag: 55 | 56 | ~ => \\\\wsl$\\Ubuntu\\home\\[user] 57 | /usr/local/bin/ => \\\\wsl$\\Ubuntu\\usr\\local\\bin 58 | /usr/local/bin/command.sh => \\\\wsl$\\Ubuntu\\usr\\local\\bin\\command.sh 59 | 60 | ./bar (assuming cwd /home/m/foo) => \\\\wsl$\\Ubuntu\\home\\m\\foo\\bar 61 | ./bar (assuming cwd /mnt/d/Foo) => D:\\Foo\\bar 62 | 63 | $PROGNAME differs from the built-in wslpath utility in several respects: 64 | 65 | - \`wslpath -w\` throws an error if the input path does not exist. 66 | - \`wslpath -w\` always converts WSL-specific paths to UNC paths. 67 | - \`wslpath -w\` throws an error if the input path is in Windows format. 68 | 69 | Limitations: 70 | 71 | Input paths do not have to exist, but they are expected to be valid paths and do not pass an additional sanity check. Invalid paths may lead to unexpected output, rather than an error. 72 | 73 | HELP_TEXT 74 | exit 0 75 | fi 76 | 77 | fatal_error() { echo "$PROGNAME: $1" >&2; exit 1; } 78 | 79 | # Checks if a path is absolute and in Windows format. Ie, it begins with 80 | # [Drive letter]:\ or \\[UNC host name]. A path consisting solely of 81 | # [Drive letter]: is also accepted. 82 | is_abs_path_in_windows_format() { [[ "$1" =~ ^[a-zA-Z]:(\\|$)|^\\\\[a-zA-Z] ]]; } 83 | 84 | # Checks if a path is in Windows format. 85 | # 86 | # In ambiguous cases, the following rules apply: 87 | # 88 | # - If the path contains a backslash, it is treated as a Windows path (even if 89 | # forward slashes are present, too). 90 | # - But there is an exception: If the path begins with a single forward slash, 91 | # it is considered to be a Linux path, even if backslashes are present 92 | # (because a path beginning with a directory separator does not make sense in 93 | # Windows). 94 | # - If the path does not contain any path separator, it is considered to be a 95 | # Linux path. 96 | # 97 | # NB These rules are somewhat different from the `is_in_windows_format()` 98 | # function in `wsl-linux-path`. 99 | is_in_windows_format() { { [[ "$1" =~ \\ ]] && [[ ! "$1" =~ ^/[^/\\]+ ]]; }; } 100 | 101 | # Checks if a WSL path points to a location in the Windows file system. Ie, it 102 | # begins with /mnt/[drive letter]. Expects an absolute path. If necessary, 103 | # resolve it with `realpath -m` first. 104 | is_in_windows_filesystem() { [[ "$1" =~ ^/mnt/[a-zA-Z] ]]; } 105 | 106 | # Option default values 107 | escape_backslash=false 108 | force_unc=false 109 | 110 | while getopts ":ef" option; do 111 | case $option in 112 | e) 113 | escape_backslash=true 114 | ;; 115 | f) 116 | force_unc=true 117 | ;; 118 | \?) 119 | fatal_error "Option '-$OPTARG' is invalid." 120 | ;; 121 | :) 122 | fatal_error "The argument for option '-$OPTARG' is missing." 123 | ;; 124 | esac 125 | done 126 | 127 | # After removing options from the arguments, get the input path from the 128 | # remaining argument or, if there is none, from stdin (ie, from a pipe). See 129 | # - https://stackoverflow.com/a/35512655/508355 130 | # - https://stackoverflow.com/a/36432966/508355 131 | # - https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion 132 | # 133 | # NB Multi-line input may come from stdin (or a redirected file). Therefore, 134 | # multiple paths are handled from here on out, separated by newlines - one path 135 | # per line. 136 | shift $(($OPTIND - 1)) 137 | paths="${1:-$( 2 | 3 | 4 | 5 | 6h0SAOjV6kmyiSK5RSYGVw== 6 | Cloudypass: Warn if WSL bash is not available 7 | Tests if WSL is available and a functioning Linux distro is installed. The cloud sync scripts rely on it. 8 | 9 | Disables cloud sync until the problem is fixed. Once the test passes, cloud sync is re-enabled automatically. 10 | 11 | 12 | 1M7NtUuYT/KmqeJVJh7I6A== 13 | 14 | 15 | 16 | 17 | 18 | uQ/4B3M4T+q7LrwL6juYww== 19 | 20 | {CMD:/Powershell.exe -Command "wsl which bash >$null 2>&1; Write-Host $? -nonewline"/M=C,WS=H/} 21 | 0 22 | True 23 | 24 | true 25 | 26 | 27 | 28 | 29 | CfePcyTsT+yItiXVMPQ0bg== 30 | 31 | Cloud Sync is disabled: WSL Linux is not available 32 | The Windows Subsystem for Linux (WSL) is not set up with a functioning Linux distribution. Cloud sync with Cloudypass won't work and is disabled. 33 | 48 34 | 0 35 | 0 36 | 0 37 | 0 38 | 39 | 40 | 41 | 42 | tkamn96US7mbrjykfswQ6g== 43 | 44 | Cloudypass: Import on Open 45 | 0 46 | 47 | 48 | 49 | tkamn96US7mbrjykfswQ6g== 50 | 51 | Cloudypass: Sync on Save 52 | 0 53 | 54 | 55 | 56 | tkamn96US7mbrjykfswQ6g== 57 | 58 | Cloudypass: Monitor Sync Errors 59 | 0 60 | 61 | 62 | 63 | 64 | 65 | ep38X3gnhUav8V/tLMTZvw== 66 | Cloudypass: Import on Open 67 | Imports changes from a cloud drive. 68 | 69 | When the local DB is opened, the cloud copy is examined for changes which have been made on another machine. If present, they are imported into the local DB. 70 | 71 | 72 | 5f8TBoW4QYm5BvaeKztApw== 73 | 74 | 0 75 | 76 | 77 | 78 | 79 | 80 | 81 | uQ/4B3M4T+q7LrwL6juYww== 82 | 83 | {CMD:/wsl "$(wslpath "{DB_DIR}\.admin\.scripts\has-cloud-copy-changed.sh")" "{DB_NAME}" && echo true || echo false/M=C,WS=H/} 84 | 0 85 | true 86 | 87 | false 88 | 89 | 90 | 91 | 92 | tkamn96US7mbrjykfswQ6g== 93 | 94 | 95 | 0 96 | 97 | 98 | 99 | tkamn96US7mbrjykfswQ6g== 100 | 101 | Cloudypass: Sync on Save 102 | 0 103 | 104 | 105 | 106 | 2uX4OwcwTBOe7y66y27kxw== 107 | 108 | wsl 109 | "$(wslpath "{DB_DIR}\.admin\.scripts\sync-incoming-changes--pre-sync.sh")" "{DB_NAME}" 110 | True 111 | 1 112 | 113 | 114 | 115 | 116 | Iq135Bd4Tu2ZtFcdArOtTQ== 117 | 118 | {CMD:/wsl "$(wslpath "{DB_DIR}\.admin\.scripts\get-import-dir.sh")"/M=C,WS=H/}\{DB_NAME} 119 | 120 | 121 | 122 | 123 | 124 | 2uX4OwcwTBOe7y66y27kxw== 125 | 126 | wsl 127 | "$(wslpath "{DB_DIR}\.admin\.scripts\sync-incoming-changes--post-sync.sh")" "{DB_NAME}" 128 | True 129 | 1 130 | 131 | 132 | 133 | 134 | tkamn96US7mbrjykfswQ6g== 135 | 136 | Cloudypass: Sync on Save 137 | 1 138 | 139 | 140 | 141 | tkamn96US7mbrjykfswQ6g== 142 | 143 | 144 | 1 145 | 146 | 147 | 148 | 149 | 150 | /43/eUNJXUKFtWBJOB2WVw== 151 | Cloudypass: Sync on Save 152 | Syncs changes to a cloud drive. 153 | 154 | After the user has made changes to the local DB and has saved them, the local DB is synced to the cloud copy. 155 | 156 | 157 | s6j9/ngTSmqcXdW6hDqbjg== 158 | 159 | 0 160 | 161 | 162 | 163 | 164 | 165 | 166 | y0qeNFaMTJWtZ00coQQZvA== 167 | 168 | {DB_DIR}\.admin\.scripts\is-included-db.sh 169 | 170 | false 171 | 172 | 173 | uQ/4B3M4T+q7LrwL6juYww== 174 | 175 | {CMD:/wsl "$(wslpath "{DB_DIR}\.admin\.scripts\is-included-db.sh")" "{DB_NAME}" && echo true || echo false/M=C,WS=H/} 176 | 0 177 | true 178 | 179 | false 180 | 181 | 182 | 183 | 184 | tkamn96US7mbrjykfswQ6g== 185 | 186 | 187 | 0 188 | 189 | 190 | 191 | 2uX4OwcwTBOe7y66y27kxw== 192 | 193 | wsl 194 | "$(wslpath "{DB_DIR}\.admin\.scripts\sync-outgoing-changes--pre-sync.sh")" "{DB_NAME}" 195 | True 196 | 1 197 | 198 | 199 | 200 | 201 | Iq135Bd4Tu2ZtFcdArOtTQ== 202 | 203 | {CMD:/wsl "$(wslpath "{DB_DIR}\.admin\.scripts\get-cloud-sync-dir.sh")"/M=C,WS=H/}\{DB_NAME} 204 | 205 | 206 | 207 | 208 | 209 | 2uX4OwcwTBOe7y66y27kxw== 210 | 211 | wsl 212 | "$(wslpath "{DB_DIR}\.admin\.scripts\sync-outgoing-changes--post-sync.sh")" "{DB_NAME}" 213 | True 214 | 1 215 | 216 | 217 | 218 | 219 | tkamn96US7mbrjykfswQ6g== 220 | 221 | 222 | 1 223 | 224 | 225 | 226 | 227 | 228 | l8FbsNj8GU2N8eS9hAEiHQ== 229 | Cloudypass: Monitor Sync Errors 230 | Detects errors which have occurred during cloud sync operations. Alerts the user if any are found. 231 | 232 | 233 | s6j9/ngTSmqcXdW6hDqbjg== 234 | 235 | 0 236 | 237 | 238 | 239 | 240 | lPpw5bE/QSamTgZP2MNslQ== 241 | 242 | 0 243 | 244 | 245 | 246 | 247 | 248 | 249 | y0qeNFaMTJWtZ00coQQZvA== 250 | 251 | {DB_DIR}\.admin\.scripts\has-new-errors.sh 252 | 253 | false 254 | 255 | 256 | uQ/4B3M4T+q7LrwL6juYww== 257 | 258 | {CMD:/wsl "$(wslpath "{DB_DIR}\.admin\.scripts\has-new-errors.sh")" && echo true || echo false/M=C,WS=H/} 259 | 0 260 | true 261 | 262 | false 263 | 264 | 265 | 266 | 267 | tkamn96US7mbrjykfswQ6g== 268 | 269 | 270 | 0 271 | 272 | 273 | 274 | CfePcyTsT+yItiXVMPQ0bg== 275 | 276 | Database Sync Error 277 | {CMD:/wsl "$(wslpath "{DB_DIR}\.admin\.scripts\process-new-errors.sh")"/M=C,WS=H/} 278 | 16 279 | 4 280 | 0 281 | 0 282 | 2 283 | {CMD:/Powershell.exe -Command "Start-Process -FilePath 'notepad' -ArgumentList '""{DB_DIR}\.admin\Config\sync.defaults.conf""' -WindowStyle Normal; Start-Process -FilePath 'notepad' -ArgumentList '""{DB_DIR}\.admin\Config\sync.conf""' -WindowStyle Normal"/M=C,WS=H,W=0/} 284 | 285 | 286 | 287 | tkamn96US7mbrjykfswQ6g== 288 | 289 | 290 | 1 291 | 292 | 293 | 294 | 295 | 296 | -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- 1 | # Cloudypass 2 | 3 | **DIY Cloud Sync for Keepass** 4 | 5 | Out of the box, Keepass doesn't synchronize its data across multiple devices. Cloudypass is a set of scripts which take care of that. 6 | 7 | ##### What does it do? 8 | 9 | Cloudypass is a connecting layer for making the synchronization work. You provide the parts: 10 | 11 | - a local Keepass install on each machine, 12 | - a local `.kdbx` database file, or perhaps several of them, 13 | - a way to synchronize files across devices, using a cloud service like Dropbox or a "cloud-free" mechanism like [Syncthing](https://syncthing.net/). The choice is yours. 14 | 15 | Cloudypass serves as the glue between these parts. 16 | 17 | You set up Cloudypass in the local directory where you keep your `.kdbx` file(s), e.g. `C:\Users\MyUserName\Documents\Keepass`. 18 | 19 | Next, you tell Cloudypass which directory to use for synchronization, e.g. `D:\Dropox\Keepass Sync`. The directory must be different from the one where you keep your `.kdbx` files. The local database is indeed strictly local, i.e. it must remain _outside_ of the cloud sync directory. 20 | 21 | Whenever you edit a local database, Cloudypass copies it to the sync directory. Cloudypass also monitors the sync directory for changes made on another machine. Edits made elsewhere are merged into your local database. 22 | 23 | That is the basic pattern. Under the hood, a few additional steps are taken. They protect the local database against file corruption from a botched network transfer. They also allow for near-simultaneous edits on more than one computer. 24 | 25 | ##### What about mobile? 26 | 27 | Cloudypass extends the functionality of Keepass. Just like Keepass, it runs on Windows, and that is the end of it. That said, mobile clients tie in nicely with such a setup. Personally, I use [Keepassium](https://keepassium.com/), but others should work just fine, too. 28 | 29 | Mobile clients usually check if the Keepass database on the mobile device is up-to-date when you access it, and download the most recent version from the cloud if necessary. When you edit password entries on your mobile device, the updated database is saved back to the cloud. Windows clients running Cloudypass pick up these changes and merge them into their local databases. 30 | 31 | ##### Is it safe? 32 | 33 | I have written these scripts for my own use. They are tried and tested, in particular with my own setup – Dropbox, ~~Boxcryptor~~ Cryptomator, Keepassium on mobile –, and designed to be reliable. If things go wrong, as they eventually always do, the scripts don't fail silently, but make a fuss. Decent error handling and notifications are part of the package. 34 | 35 | That said, Cloudypass basically just copies files around and orchestrates the process. The actual synchronization across a network is done by a service of your choice (e.g. Dropbox). Merging data from another machine into the local Keepass database is handled by Keepass itself. The scripts don't touch, know or care about passwords and keyfiles. (Keyfiles should not be synchronized anyway. If you need to move them to a new machine, do it manually.) There is little which could go wrong, security-wise, because the scope of the scripts is so limited. 36 | 37 | Finally, there is the question of trust and transparency. The code can easily be audited by anyone. It is not compiled, so you can just read the source code, beforehand and in place on your own machine, and what you see is what you get. Cloudypass consists of Bash scripts, an extremely widespread way of automating stuff in the IT world. Plenty of people should be able to judge for themselves what the scripts do. Comments guide you through them, so if you know a bit about Bash, it is easy to make sense of the scripts. 38 | 39 | ##### What you should know up front 40 | 41 | Setup is not a matter of a couple of quick clicks. It is easy enough, but if you have read this far, you can already guess that you need to wire a few things up yourself. The setup process requires some degree of computer literacy. 42 | 43 | Which brings me to the important question of **support**. Feel free to raise issues and suggest improvements in the issue tracker, but please don't expect a swift (or perhaps any) response. Let me be upfront about it: I needed this thing to work for myself and put in quite a bit of effort, but I am too busy with other (non-IT) stuff to really properly run this as an open-source project. I simply lack the time. 44 | 45 | **So here's the deal.** I have tried to provide all the info to get things going, but please don't expect any more than that. To put it bluntly, consider yourself to be on your own from here on out. Of course, you can get in touch if you run into problems, and of course I'll try to help if time allows, but please don't count on it. 46 | 47 | If you are fine with that, here's what you need to do. 48 | 49 | ## Setup 50 | 51 | ##### What you need 52 | 53 | - Windows 10 (most recent version) or Windows 11 54 | - The "[Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/about)" (WSL 2) with Ubuntu Linux 55 | 56 | The Windows Subsystem for Linux is provided for free by Microsoft. It is part of Windows 10 or newer, but must be installed separately. 57 | - Keepass 58 | 59 | You can't use a Keepass replacement like [KeePassXC](https://keepassxc.org/), which unfortunately lacks some crucial features for this task. 60 | - Cloudypass 61 | - A service to synchronize files across devices, like Dropbox, iCloud, Box, Syncthing etc. 62 | 63 | ##### Wiring it up 64 | 65 | 1. Install the "Windows Subsystem for Linux" (WSL 2). 66 | 67 | Instructions are available [from Microsoft](https://docs.microsoft.com/en-us/windows/wsl/install) (and plenty of [other](https://ubuntu.com/tutorials/install-ubuntu-on-wsl2-on-windows-10) [sources](https://www.omgubuntu.co.uk/how-to-install-wsl2-on-windows-10)). Please make sure to 68 | + install WSL 2, not the outdated version WSL 1 69 | + set up Ubuntu within WSL (which happens by default). 70 | 2. Install Cloudypass. 71 | 72 | Install it in the directory where you keep your `.kdbx` Keepass database(s). The directory containing the Cloudypass files must be named `.admin`. 73 | + If you are a Git user, just `cd` into the directory with your Keepass databases and run 74 | 75 | git clone https://github.com/hashchange/cloudypass.git .admin 76 | 77 | + Otherwise, [download the files manually](https://github.com/hashchange/cloudypass/archive/refs/heads/master.zip) and extract the zip file in the directory with your Keepass databases. 78 | 79 | When you are done, this is what your install must look like: 80 | + On the top level, there is the directory containing your `.kdbx` Keepass databases. 81 | + In addition to the databases, there is a subdirectory named `.admin` in it. 82 | + In the `.admin` directory you'll find the Cloudypass subdirectories, like `Config`, `Trigger Definitions` etc. 83 | 84 | NB If you use more than one `.kdbx` database, they mustn't be spread out across the system. They all have to be located in the same directory. 85 | 3. Adjust the default configuration to match your individual setup. 86 | + The configuration is stored in plain text files. They are located in the `.admin\Config` directory. 87 | + [Have a look](https://github.com/hashchange/cloudypass/blob/master/Config/sync.defaults.conf) at the default settings. They are stored in the file `sync.defaults.conf`, along with explanations. But please do not change the settings there. 88 | + **Rename** the file `sample.sync.conf` to `sync.conf`. Store your own settings in the `sync.conf` file. 89 | + You will almost certainly need to define the directory which you want to use for the cloud synchronization. If left unconfigured, Cloudypass attempts to use to the directory where Dropbox, in a standard setup, usually keeps your files: `[Your Windows user directory]\Dropbox`. 90 | 4. Create the Keepass triggers. 91 | + [Keepass triggers](https://keepass.info/help/v2/triggers.html) are executed by Keepass, e.g. when the application is started, a database is opened or one is saved. Keepass triggers run the appropriate Cloudypass scripts. 92 | + Locate the file containing the Cloudypass trigger definitions:
93 | `.admin\Trigger Definitions\sync-triggers.xml` 94 | + Open the file with a text editor and copy its content. 95 | + Open Keepass. Access the trigger settings via the Keepass menu:
96 | `Tools` | `Triggers...` 97 | + In the trigger settings window, click on the `Tools` button and select `Paste Triggers from Clipboard`. 98 | 5. Make sure the local databases are in place. 99 | + If the Keepass databases are already in your local Keepass directory, you are done now. The synchronization will start by itself when you open a database. 100 | + If you are connecting a new computer to an existing synchronization setup: 101 | 102 | Copy the `.kdbx` databases from the sync directory (the one inside Dropbox or similar) into the local directory (the one where you installed Cloudypass). 103 | 104 | In other words: Databases on your local computer will make their way to the synchronization directory by themselves. But not vice versa. Remote databases, which appear in the synchronization directory, need to be copied to the local directory by hand. They are not picked up automatically on a machine which doesn't have them yet. 105 | 106 | ## How to update 107 | 108 | You can update with Git or by downloading and overwriting the local installation. 109 | 110 | - If you use Git, `cd` into the `.admin` directory. A plain 111 | 112 | git pull 113 | 114 | will work and keep your local configuration intact. 115 | 116 | NB It is slightly more elegant to store your setup, including your configuration, in a private branch. You need to remove `Config/sync.conf` from `.gitignore` for that. When you update, just pull and merge the current version of Cloudypass into your private branch. 117 | 118 | - Otherwise, [download the lastest version](https://github.com/hashchange/cloudypass/archive/refs/heads/master.zip), extract the zip file and copy its content into the `.admin` directory. Just overwrite everything that is there. Your configuration will remain intact. 119 | 120 | You might have to **update the Keepass triggers**, too. Just remove the old ones and [recreate them](#create-the-keepass-triggers) from their new definition. Have a look at the [release notes](#release-notes) to find out if it is necessary. 121 | 122 | ## What do I do if ... ? 123 | 124 | ### Edits in a local database don't show up elsewhere 125 | 126 | Perhaps you encounter this issue very early on. You might notice it immediately after setup if your databases on different machines are not in sync to begin with. 127 | 128 | **Edits in a local database are pushed to the cloud and to other computers when you save the database.** They don't make it to the cloud just because the local database is somehow different from the others. You need to hit that save button. 129 | 130 | During normal operation, pushing data when saving is exactly what should happen. But if your databases are not in sync to begin with, you need to make at least one edit in each database to bring them into the same state in every location. That will most likely happen over time, but you can bring it forward and complete the initial synchronization with a pseudo edit in each database (like adding and deleting a space, which enables you to save the database and force the data push). 131 | 132 | Likewise, an additional save can help you if your latest edits in the local database have failed to make it to the cloud, for whatever reason. Your machine might have crashed the very moment you saved the database. Or perhaps an encryption layer like Boxcryptor has frozen and blocked the transfer. No matter what caused it: Once the problem is sorted out, just save the local database again, and your edits will be passed on to your other devices. 133 | 134 | NB You don't need to worry about any of this in order to _receive_ edits which have been made elsewhere. They get pulled into the local database as soon as you open it. (And every time you save it, too.) There is no need to help the import along, ever. 135 | 136 | ### Sync conflict on mobile 137 | 138 | Assume you are editing an entry in the Keepass database on your phone. Unfortunately, due to a failed sync (perhaps because of a weak signal), the file you are editing is an old one, rather than the latest version of the database. While you have been typing, the signal has come back. You are online again and hit save. 139 | 140 | What should you do if your mobile application, such as Keepassium, reports a conflict when you save the file? 141 | 142 | The answer is straightforward but perhaps a little counter-intuitive: When asked, do _not_ create a separate copy, i.e. a renamed version of the database file. Instead, choose "overwrite", which replaces the version of the database in the cloud with the one you just edited on your mobile. 143 | 144 | But what about the lost entries in the overwritten file? Well, they have vanished from the cloud copy, but not from the universe. They still exist in the local database on your computer. The next time you open it, the content of the cloud copy is imported into the local database. And the next time you edit and save it, the content of the local database is merged back into the copy in the cloud. 145 | 146 | The latter part is restoring the "lost" edits to the cloud copy. Everything you had overwritten previously, back then when you hit "save" on your mobile, is resurrected now. But please be aware that there may be a delay: [Only after the local database is edited and saved](#edits-in-a-local-database-dont-show-up-elsewhere), the "lost" edits will show up everywhere else again. 147 | 148 | Details aside, the key takeaway is this: **Just overwrite conflicting files.** Your data is safe, and all will be well. 149 | 150 | But there is one exception. For that scenario, read on. 151 | 152 | ### Synchronization troubles: Can data ever be lost? 153 | 154 | The only way to lose data forever is when Cloudypass can't get involved. 155 | 156 | Assume you are offline for some reason, and you edit the Keepass database on your mobile ... and a little later, still offline, on your tablet. When connectivity is restored, one of these devices will sync its database to the cloud first, followed by the other one. It is at that moment that you will see a message about a sync conflict on the device. If you decide to overwrite the copy in the cloud, the edits you made on the first device will be gone. 157 | 158 | The best way to handle this problem is to avoid the situation altogether: Do not edit the database on multiple mobile devices while you are offline. 159 | 160 | (Otherwise, to preserve your edits, you would have to save a renamed copy of the file rather than overwrite the version in the cloud. That course of action contradicts what you should normally do – [see above](#sync-conflict-on-mobile). When you are back home at your computer, you would also have to [merge](https://keepass.info/help/v2/sync.html) the renamed database into your local database manually.) 161 | 162 | That may sound complicated, but the good news is that you are unlikely to run into this issue, at least as a single user. When you are offline for an extended period of time, there probably won't be a reason for you to change entries in Keepass. 163 | 164 | However, if a database is shared between several people and everyone edits it on the go, data loss is much more likely. Not having a signal is a frequent problem. There is a realistic chance that two people will each be working on an outdated copy at some point, without having access to each others edits. To avoid it, you have to establish a safe workflow among your group, or use a separate database for each member which is read-only for anyone but the owner. 165 | 166 | In any event, you can't mitigate the problem with Cloudypass. The issue occurs when you use mobile devices exclusively. Cloudypass does not run on them. 167 | 168 | ### File corruption in the cloud 169 | 170 | This has happened to me a couple of times over the years. It didn't have anything to do with Cloudypass, but that doesn't make it any less annoying. If the database in the cloud directory is corrupt, you will see an error message telling you about it. 171 | 172 | The solution is simple: Provided that your local copy is intact, just copy it to the cloud directory, overwriting the corrupt database. 173 | 174 | ### File corruption of your local database 175 | 176 | I have never observed this, but then again, never is a long time. 177 | 178 | If the database in the cloud directory is still okay, you can use it to replace your local database, of course. But in case it is messed up as well, you still have another option: the backup which Cloudypass creates as part of its normal operation. 179 | 180 | Inside the directory with your local `.kdbx` file(s), this is where you find the backup: 181 | 182 | [Local .kdbx database dir]\.admin\.sync\last-known-good 183 | 184 | You can use the file you find there to restore the local database and the cloud copy. 185 | 186 | ### Something failed without an error message 187 | 188 | Unless things have gone wrong on a fairly fundamental level, you will see an error message – either immediately or when you close Keepass. But if that doesn't happen, you can check the error log. To display its contents, open Powershell or a Windows command prompt in the directory 189 | 190 | [Local .kdbx database dir]\.admin\.scripts\utils 191 | 192 | and run the command 193 | 194 | ```bat 195 | wsl ./show-error-log 196 | ``` 197 | 198 | It is unlikely you will ever have to access the log directly. It is located in the Linux filesystem of WSL. You can find the log file `sync.error.log` in a subdirectory of 199 | 200 | ~/.local/state/cloudypass/logs 201 | 202 | ### Can Keepass databases be kept in other locations? 203 | 204 | You can open a database in a location where Cloudypass is not installed and use it normally. There won't be any malfunctions, warnings or error messages. The database just won't be synced. 205 | 206 | In theory, you can even set up multiple database folders, each containing a separate Cloudypass installation with its own configuration. E.g., you could use one folder for synchonization with Dropbox and another folder for synchonization with OneDrive. The Keepass triggers work for each of these installs, without duplication or adjustment. Whether anyone actually needs such a setup is debatable, but it would work. 207 | 208 | ## Background 209 | 210 | ##### Which setups has Cloudypass been tested with? 211 | 212 | Cloudypass works in a generic way, so it should integrate with pretty much every cloud or synchronization service. But that's just theory. 213 | 214 | I have tested Cloudypass with these synchronisation services: 215 | 216 | - Dropbox 217 | - Dropbox + Boxcryptor 218 | - Dropbox + Cryptomator 219 | - iCloud 220 | - OneDrive 221 | - Syncthing 222 | 223 | I have tested and used Cloudypass in conjunction with these mobile apps: 224 | 225 | - Keepassium on iOS 226 | 227 | If you use another sync service and are happy with the results, feel free to let me know, and I will mention it here. 228 | 229 | ##### Why WSL? 230 | 231 | It might seem that using Bash scripts in WSL is a pretty roundabout way of automating the synchronization, and in fact it is. I simply didn't want to spend the time to get familiar with Powershell for this project. I know my way around Bash scripting, so that's what I used. 232 | 233 | As it turned out, working with WSL [had its own set of challenges](.scripts/dev-support/Notes/Developer%20Notes.md). 234 | 235 | If Keepass should ever be ported to macOS or its feature set is implemented in another program for the Mac, then the WSL/Bash approach will be an asset. Linux Bash scripts usually don't need much adjustment to run on a Mac. 236 | 237 | ##### Finally, I would like to thank ... 238 | 239 | - ... the contributors to the Keepass forum who provided their thoughts [in this discussion](https://sourceforge.net/p/keepass/discussion/329220/thread/a9aab281bd/), years ago. It eventually led to this project. 240 | - ... [the developer behind Keepass](https://keepass.info/contact.html) who put all that work into it, consistently over many years. Thank you. 241 | 242 | ## Release Notes 243 | 244 | ### v.1.1.4 245 | 246 | - Fixed handling of sync directories set to a drive root 247 | 248 | ### v.1.1.3 249 | 250 | - Moved logs to Linux filesystem, fixing slow performance 251 | - Added log viewer utility 252 | 253 | ### v.1.1.2 254 | 255 | - Improved error handling and logging 256 | - Added update instructions to documentation 257 | 258 | ### v.1.1.1 259 | 260 | You need to [reinstall the Keepass triggers](#how-to-update) when updating to this version. 261 | 262 | - Added support for WSL distro Ubuntu 22.04 LTS 263 | - Improved check for new errors 264 | - Fixed optional opening of config files 265 | - Excluded user config from being tracked by Git 266 | - Improved documentation 267 | 268 | ### v.1.1.0 269 | 270 | You need to [reinstall the Keepass triggers](#how-to-update) when updating to this version. 271 | 272 | - Adjusted trigger naming conventions 273 | - Added trigger verifying that WSL is available 274 | 275 | ### v.1.0.0 276 | 277 | - Initial stable release 278 | 279 | ## License 280 | 281 | MIT. 282 | 283 | Copyright (c) 2021-2025 Michael Heim. 284 | 285 | 286 | -------------------------------------------------------------------------------- /.scripts/dev-support/Notes/Developer Notes.md: -------------------------------------------------------------------------------- 1 | # Developer Notes 2 | 3 | ## Contents 4 | 5 | 1. [Missing drives in WSL (e.g. Boxcryptor)](#missing-drives-in-wsl-eg-boxcryptor) 6 | 2. [Last Modified timestamp anomalies in Boxcryptor](#last-modified-timestamp-anomalies-in-boxcryptor) 7 | 3. [Preserving timestamps when copying files on Windows drives with WSL](#preserving-timestamps-when-copying-files-on-windows-drives-with-wsl) 8 | 4. [Writing errors to stderr and a log file](#writing-errors-to-stderr-and-a-log-file) 9 | - [Using `exec`](#using-exec) 10 | - [Using a code block](#using-a-code-block) 11 | - [Where to place it](#where-to-place-it) 12 | - [Explanation of the logging code](#explanation-of-the-logging-code) 13 | - [Preventing log entry duplication (or multiplication)](#preventing-log-entry-duplication-or-multiplication) 14 | - [Process substitution leads to race conditions](#process-substitution-leads-to-race-conditions) 15 | - [Race conditions while logging #1: Messed-up log entry order](#race-conditions-while-logging-1-messed-up-log-entry-order) 16 | - [Race conditions while logging #2: Still writing errors after the prompt is back](#race-conditions-while-logging-2-still-writing-errors-after-the-prompt-is-back) 17 | - [Race conditions while logging #3: Lost log entries when the shell is closed quickly](#race-conditions-while-logging-3-lost-log-entries-when-the-shell-is-closed-quickly) 18 | - [A simpler solution: writing to the log file only](#a-simpler-solution-writing-to-the-log-file-only) 19 | 5. [Useful functions, tested but not used in the project](#useful-functions-tested-but-not-used-in-the-project) 20 | - [Extracting the drive letter from a WSL path](#extracting-the-drive-letter-from-a-wsl-path) 21 | - [Checking if a Windows drive has been mounted in WSL](#checking-if-a-windows-drive-has-been-mounted-in-wsl) 22 | - [Mounting a drive in WSL](#mounting-a-drive-in-wsl) 23 | - [Calculating a file hash if the drive is not mounted in WSL](#calculating-a-file-hash-if-the-drive-is-not-mounted-in-wsl) 24 | - [Reading the Last Modified timestamp of a file if the drive is not mounted in WSL](#reading-the-last-modified-timestamp-of-a-file-if-the-drive-is-not-mounted-in-wsl) 25 | - [Converting a Windows path to Linux format even if the drive is not mounted in WSL](#converting-a-windows-path-to-linux-format-even-if-the-drive-is-not-mounted-in-wsl) 26 | - [Testing if Powershell scripts can execute](#testing-if-powershell-scripts-can-execute) 27 | 28 | 29 | ## Missing drives in WSL (e.g. Boxcryptor) 30 | 31 | If a drive is not available in WSL, the drive might still exist. Network drives and folders mounted as a drive – like Boxcryptor does it – are not mounted automatically in WSL. 32 | 33 | - See https://github.com/Microsoft/WSL/issues/2788#issuecomment-355105691 34 | - Checking if a drive is mounted: See function `is_mounted_in_wsl`, below. 35 | 36 | This could be fixed by explicitly mounting the missing drive. 37 | 38 | See 39 | - https://boxcryptor.community/d/14709-windows-linux-subsystem-and-mounting-a-boxryptor-drive 40 | - https://unix.stackexchange.com/a/390658/297737 41 | - Mounting the drive: See function `mount_in_wsl`, below. 42 | 43 | But mounting the drive requires `sudo` priviliges and therefore isn't a viable solution in a script which is supposed to run without user intervention. 44 | 45 | Instead, the drive can be accessed indirecty via Windows, using DOS or (preferably) Powershell commands. They "see" the drive even if it is not mounted in WSL. 46 | 47 | 48 | ## Last Modified timestamp anomalies in Boxcryptor 49 | 50 | For files encrypted with Boxcryptor, the "Last modified" timestamp is prone to a race condition and not stable. 51 | 52 | The observations: 53 | 54 | - When an encrypted file is modified, the modification time is set. If the file is copied then, to a location outside of Boxcryptor, the copy will retain that modification time forever. Business as usual, so far. 55 | - When reading the modification time of the encrypted file for the first time, it always matches that of the copy (as it should). 56 | - But when reading the modification time of the encrypted file for a second time, it often – but not always – returns a different value. Compared to the copied file, approximately 0.15-0.25 s have been added to the modification time. 57 | - Sometimes, that shift becomes apparent only after a terminal has been closed and a new one has been opened, suggesting that some kind of intermediate timestamp caching is at play here, too. I.e., the timestamp in the first terminal returns the initial value more than just once, but in the second terminal, the timestamp has changed. 58 | - On other occasions, the initial timestamp does not change, ever. It is as stable as one would expect. That suggests that a race condition plays a crucial role. 59 | - On a positive note, the file content is unaffected by all this. Even if the file is copied immediately after creation, i.e. before the source file is modified again after > 0.15 s, the contents of both files are identical (binary comparison). So the second modification only relates to file metadata, not file content. 60 | 61 | 62 | What _might_ be going on with Boxcryptor: 63 | 64 | - Whenever the timestamp stays the same, perhaps the encryption is completed quickly enough. The post-encryption modification time falls into the same timeframe as the pre-encryption timestamp, given the 100 ns resolution of the NTFS filesystem. On the other hand, if the timestamp shifts, encryption has been delayed significantly for some reason (0.25 s = 2'500'000 timeframes à 100 ns). Timestamp caching might then mask that shift for a while, explaining the unaltered first readout. 65 | - A somewhat related, but more conspicuous Boxcryptor bug [has long been fixed](https://boxcryptor.community/d/11758-bc-14-does-not-use-timestamp-from-original-file). Long encryption delays, to the tune of more than 30 seconds, have again [been observed](https://boxcryptor.community/d/12627-file-modification-time-confusion-for-emacs) a while later. However, sub-second delays are much less likely to be noticed. Indeed, there isn't any public discussion or documentation about this phenomenon. 66 | 67 | Why Keepass seems to play a role, too: 68 | 69 | - The weird behaviour was impossible to reproduce with a generic test (boxcryptor-timestamp-test.sh, in Experiments). 70 | - It did, however, happen when the file in Boxcryptor was a Keepass DB, and it was one of the two files involved in a sync (the external DB being synced to). 71 | - So the Keepass sync or save process is somehow relevant, too. Yet it can't be the only ingredient. Tests in a Dropbox directory, without Boxcryptor involvement, did not show the odd behaviour. (Caveat: The tests have not been very extensive. Maybe it has just been down to luck.) 72 | - Most likely neither Boxcryptor nor Keepass Sync is the sole culprit, but the combination of the two. 73 | 74 | In theory, when a file has been copied, the "last modified" timestamp could be used to check if the file and its copy are identical, or if the original file has been modified since. That kind of comparison is fast, and it is also reliable: timestamps are quite precise on NTFS drives, with a filetime resolution of 100 ns. In Boxcryptor, though, it doesn't work. 75 | 76 | The timestamp of the original may or may not have changed sometime after the encryption, and identical files which have been modified at the exact same time may nonetheless have different timestamps. And even worse, as the first timestamp read-out usually returns identical timestamps, the condition can't be detected immediately after creating the copy. 77 | 78 | So file identity has to be checked with a binary file comparison, or by comparing checksums. It is slower, but timestamps can't be relied upon even as a shortcut. 79 | 80 | 81 | ## Preserving timestamps when copying files on Windows drives with WSL 82 | 83 | The task: Copying a file while preserving its "last modified" timestamp. 84 | 85 | If the file is located in the WSL (Linux) filesystem, not on a mounted Windows drive: 86 | 87 | - `cp --preserve=timestamps ...` or `cp -p ...` both work, as expected. 88 | - Without these options, `cp` changes the "last modified" timestamp of the copy to the time it is created. This is documented behaviour. 89 | 90 | So no surprises here. 91 | 92 | If the file is located on a mounted Windows drive: 93 | 94 | - Use Windows utilities only, NOT `cp`. 95 | - `Powershell.exe -command "Copy-Item ..."` works. 96 | - `cmd.exe /c copy /B ...` works. 97 | - `cp --preserve=timestamps ...` or `cp -p ...` both kill sub-second precision. The timestamp of the copied file is set to a time in full seconds. Fractional seconds are simply discarded, not even rounded. 98 | 99 | Powershell `Copy-Item` works with `\\wsl$` UNC paths, too, so it can deal with all locations, on Windows drives as well as inside the WSL filesystem. The DOS `copy` command only works for Windows paths. So Powershell is the safer option. 100 | 101 | In summary: If the timestamp matters, use Powershell `Copy-Item` on Windows drives. If the timestamp of the copy doesn't have to be precise or doesn't matter at all, use `cp`. A native command is much faster than a executing a Powershell command from inside WSL. 102 | 103 | 104 | ## Writing errors to stderr and a log file 105 | 106 | There are two ways of achieving this – with an `exec` call for the remainder of the script, or with a code block (`{ ... }`) for a limited part of it. Apart from the scope, both methods are equivalent. 107 | 108 | ### Using `exec` 109 | 110 | The most common method is the initial `exec` statement: 111 | 112 | ```bash 113 | exec 2> >(tee -a "$ERROR_LOG" >&2) 114 | ``` 115 | The `exec` call, when not followed by the name of another command, [sets up redirections for the current script](https://askubuntu.com/a/525788/1027405). More precisely, it creates the redirections for the current shell (the one created as context for the script), which is then also applied to child processes, e.g. commands called from the script, subshells etc. 116 | 117 | For the logging construct as a whole, see 118 | - https://askubuntu.com/a/1027519/1027405 119 | - http://mywiki.wooledge.org/BashFAQ/106 120 | 121 | ### Using a code block 122 | 123 | Alternatively, the relevant parts of the code can be wrapped in a code block (`{ ... }`), redirecting the errors which happen within it: 124 | 125 | ```bash 126 | { 127 | # Code here 128 | # ... 129 | } 2> >(tee -a "$ERROR_LOG" >&2) 130 | ``` 131 | 132 | The `{ ... }` code block acts as an anonymous function. Error redirection is set up at its end. See https://stackoverflow.com/a/315113/508355 133 | 134 | ### Where to place it 135 | 136 | Either logging solution **must only appear in a top-level script**, not in the utility scripts called by it. Never allow a script with logging code to call another script which does the same. If that is unavoidable, 137 | 138 | - move all the actual work from the top-level scripts into utility scripts 139 | - turn the top-level scripts into dumb wrappers which take care of the logging and then call the appropriate utility script 140 | - use the top-level scripts as exclusive entry points which never call one another. 141 | 142 | ### Explanation of the logging code 143 | 144 | - `2>`: stderr is directed ... 145 | - `>( )`: ... to a subshell (using process substitution), which runs asynchonously. 146 | - `tee -a "$ERROR_LOG"`: There the input is split with `tee`, with one stream being written to the log file. The other stream is directed to stdout, as `tee` always does it. 147 | - ` >&2`: stout is then redirected to stderr (`>&2`). 148 | 149 | See https://stackoverflow.com/a/692407/508355 150 | 151 | ### Preventing log entry duplication (or multiplication) 152 | 153 | Writing stderr to the log can't be done in multiple scripts which are calling each other – it **must be restricted to the top-level script**. All other scripts must write to stderr only. Otherwise, two, three or even more entries of the same error would clutter the log, depending on the depth of the call stack when an error occurs. 154 | 155 | Suppose the main script, A, calls script B, which then calls script C where an error occurs, with an error message written to stderr. Script C captures the stderr output, initiates the logging and also writes the error message back to stderr (`tee ... >&2`). Then it exits. Script B also captures stderr output, writes it to the log and back to stderr. Script A does the same thing. After that, no more capturing takes place, and stderr is written to the terminal. So on the terminal, just a single error message is displayed, but it has been written to the log three times. 156 | 157 | There are just two ways out of it. Either the error message is logged _only_, and not written back to stderr. Then it can be logged when it occurs (script C), but it won't appear in the terminal. The other option is to capture and process stderr only in the top-level script, as required above. 158 | 159 | Run the `error-log-test.sh` experiment as an practical example. 160 | 161 | ### Process substitution leads to race conditions 162 | 163 | Process substitution (`>(...)`) creates a child process (subshell) which runs in the background. In other words, the commands in it are executed asynchronously. As a result, process substitution it might (and in fact does) lead to race conditions. 164 | 165 | See 166 | - https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html 167 | - http://mywiki.wooledge.org/ProcessSubstitution 168 | 169 | Race conditions are more likely to be observed when the command inside the process substitution takes a long time to finish. In WSL, file access is exceedingly slow on mounted Windows drives (accessed via the a 9P protocol file server, i.e. as a filesystem on a network). So `tee` takes a long time to finish when the log file resides on a Windows drive, and it shows. 170 | 171 | ### Race conditions while logging #1: Messed-up log entry order 172 | 173 | If every script sets up its own logging with process substitution (i.e., asynchronously), the following sequence of events has indeed been observed: 174 | 175 | - Script A calls script B, which writes an error message to stderr, and the whole stderr capture/tee/redirect dance unfolds. The latter part happens asynchronously and takes a while. Meanwhile, script B exits with an error status, reverting control to script A. 176 | 177 | - Script A detects the error exit code and writes an error message of its own to stderr. Again, the capture/process substitution/`tee`/redirect sequence is initiated. Now we have two `tee` commands trying to write to the log in independent, asynchronous processes, both competing which gets to the log first. 178 | 179 | - And indeed, for whatever reason, tests on WSL have shown that script A, which generates its error message last, gets it into the log first – before script B has logged the error preceding it. 180 | 181 | So if logging is not restricted to the top-level script, the ordering of the log entries [does not necessarily reflect the order in which the errors have occurred](http://mywiki.wooledge.org/BashFAQ/106). 182 | 183 | Again, run the `error-log-test.sh` experiment as an practical example. Both the error messages on the screen and the log content demonstrate the confused order (with a little help from `sleep` to show it consistently, see script comments). 184 | 185 | ### Race conditions while logging #2: Still writing errors after the prompt is back 186 | 187 | If every script sets up its own logging with process substitution (i.e., asynchronously), the terminal regains the command prompt before the logging is done. Some or all of the log output is written to the terminal after the prompt. 188 | 189 | Visually, the terminal seems to "hang" with an unterminated process. In fact it does not, and the terminal is ready for input. Given that the cursor ends up in a new line, a command line which is typed there will be executed normally. However, the command prompt symbol (at the beginning of the line) can be regained when Ctrl-C (SIGINT) is pressed. 190 | 191 | Again, this mostly happens in WSL on a Windows file system. But it can be demonstrated with the following one-liner in any Bash terminal, where slow filesystem access is simply simulated with `sleep`: 192 | 193 | ```bash 194 | (exec 2> >(sleep 1; tee -a "EXAMPLE_ERROR.log" >&2); echo "Oops" >&2; exit 1) 195 | ``` 196 | 197 | The sequence of events is the same as in [race condition #1](#race-conditions-while-logging-1-messed-up-log-entry-order). But here, the competitors are different: 198 | 199 | - The logging processes do their thing asynchronously. While they are at it, control reverts to the scripts in the call stack, until finally the top-level script exits. 200 | 201 | - If the logging is done by then, it is business as usual. The top-level script exits, and the command prompt reappears in the terminal. The error messages have already been printed above it. 202 | 203 | - However, if the script exit comes first, a command prompt appears, but then the logging processes resurface. They print their output after the prompt now, leaving the terminal seemingly "hanging" without a prompt (but in fact ready for input). The prompt can be regained with Ctrl-C (SIGINT). 204 | 205 | The root cause of the problem is async execution of the substituted process, which returns so late that the original process has already exited. The substituted process then writes stderr to the terminal. The async nature of the setup is the same everywhere, but it manifests only if there is a significant delay for the stuff done in the substituted process, like writing to a file in a slow 9p-driven network filesystem (WSL). 206 | 207 | This issue is **not remedied** by restricting the logging to the top-level script. It can even occur in a single-script setup. Even then, the process substitution is pitted against the main script in a race to exit first. But it mostly seems to happen (even in WSL) if the error occurs very soon after the logging is set up. Further down the line, logging is usually faster than the exit from the script (although this is just a casual observation and cannot be not guaranteed). 208 | 209 | However, there is a way to prevent the problem entirely. It is covered next. 210 | 211 | ### Race conditions while logging #3: Lost log entries when the shell is closed quickly 212 | 213 | Log entries [may still appear](#race-conditions-while-logging-2-still-writing-errors-after-the-prompt-is-back) after the prompt is back. But if the terminal is closed quickly, there is no prompt any more. The messages which are directed to the terminal can't be printed and are lost. But what about the log entries directed to a file? 214 | 215 | In a Bash terminal, after the terminal exits, the entries still make it to the file. The following command sequence is the same as above (in [race condition #2](#race-conditions-while-logging-2-still-writing-errors-after-the-prompt-is-back)), but the final `exit` command closes the terminal itself. The log file gets created nonetheless. 216 | 217 | ```bash 218 | (exec 2> >(sleep 1; tee -a "EXAMPLE_ERROR.log" >&2); echo "Oops" >&2; exit 1); exit 219 | ``` 220 | 221 | But things change when the call is made from a Windows command prompt, using WSL. When the parent process – the `wsl` command – exits, it takes all child processes with it _immediately_, including any asynchronous subshells. And `wsl` dies as soon as the top-level script returns. 222 | 223 | At that moment the log entries are still waiting to be written. But WSL is gone, and so is the log process. The entries don't make it to the log file and are lost. 224 | 225 | In the following example, the latency of writing to the file system is simulated with `sleep`. In real-world use, long delays are the norm because writing to the Windows file system is so excruciatingly slow. (NB: `^` escapes the following character for the cmd command line.) 226 | 227 | ```bat 228 | wsl (exec 2^> ^>(sleep 1; tee -a "EXAMPLE_ERROR.log" ^>^&2); echo "Oops" ^>^&2; exit 1) 229 | ``` 230 | 231 | The file won't be created in the example above. 232 | 233 | However, if the `wsl` process is forced to hang around long enough (e.g. with `sleep`), the log will show up in both the terminal and the file: 234 | 235 | ```bat 236 | wsl (exec 2^> ^>(sleep 1; tee -a "EXAMPLE_ERROR.log" ^>^&2); echo "Oops" ^>^&2; exit 1); sleep 3 237 | ``` 238 | 239 | That works. It is not a proper solution, though. The problem should be fixed in the script which sets up the logging, not by a hack bolted onto the command line. The task is to keep the script itself from exiting before the logging is done. 240 | 241 | To that end, the script must `wait` for the logger subshell to complete (the [process ID of the logger subshell](http://mywiki.wooledge.org/BashFAQ/106#:~:text=In%20bash%204.4%2C%20a%20ProcessSubstitution%20sets%20the%20%24!%20parameter) is needed for that). 242 | 243 | But there is a catch. Because the logger subshell is waiting for input from stderr, it is never "done". It will keep running unless it is disconnected from stderr. So one more step is needed: When it is certain that no more error logging will be initiated, the file descriptor for stderr has to be closed. Then the subshell will close shop. Once all the data queued for logging (in the output buffer, waiting to be written to the log file) has been processed, the process substitution subshell exits. The `wait` is over and the top-level script is allowed to exit, too. 244 | 245 | That two-step procedure – closing stderr and waiting for the subshell to finish its tasks – has to be placed at every conceivable exit point of the top-level script. Usually, there are just two of them: 246 | 247 | - at the end of the script 248 | - when a fatal error is handled (it is logged first, then the script exits). 249 | 250 | This is what the solution looks like: 251 | 252 | ```bash 253 | close_log() { exec 2>&-; wait $log_process_id; } 254 | 255 | # Close the log after a fatal error is logged. Only in the top-level script! 256 | fatal_error() { echo -e "$1" >&2; close_log; exit 1; } 257 | 258 | exec 2> >(tee -a "$ERROR_LOG" >&2) 259 | log_process_id=$! 260 | 261 | # ... do stuff, e.g. 262 | # some_command || fatal_error "some_command has failed" 263 | 264 | # Close the log when the script has reached its end 265 | close_log 266 | ``` 267 | 268 | Important: `close_log` has to be called in `fatal_error`, but _only in the top-level script_. 269 | 270 | The second `close_log` invocation has to be the last command in the script. But there is one exception: in scripts where the result of the script is conveyed by its exit code (e.g. boolean tests). Normally, the exit code would be set by the last command in the script. Now that status has to be captured and transferred to the end: 271 | 272 | ```bash 273 | # ... do stuff ... 274 | 275 | # The exit code of the command below should be returned, but we still have to 276 | # close the log at the very end 277 | final_command 278 | result=$? 279 | 280 | close_log 281 | exit $result 282 | ``` 283 | 284 | Ending the top-level script like that is good houskeeping in most situations, not just for boolean tests. E.g., the last call in the script might exit with an error status. Unless it is handled with `... || fatal_error`, the exit status should be captured and passed on. 285 | 286 | The final implementation can be found in the top-level scripts of Cloudypass. It is a bit cleaner but functionally identical. 287 | 288 | See 289 | - http://mywiki.wooledge.org/BashFAQ/106 290 | - http://mywiki.wooledge.org/ProcessSubstitution 291 | - https://unix.stackexchange.com/questions/131801/closing-a-file-descriptor-vs 292 | 293 | ### A simpler solution: writing to the log file only 294 | 295 | All of these issues don't matter if errors are just logged to a file and not written to stderr as well. 296 | 297 | ```bash 298 | exec 2>"$ERROR_LOG" 299 | ``` 300 | 301 | Logging to a file ONLY, without redirection to stderr, eliminates the need for `tee` and hence for process substitution, so no race condition. Things are simpler then: 302 | 303 | - We no longer end up [without a command prompt](#race-conditions-while-logging-2-still-writing-errors-after-the-prompt-is-back). The issue was caused by asynchronous delays in the substituted process. 304 | - Proper [ordering of log entries](#race-conditions-while-logging-1-messed-up-log-entry-order) is guaranteeed because it's not async. 305 | - No [duplicate entries](#preventing-log-entry-duplication-or-multiplication). The error message is not fed back into stderr once logged, so it isn't recaptured by stderr logging in higher-level scripts. 306 | 307 | However, if error messages are supposed to appear in the terminal, too, then logging must be limited to the top-level script. And the top-level script must wait for the logging to complete. 308 | 309 | 310 | ## Useful functions, tested but not used in the project 311 | 312 | #### Extracting the drive letter from a WSL path 313 | 314 | Extracts the drive letter from a WSL path string. The path and the drive don't need to exist. Returns the drive letter as it is found in the path, i.e. usually in lower case. 315 | 316 | Usage: 317 | 318 | get_wsl_drive_letter "/mnt/x/foo/bar" 319 | 320 | Code: 321 | 322 | ```bash 323 | get_wsl_drive_letter() { sed -rn 's_^/mnt/([a-zA-Z])($|/.*)_\1_p' <<<"$1"; } 324 | ``` 325 | 326 | Example: 327 | ```bash 328 | some_wsl_path_on_mnt="/mnt/x/foo/bar" 329 | drive_letter="$(get_wsl_drive_letter "$some_wsl_path_on_mnt")" 330 | [ $? -ne 0 -o -z "$drive_letter" ] && fatal_error "The path does not refer to a Windows drive, or is malformed. The drive letter could not be identified." 331 | ``` 332 | 333 | #### Checking if a Windows drive has been mounted in WSL 334 | 335 | This is useful for drives other than physical volumes, e.g. the Boxcryptor drive, which are NOT mounted in WSL by default. To be used in `if` statements etc. 336 | 337 | Usage: 338 | 339 | is_mounted_in_wsl "x" || fatal_error "Drive not mounted" 340 | if is_mounted_in_wsl "X"; then ... 341 | 342 | Argument: The drive letter. Is not case-sensitive. 343 | 344 | ```bash 345 | is_mounted_in_wsl() { [[ "$(findmnt -lfno TARGET -T "/mnt/${1,}")" =~ ^/mnt/${1,}$ ]]; } 346 | ``` 347 | 348 | Notes: 349 | - `${1,}` = argument #1, with first letter converted to lower case 350 | - findmnt options in long, readable form: 351 | `findmnt --list --first-only --noheadings --output TARGET --target "/mnt/$drive_letter_lc"` 352 | - If matching, findmnt returns: `/mnt/$drive_letter_lc`, e.g. `/mnt/c` 353 | - If not matching, findmnt returns nothing or `/` 354 | 355 | 356 | #### Mounting a drive in WSL 357 | 358 | This could be useful for mounting missing, but existing drives (see above). BUT mounting a drive almost always requires sudo priviliges! 359 | 360 | Usage: 361 | 362 | sudo mount_in_wsl "x" 363 | 364 | Argument: The drive letter. Is not case-sensitive. 365 | 366 | ```bash 367 | mount_in_wsl() { mount -t drvfs "${1^^}:\\" "/mnt/${1,,}"; } 368 | ``` 369 | 370 | Notes: 371 | - `${1,,}` = argument #1, converted to lower case 372 | - `${1^^}` = argument #1, converted to upper case 373 | 374 | 375 | #### Calculating a file hash if the drive is not mounted in WSL 376 | 377 | Again, this is useful for unmounted, but existing drives like the Boxcryptor drive (see above). The operation needs to be delegated to Powershell, as it can "see" the drive while Linux commands can't. 378 | 379 | Usage: 380 | 381 | get_hash filepath 382 | 383 | Argument: The file path. The path, in the stand-alone version below, must be provided in Windows format. 384 | 385 | ```bash 386 | get_hash() { Powershell.exe -command "(Get-FileHash -Algorithm MD5 -LiteralPath '"$1"').Hash" | tr -d '\r'; } 387 | ``` 388 | 389 | In the alternative version, the path can be passed either in Windows format or as a WSL/Linux path (`/mnt/[drive]/...`). BUT it requires the `wsl-windows-path` utility. 390 | 391 | ```bash 392 | get_hash() { Powershell.exe -command "(Get-FileHash -Algorithm MD5 -LiteralPath '"$(wsl-windows-path -f "$1")"').Hash" | tr -d '\r'; } 393 | ``` 394 | 395 | Notes: 396 | - The `wsl-windows-path` utility is available in the .scripts/lib directory, or [as a gist on Github](https://gist.github.com/hashchange/f4cd619def08def6e90704e9905ce3d0). 397 | - `tr -d '\r'`: Windows newlines (\\r\\n) in Powershell output must be fixed by removing \\r. Otherwise, comparisons based on the output can fail. 398 | - Instead of MD5, another algorithm can be used. See the [`Get-FileHash` documentation](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/get-filehash?view=powershell-7.2#parameters)(`-Algorithm` parameter). 399 | 400 | 401 | #### Reading the Last Modified timestamp of a file if the drive is not mounted in WSL 402 | 403 | Again, useful for Boxcryptor drives. Returns the timestamp with maximum precision. 404 | 405 | Usage: 406 | 407 | get_last_modified filepath 408 | 409 | Argument: The file path. The path, in the stand-alone version below, must be provided in Windows format. 410 | 411 | ```bash 412 | get_last_modified() { Powershell.exe -command "(Get-Item '"$1"').LastWriteTime.ToString('yyyy-MM-dd HH:mm:ss.fffffff')" | tr -d '\r'; } 413 | ``` 414 | 415 | In the alternative version, the path can be passed either in Windows format or as a WSL/Linux path (`/mnt/[drive]/...`). BUT it requires the `wsl-windows-path` utility. 416 | 417 | ```bash 418 | get_last_modified() { Powershell.exe -command "(Get-Item '"$(wsl-windows-path -f "$1")"').LastWriteTime.ToString('yyyy-MM-dd HH:mm:ss.fffffff')" | tr -d '\r'; } 419 | ``` 420 | 421 | Notes: 422 | - The date format of the output, and the precision, can easily be changed by altering the [`ToString` format string](https://docs.microsoft.com/en-us/dotnet/standard/base-types/formatting-types#custom-format-strings). 423 | - `wsl-windows-path`, `tr -d '\r'`: see `get_hash` notes, above. 424 | 425 | 426 | #### Converting a Windows path to Linux format even if the drive is not mounted in WSL 427 | 428 | The problem: `wslpath` throws an error if the drive is not mounted in WSL. So `wslpath` doesn't work for the Boxcryptor drive. 429 | 430 | The function below does handle that special case and works for ordinary, mounted drives, too. It converts a Windows path to Linux format. The path does not have to exist. 431 | 432 | If the path does not conform to an absolute Windows path pattern (i.e., it doesn't begin with `[Drive letter]:\\`), then backslashes are converted to forward slashes, but otherwise the path is returned as it was passed in. A Linux path is returned unchanged. 433 | 434 | The function expects the path as an argument or from stdin, i.e. it works in a pipe. 435 | 436 | Usage: 437 | 438 | to_linux_path filepath 439 | ... | to_linux_path 440 | 441 | ```bash 442 | to_linux_path() { 443 | local path="${1:-$(