├── timediff ├── README.md ├── UsingTimecopy.md ├── timedog ├── LICENSE └── timecopy.py /timediff: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Copyright (c) 2009 Nathan Fiedler 4 | # 5 | # This program is free software: you can redistribute it and/or modify 6 | # it under the terms of the GNU General Public License as published by 7 | # the Free Software Foundation, either version 3 of the License, or 8 | # (at your option) any later version. 9 | # 10 | # This program is distributed in the hope that it will be useful, 11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 13 | # GNU General Public License for more details. 14 | # 15 | # You should have received a copy of the GNU General Public License 16 | # along with this program. If not, see . 17 | # 18 | # $Id$ 19 | # 20 | ###################################################################### 21 | # 22 | # Given two supposedly identical Time Machine volumes, run the timedog 23 | # script on each backup (e.g. 2008-02-19-005705) in both volumes and 24 | # compare the output to determine that they are indeed truly identical. 25 | # 26 | # Usage: sudo timediff.sh /usr/local/bin/timedog \ 27 | # /Volume/OldBackup/Backups.backupdb/hostname \ 28 | # /Volume/NewBackup/Backups.backupdb/hostname 29 | # 30 | # You must supply the path to the timedog script as the first argument 31 | # to this script, and the paths to the old and new Time Machine volumes, 32 | # including the host name of the backups to compare. Run the 'hostname' 33 | # command in the Terminal to get the name of your machine. 34 | # 35 | # Typically must be run with sudo to visit certain directories, otherwise 36 | # you will get false negatives. 37 | # 38 | ###################################################################### 39 | 40 | [ -z "$1" ] && echo "Missing path to timedog script!" && exit 3 41 | if [ ! -x "$1" ]; then 42 | echo "$1 must exist and be executable!" 43 | exit 3 44 | fi 45 | TIMEDOG=$1 46 | [ -z "$2" ] && echo "Missing required source path!" && exit 3 47 | SRC=`echo $2 | sed -e 's|/$||'` 48 | [ -z "$3" ] && echo "Missing required target path!" && exit 3 49 | TGT=`echo $3 | sed -e 's|/$||'` 50 | 51 | PWD=`pwd` 52 | SCRIPT=`basename $0` 53 | TMP1=`mktemp -t ${SCRIPT}` 54 | if [ $? != 0 ]; then 55 | echo "Failed to create temporary file!" 56 | exit 1 57 | fi 58 | TMP2=`mktemp -t ${SCRIPT}` 59 | if [ $? != 0 ]; then 60 | echo "Failed to create temporary file!" 61 | exit 1 62 | fi 63 | 64 | # Get all of the snapshots, except the first one, and the other cruft. 65 | # We trust that ls sorts the entries for us, as stated in the man page. 66 | for SNAP in `ls $SRC | egrep -v '(Latest|*.inProgress)' | sed -e '1d'`; do 67 | if [ ! -d "$TGT/$SNAP" ]; then 68 | echo "$SNAP missing in $TGT location!" 69 | exit 70 | fi 71 | 72 | echo "Processing $SNAP, please be patient..." 73 | cd $SRC 74 | $TIMEDOG $SNAP > $TMP1 75 | cd $TGT 76 | $TIMEDOG $SNAP > $TMP2 77 | 78 | diff -q $TMP1 $TMP2 79 | if [ $? = 1 ]; then 80 | echo "$SNAP differs between $SRC and $TGT!" 81 | exit 2 82 | fi 83 | done 84 | 85 | cd $PWD 86 | rm -f $TMP1 87 | rm -f $TMP2 88 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # timedog 2 | 3 | timedog is a Perl script that displays the set of files that were saved for any given backup created by Mac OS X Time Machine. By default it shows those files that were saved in the most recent backup. The listing includes the file sizes before and after, as well as a total file count and size. The script includes an option to summarize changes to a particular directory depth, producing a more concise display, helping to get an understanding of which areas of your system are taking up the most space in the backups. It can also sort by size, and/or omit files below a given size. 4 | 5 | ## Usage 6 | 7 | 1. Open Terminal (in `/Applications/Utilities`) 8 | 1. `/path/to/timedog -h` 9 | * Displays the interactive help, using a pager, from which you can exit by pressing `q` 10 | 1. `/path/to/timedog -d 5 -l` 11 | * For instance, if you unzipped `timedog` to your Desktop, the path would be `~/Desktop/timedog` 12 | * If you change to the directory containing `timedog`, then `./timedog` also works. 13 | 14 | The example above uses the options `-d 5 -l` which will summarize the changes up to five directory levels deep, and hide rows that pertain to symbolic links. These links are often meaningless and can safely be ignored. 15 | 16 | By default `timedog` shows the file changes in the most recent backup. It locates and changes to your Time Machine directory automatically (typically `/Volumes/Time\ Machine/Backups.backupdb/[Computer Name]`). Timestamped backup directories like `2013-05-01-163402` can be passed to `timedog` as an argument: 17 | 18 | * `/path/to/timedog -d 5 -l 2013-05-01-163402` 19 | 20 | You can get a list of these with the `-t` option: 21 | 22 | * `/path/to/timedog -t` 23 | 24 | Below is an example of the output. 25 | 26 | ```shell 27 | $ ~/Desktop/timedog -d 5 -l 28 | ==> Comparing TM backup 2009-01-15-080533 to 2009-01-15-070632 29 | 1.6KB-> 2.9KB /.Backup.log 30 | 0B-> 0B /.com.apple.TMCheckpoint 31 | 956B-> 956B /.exclusions.plist 32 | 0B-> 0B /Macintosh HD/.com.apple.timemachine.supported 33 | 1.1KB-> 1.1KB /Macintosh HD/private/var/db/.TimeMachine.Results.plist 34 | 1.1KB-> 1.1KB [1] /Macintosh HD/private/var/db/ 35 | 12.0KB-> 12.0KB /Macintosh HD/Users/nfiedler/.DS_Store 36 | 6.5MB-> 6.6MB [26] /Macintosh HD/Users/nfiedler/Library/Application Support/ 37 | 0B-> 245B [1] /Macintosh HD/Users/nfiedler/Library/Favorites/ 38 | 40.3KB-> 42.7KB [29] /Macintosh HD/Users/nfiedler/Library/Preferences/ 39 | 1.4MB-> 1.4MB [27] /Macintosh HD/Users/nfiedler/Library/Thunderbird/ 40 | ==> Total Backup: 111 changed files/directories, 8.08MB 41 | ``` 42 | 43 | The number in square brackets (e.g. `[26]`) indicates the number of files and/or directories that changed within that particular directory tree. So in the example above, 26 entries under `Application Support` where changed. 44 | 45 | ## Time Machine over the Network 46 | 47 | If you are using Time Machine over the network, such as with the Time Capsule product, you will probably need to mount the backup disk image before you can use the `timedog` script. To do this, open the Disk Utility application (from Spotlight, type "disk utility" and press Enter; or use Finder navigate to `/Applications/Utilities` and launch Disk Utility), then open Finder and navigate to the network share that contains your backup image. Select and drag the disk image to the Disk Utility window and drop it. You should then see the image name in left pane of the Disk Utility window. Now select that row and click the *Open* button in the toolbar. A small window will appear that shows the progress. When it shows "verifying", click the *Skip* button; another dialog appears to report a warning, just click *Ok*. 48 | 49 | At this point you will have the Time Machine backup image mounted and available for browsing. You can now follow the example usage shown in the above section. 50 | 51 | ## Copying Time Machine volumes 52 | 53 | If you have a need to copy a Time Machine volume without using a disk block copy utility, then [timecopy.py](./timecopy.py) might be for you. See the [UsingTimecopy](./UsingTimecopy.md) page for details on how this script can be used and why. 54 | 55 | ## Files Accessibility 56 | 57 | If your time machine backup includes files which are not reachable or readable as a normal user, you should run `timedog` using `sudo`, like so: 58 | 59 | ``` 60 | $ sudo /path/to/timedog -d 5 -l 61 | ``` 62 | 63 | ## Troubleshooting 64 | 65 | ### Operation not permitted on Mojave 66 | 67 | By default the Mojave release of macOS does not grant access to certain areas of the file system. To get around this restriction, simply grant the terminal application full disk access prior to using `timedog`. The steps are outlined below. 68 | 69 | 1. Open the Apple menu and choose **System Preferences** 70 | 1. Select the **Security & Privacy** control panel 71 | 1. Select the **Privacy** tab 72 | 1. In the left panel choose _Full Disk Access_ 73 | 1. Click the lock icon in the corner of the window to gain admin privileges 74 | 1. Click the **+** button to add an application with full disk access 75 | 1. Find and select the terminal application that you normally use (e.g. `Terminal.app` or `iTerm`) 76 | 1. Start your terminal (again) and you should have full access 77 | -------------------------------------------------------------------------------- /UsingTimecopy.md: -------------------------------------------------------------------------------- 1 | # Introduction # 2 | 3 | The timecopy.py script is a tool that makes a faithful copy of a Time Machine volume to a new disk (or disk image). It can be useful if you need to recover from a file system error, such as the dreaded **invalid sibling link** error, in which you can probably read the backups but you cannot create new ones. If this is the case, most of the disk repair tools cannot fix this particular error (some people have had luck with fsck and Disk Warrior), so you are left with having to make a copy of the corrupt disk (image). Using a tool that performs a block-for-block copy will in fact copy the file system error to the new disk, which is of no use at all. What's needed is a way to copy the file system to a new location using traditional file copy. The only problem with that is the Time Machine backups are full of hard links, which will appear as normal files and directories, and performing a simple file copy will result in an enormous waste of disk space. 4 | 5 | This is where the timecopy.py script comes in. It understands the format of the Time Machine volumes and will intelligently reproduce the backups, to be best of its ability, in a new location. This includes all of the hard links, backed up files, and the extended attributes that form the basis of the "magic" of Time Machine. 6 | 7 | # Getting Started # 8 | 9 | This procedure is not for the faint of heart, and involves use of the command line and running scripts as the root user. If you have any doubts, stop now. Likewise, this script is **not guaranteed to work**. In fact, the author makes no guarantee that it will not cause harm. While every step has been taken to prevent such an event, use of this script is at **your own risk**. With that being said, it has been tested on multiple systems and found to work as expected. 10 | 11 | Make backups. Before going any further, make sure you have saved your important data to removable media that is locked away in a safe place. Who knows, your computer may catch on fire while using this script. Okay, probably not, but please do make backups. 12 | 13 | # Using timecopy # 14 | 15 | So you need to make a copy of a Time Machine volume, and you want to use timecopy. Now what? Start by determining the size of your Time Machine volume and locating a disk, or some free space where you can create a disk image, that has enough room to hold a copy of the Time Machine backups. If you have another disk that you can use, make sure it is formatted with the HFS+J disk format. That is, it must be HFS+ and have journaling enabled (just like the disk used for Time Machine). 16 | 17 | Next, either attach that disk and ensure it is in HFS+J format, or create a disk image to hold the copy of your TM volume. I recommend creating a disk image using the following command, entered in the Terminal window (Terminal is found in `/Applications/Utilities`): 18 | 19 | ``` 20 | sudo hdiutil create -nospotlight -type SPARSEBUNDLE -imagekey \ 21 | sparse-band-size=131072 -size 300g -fs "HFS+J" -volname "NewBackup" \ 22 | mymac_012345678987.sparsebundle 23 | ``` 24 | 25 | * In place of the "300g" argument, enter whatever size you want for the new volume. The typical recommendation is twice the size of your system disk. The 'g' stands for gigabytes. See the hdiutil man page (`man hdiutil`) for more details. 26 | * In place of the "mymac" name, enter the host name of your computer, as found with the command: `hostname -s` 27 | * In place of the "012345678987" sequence, enter the MAC address of your computer, as found with the command: `ifconfig en0 | grep "ether" | sed 's/://g' | awk '{print $2}'` 28 | * When prompted, enter the password for your user account, which authenticates you and allows you to run the `hdiutil` command as the root user. 29 | * If you are using a remote server to hold your TM backups, you can now move the disk image to that server. Note that you (probably) cannot create the disk image directly on the remote server. 30 | 31 | Now, "attach" the disk image so it is visible. 32 | 33 | ``` 34 | sudo hdiutil attach -noverify -noautofsck /path/to/mymac_012345678987.sparsebundle 35 | ``` 36 | 37 | * The command options are there to avoid the expense of scanning and verifying the disk image, which you almost never need to do anyway. 38 | * At this point you should see the new backup volume mounted under `/Volumes` on your computer. 39 | 40 | If your original TM volume is not mounted yet, do so now. If you have a corrupt volume, you may need to mount it in read-only mode. Chances are, when you try to mount it, the file system check will show an error and mount it in read-only form anyway. If this volume is on a remote share, you can quickly mount it this way: 41 | 42 | ``` 43 | sudo hdiutil attach -noverify -noautofsck -readonly /path/to/mymac_012345678987.sparsebundle 44 | ``` 45 | 46 | Finally, after all of that, you can run the [timecopy.py](./timecopy.py) script (click the link to download the script). 47 | 48 | ``` 49 | sudo timecopy.py /Volumes/OldBackup /Volumes/NewBackup 50 | ``` 51 | 52 | Now, depending on how large your Time Machine volume is, prepare to wait a long time for the copy to complete. If you see any warnings about xattr not working, you can probably ignore them. Most of the time it's just Finder information associated with a file or directory. 53 | 54 | Once the copy is complete, you should be able to point Time Machine to the new volume, make new backups and browse the backup history as usual. 55 | -------------------------------------------------------------------------------- /timedog: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | ############################################################################### 4 | # 5 | # LICENSE 6 | # 7 | # Copyright (C) 2008, 2009, 2013, 2014 J.D. Smith 8 | # 9 | # This file is free software; you can redistribute it and/or modify 10 | # it under the terms of the GNU General Public License as published 11 | # by the Free Software Foundation; either version 2, or (at your 12 | # option) any later version. 13 | # 14 | # This File is distributed in the hope that it will be useful, but 15 | # WITHOUT ANY WARRANTY; without even the implied warranty of 16 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 17 | # General Public License for more details. 18 | # 19 | # You should have received a copy of the GNU General Public License 20 | # along with this file; see the file COPYING. If not, write to the 21 | # Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 22 | # Boston, MA 02110-1301, USA. 23 | # 24 | ############################################################################### 25 | 26 | ################################################################################ 27 | 28 | =head1 NAME 29 | 30 | timedog - script to display the files backed up by time machine 31 | 32 | =head1 SYNOPSIS 33 | 34 | timedog -h 35 | timedog -t 36 | timedog [-vlsHn] [-d depth] [-S sort] [-m limit] [timestamp] 37 | 38 | =head1 DESCRIPTION 39 | 40 | Display the files which time machine backed up in its most recent 41 | backup (or any of your choosing). Uses the fact that Time Machine 42 | creates hard links for unchanged files and directories. Reports old 43 | and new sizes of the changed files, along with the total file count 44 | and backup size. 45 | 46 | The script will locate the Time Machine directory using the system 47 | utility tmutil. To run the script, you must ensure the Time Machine 48 | volume is mounted. The Time Machine backup can normally be found in a 49 | directory called e.g., /Volumes/TM/Backups.backupdb/hostname. 50 | 51 | Written by J.D. Smith (jdtsmith A@T gmail _d0t_ com), adaptations by 52 | Hayo Baan (info A@T hayobaan _d0t_ nl). 53 | 54 | =head2 Options 55 | 56 | =over 8 57 | 58 | =item B<-?> or B<-h> or B<--help> 59 | 60 | Show full help. 61 | 62 | =item B<-t> 63 | 64 | List all available backup timestamps 65 | 66 | =item B<-v> 67 | 68 | Verbose (display depth and minimum size used, if any) 69 | 70 | =item B<-d > 71 | 72 | By default, all files are printed, which can get lengthy. With this 73 | option, all changed files/directories deeper than I will be 74 | summarized to the parent directory at level I. 75 | 76 | =item B<-l> 77 | 78 | Omit symbolic links from the summary. For whatever reason, Time 79 | Machine creates a new version of a symbolic link each and every time 80 | it backs up. 81 | 82 | =item B<-s> 83 | 84 | Sort the output by current size. 85 | 86 | =item B<-S <0..2>> 87 | 88 | Sort by: 0=old size, 1=current size, 2=name (default) 89 | 90 | =item B<-m > 91 | 92 | Omit files/directories whose current size is less than limit (limit in 93 | bytes by default, but can be specified as K, M, G, T for Kilo-, Mega-, 94 | Giga-, Tera-, or Peta- Bytes). 95 | 96 | =item B<-H> 97 | 98 | Display sizes using base 10 sizes (default is base 2). 99 | 100 | =item B<-n> 101 | 102 | Use simple fixed width formatting (useful for spreadsheets or other 103 | parsing). 104 | 105 | =item B 106 | 107 | The backup directory for which you'd like to see the changed contents. Defaults to the most recent (the one linked to by "Latest" in the backup directory) 108 | 109 | =back 110 | 111 | =head2 Acknowledgements 112 | 113 | =over 114 | 115 | =item Nathan Fielder: 116 | 117 | Packaging and google code repo. 118 | 119 | =item Lanny Rosicky: 120 | 121 | Use of tmutil to locate the time machine backup directory. 122 | 123 | =item Hayo Baan: 124 | 125 | Code rewrite & bug fixes. 126 | Additional help (-h), verbose (-v), sort (-S), and display (-H) options. 127 | Documentation and help (perlpod). 128 | 129 | =back 130 | 131 | =cut 132 | 133 | ################################################################################ 134 | 135 | use warnings; # Enable warnings 136 | use strict; # Enable strict 137 | 138 | use Pod::Usage; 139 | use Fcntl ':mode'; 140 | use Getopt::Std; 141 | 142 | use Cwd; 143 | use File::Find; 144 | no warnings "File::Find"; # Do not report File::Find errors (e.g., opendir errors due to lack of directory permissions) 145 | 146 | # Usage and Help 147 | sub usage { pod2usage(-exitstatus => 1, -verbose => 0); } 148 | sub help { pod2usage(-exitstatus => 0, -verbose => 2); } 149 | *HELP_MESSAGE = *VERSION_MESSAGE = \&help; 150 | $Getopt::Std::STANDARD_HELP_VERSION = 1; 151 | 152 | ################################################################################ 153 | # Option processing and Initialisation ######################################### 154 | ################################################################################ 155 | 156 | # Process the options 157 | our ($opt_h, $opt_t, $opt_v, $opt_d, $opt_l, $opt_s, $opt_S, $opt_m, $opt_H, $opt_n); 158 | getopts('?htvd:lsS:m:Hn') || usage(); 159 | { 160 | no strict 'refs'; 161 | help() if ${'opt_?'} || $opt_h; 162 | } 163 | 164 | # Interpret depth option 165 | usage() if defined $opt_d && $opt_d !~ /^[0-9]+$/; 166 | 167 | # Interpret sort option 168 | $opt_S //= 2; # default sort by name 169 | $opt_S = 1 if $opt_s; # sort by current size if -s given 170 | usage() if $opt_S && ($opt_S !~ /^[0-9]+$/ || $opt_S > 2); 171 | 172 | # Interpret min size option 173 | my $base = ($opt_H?1000.0:1024.0); 174 | if ($opt_m) { 175 | my %bases=('K' => $base, 'M' => $base**2, 'G'=>$base**3, 'T'=>$base**4); 176 | usage if $opt_m !~ /^([0-9.]+)([KMGT]?)$/i; 177 | $opt_m = $1 || 1; 178 | $opt_m *= $bases{uc $2} if $2; 179 | } 180 | 181 | # Get all available backups 182 | my @backups=`tmutil listbackups -m`; 183 | chomp @backups; 184 | die "None or only one Time Machine backups found" if @backups <= 1; 185 | 186 | # With -t, just list all available backups and exit 187 | if ($opt_t) { 188 | print join("\n", map { s/.*(\d{4}-\d{2}-\d{2}-\d{6}).*/$1/r } @backups),"\n"; 189 | exit; 190 | } 191 | 192 | # Determine backups to compare 193 | my ($last, $latest); 194 | if (@ARGV) { 195 | $latest = $ARGV[0]; 196 | $latest =~ s|/$||; 197 | $latest =~ s/([0-9]{4})-?([0-9]{2})-?([0-9]{2})-?([0-9]{6})/$1-$2-$3-$4/; # So that timestamp can be given with and without dashes 198 | for (@backups) { 199 | if (/$latest/) { $latest = $_; last; } 200 | $last=$_; 201 | } 202 | die "Invalid backup directory $latest specified" if !defined($last) || $last eq $latest; 203 | } else { 204 | ($last,$latest)=@backups[$#backups-1..$#backups]; 205 | } 206 | 207 | print "==> Comparing TM backup ". 208 | $latest =~ s/.*(\d{4}-\d{2}-\d{2}-\d{6}).*/$1/r ." to ". 209 | $last =~ s/.*(\d{4}-\d{2}-\d{2}-\d{6}).*/$1/r . "\n" unless $opt_n; 210 | 211 | unless ($opt_n || !$opt_v) { 212 | print " Depth: $opt_d directories\n" if defined $opt_d; 213 | print " Omitting if smaller than: ", bytes($opt_m), "\n" if ($opt_m); 214 | } 215 | 216 | ################################################################################ 217 | # Find all files/directories in latest backup and compare to previous ########## 218 | ################################################################################ 219 | 220 | my ($rsize,$rsize_old,$rcnt); # Counters/sizes for files/dirs below max depth 221 | my $total_size = 0; # Total size of changed files/dirs in latest backup 222 | my $total_cnt = 0; # Total number of changed files/dirs in latest backup 223 | my @summary = (); # List of summary lines 224 | 225 | 226 | find( 227 | { 228 | wanted => 229 | sub { 230 | (my $old=$_) =~ s/^$latest/$last/; 231 | (my $name=$_) =~ s/^$latest//; 232 | my (undef, $ino, $mode, undef, undef, undef, undef, $size) = lstat($_); 233 | # Silently ignore lstat errors (e.g., due to permission problems) 234 | $mode //= 0; 235 | $size //= 0; 236 | my ($ino_old, $size_old); 237 | if ($ino && -e $old) { 238 | (undef, $ino_old, undef, undef, undef, undef, undef, $size_old) = lstat($old); 239 | if ($ino_old && $ino == $ino_old) { 240 | $File::Find::prune=1 if S_ISDIR($mode); # Prune matching dirs 241 | return; 242 | } 243 | } 244 | 245 | # Add found size to total size 246 | $total_size += $size; 247 | 248 | # With -l, skip symbolic links 249 | my $link = S_ISLNK($mode); 250 | return if $opt_l && $link; 251 | 252 | # Increase count 253 | $total_cnt++; 254 | 255 | if (defined $opt_d) { 256 | my $depth = $name=~tr|/||; 257 | $rcnt ||= 0; 258 | if ($depth > $opt_d || (S_ISDIR($mode) && $depth == $opt_d)) { 259 | # Count files in directory 260 | $rsize += $size; 261 | $rsize_old += $size_old if defined $size_old; # Silently ignore lstat errors (e.g., due to permission problems) 262 | $rcnt++; 263 | return # Post will handle summarizing the directory 264 | } 265 | } 266 | # Add type indicator to name 267 | $name .= "/" if S_ISDIR($mode); 268 | $name .= "@" if $link; 269 | summarize($size,$size_old,$name); 270 | }, 271 | preprocess => 272 | (!defined $opt_d) ? 0: 273 | sub { 274 | (my $name=$File::Find::dir) =~ s/^$latest//; 275 | my $depth = $name =~ tr|/||; 276 | if ($depth <= $opt_d && !(-d && $depth == $opt_d)) { 277 | # Starting a new printable directory level; zero out recursive sizes 278 | $rsize=$rsize_old=$rcnt=undef; 279 | } 280 | @_; 281 | }, 282 | postprocess => 283 | (!defined $opt_d) ? 0: 284 | sub { 285 | (my $name=$File::Find::dir) =~ s/^$latest//; 286 | my $depth = $name =~ tr|/||; 287 | if ($depth == $opt_d) { 288 | # This directory is at the given depth, summarize it 289 | summarize($rsize,$rsize_old,"$name/",$rcnt) if $rsize || $rsize_old; 290 | $rsize = $rsize_old = $rcnt = undef; 291 | } 292 | }, 293 | no_chdir => 1, 294 | }, 295 | $latest); 296 | 297 | ################################################################################ 298 | # Display the results ########################################################## 299 | ################################################################################ 300 | 301 | my $format = $opt_n ? 14 : 11; 302 | my $div = $opt_n ? " " : " -> "; 303 | my $cnt_l = $opt_n ? "" : "["; 304 | my $cnt_r = $opt_n ? "" : "]"; 305 | 306 | # Header 307 | printf("%${format}.${format}s%s%${format}.${format}s", "Old Size" , $div, "New Size"); 308 | printf(" %9s", "# in Dir") if defined $opt_d; 309 | printf(" File/Directory\n"); 310 | if (!$opt_n) { 311 | print("=" x $format . $div . "=" x $format); 312 | print(" " . "=" x 9) if defined $opt_d; 313 | print(" " . "=" x 20 . "\n"); 314 | } 315 | 316 | # List sorted summary lines 317 | for (sort { ($opt_S != 2 ? ($a->[$opt_S]//-1) <=> ($b->[$opt_S]//-1) : 0) || $a->[2] cmp $b->[2]} @summary) { 318 | printf("%${format}s%s%${format}s", bytes($_->[0]), $div, bytes($_->[1])); 319 | printf(" %9s", ($_->[3]&&$_->[3]>1) ? $cnt_l . ($_->[3]-1) . $cnt_r : "") if defined $opt_d; 320 | print " $_->[2]\n"; 321 | } 322 | 323 | # Total 324 | printf "==> Total Backup: $total_cnt changed files/directories, %s\n", bytes($total_size,".2") unless $opt_n; 325 | 326 | exit 0; 327 | 328 | 329 | ################################################################################ 330 | # Helper functions ############################################################# 331 | ################################################################################ 332 | 333 | # Function to format number of bytes 334 | sub bytes { 335 | my $bytes=shift; 336 | my $format=shift || ".1"; 337 | 338 | $bytes //= 0 if ($opt_n); 339 | if (defined $bytes) { 340 | my @suffix = ("", "K", "M", "G", "T", "P"); 341 | my $suf = 0; 342 | 343 | while (!$opt_n && $bytes >= $base && $suf <= @suffix) { 344 | ++$suf; 345 | $bytes /= $base; 346 | } 347 | 348 | my $suffix = $suffix[$suf]; 349 | $suffix .= "i" if $suf && !$opt_H; 350 | $suffix .= "B" if !$opt_n; 351 | return ($suf ? sprintf("%${format}f", $bytes) : sprintf("%d", $bytes)) . $suffix; 352 | } else { 353 | return "..."; 354 | } 355 | } 356 | 357 | # Function to add the summary of a file/directory 358 | sub summarize { 359 | my ($size,$size_old,$name,$cnt) = @_; 360 | return if $opt_m && $size < $opt_m; 361 | push @summary, [$size_old, $size, $name, $cnt]; 362 | } 363 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | -------------------------------------------------------------------------------- /timecopy.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | """Copy a Time Machine volume. 3 | 4 | Invoke this script with '--help' option for detailed description of 5 | what it does and how you can use it. 6 | 7 | """ 8 | 9 | # 10 | # Copyright (c) 2009-2017 Nathan Fiedler 11 | # 12 | # This program is free software: you can redistribute it and/or modify 13 | # it under the terms of the GNU General Public License as published by 14 | # the Free Software Foundation, either version 3 of the License, or 15 | # (at your option) any later version. 16 | # 17 | # This program is distributed in the hope that it will be useful, 18 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 19 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 20 | # GNU General Public License for more details. 21 | # 22 | # You should have received a copy of the GNU General Public License 23 | # along with this program. If not, see . 24 | # 25 | 26 | import errno 27 | import getopt 28 | import os 29 | import os.path 30 | import re 31 | import shutil 32 | import stat 33 | import subprocess 34 | import sys 35 | import time 36 | import xattr 37 | import xattr.constants 38 | 39 | # 40 | # TODO: "too many open files" says get_tm_bandsize() on server the last time I tried using this 41 | # 42 | 43 | 44 | class TreeVisitor: 45 | 46 | """Visitor pattern for visitfiles function. 47 | 48 | As tree is traversed, methods of the visitor are invoked. 49 | """ 50 | 51 | def dir(self, dir): 52 | """A directory has been encountered.""" 53 | pass 54 | 55 | def file(self, file): 56 | """A file has been encountered.""" 57 | pass 58 | 59 | def link(self, link): 60 | """A symbolic link has been encountered.""" 61 | pass 62 | 63 | 64 | def visitfiles(dir, visitor): 65 | """Calls the visitor for each entry encountered in the directory.""" 66 | for entry in os.listdir(dir): 67 | pathname = os.path.join(dir, entry) 68 | try: 69 | mode = os.lstat(pathname)[stat.ST_MODE] 70 | if stat.S_ISDIR(mode): 71 | visitor.dir(pathname) 72 | elif stat.S_ISLNK(mode): 73 | visitor.link(pathname) 74 | elif stat.S_ISREG(mode): 75 | visitor.file(pathname) 76 | else: 77 | print 'WARNING: unknown file %s' % pathname 78 | except OSError, e: 79 | print "ERROR '{}' processing {}".format(e, pathname) 80 | 81 | 82 | def chown(path, uid, gid): 83 | """Attempt to change the owner/group of the given file/directory. 84 | 85 | Uses os.lchown(). If this fails due to insufficient permissions, display 86 | an appropriate error message and exit. Otherwise, raise the error. 87 | """ 88 | try: 89 | # Use lchown so we do not follow symbolic links, just change the 90 | # target as specified by the caller. 91 | os.lchown(path, uid, gid) 92 | # Note that it is possible the destination volume was mounted with 93 | # the MNT_IGNORE_OWNERSHIP flag, in which case everything we create 94 | # there will be owned by the _unknown user and group, no matter what 95 | # we might want it to be. This is built into the XNU kernel. 96 | except OSError, e: 97 | if e.errno == errno.EPERM: 98 | # Strangely root has problems changing symlinks that point 99 | # to non-existent entries, need to filter out and ignore 100 | # (we are most likely copying to an external disk anyway, 101 | # in which case all files are owned by the _unknown user). 102 | mode = os.lstat(path)[stat.ST_MODE] 103 | if not stat.S_ISLNK(mode): 104 | # Sometimes mysteriously fails to chown directories. 105 | # Try again in one second; if it fails again ignore 106 | # the problem and move on. 107 | time.sleep(1) 108 | try: 109 | os.chown(path, uid, gid) 110 | except OSError: 111 | pass 112 | else: 113 | raise e 114 | 115 | 116 | def link(src, dst): 117 | """Create a hard link called 'dst' that points to 'src'. 118 | 119 | Ensures that the src entry exists and raises an error if not. 120 | """ 121 | if os.path.exists(src): 122 | os.link(src, dst) 123 | else: 124 | raise OSError(errno.ENOENT, "%s missing!" % src) 125 | 126 | 127 | def copyxattr(src, dst): 128 | """Copy the extended attributes from src to dst using xattr.""" 129 | # See http://pypi.python.org/pypi/xattr for a (possibly outdated) 130 | # version of xattr. A (possibly newer) version is included with 131 | # Python on the Mac. 132 | sx = xattr.xattr(src) 133 | dx = xattr.xattr(dst) 134 | # Make sure not to follow symbolic links as we always work on the 135 | # links themselves, not the (possibly) non-existent target. 136 | attrs = sx.list(xattr.constants.XATTR_NOFOLLOW) 137 | try: 138 | for name in attrs: 139 | value = sx.get(name, xattr.constants.XATTR_NOFOLLOW) 140 | dx.set(name, value, xattr.constants.XATTR_NOFOLLOW) 141 | except IOError: 142 | # Fails for certain directories which we will ignore. 143 | # All others, show a warning. 144 | if not dst.endswith(("/etc", "/tmp", "/var")): 145 | print "WARNING: cannot xattr %s" % dst 146 | 147 | 148 | class CopyInitialVisitor(TreeVisitor): 149 | """Copies a directory tree from one place to another.""" 150 | 151 | def __init__(self, verbose, dryrun, extattr): 152 | """Initialize a CopyInitialVisitor. 153 | 154 | If verbose is True, display operations as they are performed 155 | If dryrun is True, do not make any modifications on disk. 156 | If extattr is True, just copy the extended attributes. 157 | """ 158 | self.verbose = verbose 159 | self.dryrun = dryrun 160 | self.extattr = extattr 161 | 162 | def copytree(self, src, dst): 163 | """Copy the directory tree rooted at src to dst.""" 164 | self.src = src 165 | self.dst = dst 166 | visitfiles(src, self) 167 | 168 | def dir(self, dir): 169 | """Create destination directory, copying stats and ownership.""" 170 | dst = re.sub(self.src, self.dst, dir) 171 | if self.verbose: 172 | print "mkdir <%s>" % dst 173 | if not self.dryrun: 174 | os.mkdir(dst) 175 | shutil.copystat(dir, dst) 176 | stats = os.lstat(dir) 177 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 178 | if not self.dryrun or self.extattr: 179 | copyxattr(dir, dst) 180 | # Continue traversal... 181 | visitfiles(dir, self) 182 | 183 | def file(self, file): 184 | """Process a single file.""" 185 | dst = re.sub(self.src, self.dst, file) 186 | if self.verbose: 187 | print "cp <%s> <%s>" % (file, dst) 188 | if not self.dryrun: 189 | try: 190 | # Copy file contents from snapshot to destination. 191 | shutil.copyfile(file, dst) 192 | # Copy the permissions and accessed/modified times. 193 | shutil.copystat(file, dst) 194 | # Copy the owner/group values to destination. 195 | stats = os.lstat(file) 196 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 197 | except IOError, e: 198 | print "ERROR '{}' processing file {}".format(e, file) 199 | if not self.dryrun or self.extattr: 200 | copyxattr(file, dst) 201 | 202 | def link(self, link): 203 | """Copy link to destination.""" 204 | lnk = os.readlink(link) 205 | dst = re.sub(self.src, self.dst, link) 206 | if self.verbose: 207 | print "ln -s <%s> <%s>" % (lnk, dst) 208 | if not self.dryrun: 209 | os.symlink(lnk, dst) 210 | stats = os.lstat(link) 211 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 212 | if not self.dryrun or self.extattr: 213 | copyxattr(link, dst) 214 | 215 | 216 | class CopyBackupVisitor(TreeVisitor): 217 | """Copy a directory tree and its files. 218 | 219 | If any entry has the same inode value as the corresponding entry in the 220 | reference tree, a new hard link is made in the destination, and nothing 221 | further is done with that entry (directories are not traversed, files are 222 | not copied). 223 | 224 | """ 225 | 226 | def __init__(self, old, prev, curr, verbose, dryrun, extattr): 227 | """Initialize a CopyBackupVisitor. 228 | 229 | If verbose is True, display operations as they are performed 230 | If dryrun is True, do not make any modifications on disk. 231 | If extattr is True, just copy the extended attributes. 232 | old is the reference tree to which src will be compared. 233 | prev is the entry name of the previous backup. 234 | curr is the entry name of the backup being copied. 235 | 236 | """ 237 | self.verbose = verbose 238 | self.dryrun = dryrun 239 | self.old = old 240 | self.prev = prev 241 | self.curr = curr 242 | self.extattr = extattr 243 | 244 | def copytree(self, src, dst): 245 | """Copy the tree rooted at src to dst.""" 246 | self.src = src 247 | self.dst = dst 248 | visitfiles(src, self) 249 | 250 | def dir(self, dir): 251 | """Process a directory.""" 252 | stats = os.lstat(dir) 253 | old = re.sub(self.src, self.old, dir) 254 | try: 255 | ostats = os.lstat(old) 256 | except OSError, e: 257 | if e.errno in (errno.ENOENT, errno.ENOTDIR, errno.EISDIR): 258 | # File became directory, or vice versa, or just isn't there. 259 | ostats = None 260 | else: 261 | raise e 262 | dst = re.sub(self.src, self.dst, dir) 263 | if ostats is None or stats[stat.ST_INO] != ostats[stat.ST_INO]: 264 | if self.verbose: 265 | print "mkdir <%s>" % dst 266 | if not self.dryrun: 267 | # Create destination directory, copying stats and ownership. 268 | os.mkdir(dst) 269 | shutil.copystat(dir, dst) 270 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 271 | if not self.dryrun or self.extattr: 272 | copyxattr(dir, dst) 273 | # Continue traversal... 274 | visitfiles(dir, self) 275 | else: 276 | odst = re.sub(self.curr, self.prev, dst) 277 | if self.verbose: 278 | print "ln <%s> <%s>" % (dst, odst) 279 | if not self.dryrun: 280 | # Create hard link in destination. 281 | link(odst, dst) 282 | 283 | def file(self, file): 284 | """Process a file.""" 285 | stats = os.lstat(file) 286 | old = re.sub(self.src, self.old, file) 287 | dst = re.sub(self.src, self.dst, file) 288 | try: 289 | ostats = os.lstat(old) 290 | except OSError, e: 291 | if e.errno in (errno.ENOENT, errno.ENOTDIR, errno.EISDIR): 292 | # File became directory, or vice versa, or just isn't there. 293 | ostats = None 294 | else: 295 | raise e 296 | if ostats is None or stats[stat.ST_INO] != ostats[stat.ST_INO]: 297 | if self.verbose: 298 | print "cp <%s> <%s>" % (file, dst) 299 | if not self.dryrun: 300 | try: 301 | # Copy file contents from snapshot to destination. 302 | shutil.copyfile(file, dst) 303 | # Copy the permissions and accessed/modified times. 304 | shutil.copystat(file, dst) 305 | # Copy the owner/group values to destination. 306 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 307 | except IOError, e: 308 | print "ERROR '{}' processing file {}".format(e, file) 309 | if not self.dryrun or self.extattr: 310 | copyxattr(file, dst) 311 | else: 312 | odst = re.sub(self.curr, self.prev, dst) 313 | if self.verbose: 314 | print "ln <%s> <%s>" % (dst, odst) 315 | if not self.dryrun: 316 | # Create hard link in destination. 317 | link(odst, dst) 318 | 319 | def link(self, link): 320 | """Process a link.""" 321 | stats = os.lstat(link) 322 | old = re.sub(self.src, self.old, link) 323 | dst = re.sub(self.src, self.dst, link) 324 | try: 325 | ostats = os.lstat(old) 326 | except OSError, e: 327 | if e.errno in (errno.ENOENT, errno.ENOTDIR, errno.EISDIR): 328 | # File became directory, or vice versa, or just isn't there. 329 | ostats = None 330 | else: 331 | raise e 332 | if ostats is None or stats[stat.ST_INO] != ostats[stat.ST_INO]: 333 | lnk = os.readlink(link) 334 | if self.verbose: 335 | print "ln -s <%s> <%s>" % (lnk, dst) 336 | if not self.dryrun: 337 | # Copy link to destination. 338 | os.symlink(lnk, dst) 339 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 340 | if not self.dryrun or self.extattr: 341 | copyxattr(link, dst) 342 | else: 343 | odst = re.sub(self.curr, self.prev, dst) 344 | if self.verbose: 345 | print "ln <%s> <%s>" % (dst, odst) 346 | if not self.dryrun: 347 | # Create hard link in destination. 348 | link(odst, dst) 349 | 350 | 351 | def copybackupdb(srcbase, dstbase, verbose, dryrun, extattr): 352 | """Copy the backup database found in srcbase to dstbase.""" 353 | # Validate that srcbase contains a backup database. 354 | srcdb = os.path.join(srcbase, 'Backups.backupdb') 355 | if not os.path.exists(srcdb): 356 | print "ERROR: %s does not contain a Time Machine backup!" % srcbase 357 | sys.exit(2) 358 | dstdb = os.path.join(dstbase, 'Backups.backupdb') 359 | # Get a list of entries in the backupdb (typically just one). 360 | hosts = os.listdir(srcdb) 361 | 362 | def goodhost(host): 363 | src = os.path.join(srcdb, host) 364 | mode = os.lstat(src)[stat.ST_MODE] 365 | if len(host) > 0 and host[0] != '.' and stat.S_ISDIR(mode): 366 | return True 367 | return False 368 | hosts = [host for host in hosts if goodhost(host)] 369 | for host in hosts: 370 | # Get the list of backup snapshots sorted by name (i.e. date). 371 | src = os.path.join(srcdb, host) 372 | entries = os.listdir(src) 373 | 374 | def goodsnap(snap): 375 | if snap == '.DS_Store' or snap == 'Latest'\ 376 | or snap.endswith('.inProgress'): 377 | return False 378 | return True 379 | entries = [entry for entry in entries if goodsnap(entry)] 380 | entries.sort() 381 | 382 | def mkdest(source, target): 383 | stats = os.lstat(source) 384 | if verbose: 385 | print "mkdir <%s>" % target 386 | if not dryrun: 387 | os.makedirs(target) 388 | shutil.copystat(source, target) 389 | chown(target, stats[stat.ST_UID], stats[stat.ST_GID]) 390 | if not dryrun or extattr: 391 | copyxattr(source, target) 392 | # Copy initial backup. 393 | dst = os.path.join(dstdb, host) 394 | srcbkup = os.path.join(src, entries[0]) 395 | dstbkup = os.path.join(dst, entries[0]) 396 | if not extattr and os.path.exists(dstbkup): 397 | print "%s already exists, skipping..." % entries[0] 398 | else: 399 | mkdest(srcbkup, dstbkup) 400 | visitor = CopyInitialVisitor(verbose, dryrun, extattr) 401 | print "Copying backup %s -- this may take a while..." % entries[0] 402 | visitor.copytree(srcbkup, dstbkup) 403 | # Copy all subsequent backup snapshots. 404 | prev = entries[0] 405 | for entry in entries[1:]: 406 | # Here previous is the backup before the one we are about to 407 | # copy; it is used to determine which entries are hard links. 408 | previous = srcbkup 409 | srcbkup = os.path.join(src, entry) 410 | dstbkup = os.path.join(dst, entry) 411 | if not extattr and os.path.exists(dstbkup): 412 | print "%s already exists, skipping..." % entry 413 | else: 414 | mkdest(srcbkup, dstbkup) 415 | visitor = CopyBackupVisitor(previous, prev, entry, 416 | verbose, dryrun, extattr) 417 | print "Copying backup %s..." % entry 418 | visitor.copytree(srcbkup, dstbkup) 419 | prev = entry 420 | # Create Latest symlink pointing to last entry. 421 | latest = os.path.join(dst, 'Latest') 422 | if verbose: 423 | print "ln -s <%s> <%s>" % (entries[-1], latest) 424 | if not dryrun: 425 | if os.path.lexists(latest): 426 | # Seems root cannot delete the symlink, so have the real 427 | # user perform the delete for us. 428 | user = subprocess.Popen( 429 | ["who", "am", "i"], stdout=subprocess.PIPE 430 | ).communicate()[0] 431 | user = user.split()[0] 432 | os.system("sudo -u %s unlink %s" % (user, latest)) 433 | os.symlink(entries[-1], latest) 434 | # Copy the MAC address dotfile(s) that TM creates. 435 | entries = os.listdir(srcbase) 436 | regex = re.compile('^\.[0-9a-f]{12}$') 437 | for entry in entries: 438 | if regex.match(entry): 439 | src = os.path.join(srcbase, entry) 440 | dst = os.path.join(dstbase, entry) 441 | if verbose: 442 | print "cp <%s> <%s>" % (src, dst) 443 | if not dryrun: 444 | try: 445 | shutil.copyfile(src, dst) 446 | shutil.copystat(src, dst) 447 | stats = os.lstat(src) 448 | chown(dst, stats[stat.ST_UID], stats[stat.ST_GID]) 449 | except IOError, e: 450 | print "ERROR '{}' processing file {}".format(e, src) 451 | if not dryrun or extattr: 452 | copyxattr(src, dst) 453 | 454 | 455 | def usage(): 456 | """Display a usage summary.""" 457 | print """Usage: timecopy.py [-hnvx] [--nochown] 458 | 459 | Copies a Mac OS X Time Machine volume (set of backups) from one location 460 | to another, such as from one disk to another, or from one disk image to 461 | another. This can be useful when block copying the disk is not feasible 462 | (i.e. the destination disk is smaller than the original). 463 | 464 | The location must be the root directory of the source Time 465 | Machine volume, that which contains the 'Backups.backupdb' directory 466 | (e.g. /Volumes/Backup, not /Volumes/Backup/Backups.backupdb/gojira). 467 | You must have sufficient privileges to access this directory, and the 468 | Time Machine volume must already be mounted (read-only mode is okay). 469 | 470 | The location should be the root of an empty volume to which the 471 | Time Machine backups will be copied. You must have sufficient privileges 472 | to write to this location. Chances are you will need to be using `sudo` 473 | to gain the necessary privileges, unless -n or --dry-run is given. 474 | 475 | -h|--help 476 | \tPrints this usage information. 477 | 478 | -n|--dry-run 479 | \tDo not make any changes on disk. 480 | 481 | --nochown 482 | \tDo not use chown to change the owner/group of the destination 483 | \tfiles. Generally only root can do that, and on network volumes 484 | \tthe Mac will make everything owned by the 'unknown' user anyway. 485 | 486 | -v|--verbose 487 | \tPrints information about what the script is doing at each step. 488 | 489 | -x|--xattr 490 | \tCopies the extended attributes from the source volume to the target 491 | \tvolume, assuming that the target is an exact copy of the source. 492 | \tThis is useful if you have a copy of a Time Machine volume that is 493 | \tmissing the necessary extended attributes. Normally this script 494 | \twill already have copied the extended attributes as part of the 495 | \tcopying process, so this option is only needed when you have created 496 | \tthe copy using some other means.""" 497 | 498 | 499 | def main(): 500 | """Parse command line arguments and do the work.""" 501 | # Parse the command line arguments. 502 | shortopts = "hnvx" 503 | longopts = ["help", "dry-run", "nochown", "verbose", "xattr"] 504 | try: 505 | opts, args = getopt.getopt(sys.argv[1:], shortopts, longopts) 506 | except getopt.GetoptError, err: 507 | print str(err) 508 | print "Invoke with -h for help." 509 | sys.exit(2) 510 | verbose = False 511 | dryrun = False 512 | extattr = False 513 | for opt, val in opts: 514 | if opt in ("-v", "--verbose"): 515 | verbose = True 516 | elif opt in ("-h", "--help"): 517 | usage() 518 | sys.exit() 519 | elif opt in ("-n", "--dry-run"): 520 | dryrun = True 521 | elif opt == '--nochown': 522 | # Nullify the chown function defined above. 523 | global chown 524 | chown = lambda path, uid, gid: "" 525 | elif opt in ("-x", "--xattr"): 526 | extattr = True 527 | # Copying only the extended attributes means that no other 528 | # file system changes will be made in the process. 529 | dryrun = True 530 | else: 531 | assert False, "unhandled option: %s" % opt 532 | if len(args) != 2: 533 | print "Missing required arguments. Invoke with -h for help." 534 | sys.exit(2) 535 | # Check that the given source and destination exist. 536 | src = args[0] 537 | if not os.path.exists(src): 538 | print "%s does not exist!" % src 539 | sys.exit(1) 540 | if not os.path.isdir(src): 541 | print "%s is not a directory!" % src 542 | sys.exit(1) 543 | dst = args[1] 544 | if not os.path.exists(dst): 545 | print "%s does not exist!" % dst 546 | sys.exit(1) 547 | if not os.path.isdir(dst): 548 | print "%s is not a directory!" % dst 549 | sys.exit(1) 550 | try: 551 | copybackupdb(src, dst, verbose, dryrun, extattr) 552 | except KeyboardInterrupt: 553 | print "Exiting..." 554 | sys.exit(1) 555 | 556 | 557 | if __name__ == '__main__': 558 | main() 559 | --------------------------------------------------------------------------------