├── README.MD ├── README ├── CHANGELOG ├── automysqlbackup.conf ├── LICENSE ├── install.sh └── automysqlbackup /README.MD: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /README: -------------------------------------------------------------------------------- 1 | AutomysqlBackup 2 | ------------------------- 3 | .. INDEX 4 | ------------------------- 5 | Disclaimer 6 | Install 7 | Usage 8 | Configuration Options 9 | Encryption 10 | Backup rotation 11 | Restoring 12 | 13 | 14 | 15 | .. DISCLAIMER 16 | ------------------------- 17 | I take no resposibility for any data loss or corruption when using this script. 18 | This script will not help in the event of a hard drive crash. If a copy of the 19 | backup has not been stored offline or on another PC. You should copy your backups 20 | offline regularly for best protection. 21 | 22 | Happy backing up... 23 | 24 | 25 | 26 | .. INSTALL 27 | ------------------------- 28 | Extract the package into a directory. If you are reading this you have probably done 29 | this already. 30 | 31 | To install the Automysqlbackup the easy way. 32 | 1. Run the install.sh script. 33 | 2. Edit the /etc/automysqlbackup/myserver.conf file to customise your settings. 34 | 3. See usage section. 35 | 36 | To install it manually (the hard way). 37 | 1. Create the /etc/automysqlbackup directory. 38 | 2. Copy in the automysqlbackup.conf file. 39 | 3. Copy the automysqlbackup file to /usr/local/bin and make executable. 40 | 4. cp /etc/automysqlbackup/automysqlbackup.conf /etc/automysqlbackup/myserver.conf 41 | 5. Edit the /etc/automysqlbackup/myserver.conf file to customise your settings. 42 | 6. See usage section. 43 | 44 | 45 | 46 | .. USAGE 47 | ------------------------- 48 | 49 | Automysqlbackup can be run a number of ways, you can choose which is best for you. 50 | 51 | 1. Create a script as below called runmysqlbackup using the lines below: 52 | 53 | #~~~~ Copy From Below Here ~~~~ 54 | #!/bin/sh 55 | 56 | /usr/local/bin/automysqlbackup /etc/automysqlbackup/myserver.conf 57 | 58 | chown root.root /var/backup/db* -R 59 | find /var/backup/db* -type f -exec chmod 400 {} \; 60 | find /var/backup/db* -type d -exec chmod 700 {} \; 61 | 62 | #~~~~~ Copy To Above Here ~~~~ 63 | 64 | 2. Save it to a suitable location or copy it to your /etc/cron.daily folder. 65 | 66 | 3. Make it executable, i.e. chmod +x /etc/cron.daily/runmysqlbackup. 67 | 68 | 69 | The backup can be run from the command line simply by running the following command. 70 | 71 | automysqlbackup /etc/automysqlbackup/myserver.conf 72 | 73 | If you don't supply an argument for automysqlbackup, the default configuration 74 | in the program automysqlbackup will be used unless a global file 75 | 76 | CONFIG_configfile="/etc/automysqlbackup/automysqlbackup.conf" 77 | 78 | exists. 79 | 80 | You can just copy the supplied automysqlbackup.conf as many times as you want 81 | and use for separate configurations, i.e. for example different mysql servers. 82 | 83 | !!! NEW !!! 84 | ---------- 85 | As of version 3.0 we have added differential backups using the program diff. In an 86 | effort to make the reconstruction of the full archives more user friendly, we 87 | added new functionality to the script. Therefore, while preserving the old syntax, 88 | we created options for the script, so that the new functions can be accessed. 89 | 90 | Usage automysqlbackup options -cblh 91 | -c CONFIG_FILE Specify optional config file. 92 | -b Use backup method. 93 | -l List manifest entries. 94 | -h Show this help. 95 | 96 | If you use these options, you have to specify everything according to them and can't 97 | mix the old syntax with the new one. Example: 98 | 99 | before (still valid!): 100 | 101 | >> automysqlbackup "myconfig.conf" 102 | 103 | now: 104 | 105 | >> automysqlbackup -c "myconfig.conf" -b 106 | 107 | which is equivalent to 108 | 109 | >> automysqlbackup -bc "myconfig.conf" 110 | 111 | or in English: The order of the options doesn't matter, however those options expecting 112 | arguments, have to be placed right before the argument (as seen above). 113 | 114 | The option '-l' (List manifest entries) finds all Manifest files in your configuration 115 | directory (you need to specify your optional config file - otherwise a fallback will be 116 | used: global config file -> program internal default options). It then filters from which 117 | databases these are and presents you with a list (you can select more than one!) of them. 118 | Once you have chosen, you will be given a list of Manifest files, from which you choose 119 | again and after that from which you choose differential files. When you have completed 120 | all your selections, a list of selected differential files will be shown. You may then 121 | choose what you want to be done with/to those files. At the moment the options are: 122 | - create full backup out of differential one 123 | - remove the differential backup and its Manifest entry. 124 | 125 | 126 | .. CONFIGURATION OPTIONS 127 | ------------------------- 128 | 129 | !! "automysqlbackup" program contains a default configuration that should not be changed: 130 | 131 | The global config file which overwrites the default configuration is located here 132 | "/etc/automysqlbackup/automysqlbackup.conf" by default. 133 | 134 | Please take a look at the supplied "automysqlbackup.conf" for information about the configuration options. 135 | 136 | Default configuration 137 | CONFIG_configfile="/etc/automysqlbackup/automysqlbackup.conf" 138 | CONFIG_backup_dir='/var/backup/db' 139 | CONFIG_do_monthly="01" 140 | CONFIG_do_weekly="5" 141 | CONFIG_rotation_daily=6 142 | CONFIG_rotation_weekly=35 143 | CONFIG_rotation_monthly=150 144 | CONFIG_mysql_dump_usessl='yes' 145 | CONFIG_mysql_dump_username='root' 146 | CONFIG_mysql_dump_password='' 147 | CONFIG_mysql_dump_host='localhost' 148 | CONFIG_mysql_dump_socket='' 149 | CONFIG_mysql_dump_create_database='no' 150 | CONFIG_mysql_dump_use_separate_dirs='yes' 151 | CONFIG_mysql_dump_compression='gzip' 152 | CONFIG_mysql_dump_commcomp='no' 153 | CONFIG_mysql_dump_latest='no' 154 | CONFIG_mysql_dump_max_allowed_packet='' 155 | CONFIG_db_names=() 156 | CONFIG_db_month_names=() 157 | CONFIG_db_exclude=( 'information_schema' ) 158 | CONFIG_mailcontent='log' 159 | CONFIG_mail_maxattsize=4000 160 | CONFIG_mail_address='root' 161 | CONFIG_encrypt='no' 162 | CONFIG_encrypt_password='password0123' 163 | 164 | !! automysqlbackup (the shell program) accepts one parameter, the filename of a configuration file. The entries in there will supersede all others. 165 | 166 | Please take a look at the supplied "automysqlbackup.conf" for information about the configuration options. 167 | 168 | 169 | 170 | .. ENCRYPTION 171 | ------------------------- 172 | 173 | To decrypt run (replace bz2 with gz if using gzip): 174 | 175 | openssl enc -aes-256-cbc -d -in encrypted_file_name(ex: *.enc.bz2) -out outputfilename.bz2 -pass pass:PASSWORD-USED-TO-ENCRYPT 176 | 177 | 178 | 179 | .. BACKUP ROTATION 180 | ------------------------- 181 | 182 | Daily Backups are rotated weekly. 183 | Weekly Backups are run on fridays, unless otherwise specified via CONFIG_do_weekly. 184 | Weekly Backups are rotated on a 5 week cycle, unless otherwise specified via CONFIG_rotation_weekly. 185 | Monthly Backups are run on the 1st of the month, unless otherwise specified via CONFIG_do_monthly. 186 | Monthly Backups are rotated on a 5 month cycle, unless otherwise specified via CONFIG_rotation_monthly. 187 | 188 | Suggestion: It may be a good idea to copy monthly backups offline or to another server. 189 | 190 | 191 | 192 | .. RESTORING 193 | ------------------------- 194 | 195 | Firstly you will need to uncompress the backup file and decrypt it if encryption was used (see encryption section). 196 | 197 | eg. 198 | gunzip file.gz (or bunzip2 file.bz2) 199 | 200 | Next you will need to use the mysql client to restore the DB from the sql file. 201 | 202 | eg. 203 | mysql --user=username --pass=password --host=dbserver database < /path/file.sql 204 | or 205 | mysql --user=username --pass=password --host=dbserver -e "source /path/file.sql" database 206 | 207 | NOTE: Make sure you use "<" and not ">" in the above command because you are piping the file.sql to mysql and not the other way around. 208 | 209 | -------------------------------------------------------------------------------- /CHANGELOG: -------------------------------------------------------------------------------- 1 | #===================================================================== 2 | # Change Log 3 | #===================================================================== 4 | # 5 | # version 3.0_rc4 - (2011-11-24) 6 | # - Removing mkfifo commands, thereby improving portability. 7 | # - Fixing not working table exclude feature for wildcards. 8 | # version 3.0_rc3 - (2011-11-24) 9 | # - Changed code to make it more portable, thereby resolving FreeBSD issue. 10 | # version 3.0_rc2 - (2011-11-21) 11 | # - Added multicore support for bzip and gzip2. 12 | # - Fixed error in README file. 13 | # version 3.0_rc1 - (2011-11-15) 14 | # - Added differential backup method. 15 | # - Added user-friendly method to recreate full backups out of differential ones. 16 | # - Changed paramters, with which the script can be called, to make the new 17 | # methods available. 18 | # - Fixed some bugs. 19 | # version 3.0_beta2 - (2011-08-21) 20 | # - Added possibility to backup local files. 21 | # - Added full schema backup. 22 | # - Added master-data option. 23 | # - Fixed some bugs. 24 | # version 3.0_beta1 - (2011-08-15) 25 | # - REMOVED: Implementation of Variables containing full path to binaries to 26 | # avoid possibly confusion with aliases or builtins. (by Johannes Kolter) 27 | # Inside a bash script no aliases are used! This didn't make ANY sense! 28 | # Thereby resolved bug item #3074425. 29 | # - Changed some variables to be arrays, i.e. lists, and removed the ugly 30 | # sed stuff. 31 | # Fixed bug item #3169562 32 | # - Added check for weekly and monthly backups, so that they are unique per day. 33 | # bug item #3185389 34 | # - Changed SHEBANG to #!/usr/bin/env bash for portability reasons. 35 | # bug item #3292873 36 | # - Changed config file structure: read /etc/automysqlbackup/mysqlbackup.conf, 37 | # if supplied read configfile parameter (no more -c or sth. like that, just 38 | # the name of the file!), 39 | # whatever isn't set yet, set in here to default values. 40 | # - bug item #3110715: create a file in /etc/cron.d/daily and call the script 41 | # from in there, i.e. place the script in /usr/local/bin 42 | # - bug item #3082899: the PATH variables are different in ssh, you have to 43 | # supply the complete path 44 | # - Fixed bug item #3064547, suggestion accepted. 45 | # - Fixed bug item #3031023, suggestion accepted. 46 | # - Fixed bug item #3030604, resolved due to design correction. 47 | # - Fixed bug item #3025849, as long as basename is in $PATH on your system. 48 | # - Fixed bug item #3030478. 49 | # - bug item #3054633: .muttrc entry save=yes will result in saving sent files! 50 | # - Feature request item #1538588. 51 | # - Feature request item #1538138. 52 | # - Feature request item #1538142. 53 | # - Feature request item #1541843 was already included. 54 | # - Feature request item #2808012. 55 | # - Feature request item #2831465. 56 | # - Feature request item #3052484. Mysqldump already has an ssl option. 57 | # - Feature request item #3190079. I hope cleaning up everything older than 24 58 | # hours as a lower limit is good enough. 59 | # - Feature request item #3284779 was already included. See CONFIG_mysql_dump_latest. 60 | # - Feature request item #3053623. 61 | # version 2.6.0 - (2011-07-19) 62 | # - Fixed bug where files would not email correctly (Fix by Jesse Vaughan) 63 | # - Added section to encrypt .gz and .bz2 files using openssl (added by Jesse Vaughan) 64 | # version 2.5.1-01 - (2010-07-06) 65 | # - Fixed pathname bug item #3025849 (by Johannes Kolter) 66 | # version 2.5.1 - (2010-07-04) 67 | # - Added support for default and optional config file (by Johannes Kolter) 68 | # - Rotating after backup was successful whith find(1) (by Johannes Kolter) 69 | # - Implementation of Variables containing full path to binaries to 70 | # avoid possibly confusion with aliases or builtins. (by Johannes Kolter) 71 | # - Fixed bug where weekly backups were not being rotated. 72 | # Added rotation of 5 monthly backups 73 | # Now all old backups are deleted, not only the most recent one 74 | # (inspired by oleg@bintime.com) 75 | # - Use Debian special-file to access database (by Johannes Kolter) 76 | # - Fixed bug ID: 1438565 77 | # Moved IO redirection to a place before decicions are made and actions are taken. 78 | # (inspired by Derk Bernhardt) 79 | # - Fixed bug ID: #3000316 (reported by Sascha Feldhorst) 80 | # - Fixed bug ID: #1529458 (reported by Natalie ( njwood )) 81 | # - Fixed bug ID: #1548919 (reported by Piotr Kuczynski) 82 | # version 2.5 - (2006-01-15) 83 | # Added support for setting MAXIMUM_PACKET_SIZE and CONFIG_mysql_dump_socket parameters (suggested by Yvo van Doorn) 84 | # version 2.4 - (2006-01-23) 85 | # Fixed bug where weekly backups were not being rotated. (Fix by wolf02) 86 | # Added hour an min to backup filename for the case where backups are taken multiple 87 | # times in a day. NOTE This is not complete support for mutiple executions of the script 88 | # in a single day. 89 | # Added MAILCONTENT="quiet" option, see docs for details. (requested by snowsam) 90 | # Updated path statment for compatibility with OSX. 91 | # Added "CONFIG_mysql_dump_latest" to additionally store the last backup to a standard location. (request by Grant29) 92 | # version 2.3 - (2005-11-07) 93 | # Better error handling and notification of errors (a long time coming) 94 | # Compression on Backup server to MySQL server communications. 95 | # version 2.2 - (2004-12-05) 96 | # Changed from using depricated "-N" to "--skip-column-names". 97 | # Added ability to have compressed backup's emailed out. (code from Thomas Heiserowski) 98 | # Added maximum attachment size setting. 99 | # version 2.1 - (2004-11-04) 100 | # Fixed a bug in daily rotation when not using gzip compression. (Fix by Rob Rosenfeld) 101 | # version 2.0 - (2004-07-28) 102 | # Switched to using IO redirection instead of pipeing the output to the logfile. 103 | # Added choice of compression of backups being gzip of bzip2. 104 | # Switched to using functions to facilitate more functionality. 105 | # Added option of either gzip or bzip2 compression. 106 | # version 1.10 - (2004-07-17) 107 | # Another fix for spaces in the paths (fix by Thomas von Eyben) 108 | # Fixed bug when using PREBACKUP and POSTBACKUP commands containing many arguments. 109 | # version 1.9 - (2004-05-25) 110 | # Small bug fix to handle spaces in LOGFILE path which contains spaces (reported by Thomas von Eyben) 111 | # Updated docs to mention that Log email can be sent to multiple email addresses. 112 | # version 1.8 - (2004-05-01) 113 | # Added option to make backups restorable to alternate database names 114 | # meaning that a copy of the database can be created (Based on patch by Rene Hoffmann) 115 | # Seperated options into standard and advanced. 116 | # Removed " from single file dump DBMANES because it caused an error but 117 | # this means that if DB's have spaces in the name they will not dump when CONFIG_mysql_dump_use_separate_dirs=no. 118 | # Added -p option to mkdir commands to create multiple subdirs without error. 119 | # Added disk usage and location to the bottom of the backup report. 120 | # version 1.7 - (2004-04-22) 121 | # Fixed an issue where weelky backups would only work correctly if server 122 | # locale was set to English (issue reported by Tom Ingberg) 123 | # used "eval" for "rm" commands to try and resolve rotation issues. 124 | # Changed name of status log so multiple scripts can be run at the same time. 125 | # version 1.6 - (2004-03-14) 126 | # Added PREBACKUP and POSTBACKUP command functions. (patch by markpustjens) 127 | # Added support for backing up DB's with Spaces in the name. 128 | # (patch by markpustjens) 129 | # version 1.5 - (2004-02-24) 130 | # Added the ability to exclude DB's when the "all" option is used. 131 | # (Patch by kampftitan) 132 | # version 1.4 - (2004-02-02) 133 | # Project moved to Sourceforge.net 134 | # version 1.3 - (2003-09-25) 135 | # Added support for backing up "all" databases on the server without 136 | # having to list each one seperately in the configuration. 137 | # Added DB restore instructions. 138 | # version 1.2 - (2003-03-16) 139 | # Added server name to the backup log so logs from multiple servers 140 | # can be easily identified. 141 | # version 1.1 - (2003-03-13) 142 | # Small Bug fix in monthly report. (Thanks Stoyanski) 143 | # Added option to email log to any email address. (Inspired by Stoyanski) 144 | # Changed Standard file name to .sh extention. 145 | # Option are set using yes and no rather than 1 or 0. 146 | # version 1.0 - (2003-01-30) 147 | # Added the ability to have all databases backup to a single dump 148 | # file or seperate directory and file for each database. 149 | # Output is better for log keeping. 150 | # version 0.6 - (2003-01-22) 151 | # Bug fix for daily directory (Added in version 0.5) rotation. 152 | # version 0.5 - (2003-01-20) 153 | # Added "daily" directory for daily backups for neatness (suggestion by Jason) 154 | # Added CONFIG_mysql_dump_host option to allow backing up a remote server (Suggestion by Jason) 155 | # Added "--quote-names" option to mysqldump command. 156 | # Bug fix for handling the last and first of the year week rotation. 157 | # version 0.4 - (2002-11-06) 158 | # Added the abaility for the script to create its own directory structure. 159 | # version 0.3 - (2002-10-01) 160 | # Changed Naming of Weekly backups so they will show in order. 161 | # version 0.2 - (2002-09-27) 162 | # Corrected weekly rotation logic to handle weeks 0 - 10 163 | # version 0.1 - (2002-09-21) 164 | # Initial Release -------------------------------------------------------------------------------- /automysqlbackup.conf: -------------------------------------------------------------------------------- 1 | #version=3.0_rc2 2 | # DONT'T REMOVE THE PREVIOUS VERSION LINE! 3 | # 4 | # Uncomment to change the default values (shown after =) 5 | # WARNING: 6 | # This is not true for UMASK, CONFIG_prebackup and CONFIG_postbackup!!! 7 | # 8 | # Default values are stored in the script itself. Declarations in 9 | # /etc/automysqlbackup/automysqlbackup.conf will overwrite them. The 10 | # declarations in here will supersede all other. 11 | 12 | # Edit $PATH if mysql and mysqldump are not located in /usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 13 | #PATH=${PATH}:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQL:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQLDUMP 14 | 15 | # Basic Settings 16 | 17 | # Username to access the MySQL server e.g. dbuser 18 | #CONFIG_mysql_dump_username='root' 19 | 20 | # Password to access the MySQL server e.g. password 21 | #CONFIG_mysql_dump_password='' 22 | 23 | # Host name (or IP address) of MySQL server e.g localhost 24 | #CONFIG_mysql_dump_host='localhost' 25 | 26 | # "Friendly" host name of MySQL server to be used in email log 27 | # if unset or empty (default) will use CONFIG_mysql_dump_host instead 28 | #CONFIG_mysql_dump_host_friendly='' 29 | 30 | # Backup directory location e.g /backups 31 | #CONFIG_backup_dir='/var/backup/db' 32 | 33 | # This is practically a moot point, since there is a fallback to the compression 34 | # functions without multicore support in the case that the multicore versions aren't 35 | # present in the system. Of course, if you have the latter installed, but don't want 36 | # to use them, just choose no here. 37 | # pigz -> gzip 38 | # pbzip2 -> bzip2 39 | #CONFIG_multicore='yes' 40 | 41 | # Number of threads (= occupied cores) you want to use. You should - for the sake 42 | # of the stability of your system - not choose more than (#number of cores - 1). 43 | # Especially if the script is run in background by cron and the rest of your system 44 | # has already heavy load, setting this too high, might crash your system. Assuming 45 | # all systems have at least some sort of HyperThreading, the default is 2 threads. 46 | # If you wish to let pigz and pbzip2 autodetect or use their standards, set it to 47 | # 'auto'. 48 | #CONFIG_multicore_threads=2 49 | 50 | # Databases to backup 51 | 52 | # List of databases for Daily/Weekly Backup e.g. ( 'DB1' 'DB2' 'DB3' ... ) 53 | # set to (), i.e. empty, if you want to backup all databases 54 | #CONFIG_db_names=() 55 | # You can use 56 | #declare -a MDBNAMES=( "${DBNAMES[@]}" 'added entry1' 'added entry2' ... ) 57 | # INSTEAD to copy the contents of $DBNAMES and add further entries (optional). 58 | 59 | # List of databases for Monthly Backups. 60 | # set to (), i.e. empty, if you want to backup all databases 61 | #CONFIG_db_month_names=() 62 | 63 | # List of DBNAMES to EXLUCDE if DBNAMES is empty, i.e. (). 64 | #CONFIG_db_exclude=( 'information_schema' ) 65 | 66 | # List of tables to exclude, in the form db_name.table_name 67 | # You may use wildcards for the table names, i.e. 'mydb.a*' selects all tables starting with an 'a'. 68 | # However we only offer the wildcard '*', matching everything that could appear, which translates to the 69 | # '%' wildcard in mysql. 70 | #CONFIG_table_exclude=() 71 | 72 | 73 | # Advanced Settings 74 | 75 | # Rotation Settings 76 | 77 | # Which day do you want monthly backups? (01 to 31) 78 | # If the chosen day is greater than the last day of the month, it will be done 79 | # on the last day of the month. 80 | # Set to 0 to disable monthly backups. 81 | #CONFIG_do_monthly="01" 82 | 83 | # Which day do you want weekly backups? (1 to 7 where 1 is Monday) 84 | # Set to 0 to disable weekly backups. 85 | #CONFIG_do_weekly="5" 86 | 87 | # Set rotation of daily backups. VALUE*24hours 88 | # If you want to keep only today's backups, you could choose 1, i.e. everything older than 24hours will be removed. 89 | #CONFIG_rotation_daily=6 90 | 91 | # Set rotation for weekly backups. VALUE*24hours 92 | #CONFIG_rotation_weekly=35 93 | 94 | # Set rotation for monthly backups. VALUE*24hours 95 | #CONFIG_rotation_monthly=150 96 | 97 | 98 | # Server Connection Settings 99 | 100 | # Set the port for the mysql connection 101 | #CONFIG_mysql_dump_port=3306 102 | 103 | # Compress communications between backup server and MySQL server? 104 | #CONFIG_mysql_dump_commcomp='no' 105 | 106 | # Use ssl encryption with mysqldump? 107 | #CONFIG_mysql_dump_usessl='yes' 108 | 109 | # For connections to localhost. Sometimes the Unix socket file must be specified. 110 | #CONFIG_mysql_dump_socket='' 111 | 112 | # The maximum size of the buffer for client/server communication. e.g. 16MB (maximum is 1GB) 113 | #CONFIG_mysql_dump_max_allowed_packet='' 114 | 115 | # This option sends a START TRANSACTION SQL statement to the server before dumping data. It is useful only with 116 | # transactional tables such as InnoDB, because then it dumps the consistent state of the database at the time 117 | # when BEGIN was issued without blocking any applications. 118 | # 119 | # When using this option, you should keep in mind that only InnoDB tables are dumped in a consistent state. For 120 | # example, any MyISAM or MEMORY tables dumped while using this option may still change state. 121 | # 122 | # While a --single-transaction dump is in process, to ensure a valid dump file (correct table contents and 123 | # binary log coordinates), no other connection should use the following statements: ALTER TABLE, CREATE TABLE, 124 | # DROP TABLE, RENAME TABLE, TRUNCATE TABLE. A consistent read is not isolated from those statements, so use of 125 | # them on a table to be dumped can cause the SELECT that is performed by mysqldump to retrieve the table 126 | # contents to obtain incorrect contents or fail. 127 | #CONFIG_mysql_dump_single_transaction='no' 128 | 129 | # http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_master-data 130 | # --master-data[=value] 131 | # Use this option to dump a master replication server to produce a dump file that can be used to set up another 132 | # server as a slave of the master. It causes the dump output to include a CHANGE MASTER TO statement that indicates 133 | # the binary log coordinates (file name and position) of the dumped server. These are the master server coordinates 134 | # from which the slave should start replicating after you load the dump file into the slave. 135 | # 136 | # If the option value is 2, the CHANGE MASTER TO statement is written as an SQL comment, and thus is informative only; 137 | # it has no effect when the dump file is reloaded. If the option value is 1, the statement is not written as a comment 138 | # and takes effect when the dump file is reloaded. If no option value is specified, the default value is 1. 139 | # 140 | # This option requires the RELOAD privilege and the binary log must be enabled. 141 | # 142 | # The --master-data option automatically turns off --lock-tables. It also turns on --lock-all-tables, unless 143 | # --single-transaction also is specified, in which case, a global read lock is acquired only for a short time at the 144 | # beginning of the dump (see the description for --single-transaction). In all cases, any action on logs happens at 145 | # the exact moment of the dump. 146 | # ================================================================================================================== 147 | # possible values are 1 and 2, which correspond with the values from mysqldump 148 | # VARIABLE= , i.e. no value, turns it off (default) 149 | # 150 | #CONFIG_mysql_dump_master_data= 151 | 152 | # Included stored routines (procedures and functions) for the dumped databases in the output. Use of this option 153 | # requires the SELECT privilege for the mysql.proc table. The output generated by using --routines contains 154 | # CREATE PROCEDURE and CREATE FUNCTION statements to re-create the routines. However, these statements do not 155 | # include attributes such as the routine creation and modification timestamps. This means that when the routines 156 | # are reloaded, they will be created with the timestamps equal to the reload time. 157 | # 158 | # If you require routines to be re-created with their original timestamp attributes, do not use --routines. Instead, 159 | # dump and reload the contents of the mysql.proc table directly, using a MySQL account that has appropriate privileges 160 | # for the mysql database. 161 | # 162 | # This option was added in MySQL 5.0.13. Before that, stored routines are not dumped. Routine DEFINER values are not 163 | # dumped until MySQL 5.0.20. This means that before 5.0.20, when routines are reloaded, they will be created with the 164 | # definer set to the reloading user. If you require routines to be re-created with their original definer, dump and 165 | # load the contents of the mysql.proc table directly as described earlier. 166 | # 167 | #CONFIG_mysql_dump_full_schema='yes' 168 | 169 | # Backup status of table(s) in textfile. This is very helpful when restoring backups, since it gives an idea, what changed 170 | # in the meantime. 171 | #CONFIG_mysql_dump_dbstatus='yes' 172 | 173 | # Backup dump settings 174 | 175 | # Include CREATE DATABASE in backup? 176 | #CONFIG_mysql_dump_create_database='no' 177 | 178 | # Separate backup directory and file for each DB? (yes or no) 179 | #CONFIG_mysql_dump_use_separate_dirs='yes' 180 | 181 | # Choose Compression type. (gzip or bzip2) 182 | #CONFIG_mysql_dump_compression='gzip' 183 | 184 | # Store an additional copy of the latest backup to a standard 185 | # location so it can be downloaded by third party scripts. 186 | #CONFIG_mysql_dump_latest='no' 187 | 188 | # Remove all date and time information from the filenames in the latest folder. 189 | # Runs, if activated, once after the backups are completed. Practically it just finds all files in the latest folder 190 | # and removes the date and time information from the filenames (if present). 191 | #CONFIG_mysql_dump_latest_clean_filenames='no' 192 | 193 | # Create differential backups. Master backups are created weekly at #$CONFIG_do_weekly weekday. Between master backups, 194 | # diff is used to create differential backups relative to the latest master backup. In the Manifest file, you find the 195 | # following structure 196 | # $filename md5sum $md5sum diff_id $diff_id rel_id $rel_id 197 | # where each field is separated by the tabular character '\t'. The entries with $ at the beginning mean the actual values, 198 | # while the others are just for readability. The diff_id is the id of the differential or master backup which is also in 199 | # the filename after the last _ and before the suffixes begin, i.e. .diff, .sql and extensions. It is used to relate 200 | # differential backups to master backups. The master backups have 0 as $rel_id and are thereby identifiable. Differential 201 | # backups have the id of the corresponding master backup as $rel_id. 202 | # 203 | # To ensure that master backups are kept long enough, the value of $CONFIG_rotation_daily is set to a minimum of 21 days. 204 | # 205 | #CONFIG_mysql_dump_differential='no' 206 | 207 | 208 | # Notification setup 209 | 210 | # What would you like to be mailed to you? 211 | # - log : send only log file 212 | # - files : send log file and sql files as attachments (see docs) 213 | # - stdout : will simply output the log to the screen if run manually. 214 | # - quiet : Only send logs if an error occurs to the MAILADDR. 215 | #CONFIG_mailcontent='stdout' 216 | 217 | # Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs]) 218 | #CONFIG_mail_maxattsize=4000 219 | 220 | # Allow packing of files with tar and splitting it in pieces of CONFIG_mail_maxattsize. 221 | #CONFIG_mail_splitandtar='yes' 222 | 223 | # Use uuencode instead of mutt. WARNING: Not all email clients work well with uuencoded attachments. 224 | #CONFIG_mail_use_uuencoded_attachments='no' 225 | 226 | # Email Address to send mail to? (user@domain.com) 227 | #CONFIG_mail_address='root' 228 | 229 | 230 | # Encryption 231 | 232 | # Do you wish to encrypt your backups using openssl? 233 | #CONFIG_encrypt='no' 234 | 235 | # Choose a password to encrypt the backups. 236 | #CONFIG_encrypt_password='password0123' 237 | 238 | # Other 239 | 240 | # Backup local files, i.e. maybe you would like to backup your my.cnf (mysql server configuration), etc. 241 | # These files will be tar'ed, depending on your compression option CONFIG_mysql_dump_compression compressed and 242 | # depending on the option CONFIG_encrypt encrypted. 243 | # 244 | # Note: This could also have been accomplished with CONFIG_prebackup or CONFIG_postbackup. 245 | #CONFIG_backup_local_files=() 246 | 247 | # Command to run before backups (uncomment to use) 248 | #CONFIG_prebackup="/etc/mysql-backup-pre" 249 | 250 | # Command run after backups (uncomment to use) 251 | #CONFIG_postbackup="/etc/mysql-backup-post" 252 | 253 | # Uncomment to activate! This will give folders rwx------ 254 | # and files rw------- permissions. 255 | #umask 0077 256 | 257 | # dry-run, i.e. show what you are gonna do without actually doing it 258 | # inactive: =0 or commented out 259 | # active: uncommented AND =1 260 | #CONFIG_dryrun=1 261 | 262 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | 4 | # return true, if variable is set; else false 5 | isSet() { 6 | if [[ ! ${!1} && ${!1-_} ]]; then return 1; else return 0; fi 7 | } 8 | 9 | activateIO() { 10 | touch "$1" 11 | exec 6>&1 12 | exec > "$1" 13 | } 14 | removeIO() { 15 | exec 1>&6 6>&- 16 | } 17 | 18 | upgrade_config_file () { 19 | ( # execute in subshell, so that sourced variables are only available inside () 20 | source "$1" 21 | #declare -p 22 | local temp 23 | temp=$(mktemp /tmp/tmp.XXXXXX) 24 | (( $? != 0 )) && return 1 25 | activateIO "$temp" 26 | echo "#version=3.0_rc2" 27 | echo "# Uncomment to change the default values (shown after =)" 28 | echo "# WARNING:" 29 | echo "# This is not true for UMASK, CONFIG_prebackup and CONFIG_postbackup!!!" 30 | echo "#" 31 | echo "# Default values are stored in the script itself. Declarations in" 32 | echo "# /etc/automysqlbackup/automysqlbackup.conf will overwrite them. The" 33 | echo "# declarations in here will supersede all other." 34 | echo "" 35 | echo "# Edit \$PATH if mysql and mysqldump are not located in /usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin" 36 | echo "#PATH=\${PATH}:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQL:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQLDUMP" 37 | echo "" 38 | echo "# Basic Settings" 39 | echo "" 40 | echo "# Username to access the MySQL server e.g. dbuser" 41 | if isSet USERNAME; then 42 | printf '%s=%q\n' CONFIG_mysql_dump_username "${USERNAME-}" 43 | else 44 | echo "#CONFIG_mysql_dump_username='root'" 45 | fi 46 | echo "" 47 | echo "# Password to access the MySQL server e.g. password" 48 | if isSet PASSWORD; then 49 | printf '%s=%q\n' CONFIG_mysql_dump_password "${PASSWORD-}" 50 | else 51 | echo "#CONFIG_mysql_dump_password=''" 52 | fi 53 | echo "" 54 | echo "# Host name (or IP address) of MySQL server e.g localhost" 55 | if isSet DBHOST; then 56 | printf '%s=%q\n' CONFIG_mysql_dump_host "${DBHOST-}" 57 | else 58 | echo "#CONFIG_mysql_dump_host='localhost'" 59 | fi 60 | echo "" 61 | echo "# \"Friendly\" host name of MySQL server to be used in email log" 62 | echo "# if unset or empty (default) will use CONFIG_mysql_dump_host instead" 63 | if isSet CONFIG_mysql_dump_host_friendly; then 64 | printf '%s=%q\n' CONFIG_mysql_dump_host_friendly "${CONFIG_mysql_dump_host_friendly-}" 65 | else 66 | echo "#CONFIG_mysql_dump_host_friendly=''" 67 | fi 68 | echo "" 69 | echo "# Backup directory location e.g /backups" 70 | if isSet BACKUPDIR; then 71 | printf '%s=%q\n' CONFIG_backup_dir "${BACKUPDIR-}" 72 | else 73 | echo "#CONFIG_backup_dir='/var/backup/db'" 74 | fi 75 | echo "" 76 | echo "# This is practically a moot point, since there is a fallback to the compression" 77 | echo "# functions without multicore support in the case that the multicore versions aren't" 78 | echo "# present in the system. Of course, if you have the latter installed, but don't want" 79 | echo "# to use them, just choose no here." 80 | echo "# pigz -> gzip" 81 | echo "# pbzip2 -> bzip2" 82 | echo "#CONFIG_multicore='yes'" 83 | echo "" 84 | echo "# Number of threads (= occupied cores) you want to use. You should - for the sake" 85 | echo "# of the stability of your system - not choose more than (#number of cores - 1)." 86 | echo "# Especially if the script is run in background by cron and the rest of your system" 87 | echo "# has already heavy load, setting this too high, might crash your system. Assuming" 88 | echo "# all systems have at least some sort of HyperThreading, the default is 2 threads." 89 | echo "# If you wish to let pigz and pbzip2 autodetect or use their standards, set it to" 90 | echo "# 'auto'." 91 | echo "#CONFIG_multicore_threads=2" 92 | echo "" 93 | echo "# Databases to backup" 94 | echo "" 95 | echo "# List of databases for Daily/Weekly Backup e.g. ( 'DB1' 'DB2' 'DB3' ... )" 96 | echo "# set to (), i.e. empty, if you want to backup all databases" 97 | if isSet DBNAMES; then 98 | if [[ "x$DBNAMES" = "xall" ]]; then 99 | echo "#CONFIG_db_names=()" 100 | else 101 | declare -a CONFIG_db_names 102 | for i in $DBNAMES; do 103 | CONFIG_db_names=( "${CONFIG_db_names[@]}" "$i" ) 104 | done 105 | declare -p CONFIG_db_names | sed -e 's/\[[^]]*]=//g' 106 | fi 107 | else 108 | echo "#CONFIG_db_names=()" 109 | fi 110 | echo "# You can use" 111 | echo "#declare -a MDBNAMES=( \"\${DBNAMES[@]}\" 'added entry1' 'added entry2' ... )" 112 | echo "# INSTEAD to copy the contents of \$DBNAMES and add further entries (optional)." 113 | echo "" 114 | echo "# List of databases for Monthly Backups." 115 | echo "# set to (), i.e. empty, if you want to backup all databases" 116 | if isSet MDBNAMES; then 117 | if [[ "x$MDBNAMES" = "xall" ]]; then 118 | echo "#CONFIG_db_month_names=()" 119 | else 120 | declare -a CONFIG_db_month_names 121 | for i in $MDBNAMES; do 122 | CONFIG_db_month_names=( "${CONFIG_db_month_names[@]}" "$i" ) 123 | done 124 | declare -p CONFIG_db_month_names | sed -e 's/\[[^]]*]=//g' 125 | fi 126 | else 127 | echo "#CONFIG_db_month_names=()" 128 | fi 129 | echo "" 130 | echo "# List of DBNAMES to EXLUCDE if DBNAMES is empty, i.e. ()." 131 | if isSet DBEXCLUDE; then 132 | declare -a CONFIG_db_exclude 133 | for i in $DBEXCLUDE; do 134 | CONFIG_db_exclude=( "${CONFIG_db_exclude[@]}" "$i" ) 135 | done 136 | declare -p CONFIG_db_exclude | sed -e 's/\[[^]]*]=//g' 137 | else 138 | echo "#CONFIG_db_exclude=( 'information_schema' )" 139 | fi 140 | echo "" 141 | echo "# List of tables to exclude, in the form db_name.table_name" 142 | echo "#CONFIG_table_exclude=()" 143 | echo "" 144 | echo "" 145 | echo "# Advanced Settings" 146 | echo "" 147 | echo "# Rotation Settings" 148 | echo "" 149 | echo "# Which day do you want monthly backups? (01 to 31)" 150 | echo "# If the chosen day is greater than the last day of the month, it will be done" 151 | echo "# on the last day of the month." 152 | echo "# Set to 0 to disable monthly backups." 153 | echo "#CONFIG_do_monthly=\"01\"" 154 | echo "" 155 | echo "# Which day do you want weekly backups? (1 to 7 where 1 is Monday)" 156 | echo "# Set to 0 to disable weekly backups." 157 | if isSet DOWEEKLY; then 158 | printf '%s=%q\n' CONFIG_do_weekly "${DOWEEKLY-}" 159 | else 160 | echo "#CONFIG_do_weekly=\"5\"" 161 | fi 162 | echo "" 163 | echo "# Set rotation of daily backups. VALUE*24hours" 164 | echo "# If you want to keep only today's backups, you could choose 1, i.e. everything older than 24hours will be removed." 165 | echo "#CONFIG_rotation_daily=6" 166 | echo "" 167 | echo "# Set rotation for weekly backups. VALUE*24hours" 168 | echo "#CONFIG_rotation_weekly=35" 169 | echo "" 170 | echo "# Set rotation for monthly backups. VALUE*24hours" 171 | echo "#CONFIG_rotation_monthly=150" 172 | echo "" 173 | echo "" 174 | echo "# Server Connection Settings" 175 | echo "" 176 | echo "# Set the port for the mysql connection" 177 | echo "#CONFIG_mysql_dump_port=3306" 178 | echo "" 179 | echo "# Compress communications between backup server and MySQL server?" 180 | if isSet COMMCOMP; then 181 | printf '%s=%q\n' CONFIG_mysql_dump_commcomp "${COMMCOMP-}" 182 | else 183 | echo "#CONFIG_mysql_dump_commcomp='no'" 184 | fi 185 | echo "" 186 | echo "# Use ssl encryption with mysqldump?" 187 | echo "#CONFIG_mysql_dump_usessl='yes'" 188 | echo "" 189 | echo "# For connections to localhost. Sometimes the Unix socket file must be specified." 190 | if isSet SOCKET; then 191 | printf '%s=%q\n' CONFIG_mysql_dump_socket "${SOCKET-}" 192 | else 193 | echo "#CONFIG_mysql_dump_socket=''" 194 | fi 195 | echo "" 196 | echo "# The maximum size of the buffer for client/server communication. e.g. 16MB (maximum is 1GB)" 197 | if isSet MAX_ALLOWED_PACKET; then 198 | printf '%s=%q\n' CONFIG_mysql_dump_max_allowed_packet "${MAX_ALLOWED_PACKET-}" 199 | else 200 | echo "#CONFIG_mysql_dump_max_allowed_packet=''" 201 | fi 202 | echo "" 203 | echo "# This option sends a START TRANSACTION SQL statement to the server before dumping data. It is useful only with" 204 | echo "# transactional tables such as InnoDB, because then it dumps the consistent state of the database at the time" 205 | echo "# when BEGIN was issued without blocking any applications." 206 | echo "#" 207 | echo "# When using this option, you should keep in mind that only InnoDB tables are dumped in a consistent state. For" 208 | echo "# example, any MyISAM or MEMORY tables dumped while using this option may still change state." 209 | echo "#" 210 | echo "# While a --single-transaction dump is in process, to ensure a valid dump file (correct table contents and" 211 | echo "# binary log coordinates), no other connection should use the following statements: ALTER TABLE, CREATE TABLE," 212 | echo "# DROP TABLE, RENAME TABLE, TRUNCATE TABLE. A consistent read is not isolated from those statements, so use of" 213 | echo "# them on a table to be dumped can cause the SELECT that is performed by mysqldump to retrieve the table" 214 | echo "# contents to obtain incorrect contents or fail." 215 | echo "#CONFIG_mysql_dump_single_transaction='no'" 216 | echo "" 217 | echo "# http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_master-data" 218 | echo "# --master-data[=value] " 219 | echo "# Use this option to dump a master replication server to produce a dump file that can be used to set up another" 220 | echo "# server as a slave of the master. It causes the dump output to include a CHANGE MASTER TO statement that indicates" 221 | echo "# the binary log coordinates (file name and position) of the dumped server. These are the master server coordinates" 222 | echo "# from which the slave should start replicating after you load the dump file into the slave." 223 | echo "#" 224 | echo "# If the option value is 2, the CHANGE MASTER TO statement is written as an SQL comment, and thus is informative only;" 225 | echo "# it has no effect when the dump file is reloaded. If the option value is 1, the statement is not written as a comment" 226 | echo "# and takes effect when the dump file is reloaded. If no option value is specified, the default value is 1." 227 | echo "#" 228 | echo "# This option requires the RELOAD privilege and the binary log must be enabled. " 229 | echo "#" 230 | echo "# The --master-data option automatically turns off --lock-tables. It also turns on --lock-all-tables, unless" 231 | echo "# --single-transaction also is specified, in which case, a global read lock is acquired only for a short time at the" 232 | echo "# beginning of the dump (see the description for --single-transaction). In all cases, any action on logs happens at" 233 | echo "# the exact moment of the dump." 234 | echo "# ==================================================================================================================" 235 | echo "# possible values are 1 and 2, which correspond with the values from mysqldump" 236 | echo "# VARIABLE= , i.e. no value, turns it off (default)" 237 | echo "#" 238 | echo "#CONFIG_mysql_dump_master_data=" 239 | echo "" 240 | echo "# Included stored routines (procedures and functions) for the dumped databases in the output. Use of this option" 241 | echo "# requires the SELECT privilege for the mysql.proc table. The output generated by using --routines contains" 242 | echo "# CREATE PROCEDURE and CREATE FUNCTION statements to re-create the routines. However, these statements do not" 243 | echo "# include attributes such as the routine creation and modification timestamps. This means that when the routines" 244 | echo "# are reloaded, they will be created with the timestamps equal to the reload time." 245 | echo "#" 246 | echo "# If you require routines to be re-created with their original timestamp attributes, do not use --routines. Instead," 247 | echo "# dump and reload the contents of the mysql.proc table directly, using a MySQL account that has appropriate privileges" 248 | echo "# for the mysql database. " 249 | echo "#" 250 | echo "# This option was added in MySQL 5.0.13. Before that, stored routines are not dumped. Routine DEFINER values are not" 251 | echo "# dumped until MySQL 5.0.20. This means that before 5.0.20, when routines are reloaded, they will be created with the" 252 | echo "# definer set to the reloading user. If you require routines to be re-created with their original definer, dump and" 253 | echo "# load the contents of the mysql.proc table directly as described earlier." 254 | echo "#" 255 | echo "#CONFIG_mysql_dump_full_schema='yes'" 256 | echo "" 257 | echo "# Backup dump settings" 258 | echo "" 259 | echo "# Include CREATE DATABASE in backup?" 260 | if isSet CREATE_DATABASE; then 261 | printf '%s=%q\n' CONFIG_mysql_dump_create_database "${CREATE_DATABASE-}" 262 | else 263 | echo "#CONFIG_mysql_dump_create_database='no'" 264 | fi 265 | echo "" 266 | echo "# Separate backup directory and file for each DB? (yes or no)" 267 | if isSet SEPDIR; then 268 | printf '%s=%q\n' CONFIG_mysql_dump_use_separate_dirs "${SEPDIR-}" 269 | else 270 | echo "#CONFIG_mysql_dump_use_separate_dirs='yes'" 271 | fi 272 | echo "" 273 | echo "# Choose Compression type. (gzip or bzip2)" 274 | if isSet COMP; then 275 | printf '%s=%q\n' CONFIG_mysql_dump_compression "${COMP-}" 276 | else 277 | echo "#CONFIG_mysql_dump_compression='gzip'" 278 | fi 279 | echo "" 280 | echo "# Store an additional copy of the latest backup to a standard" 281 | echo "# location so it can be downloaded by third party scripts." 282 | if isSet LATEST; then 283 | printf '%s=%q\n' CONFIG_mysql_dump_latest "${LATEST-}" 284 | else 285 | echo "#CONFIG_mysql_dump_latest='no'" 286 | fi 287 | echo "" 288 | echo "# Remove all date and time information from the filenames in the latest folder." 289 | echo "# Runs, if activated, once after the backups are completed. Practically it just finds all files in the latest folder" 290 | echo "# and removes the date and time information from the filenames (if present)." 291 | echo "#CONFIG_mysql_dump_latest_clean_filenames='no'" 292 | echo "" 293 | echo "# Notification setup" 294 | echo "" 295 | echo "# What would you like to be mailed to you?" 296 | echo "# - log : send only log file" 297 | echo "# - files : send log file and sql files as attachments (see docs)" 298 | echo "# - stdout : will simply output the log to the screen if run manually." 299 | echo "# - quiet : Only send logs if an error occurs to the MAILADDR." 300 | if isSet MAILCONTENT; then 301 | printf '%s=%q\n' CONFIG_mailcontent "${MAILCONTENT-}" 302 | else 303 | echo "#CONFIG_mailcontent='stdout'" 304 | fi 305 | echo "" 306 | echo "# Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs])" 307 | if isSet MAXATTSIZE; then 308 | printf '%s=%q\n' CONFIG_mail_maxattsize "${MAXATTSIZE-}" 309 | else 310 | echo "#CONFIG_mail_maxattsize=4000" 311 | fi 312 | echo "" 313 | echo "# Email Address to send mail to? (user@domain.com)" 314 | if isSet MAILADDR; then 315 | printf '%s=%q\n' CONFIG_mail_address "${MAILADDR-}" 316 | else 317 | echo "#CONFIG_mail_address='root'" 318 | fi 319 | echo "" 320 | echo '# Create differential backups. Master backups are created weekly at #$CONFIG_do_weekly weekday. Between master backups,' 321 | echo "# diff is used to create differential backups relative to the latest master backup. In the Manifest file, you find the" 322 | echo "# following structure" 323 | echo '# $filename md5sum $md5sum diff_id $diff_id rel_id $rel_id' 324 | echo "# where each field is separated by the tabular character '\t'. The entries with $ at the beginning mean the actual values," 325 | echo "# while the others are just for readability. The diff_id is the id of the differential or master backup which is also in" 326 | echo "# the filename after the last _ and before the suffixes begin, i.e. .diff, .sql and extensions. It is used to relate" 327 | echo '# differential backups to master backups. The master backups have 0 as $rel_id and are thereby identifiable. Differential' 328 | echo '# backups have the id of the corresponding master backup as $rel_id.' 329 | echo "#" 330 | echo '# To ensure that master backups are kept long enough, the value of $CONFIG_rotation_daily is set to a minimum of 21 days.' 331 | echo "#" 332 | echo "#CONFIG_mysql_dump_differential='no'" 333 | echo "" 334 | echo "# Encryption" 335 | echo "" 336 | echo "# Do you wish to encrypt your backups using openssl?" 337 | echo "#CONFIG_encrypt='no'" 338 | echo "" 339 | echo "# Choose a password to encrypt the backups." 340 | echo "#CONFIG_encrypt_password='password0123'" 341 | echo "" 342 | echo "# Other" 343 | echo "" 344 | echo "# Backup local files, i.e. maybe you would like to backup your my.cnf (mysql server configuration), etc." 345 | echo "# These files will be tar'ed, depending on your compression option CONFIG_mysql_dump_compression compressed and" 346 | echo "# depending on the option CONFIG_encrypt encrypted." 347 | echo "#" 348 | echo "# Note: This could also have been accomplished with CONFIG_prebackup or CONFIG_postbackup." 349 | echo "#CONFIG_backup_local_files=()" 350 | echo "" 351 | echo "# Command to run before backups (uncomment to use)" 352 | if isSet PREBACKUP; then 353 | printf '%s=%q\n' CONFIG_prebackup "${PREBACKUP-}" 354 | else 355 | echo "#CONFIG_prebackup=\"/etc/mysql-backup-pre\"" 356 | fi 357 | echo "" 358 | echo "# Command run after backups (uncomment to use)" 359 | if isSet POSTBACKUP; then 360 | printf '%s=%q\n' CONFIG_postbackup "${POSTBACKUP-}" 361 | else 362 | echo "#CONFIG_postbackup=\"/etc/mysql-backup-post\"" 363 | fi 364 | echo "" 365 | echo "# Uncomment to activate! This will give folders rwx------" 366 | echo "# and files rw------- permissions." 367 | echo "#umask 0077" 368 | echo "" 369 | echo "# dry-run, i.e. show what you are gonna do without actually doing it" 370 | echo "# inactive: =0 or commented out" 371 | echo "# active: uncommented AND =1" 372 | echo "#CONFIG_dryrun=1" 373 | removeIO 374 | mv "$temp" "${1}_converted" 375 | return 0 376 | ) 377 | } 378 | 379 | 380 | parse_config_file () { 381 | printf 'Found config file %s. ' "$1" 382 | if head -n1 "$1" | egrep -o 'version=.*' >& /dev/null; then 383 | version=`head -n1 "$1" | egrep -o 'version=.*' | awk -F"=" '{print $2}'` 384 | if [[ "$version" =~ 3.* ]]; then 385 | printf 'Version 3.* determined. No conversion necessary.\n' 386 | else 387 | printf 'Unknown version. Can not convert it. You have to convert it manually.\n' 388 | fi 389 | else 390 | printf 'No version information on first line of config file. Assuming the version is <3.\n' 391 | while true; do 392 | read -p "Convert? [Y/n] " yn 393 | [[ "x$yn" = "x" ]] && { upgrade_config_file "$1" || echo "Failed to convert."; break; } 394 | case $yn in 395 | [Yy]* ) upgrade_config_file "$1" || echo "Failed to convert."; break;; 396 | [Nn]* ) break;; 397 | * ) echo "Please answer yes or no.";; 398 | esac 399 | done 400 | fi 401 | } 402 | 403 | #precheck 404 | echo "### Checking archive files for existence, readability and integrity." 405 | echo 406 | 407 | precheck_files=( automysqlbackup 447c33d2546181d07d0c0d69d76b189b 408 | automysqlbackup.conf d525efa3da15ce9fea96893e5a8ce6d5 409 | README b17740fcd3a5f8579b907a42249a83cd 410 | LICENSE 39bba7d2cf0ba1036f2a6e2be52fe3f0 411 | ) 412 | 413 | n=$(( ${#precheck_files[@]}/2 )) 414 | i=0 415 | while [ $i -lt $n ]; do 416 | printf "${precheck_files[$((2*$i))]} ... " 417 | if [ -r "${precheck_files[$((2*$i))]}" ]; then 418 | printf "exists and is readable ... " 419 | else 420 | printf "failed\n" 421 | exit 1 422 | fi 423 | if echo "${precheck_files[$((2*$i+1))]} ${precheck_files[$((2*$i))]}" | md5sum --check >/dev/null 2>&1; then 424 | printf "md5sum okay :)\n" 425 | else 426 | printf "md5sum failed :(\n" 427 | exit 1 428 | fi 429 | let i+=1 430 | done 431 | 432 | echo 433 | printf 'Select the global configuration directory [/etc/automysqlbackup]: ' 434 | read configdir 435 | configdir="${configdir%/}" # strip trailing slash if there 436 | [[ "x$configdir" = "x" ]] && configdir='/etc/automysqlbackup' 437 | printf 'Select directory for the executable [/usr/local/bin]: ' 438 | read bindir 439 | bindir="${bindir%/}" # strip trailing slash if there 440 | [[ "x$bindir" = "x" ]] && bindir='/usr/local/bin' 441 | 442 | #create global config directory 443 | echo "### Creating global configuration directory ${configdir}:" 444 | echo 445 | if [ -d "${configdir}" ]; then 446 | echo "exists already ... searching for config files:" 447 | for i in "${configdir}"/*.conf; do 448 | [[ "x$(basename $i)" = "xautomysqlbackup.conf" ]] && continue 449 | parse_config_file "$i" 450 | done 451 | else 452 | if mkdir "${configdir}" >/dev/null 2>&1; then 453 | #testing for permissions 454 | if [ -r "${configdir}" -a -x "${configdir}" ]; then 455 | printf "success\n" 456 | else 457 | printf "directory successfully created but has wrong permissions, trying to correct ... " 458 | if chmod +rx "${configdir}" >/dev/null 2>&1; then 459 | printf "corrected\n" 460 | else 461 | printf "failed. Aborting. Make sure you run the script with appropriate permissions.\n" 462 | fi 463 | fi 464 | else 465 | printf "failed ... check permissions.\n" 466 | fi 467 | fi 468 | 469 | echo 470 | #copying files 471 | echo "### Copying files." 472 | echo 473 | cp -i automysqlbackup.conf LICENSE README "${configdir}"/ 474 | cp -i automysqlbackup.conf "${configdir}"/myserver.conf 475 | cp -i automysqlbackup "${bindir}"/ 476 | [[ -f "${bindir}"/automysqlbackup ]] && [[ -x "${bindir}"/automysqlbackup ]] || chmod +x "${bindir}"/automysqlbackup || echo " failed - make sure you make the program executable, i.e. run 'chmod +x ${bindir}/automysqlbackup'" 477 | echo 478 | 479 | if echo $PATH | grep "${bindir}" >/dev/null 2>&1; then 480 | printf "if you are running automysqlbackup under the same user as you run this install script,\nyou should be able to access it by running 'automysqlbackup' from the command line.\n" 481 | printf "if not, you have to check if 'echo \$PATH' has ${bindir} in it\n" 482 | printf "\nSetup Complete!\n" 483 | else 484 | printf "if running under the current user, you have to use the full path ${bindir}/automysqlbackup since /usr/local/bin is not in 'echo \$PATH'\n" 485 | printf "\nSetup Complete!\n" 486 | fi -------------------------------------------------------------------------------- /automysqlbackup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | # This program is free software; you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation; either version 2 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program; if not, write to the Free Software 15 | # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 16 | # 17 | shopt -s extglob 18 | 19 | # BEGIN _flags 20 | let "filename_flag_encrypted=0x01" 21 | let "filename_flag_gz=0x02" 22 | let "filename_flag_bz2=0x04" 23 | let "filename_flag_diff=0x08" 24 | # END _flags 25 | 26 | # BEGIN _errors_notifications 27 | let "E=0x00" # no errors 28 | let "N=0x00" # no notifications 29 | 30 | let "E_dbdump_failed=0x01" 31 | let "E_backup_local_failed=0x02" 32 | let "E_mkdir_basedir_failed=0x04" 33 | let "E_mkdir_subdirs_failed=0x08" 34 | let "E_perm_basedir=0x10" 35 | let "E_enc_cleartext_delfailed=0x20" 36 | let "E_enc_failed=0x40" 37 | let "E_db_empty=0x80" 38 | let "E_create_pipe_failed=0x100" 39 | let "E_missing_deps=0x200" 40 | let "E_no_basedir=0x400" 41 | let "E_config_backupdir_not_writable=0x800" 42 | let "E_dump_status_failed=0x1000" 43 | let "E_dump_fullschema_failed=0x2000" 44 | 45 | let "N_config_file_missing=0x01" 46 | let "N_arg_conffile_parsed=0x02" 47 | let "N_arg_conffile_unreadable=0x04" 48 | let "N_too_many_args=0x08" 49 | let "N_latest_cleanup_failed=0x10" 50 | let "N_backup_local_nofiles=0x20" 51 | # END _errors_notifications 52 | 53 | # BEGIN _functions 54 | 55 | # @info: Default configuration options. 56 | # @deps: (none) 57 | load_default_config() { 58 | CONFIG_configfile="/etc/automysqlbackup/automysqlbackup.conf" 59 | CONFIG_backup_dir='/var/backup/db' 60 | CONFIG_multicore='yes' 61 | CONFIG_multicore_threads=2 62 | CONFIG_do_monthly="01" 63 | CONFIG_do_weekly="5" 64 | CONFIG_rotation_daily=6 65 | CONFIG_rotation_weekly=35 66 | CONFIG_rotation_monthly=150 67 | CONFIG_mysql_dump_port=3306 68 | CONFIG_mysql_dump_usessl='yes' 69 | CONFIG_mysql_dump_username='root' 70 | CONFIG_mysql_dump_password='' 71 | CONFIG_mysql_dump_host='localhost' 72 | CONFIG_mysql_dump_host_friendly='' 73 | CONFIG_mysql_dump_socket='' 74 | CONFIG_mysql_dump_create_database='no' 75 | CONFIG_mysql_dump_use_separate_dirs='yes' 76 | CONFIG_mysql_dump_compression='gzip' 77 | CONFIG_mysql_dump_commcomp='no' 78 | CONFIG_mysql_dump_latest='no' 79 | CONFIG_mysql_dump_latest_clean_filenames='no' 80 | CONFIG_mysql_dump_max_allowed_packet='' 81 | CONFIG_mysql_dump_single_transaction='no' 82 | CONFIG_mysql_dump_master_data= 83 | CONFIG_mysql_dump_full_schema='yes' 84 | CONFIG_mysql_dump_dbstatus='yes' 85 | CONFIG_mysql_dump_differential='no' 86 | CONFIG_backup_local_files=() 87 | CONFIG_db_names=() 88 | CONFIG_db_month_names=() 89 | CONFIG_db_exclude=( 'information_schema' ) 90 | CONFIG_table_exclude=() 91 | CONFIG_mailcontent='stdout' 92 | CONFIG_mail_maxattsize=4000 93 | CONFIG_mail_splitandtar='yes' 94 | CONFIG_mail_use_uuencoded_attachments='no' 95 | CONFIG_mail_address='root' 96 | CONFIG_encrypt='no' 97 | CONFIG_encrypt_password='password0123' 98 | } 99 | 100 | # @return: true, if variable is set; else false 101 | isSet() { 102 | if [[ ! ${!1} && ${!1-_} ]]; then return 1; else return 0; fi 103 | } 104 | 105 | # @return: true, if variable is empty; else false 106 | isEmpty() { 107 | if [[ ${!1} ]]; then return 1; else return 0; fi 108 | } 109 | 110 | # @info: Called when one of the signals EXIT, SIGHUP, SIGINT, SIGQUIT or SIGTERM is emitted. 111 | # It removes the IO redirection, mails any log file information and cleans up any temporary files. 112 | # @args: (none) 113 | # @return: (none) 114 | mail_cleanup () { 115 | removeIO 116 | # if the variables $log_file and $log_errfile aren't set or are empty and both associated files don't exist, skip output methods. 117 | # this might happen if 'exit' occurs before they are set. 118 | if [[ ! -e "$log_file" && ! -e "$log_errfile" ]];then 119 | echo "Skipping normal output methods, since the program exited before any log files could be created." 120 | else 121 | case "${CONFIG_mailcontent}" in 122 | 'files') 123 | # Include error log if larger than zero. 124 | if [[ -s "$log_errfile" ]]; then 125 | backupfiles=( "${backupfiles[@]}" "$log_errfile" ) 126 | errornote="WARNING: Error Reported - " 127 | fi 128 | temp="$(mktemp "$CONFIG_backup_dir"/tmp/mail_content.XXXXXX)" 129 | # Get backup size 130 | attsize=`du -c "${backupfiles[@]}" | awk 'END {print $1}'` 131 | if (( ${CONFIG_mail_maxattsize} >= ${attsize} )); then 132 | if [[ "x$CONFIG_mail_use_uuencoded_attachments" = "xyes" ]]; then 133 | cat "$log_file" > "$temp" 134 | for j in "${backupfiles[@]}"; do 135 | uuencode "$j" "$j" >> "$temp" 136 | done 137 | mail -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} < "$temp" 138 | else 139 | mutt -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" -a "${backupfiles[@]}" -- ${CONFIG_mail_address} < "$log_file" 140 | fi 141 | elif (( ${CONFIG_mail_maxattsize} <= ${attsize} )) && [[ "x$CONFIG_mail_splitandtar" = "xyes" ]]; then 142 | if sPWD="$PWD"; cd "$CONFIG_backup_dir"/tmp && pax -wv "${backupfiles[@]}" | bzip2_compression | split -b $((CONFIG_mail_maxattsize*1000)) - mail_attachment_${datetimestamp}_ && cd "$sPWD"; then 143 | files=("$CONFIG_backup_dir"/tmp/mail_attachment_${datetimestamp}_*) 144 | echo -e "\n\nThe attachments have been split into multiple files.\nUse 'cat mail_attachment_2011-08-13_13h15m_* > mail_attachment_2011-08-13_13h15m.tar.bz2' to combine them and \ 145 | 'bunzip2 "$temp" 150 | uuencode "$j" "$j" >> "$temp" 151 | else 152 | uuencode "$j" "$j" > "$temp" 153 | fi 154 | mail -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} < "$temp" 155 | else 156 | mutt -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}; Part $((j+1))/${#files[@]}" -a "${files[j]}" -- ${CONFIG_mail_address} < "$log_file" 157 | fi 158 | done 159 | else 160 | cat "$log_file" | mail -s "WARNING! - MySQL Backup exceeds set maximum attachment size on ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 161 | fi 162 | else 163 | cat "$log_file" | mail -s "WARNING! - MySQL Backup exceeds set maximum attachment size on ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 164 | fi 165 | rm "$temp" 166 | ;; 167 | 'log') 168 | cat "$log_file" | mail -s "MySQL Backup Log for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 169 | [[ -s "$log_errfile" ]] && cat "$log_errfile" | mail -s "ERRORS REPORTED: MySQL Backup error Log for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 170 | ;; 171 | 'quiet') 172 | [[ -s "$log_errfile" ]] && cat "$log_errfile" | mail -s "ERRORS REPORTED: MySQL Backup error Log for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 173 | ;; 174 | *) 175 | if [[ -s "$log_errfile" ]]; then 176 | cat "$log_file" 177 | echo 178 | echo "###### WARNING ######" 179 | echo "Errors reported during AutoMySQLBackup execution.. Backup failed" 180 | echo "Error log below.." 181 | cat "$log_errfile" 182 | else 183 | cat "$log_file" 184 | fi 185 | ;; 186 | esac 187 | ################################################################################### 188 | # Clean up and finish 189 | [[ -e "$log_file" ]] && rm -f "$log_file" 190 | [[ -e "$log_errfile" ]] && rm -f "$log_errfile" 191 | fi 192 | } 193 | 194 | # @params: #month #year 195 | # @deps: (none) 196 | days_of_month() { 197 | m="$1"; y="$2"; a=$(( 30+(m+m/8)%2 )) 198 | (( m==2 )) && a=$((a-2)) 199 | (( m==2 && y%4==0 && ( y<100 || y%100>0 || y%400==0) )) && a=$((a+1)) 200 | printf '%d' $a 201 | } 202 | 203 | # @info: Checks if a folder is writable by creating a temporary file in it and removing it afterwards. 204 | # @args: folder to test 205 | # @return: returns false if creation of temporary file failed or it can't be removed afterwards; else true 206 | # @deps: (none) 207 | chk_folder_writable () { 208 | local temp; temp="$(mktemp "$1"/tmp.XXXXXX)" 209 | if (( $? == 0 )); then 210 | rm "${temp}" || return 1 211 | return 0 212 | else 213 | return 1 214 | fi 215 | } 216 | 217 | # @info: bzip2 compression 218 | bzip2_compression() { 219 | var=("$@") 220 | re='^[0-9]*$' 221 | if [[ "x$CONFIG_multicore" = 'xyes' ]]; then 222 | if [[ "x$CONFIG_multicore_threads" != 'xauto' ]] && [[ "x$CONFIG_multicore_threads" =~ $re ]]; then 223 | var=( "-p${CONFIG_multicore_threads}" "${var[@]}" ) 224 | fi 225 | pbzip2 "${var[@]}" 226 | else 227 | bzip2 "${var[@]}" 228 | fi 229 | } 230 | 231 | # @info: gzip compression 232 | gzip_compression() { 233 | var=("$@") 234 | re='^[0-9]*$' 235 | if [[ "x$CONFIG_multicore" = 'xyes' ]]; then 236 | if [[ "x$CONFIG_multicore_threads" != 'xauto' ]] && [[ "x$CONFIG_multicore_threads" =~ $re ]]; then 237 | var=( "-p${CONFIG_multicore_threads}" "${var[@]}" ) 238 | fi 239 | pigz "${var[@]}" 240 | else 241 | gzip "${var[@]}" 242 | fi 243 | } 244 | 245 | # @info: Remove date and time information from filename by renaming it. 246 | # @args: filename 247 | # @return: (none) 248 | # @deps: (none) 249 | remove_datetimeinfo () { 250 | mv "$1" "$(echo "$1" | sed -re 's/_[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}h[0-9]{2}m_(Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday|January|February|March|April|May|June|July|August|September|October|November|December|[0-9]{1,2})//g')" 251 | } 252 | export -f remove_datetimeinfo 253 | 254 | # @info: Set time and date variables. 255 | # @args: (none) 256 | # @deps: days_of_month 257 | set_datetime_vars() { 258 | datetimestamp=`date +%Y-%m-%d_%Hh%Mm` # Datestamp e.g 2002-09-21_18h12m 259 | date_stamp=`date +%Y-%m-%d` # Datestamp e.g 2002-09-21 260 | date_day_of_week=`date +%A` # Day of the week e.g. Monday 261 | date_dayno_of_week=`date +%u` # Day number of the week 1 to 7 where 1 represents Monday 262 | date_day_of_month=`date +%e | sed -e 's/^ //'` # Date of the Month e.g. 27 263 | date_month=`date +%B` # Month e.g January 264 | date_weekno=`date +%V | sed -e 's/^0//'` # Week Number e.g 37 265 | year=`date +%Y` 266 | month=`date +%m | sed -e 's/^0//'` 267 | date_lastday_of_last_month=$(days_of_month $(( $month==1 ? 12 : $month-1 )) $(( $month==1 ? ($year-1):$year )) ) 268 | date_lastday_of_this_month=$(days_of_month $month $year) 269 | } 270 | 271 | # @info: This function is called after data has already been saved. It performs encryption and 272 | # hardlink-copying of files to a latest folder. 273 | # @return: flags 274 | # @deps: load_default_config 275 | files_postprocessing () { 276 | local flags 277 | let "flags=0x00" 278 | let "flags_files_postprocessing_success_encrypt=0x01" 279 | 280 | # -> CONFIG_encrypt 281 | [[ "${CONFIG_encrypt}" = "yes" && "${CONFIG_encrypt_password}" ]] && { 282 | if (( $CONFIG_dryrun )); then 283 | printf 'dry-running: openssl enc -aes-256-cbc -e -in %s -out %s.enc -pass pass:%s\n' ${1} ${1} "${CONFIG_encrypt_password}" 284 | else 285 | openssl enc -aes-256-cbc -e -in ${1} -out ${1}.enc -pass pass:"${CONFIG_encrypt_password}" 286 | if (( $? == 0 )); then 287 | if rm ${1} 2>&1; then 288 | echo "Successfully encrypted archive as ${1}.enc" 289 | let "flags |= $flags_files_postprocessing_success_encrypt" 290 | else 291 | echo "Successfully encrypted archive as ${1}.enc, but could not remove cleartext file ${1}." 292 | let "E |= $E_enc_cleartext_delfailed" 293 | fi 294 | else 295 | let "E |= $E_enc_failed" 296 | fi 297 | fi 298 | } 299 | # <- CONFIG_encrypt 300 | 301 | # -> CONFIG_mysql_dump_latest 302 | [[ "${CONFIG_mysql_dump_latest}" = "yes" ]] && { 303 | if (( $flags & $flags_files_postprocessing_success_encrypt )); then 304 | if (( $CONFIG_dryrun )); then 305 | printf 'dry-running: cp -al %s.enc %s/latest/\n' "${1}" "${CONFIG_backup_dir}" 306 | else 307 | cp -al "${1}${suffix}.enc" "${CONFIG_backup_dir}"/latest/ 308 | fi 309 | else 310 | if (( $CONFIG_dryrun )); then 311 | printf 'dry-running: cp -al %s %s/latest/\n' "${1}" "${CONFIG_backup_dir}" 312 | else 313 | cp -al "${1}" "${CONFIG_backup_dir}"/latest/ 314 | fi 315 | fi 316 | } 317 | # <- CONFIG_mysql_dump_latest 318 | 319 | return $flags 320 | } 321 | 322 | # @info: When called, sets error and notify strings matching their flags. It then goes through all 323 | # collected error and notify messages and displays them. 324 | # @args: (none) 325 | # @return: true if no errors were set, otherwise false 326 | # @deps: log_base2, load_default_config 327 | error_handler () { 328 | 329 | errors=( 330 | [0x01]='dbdump() failed.' 331 | [0x02]='Backup of local files failed. This is not this scripts primary objective. Continuing anyway.' 332 | [0x04]="Could not create the backup_dir ${CONFIG_backup_dir}. Please check permissions of the higher directory." 333 | [0x08]='At least one of the subdirectories (daily, weekly, monthly, latest) failed to create.' 334 | [0x10]="The backup_dir ${CONFIG_backup_dir} is not writable AND/OR executable." 335 | [0x20]='Could not remove the cleartext file after encryption. This error did not cause an abort. Remove it manually and check permissions.' 336 | [0x40]='Encryption failed. Continuing without encryption.' 337 | [0x80]='The mysql server is empty, i.e. no databases found. Check if something is wrong. Exiting.' 338 | [0x100]='Failed to create the named pipe (fifo) for reading in all databases. Exiting.' 339 | [0x200]='Dependency programs are missing. Perhaps they are not in $PATH. Exiting.' 340 | [0x400]='No basedir found, i.e. ' 341 | [0x800]="${CONFIG_backup_dir} is not writable. Exiting." 342 | [0x1000]='Running of mysqlstatus failed.' 343 | [0x2000]='Running of mysqldump full schema failed.' 344 | ) 345 | 346 | notify=( 347 | [0x01]="${CONFIG_configfile} was not found - no global config file." 348 | [0x02]="Parsed config file ${opt_config_file}." 349 | [0x04]="Unreadable config file \"${opt_config_file}\"" 350 | [0x08]='Supplied more than one argument, ignoring ALL arguments - using default and global config file only.' 351 | [0x10]='Could not remove the files in the latest directory. Please check this.' 352 | [0x20]='No local backup files were set.' 353 | [0x40]='' 354 | [0x80]='' 355 | [0x100]='' 356 | [0x200]='' 357 | [0x400]='' 358 | [0x800]='' 359 | [0x1000]='' 360 | [0x2000]='' 361 | ) 362 | 363 | local n 364 | local e 365 | 366 | n=$((${#notify[@]}-1)) 367 | while (( N > 0 )); do 368 | e=$((2**n)) 369 | if (( N&e )); then 370 | echo "Note:" ${notify[e]} 371 | let "N-=e" 372 | fi 373 | ((n--)) 374 | done 375 | unset n; 376 | 377 | n=$((${#errors[@]}-1)) 378 | if (( E > 0 )); then 379 | while (( E > 0 )); do 380 | e=$((2**n)) 381 | if (( E&e )); then 382 | echo "Error:" ${errors[e]} 383 | let "E-=e" 384 | fi 385 | ((n--)) 386 | done 387 | exit 1 388 | else 389 | exit 0 390 | fi 391 | } 392 | 393 | # @info: Packs files in array ${#CONFIG_backup_local_files[@]} into tar file with optional compression. 394 | # @args: archive file without compression suffix, i.e. ending on .tar 395 | # @return: true in case of dry-run, otherwise the return value of tar -cvf 396 | # @deps: load_default_config 397 | backup_local_files () { 398 | if ((! ${#CONFIG_backup_local_files[@]})) ; then 399 | if (( $CONFIG_dryrun )); then 400 | case "${CONFIG_mysql_dump_compression}" in 401 | 'gzip') 402 | echo "tar -czvf ${1}${suffix} ${CONFIG_backup_local_files[@]}"; 403 | ;; 404 | 'bzip2') 405 | echo "tar -cjvf ${1}${suffix} ${CONFIG_backup_local_files[@]}"; 406 | ;; 407 | *) 408 | echo "tar -cvf ${1}${suffix} ${CONFIG_backup_local_files[@]}"; 409 | ;; 410 | esac 411 | echo "dry-running: tar -cv ${1} ${CONFIG_backup_local_files[@]}" 412 | return 0; 413 | else 414 | case "${CONFIG_mysql_dump_compression}" in 415 | 'gzip') 416 | tar -czvf "${1}${suffix}" "${CONFIG_backup_local_files[@]}"; 417 | return $? 418 | ;; 419 | 'bzip2') 420 | tar -cjvf "${1}${suffix}" "${CONFIG_backup_local_files[@]}"; 421 | return $? 422 | ;; 423 | *) 424 | tar -cvf "${1}${suffix}" "${CONFIG_backup_local_files[@]}"; 425 | return $? 426 | ;; 427 | esac 428 | fi 429 | else 430 | let "N |= $N_backup_local_nofiles" 431 | echo "No local backup files specified." 432 | fi 433 | } 434 | 435 | # @info: Parses the configuration options and sets the variables appropriately. 436 | # @args: (none) 437 | # @deps: load_default_config 438 | parse_configuration () { 439 | # OPT string for use with mysqldump ( see man mysqldump ) 440 | opt=( '--quote-names' '--opt' ) 441 | 442 | # OPT string for use with mysql (see man mysql ) 443 | mysql_opt=() 444 | 445 | # OPT string for use with mysqldump fullschema 446 | opt_fullschema=( '--all-databases' '--routines' '--no-data' ) 447 | 448 | # OPT string for use with mysqlstatus 449 | opt_dbstatus=( '--status' ) 450 | 451 | [[ "${CONFIG_mysql_dump_usessl}" = "yes" ]] && { 452 | opt=( "${opt[@]}" '--ssl' ) 453 | mysql_opt=( "${mysql_opt[@]}" '--ssl' ) 454 | opt_fullschema=( "${opt_fullschema[@]}" '--ssl' ) 455 | opt_dbstatus=( "${opt_dbstatus[@]}" '--ssl' ) 456 | } 457 | [[ "${CONFIG_mysql_dump_master_data}" ]] && (( ${CONFIG_mysql_dump_master_data} == 1 || ${CONFIG_mysql_dump_master_data} == 2 )) && { opt=( "${opt[@]}" "--master-data=${CONFIG_mysql_dump_master_data}" );} 458 | [[ "${CONFIG_mysql_dump_single_transaction}" = "yes" ]] && { 459 | opt=( "${opt[@]}" '--single-transaction' ) 460 | opt_fullschema=( "${opt_fullschema[@]}" '--single-transaction' ) 461 | } 462 | [[ "${CONFIG_mysql_dump_commcomp}" = "yes" ]] && { 463 | opt=( "${opt[@]}" '--compress' ) 464 | opt_fullschema=( "${opt_fullschema[@]}" '--compress' ) 465 | opt_dbstatus=( "${opt_dbstatus[@]}" '--compress' ) 466 | } 467 | [[ "${CONFIG_mysql_dump_max_allowed_packet}" ]] && { 468 | opt=( "${opt[@]}" "--max_allowed_packet=${CONFIG_mysql_dump_max_allowed_packet}" ) 469 | opt_fullschema=( "${opt_fullschema[@]}" "--max_allowed_packet=${CONFIG_mysql_dump_max_allowed_packet}" ) 470 | } 471 | [[ "${CONFIG_mysql_dump_socket}" ]] && { 472 | opt=( "${opt[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 473 | mysql_opt=( "${mysql_opt[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 474 | opt_fullschema=( "${opt_fullschema[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 475 | opt_dbstatus=( "${opt_dbstatus[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 476 | } 477 | [[ "${CONFIG_mysql_dump_port}" ]] && { 478 | opt=( "${opt[@]}" "--port=${CONFIG_mysql_dump_port}" ) 479 | mysql_opt=( "${mysql_opt[@]}" "--port=${CONFIG_mysql_dump_port}" ) 480 | opt_fullschema=( "${opt_fullschema[@]}" "--port=${CONFIG_mysql_dump_port}" ) 481 | opt_dbstatus=( "${opt_dbstatus[@]}" "--port=${CONFIG_mysql_dump_port}" ) 482 | } 483 | 484 | # Check if CREATE DATABASE should be included in Dump 485 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 486 | if [[ "${CONFIG_mysql_dump_create_database}" = "no" ]]; then 487 | opt=( "${opt[@]}" '--no-create-db' ) 488 | else 489 | opt=( "${opt[@]}" '--databases' ) 490 | fi 491 | else 492 | opt=( "${opt[@]}" '--databases' ) 493 | fi 494 | 495 | # if differential backup is active and the specified rotation is smaller than 21 days, set it to 21 days to ensure, that 496 | # master backups aren't deleted. 497 | if [[ "x$CONFIG_mysql_dump_differential" = "xyes" ]] && (( ${CONFIG_rotation_daily} < 21 )); then 498 | CONFIG_rotation_daily=21 499 | fi 500 | 501 | # -> determine suffix 502 | case "${CONFIG_mysql_dump_compression}" in 503 | 'gzip') suffix='.gz';; 504 | 'bzip2') suffix='.bz2';; 505 | *) suffix='';; 506 | esac 507 | # <- determine suffix 508 | 509 | # -> check exclude tables for wildcards 510 | local tmp;tmp=() 511 | local z;z=0 512 | for i in "${CONFIG_table_exclude[@]}"; do 513 | r='^[^*.]+\.[^.]+$'; [[ "$i" =~ $r ]] || { printf 'The entry %s in CONFIG_table_exclude has a wrong format. Ignoring the entry.' "$i"; continue; } 514 | db=${i%.*} 515 | table=${i#"$db".} 516 | r='\*'; [[ "$i" =~ $r ]] || { tmp[z++]="$i"; continue; } 517 | while read -r; do tmp[z++]="${db}.${REPLY}"; done < <(mysql --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${mysql_opt[@]}" --batch --skip-column-names -e "select table_name from information_schema.tables where table_schema='${db}' and table_name like '${table//\*/%}';") 518 | done 519 | for l in "${tmp[@]}"; do echo "exclude $l";done 520 | CONFIG_table_exclude=("${tmp[@]}") 521 | # <- 522 | 523 | if ((${#CONFIG_table_exclude[@]})); then 524 | for i in "${CONFIG_table_exclude[@]}"; do 525 | opt=( "${opt[@]}" "--ignore-table=$i" ) 526 | done 527 | fi 528 | } 529 | 530 | # @info: Backup database status 531 | # @args: archive file without compression suffix, i.e. ending on .txt 532 | # @return: true in case of dry-run, otherwise the return value of mysqlshow 533 | # @deps: load_default_config, parse_configuration 534 | dbstatus() { 535 | if (( $CONFIG_dryrun )); then 536 | case "${CONFIG_mysql_dump_compression}" in 537 | 'gzip') 538 | echo "dry-running: mysqlshow --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_dbstatus[@]} | gzip_compression > ${1}${suffix}"; 539 | ;; 540 | 'bzip2') 541 | echo "dry-running: mysqlshow --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_dbstatus[@]} | bzip2_compression > ${1}${suffix}"; 542 | ;; 543 | *) 544 | echo "dry-running: mysqlshow --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_dbstatus[@]} > ${1}${suffix}"; 545 | ;; 546 | esac 547 | return 0; 548 | else 549 | case "${CONFIG_mysql_dump_compression}" in 550 | 'gzip') 551 | mysqlshow --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_dbstatus[@]}" | gzip_compression > "${1}${suffix}"; 552 | return $? 553 | ;; 554 | 'bzip2') 555 | mysqlshow --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_dbstatus[@]}" | bzip2_compression > "${1}${suffix}"; 556 | return $? 557 | ;; 558 | *) 559 | mysqlshow --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_dbstatus[@]}" > "${1}${suffix}"; 560 | return $? 561 | ;; 562 | esac 563 | fi 564 | } 565 | 566 | # @info: Backup of the database schema. 567 | # @args: filename to save data to 568 | # @return: true in case of dry-run, otherwise the return value of mysqldump 569 | # @deps: load_default_config, parse_configuration 570 | fullschema () { 571 | if (( $CONFIG_dryrun )); then 572 | case "${CONFIG_mysql_dump_compression}" in 573 | 'gzip') 574 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_fullschema[@]} | gzip_compression > ${1}${suffix}"; 575 | ;; 576 | 'bzip2') 577 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_fullschema[@]} | bzip2_compression > ${1}${suffix}"; 578 | ;; 579 | *) 580 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_fullschema[@]} > ${1}${suffix}"; 581 | ;; 582 | esac 583 | return 0; 584 | else 585 | case "${CONFIG_mysql_dump_compression}" in 586 | 'gzip') 587 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_fullschema[@]}" | gzip_compression > "${1}${suffix}"; 588 | return $? 589 | ;; 590 | 'bzip2') 591 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_fullschema[@]}" | bzip2_compression > "${1}${suffix}"; 592 | return $? 593 | ;; 594 | *) 595 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_fullschema[@]}" > "${1}${suffix}"; 596 | return $? 597 | ;; 598 | esac 599 | fi 600 | } 601 | 602 | # @info: Process a single db. 603 | # @args: subfolder, prefix, midfix, extension, rotation, rotation_divisor, rotation_string, 0/1 (db/dbs), db[, db ...] 604 | process_dbs() { 605 | local subfolder="$1" 606 | local prefix="$2" 607 | local midfix="$3" 608 | local extension="$4" 609 | local rotation="$5" 610 | local rotation_divisor="$6" 611 | local rotation_string="$7" 612 | local multipledbs="$8" 613 | shift 8 614 | 615 | local name 616 | local subsubfolder 617 | 618 | # only activate differential backup for daily backups 619 | [[ "x$subfolder" != "xdaily" ]] && activate_differential_backup=0 || activate_differential_backup=1 620 | 621 | if (( $multipledbs )); then 622 | # multiple dbs 623 | subsubfolder="" 624 | name="all-databases" 625 | else 626 | # single db 627 | subsubfolder="/$1" 628 | name="$@" 629 | fi 630 | 631 | [[ -d "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" ]] || { 632 | if (( $CONFIG_dryrun )); then 633 | printf 'dry-running: mkdir -p %s/${subfolder}%s\n' "${CONFIG_backup_dir}" "${subsubfolder}" 634 | else 635 | mkdir -p "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" 636 | fi 637 | } 638 | 639 | manifest_file="${CONFIG_backup_dir}/${subfolder}${subsubfolder}/Manifest" 640 | fname="${CONFIG_backup_dir}/${subfolder}${subsubfolder}/${prefix}${name}_${datetimestamp}${midfix}${extension}" 641 | 642 | (( $CONFIG_debug )) && echo "DEBUG: process_dbs >> Setting manifest file to: ${manifest_file}" 643 | 644 | if (( $multipledbs )); then 645 | # multiple databases 646 | db="all-databases" 647 | else 648 | # single db 649 | db="$1" 650 | fi 651 | 652 | if [[ "x$CONFIG_mysql_dump_differential" = "xyes" ]] && [[ "x${CONFIG_encrypt}" != "xyes" ]] && (( $activate_differential_backup )); then 653 | 654 | 655 | unset manifest_entry manifest_entry_to_check 656 | 657 | echo "## Reading in Manifest file" 658 | parse_manifest "$manifest_file" 659 | echo 660 | echo "Number of manifest entries: $(num_manifest_entries)" 661 | echo 662 | 663 | 664 | # -> generate diff file 665 | let "filename_flags=0x00" 666 | 667 | # ## -> get latest differential manifest entry for specified db 668 | # if get_latest_manifest_entry_for_db "$db" 1; then 669 | # pid="${manifest_entry[2]}" 670 | # # filename format: prefix_db_YYYY-MM-DD_HHhMMm_[A-Za-z0-9]{8}(.sql|.diff)(.gz|.bz2)(.enc) 671 | # FileStub=${manifest_entry[0]%.@(sql|diff)*} 672 | # FileExt=${manifest_entry[0]#"$FileStub"} 673 | # re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_encrypted" 674 | # re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_gz" 675 | # re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_bz2" 676 | # re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_diff" 677 | # manifest_latest_diff_entry=("${manifest_entry[@]}") 678 | # else # no entries in manifest 679 | # pid=0 680 | # fi 681 | # ## <- get latest differential manifest entry for specified db 682 | 683 | ## -> get latest master manifest entry for specified db 684 | # Create a differential backup if a master entry in the manifest exists, it isn't the day we do weekly master backups or the master file we fetched is already from today. 685 | if get_latest_manifest_entry_for_db "$db" 0 && ( (( ${date_dayno_of_week} != ${CONFIG_do_weekly} )) || [[ "${manifest_entry[0]}" = *_$(date +%Y-%m-%d)_* ]] ); then 686 | pid="${manifest_entry[2]}" 687 | # filename format: prefix_db_YYYY-MM-DD_HHhMMm_[A-Za-z0-9]{8}(.sql|.diff)(.gz|.bz2)(.enc) 688 | FileStub="${manifest_entry[0]%.@(sql|diff)*}" 689 | FileExt="${manifest_entry[0]#"$FileStub"}" 690 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_encrypted" 691 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_gz" 692 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_bz2" 693 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_diff" 694 | manifest_latest_master_entry=("${manifest_entry[@]}") 695 | else # no entries in manifest 696 | pid=0 697 | fi 698 | ## <- get latest master manifest entry for specified db 699 | 700 | fi 701 | 702 | if [[ "x$CONFIG_mysql_dump_differential" = "xyes" ]] && [[ "x${CONFIG_encrypt}" != "xyes" ]] && (( $activate_differential_backup )) && ((! ($filename_flags & $filename_flag_encrypted) )); then 703 | 704 | # the master file is encrypted ... well this just shouldn't happen ^^ not going to decrypt or stuff like that ...at least not today :) 705 | 706 | if [[ "x$pid" = "x0" ]]; then 707 | # -> create master backup 708 | cfname="$(mktemp "${fname%.sql}_"XXXXXXXX".sql${suffix}")" 709 | uid="${cfname%.@(diff|sql)*}" 710 | uid="${uid:-8:8}" 711 | case "${CONFIG_mysql_dump_compression}" in 712 | 'gzip') 713 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | gzip_compression > "$cfname"; 714 | ;; 715 | 'bzip2') 716 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | bzip2_compression > "$cfname"; 717 | ;; 718 | *) 719 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" > "$cfname"; 720 | ;; 721 | esac 722 | add_manifest_entry "$manifest_file" "$cfname" "$pid" "$db" && parse_manifest "$manifest_file" && cp -al "$cfname" "${CONFIG_backup_dir}"/latest/ && echo "Generated master backup $cfname" && return 0 || return 1 723 | # <- create master backup 724 | else 725 | cfname="$(mktemp "${fname%.sql}_"XXXXXXXX".diff${suffix}")" 726 | uid="${cfname%.@(diff|sql)*}" 727 | uid=${uid:-8:8} 728 | echo "Creating differential backup to ${manifest_entry[0]}:" 729 | case "${CONFIG_mysql_dump_compression}" in 730 | 'gzip') 731 | if (( $filename_flags & $filename_flag_gz )); then 732 | diff <(gzip_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | gzip_compression > "$cfname"; 733 | elif (( $filename_flags & $filename_flag_bz2 )); then 734 | diff <(bzip2_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | gzip_compression > "$cfname"; 735 | else 736 | diff "${manifest_latest_master_entry[0]}" <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | gzip_compression > "$cfname"; 737 | fi 738 | ;; 739 | 'bzip2') 740 | if (( $filename_flags & $filename_flag_gz )); then 741 | diff <(gzip_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | bzip2_compression > "$cfname"; 742 | elif (( $filename_flags & $filename_flag_bz2 )); then 743 | diff <(bzip2_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | bzip2_compression > "$cfname"; 744 | else 745 | diff "${manifest_latest_master_entry[0]}" <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | bzip2_compression > "$cfname"; 746 | fi 747 | ;; 748 | *) 749 | if (( $filename_flags & $filename_flag_gz )); then 750 | diff <(gzip_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") > "$cfname"; 751 | elif (( $filename_flags & $filename_flag_bz2 )); then 752 | diff <(bzip2_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") > "$cfname"; 753 | else 754 | diff "${manifest_latest_master_entry[0]}" <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") > "$cfname"; 755 | fi 756 | ;; 757 | esac 758 | add_manifest_entry "$manifest_file" "$cfname" "$pid" "$db" && parse_manifest "$manifest_file" && cp -al "$cfname" "${manifest_latest_master_entry[0]}" "${CONFIG_backup_dir}"/latest/ && echo "generated $cfname" && return 0 || return 1 759 | 760 | fi 761 | # <- generate diff filename 762 | 763 | else 764 | cfname="${fname}${suffix}" 765 | if (( $CONFIG_dryrun )); then 766 | case "${CONFIG_mysql_dump_compression}" in 767 | 'gzip') 768 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt[@]} $@ | gzip_compression > ${cfname}" 769 | ;; 770 | 'bzip2') 771 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt[@]} $@ | bzip2_compression > ${cfname}" 772 | ;; 773 | *) 774 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt[@]} $@ > ${cfname}" 775 | ;; 776 | esac 777 | return 0; 778 | else 779 | case "${CONFIG_mysql_dump_compression}" in 780 | 'gzip') 781 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | gzip_compression > "${cfname}" 782 | ret=$? 783 | ;; 784 | 'bzip2') 785 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | bzip2_compression > "${cfname}" 786 | ret=$? 787 | ;; 788 | *) 789 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" > "${cfname}" 790 | ret=$? 791 | ;; 792 | esac 793 | fi 794 | 795 | fi 796 | 797 | 798 | if (( $ret == 0 )); then 799 | echo "Rotating $(( ${rotation}/${rotation_divisor} )) ${rotation_string} backups for ${name}" 800 | if (( $CONFIG_dryrun )); then 801 | find "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" -mtime +"${rotation}" -type f -exec echo "dry-running: rm" {} \; 802 | else 803 | find "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" -mtime +"${rotation}" -type f -exec rm {} \; 804 | fi 805 | files_postprocessing "$cfname" 806 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 807 | backupfiles=( "${backupfiles[@]}" "${cfname}${var}" ) 808 | else 809 | let "E |= $E_dbdump_failed" 810 | echo "dbdump with parameters \"${CONFIG_db_names[@]}\" \"${cfname}\" failed!" 811 | fi 812 | } 813 | 814 | # @info: Save stdout and stderr 815 | # @deps: (none) 816 | activateIO() { 817 | ################################################################################### 818 | # IO redirection for logging. 819 | # $1 = $log_file, $2 = $log_errfile 820 | 821 | #(( $CONFIG_debug )) || { 822 | touch "$log_file" 823 | exec 6>&1 # Link file descriptor #6 with stdout. Saves stdout. 824 | exec > "$log_file" # stdout replaced with file $log_file. 825 | 826 | touch "$log_errfile" 827 | exec 7>&2 # Link file descriptor #7 with stderr. Saves stderr. 828 | exec 2> "$log_errfile" # stderr replaced with file $log_errfile. 829 | #} 830 | } 831 | 832 | # @info: Restore stdout and stderr redirections. 833 | # @deps: (none) 834 | removeIO() { 835 | exec 1>&6 6>&- # Restore stdout and close file descriptor #6. 836 | exec 2>&7 7>&- # Restore stdout and close file descriptor #7. 837 | } 838 | 839 | # @info: Checks directories and subdirectories for existence and activates logging to either 840 | # $CONFIG_backup_dir or /tmp depending on what exists. 841 | # @args: (none) 842 | # @deps: load_default_config, activateIO, chk_folder_writable, error_handler 843 | directory_checks_enable_logging () { 844 | ################################################################################### 845 | # Check directories and do cleanup work 846 | 847 | checkdirs=( "${CONFIG_backup_dir}"/{daily,weekly,monthly,latest,tmp} ) 848 | [[ "${CONFIG_backup_local_files[@]}" ]] && { checkdirs=( "${checkdirs[@]}" "${CONFIG_backup_dir}/backup_local_files" ); } 849 | [[ "${CONFIG_mysql_dump_full_schema}" = 'yes' ]] && { checkdirs=( "${checkdirs[@]}" "${CONFIG_backup_dir}/fullschema" ); } 850 | [[ "${CONFIG_mysql_dump_dbstatus}" = 'yes' ]] && { checkdirs=( "${checkdirs[@]}" "${CONFIG_backup_dir}/status" ); } 851 | 852 | tmp_permcheck=0 853 | printf '# Checking for permissions to write to folders:\n' 854 | 855 | 856 | # "dirname ${CONFIG_backup_dir}" exists? 857 | # Y -> ${CONFIG_backup_dir} exists? 858 | # Y -> Dry-run? 859 | # Y -> log to /tmp, proceed to test subdirs 860 | # N -> check writable ${CONFIG_backup_dir}? 861 | # Y -> proceed to test subdirs 862 | # N -> error: can't write to ${CONFIG_backup_dir}. Exit. 863 | # N -> Dry-run? 864 | # N -> proceed without testing subdirs 865 | # Y -> create directory ${CONFIG_backup_dir}? 866 | # Y -> check writable ${CONFIG_backup_dir}? 867 | # Y -> proceed to test subdirs 868 | # N -> error: can't write to ${CONFIG_backup_dir}. Exit. 869 | # N -> error: ${CONFIG_backup_dir} is not writable. Exit. 870 | # N -> Dry-run? 871 | # Y -> log to /tmp, proceed without testing subdirs 872 | # N -> error: no basedir. Exit. 873 | 874 | 875 | # -> check base folder 876 | printf 'base folder %s ... ' "$(dirname "${CONFIG_backup_dir}")" 877 | if [[ -d "$(dirname "${CONFIG_backup_dir}")" ]]; then 878 | 879 | printf 'exists ... ok.\n' 880 | printf 'backup folder %s ... ' "${CONFIG_backup_dir}" 881 | 882 | if [[ -d "${CONFIG_backup_dir}" ]]; then 883 | printf 'exists ... writable? ' 884 | if (( $CONFIG_dryrun )); then 885 | printf 'dry-running. Skipping. Logging to /tmp\n' 886 | log_file="/tmp/${CONFIG_mysql_dump_host}-`date +%N`.log" 887 | log_errfile="/tmp/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 888 | activateIO "$log_file" "$log_errfile" 889 | tmp_permcheck=1 890 | else 891 | if chk_folder_writable "${CONFIG_backup_dir}"; then 892 | printf 'yes. Proceeding.\n' 893 | log_file="${CONFIG_backup_dir}/${CONFIG_mysql_dump_host}-`date +%N`.log" 894 | log_errfile="${CONFIG_backup_dir}/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 895 | activateIO "$log_file" "$log_errfile" 896 | tmp_permcheck=1 897 | else 898 | printf 'no. Exiting.\n' 899 | let "E |= $E_config_backupdir_not_writable" 900 | error_handler 901 | fi 902 | fi 903 | 904 | else 905 | 906 | printf 'creating ... ' 907 | if (( $CONFIG_dryrun )); then 908 | printf 'dry-running. Skipping.\n' 909 | else 910 | if mkdir -p "${CONFIG_backup_dir}" >/dev/null 2>&1; then 911 | printf 'success.\n' 912 | log_file="${CONFIG_backup_dir}/${CONFIG_mysql_dump_host}-`date +%N`.log" 913 | log_errfile="${CONFIG_backup_dir}/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 914 | activateIO "$log_file" "$log_errfile" 915 | tmp_permcheck=1 916 | else 917 | printf 'failed. Exiting.\n' 918 | let "E |= $E_mkdir_basedir_failed" 919 | error_handler 920 | fi 921 | fi 922 | 923 | fi 924 | 925 | else 926 | 927 | if (( $CONFIG_dryrun )); then 928 | printf 'dry-running. Skipping. Logging to /tmp\n' 929 | log_file="/tmp/${CONFIG_mysql_dump_host}-`date +%N`.log" 930 | log_errfile="/tmp/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 931 | activateIO "$log_file" "$log_errfile" 932 | else 933 | printf 'does not exist. Exiting.\n' 934 | let "E |= $E_no_basedir" 935 | error_handler 936 | fi 937 | 938 | fi 939 | # <- check base folder 940 | 941 | 942 | # -> check subdirs 943 | if (( $tmp_permcheck == 1 )); then 944 | 945 | (( $CONFIG_dryrun )) || [[ -r "${CONFIG_backup_dir}" && -x "${CONFIG_backup_dir}" ]] || { let "E |= $E_perm_basedir"; error_handler; } 946 | 947 | for i in "${checkdirs[@]}"; do 948 | printf 'checking directory "%s" ... ' "$i" 949 | if [[ -d "$i" ]]; then 950 | printf 'exists.\n' 951 | else 952 | printf 'creating ... ' 953 | if (( $CONFIG_dryrun )); then 954 | printf 'dry-running. Skipping.\n' 955 | else 956 | if mkdir -p "$i" >/dev/null 2>&1; then 957 | printf 'success.\n' 958 | else 959 | printf 'failed. Exiting.\n' 960 | let "E |= $E_mkdir_subdirs_failed" 961 | error_handler 962 | fi 963 | fi 964 | fi 965 | done 966 | 967 | fi 968 | # <- check subdirs 969 | 970 | } 971 | 972 | # @info: If CONFIG_mysql_dump_latest is set to 'yes', the directory ${CONFIG_backup_dir}"/latest will 973 | # be cleaned. 974 | # @args: (none) 975 | # @deps: load_default_config 976 | cleanup_latest () { 977 | # -> latest cleanup 978 | if [[ "${CONFIG_mysql_dump_latest}" = "yes" ]]; then 979 | printf 'Cleaning up latest directory ... ' 980 | if (( $CONFIG_dryrun )); then 981 | printf 'dry-running. Skipping.\n' 982 | else 983 | if rm -f "${CONFIG_backup_dir}"/latest/* >/dev/null 2>&1; then 984 | printf 'success.\n' 985 | else 986 | printf 'failed. Continuing anyway, activating Note-Flag.\n' 987 | let "N |= $N_latest_cleanup_failed" 988 | fi 989 | fi 990 | fi 991 | # <- latest cleanup 992 | } 993 | 994 | # @info: Checks for dependencies in form of external programs, that need to be available when running 995 | # this program. 996 | # @args: (none) 997 | # @deps: load_default_config 998 | check_dependencies () { 999 | echo 1000 | echo "# Testing for installed programs" 1001 | dependencies=( 'mysql' 'mysqldump' ) 1002 | 1003 | if [[ "x$CONFIG_multicore" = 'xyes' ]]; then 1004 | 1005 | if [[ "x$CONFIG_mysql_dump_compression" = 'xbzip2' ]]; then 1006 | if type pbzip2 &>/dev/null; then 1007 | echo "pbzip2 ... found." 1008 | else 1009 | CONFIG_multicore='no' # turn off multicore support, since the program isn't there 1010 | echo "WARNING: Turning off multicore support, since pbzip2 isn't there." 1011 | fi 1012 | elif [[ "x$CONFIG_mysql_dump_compression" = 'xgzip' ]]; then 1013 | if type pigz &>/dev/null; then 1014 | echo "pigz ... found." 1015 | else 1016 | CONFIG_multicore='no' # turn off multicore support, since the program isn't there 1017 | echo "WARNING: Turning off multicore support, since pigz isn't there." 1018 | fi 1019 | fi 1020 | 1021 | else 1022 | [[ "x$CONFIG_mysql_dump_compression" = 'xbzip2' ]] && dependencies=("${dependencies[@]}" 'bzip2' ) 1023 | [[ "x$CONFIG_mysql_dump_compression" = 'xgzip' ]] && dependencies=("${dependencies[@]}" 'gzip' ) 1024 | fi 1025 | 1026 | if [[ "x$CONFIG_mailcontent" = 'xlog' || "x$CONFIG_mailcontent" = 'xquiet' ]]; then 1027 | dependencies=( "${dependencies[@]}" 'mail' ) 1028 | elif [[ "x$CONFIG_mailcontent" = 'xfiles' ]]; then 1029 | dependencies=( "${dependencies[@]}" 'mail' ) 1030 | if [[ "x$CONFIG_mail_use_uuencoded_attachments" != 'xyes' ]]; then 1031 | dependencies=( "${dependencies[@]}" 'mutt' ) 1032 | fi 1033 | fi 1034 | 1035 | for i in "${dependencies[@]}"; do 1036 | printf '%s ... ' "$i" 1037 | if type "$i" &>/dev/null; then 1038 | printf 'found.\n' 1039 | else 1040 | printf 'not found. Aborting.\n'; 1041 | let "E |= $E_missing_deps" 1042 | error_handler 1043 | fi 1044 | done 1045 | echo 1046 | } 1047 | 1048 | # @info: Get database list and remove excluded ones. 1049 | # @args: (none) 1050 | # @deps: load_default_config, error_handler 1051 | # 1052 | # alldbnames = array of all databases 1053 | # empty? -> error 1054 | # remove excludes from array alldbnames 1055 | # CONFIG_db_names empty? -> set to alldbnames 1056 | # CONFIG_db_month_names empty? -> set to alldbnames 1057 | # 1058 | parse_databases() { 1059 | # bash 4.x version 1060 | #mapfile -t alldbnames < <(mysql --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" --batch --skip-column-names -e "show databases") 1061 | alldbnames=() 1062 | 1063 | printf "# Parsing databases ... " 1064 | # bash 3.0 1065 | local i;i=0; 1066 | while read -r; do alldbnames[i++]="$REPLY"; done < <(mysql --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${mysql_opt[@]}" --batch --skip-column-names -e "show databases") 1067 | unset i 1068 | 1069 | # mkfifo foo || exit; trap 'rm -f foo' EXIT 1070 | 1071 | ((! "${#alldbnames[@]}" )) && { let "E |= $E_db_empty"; error_handler; } 1072 | 1073 | # -> remove excluded dbs from list 1074 | for exclude in "${CONFIG_db_exclude[@]}"; do 1075 | for i in "${!alldbnames[@]}"; do if [[ "x${alldbnames[$i]}" = "x${exclude}" ]]; then unset 'alldbnames[i]'; fi; done 1076 | done 1077 | # <- remove excluded dbs from list 1078 | 1079 | # check for empty array lists and copy all dbs 1080 | ((! ${#CONFIG_db_names[@]})) && CONFIG_db_names=( "${alldbnames[@]}" ) 1081 | ((! ${#CONFIG_db_month_names[@]})) && CONFIG_db_month_names=( "${alldbnames[@]}" ) 1082 | printf "done.\n" 1083 | } 1084 | 1085 | # @return: true if locked, false otherwise 1086 | # @param: manifest_file 1087 | status_manifest() { 1088 | if [[ -e "$1".lock ]]; then 1089 | return 0 1090 | else 1091 | return 1 1092 | fi 1093 | } 1094 | # @return: true if successfully created lock file, else false 1095 | # @param: manifest_file 1096 | lock_manifest() { 1097 | if status_manifest "$1"; then 1098 | return 0 1099 | else 1100 | if touch "$1".lock &>/dev/null; then 1101 | return 0 1102 | else 1103 | return 1 1104 | fi 1105 | fi 1106 | } 1107 | # @return: true if successfully removed lock file, else false 1108 | # @param: manifest_file 1109 | unlock_manifest() { 1110 | if status_manifest "$1"; then 1111 | if rm "$1".lock &>/dev/null; then 1112 | return 0 1113 | else 1114 | return 1 1115 | fi 1116 | else 1117 | return 0 1118 | fi 1119 | } 1120 | # @return: true if unlock_manifest or lock_manifest, depending on status_manifest, return true, else false 1121 | # @param: manifest_file 1122 | toggle_manifest() { 1123 | if status_manifest "$1"; then 1124 | unlock_manifest "$1" && return 0 || return 1 1125 | else 1126 | lock_manifest "$1" && return 0 || return 1 1127 | fi 1128 | } 1129 | 1130 | # expects manifest_entry_to_check to be an array with four entries 1131 | # @param: manifest_file 1132 | # return: 0, if all is okay 1133 | # 1, file doesn't exist - removed entry from manifest 1134 | # 2, file doesn't exist - tried to remove entry from manifest, but failed 1135 | check_manifest_entry() { 1136 | local entry_md5sum 1137 | [[ ! -e "${manifest_entry_to_check[0]}" ]] && { rm_manifest_entry_by_filename "${manifest_entry_to_check[0]}" 1 && return 1 || return 2; } 1138 | entry_md5sum="$(md5sum "${manifest_entry_to_check[0]}" | awk '{print $1}')" 1139 | if [[ "${entry_md5sum}" != "${manifest_entry_to_check[1]}" ]]; then 1140 | printf 'g/%s/s//%s/g\nw\nq' "${manifest_entry_to_check[1]}" "${entry_md5sum}" | ed -s "$1" 1141 | else 1142 | return 0 1143 | fi 1144 | } 1145 | 1146 | # parse manifest file and collect entries in manifest_array 1147 | # @param: manifest_file 1148 | # 1149 | # sort manifest file after first field (filename) -> read this line by line 1150 | # check if line matches regexp || add to array manifest_entries_corrupted && continue 1151 | # split lines at tab character \t and put into array line_arr 1152 | # check manifest entry 1153 | # -> file does not exist -> remove entry from manifest; continue no matter if this succeeds or not 1154 | # loop through previous entries in the manifest 1155 | # filename already in there? remove all entries with the same filename but the one that is already in the array 1156 | # md5sum has already occured? 1157 | # if size = 0 1158 | # then don't compare 1159 | # else 1160 | # request user action by adding entry to array manifest_entries_user_action_required with information, that identical files exist && continue 2 1161 | # fi 1162 | # add entry to manifest_array 1163 | # 1164 | parse_manifest() { 1165 | local i n re line line_arr check 1166 | unset manifest_array; manifest_array=() 1167 | local tmp_md5sum 1168 | # array ( filename_1, md5sum_1, id_1[, rel_id_1] ), ... ) 1169 | # reserving 4 members for each entry, thus each filename entry in the array has array key 4(n-1)+1 1170 | (( $CONFIG_debug )) && echo ">>>>>>> Parsing manifest file: $1" 1171 | n=1 1172 | [[ -s "$1" ]] && 1173 | while read line 1174 | do 1175 | # ANY CHANGES INSIDE HERE ON THE MANIFEST_FILE HAVE NO IMPACT ON THE LINES WE LOOP OVER; THE sort COMMAND READS THE FILE ENTIRELY AT THE BEGINNING AND PASSES THE OUTPUT TO THE LOOP 1176 | # check if line has expected format, i.e. check against regular expression 1177 | re=$'^[^\t]*\tmd5sum\t[^\t]*\tdiff_id\t[A-Za-z0-9]{8}\trel_id\t(0|[A-Za-z0-9]{8})\tdb\t[^\t]*$' 1178 | [[ $line =~ $re ]] || { echo "Corrupted line: $line"; manifest_entries_corrupted=( "${manifest_entries_corrupted[@]}" "$1" "$line" ); continue; } 1179 | IFS=$'\t' read -ra line_arr <<< "$line" 1180 | 1181 | # prepare array of the current line 1182 | manifest_entry_to_check=() 1183 | for ((i=0;i<${#line_arr[@]};i=$i+2)); do 1184 | manifest_entry_to_check[i/2]="${line_arr[i]}" 1185 | done 1186 | # check manifest entry, which uses the array manifest_entry_to_check 1187 | check_manifest_entry "$1" 1188 | check=$? 1189 | case $check in 1190 | 1) (( $CONFIG_debug )) && echo "File for manifest entry $line does not exist. Entry removed." 1191 | continue # file doesn't exist - removed entry from manifest 1192 | break;; 1193 | 2) (( $CONFIG_debug )) && echo "File for manifest entry $line does not exist. Failed to remove the entry." 1194 | continue # file doesn't exist - tried to remove entry from manifest, but failed 1195 | break;; 1196 | esac 1197 | 1198 | # loop through the manifest_array, as it has been filled by now and check if an entry already exists with the same values 1199 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1200 | if [[ "x${manifest_array[i]}" = "x${line_arr[0]}" ]]; then # found entry with the same filename 1201 | (( $CONFIG_debug )) && echo "Found multiple entries with the same filename. Removing all but the first-found entry from manifest." 1202 | # remove all entries with this filename and add a new one based on the values of the item already in the array 1203 | rm_manifest_entry_by_filename "$1" "${manifest_array[i]}" 1 && add_manifest_entry "$1" "${manifest_array[i]}" "${manifest_array[i+3]}" 1204 | continue 2 # the original entry, to which we compared, is already in the manifest_array; no matter if this is resolved or not, we don't 1205 | # need to add this entry to the manifest_array 1206 | elif [[ "x${manifest_array[i+1]}" = "x${line_arr[2]}" ]]; then # found entry with different filename but same md5sum - file copied and renamed?! 1207 | if [[ ! -s "${line_arr[0]}" ]]; then # empty file - don't start to compare md5sums ... 1208 | (( $CONFIG_debug )) && echo "Found empty file ${line_arr[0]}." 1209 | else 1210 | (( $CONFIG_debug )) && echo "Found multiple entries with the same md5sum but different filename." 1211 | (( $CONFIG_debug )) && echo -e ">> fname_manifest:\t${manifest_array[i]}\t${manifest_array[i+1]}\n>> fname_line:\t\t${line_arr[0]}\t${line_arr[2]}" 1212 | if [[ "x${line_arr[6]}" != "x0" ]]; then 1213 | if [[ "x${manifest_array[i+3]}" = "x${line_arr[6]}" ]]; then # parent id is the same; TODO inform user of this predicament and suggest solution 1214 | manifest_entries_user_action_required=( "${manifest_entries_user_action_required[@]}" "$1" "${manifest_array[i]}" "The file has an identical copy with the same parent id. If you don't know why it exists, it is safe to remove it." ) 1215 | continue 2 1216 | else 1217 | manifest_entries_user_action_required=( "${manifest_entries_user_action_required[@]}" "$1" "${manifest_array[i]}" "The file has an identical copy with different parent id. This should not happen. Remove the file, which is not the correct follow-up to the previous differential or master backup." ) 1218 | continue 2 1219 | fi 1220 | fi 1221 | fi 1222 | fi 1223 | done 1224 | 1225 | # add entry to manifest array 1226 | for ((i=0;i<${#line_arr[@]};i=$i+2)); do 1227 | manifest_array[(n-1)*${fields}+i/2]="${line_arr[i]}" 1228 | #echo "manifest array key $((($n-1)*4+$i/2)) with value ${line_arr[i]}" 1229 | done 1230 | 1231 | ((n++)) 1232 | done < <(sort -t $'\t' -k"1" "$1") 1233 | (( $CONFIG_debug )) && echo "<<<<<<< # manifest entries: $((${#manifest_array[@]}/$fields))" 1234 | (( $CONFIG_debug )) && echo "<<<<<<< FINISHED" 1235 | return 0 1236 | } 1237 | 1238 | # get_manifest_entry_by_* PATTERN [regexp] 1239 | # if second parameter 'regexp' (string!) is passed, PATTERN will be matched as regular expression 1240 | get_manifest_entry_by_filename() { 1241 | local i 1242 | if [[ "x$2" = "xregexp" ]]; then 1243 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1244 | if [[ "${manifest_array[i]}" =~ $1 ]]; then 1245 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1246 | return 0 1247 | break; 1248 | fi 1249 | done 1250 | else 1251 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1252 | if [[ "x${manifest_array[i]}" = "x$1" ]]; then 1253 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1254 | return 0 1255 | break; 1256 | fi 1257 | done 1258 | fi 1259 | return 1 1260 | } 1261 | get_manifest_entry_by_md5sum() { 1262 | local i 1263 | if [[ "x$2" = "xregexp" ]]; then 1264 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1265 | if [[ "${manifest_array[i+1]}" =~ $1 ]]; then 1266 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1267 | return 0 1268 | break; 1269 | fi 1270 | done 1271 | else 1272 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1273 | if [[ "x${manifest_array[i+1]}" = "x$1" ]]; then 1274 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1275 | return 0 1276 | break; 1277 | fi 1278 | done 1279 | fi 1280 | return 1 1281 | } 1282 | get_manifest_entry_by_id() { 1283 | local i 1284 | if [[ "x$2" = "xregexp" ]]; then 1285 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1286 | if [[ "${manifest_array[i+2]}" =~ $1 ]]; then 1287 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1288 | return 0 1289 | break; 1290 | fi 1291 | done 1292 | else 1293 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1294 | if [[ "x${manifest_array[i+2]}" = "x$1" ]]; then 1295 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1296 | return 0 1297 | break; 1298 | fi 1299 | done 1300 | fi 1301 | return 1 1302 | } 1303 | get_manifest_entry_by_rel_id() { 1304 | local i 1305 | if [[ "x$2" = "xregexp" ]]; then 1306 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1307 | if [[ "${manifest_array[i+3]}" =~ $1 ]]; then 1308 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1309 | return 0 1310 | break; 1311 | fi 1312 | done 1313 | else 1314 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1315 | if [[ "x${manifest_array[i+3]}" = "x$1" ]]; then 1316 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1317 | return 0 1318 | break; 1319 | fi 1320 | done 1321 | fi 1322 | return 1 1323 | } 1324 | get_manifest_entry_by_db() { 1325 | local i 1326 | if [[ "x$2" = "xregexp" ]]; then 1327 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1328 | if [[ "${manifest_array[i+4]}" =~ $1 ]]; then 1329 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1330 | return 0 1331 | break; 1332 | fi 1333 | done 1334 | else 1335 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1336 | if [[ "x${manifest_array[i+4]}" = "x$1" ]]; then 1337 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1338 | return 0 1339 | break; 1340 | fi 1341 | done 1342 | fi 1343 | return 1 1344 | } 1345 | 1346 | # @params: db, master/diff (0,1) 1347 | # @return: 2: no entries in manifest for the specified database 'db' 1348 | # 1: could not get manifest element by filename 1349 | # 0: all fine, match is in array 'manifest_entry' 1350 | get_latest_manifest_entry_for_db() { 1351 | local db_array newarray i 1352 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1353 | if (( $2 )); then # latest differential or master backup, i.e. just take the latest one! 1354 | if [[ "x${manifest_array[i+4]}" = "x$1" ]]; then 1355 | db_array=( "${db_array[@]}" "${manifest_array[i]}" ) 1356 | fi 1357 | else # latest master backup, pid=0 1358 | if [[ "x${manifest_array[i+4]}" = "x$1" && "x${manifest_array[i+3]}" = "x0" ]]; then 1359 | db_array=( "${db_array[@]}" "${manifest_array[i]}") 1360 | fi 1361 | fi 1362 | done 1363 | if (( "${#db_array[@]}" == 0 )); then return 2; 1364 | else 1365 | #newarray=(); while IFS= read -r -d '' line; do newarray+=("$line"); done < <(printf '%s\0' "${db_array[@]}" | sort -z) 1366 | get_manifest_entry_by_filename "${db_array[@]:(-1)}" # last entry of db_array, has, due to the way sort works, to be the latest one 1367 | return $? 1368 | fi 1369 | } 1370 | 1371 | # @params: manifest_file filename/md5sum/id/rel_id [1(=don't parse manifest after finished)] 1372 | # if second parameters 1373 | # 1374 | # lock manifest -> use awk, print all lines that don't have second parameter at the appropriate field -> unlock manifest 1375 | # param3=0 -> parse manifest 1376 | # 1377 | rm_manifest_entry_by_filename() { 1378 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$1 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1379 | (( "$3" )) || parse_manifest "$1" 1380 | return 0 1381 | } 1382 | rm_manifest_entry_by_md5sum() { 1383 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$3 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1384 | (( "$3" )) || parse_manifest "$1" 1385 | return 0 1386 | } 1387 | rm_manifest_entry_by_id() { 1388 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$5 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1389 | (( "$3" )) || parse_manifest "$1" 1390 | return 0 1391 | } 1392 | rm_manifest_entry_by_rel_id() { 1393 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$7 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1394 | (( "$3" )) || parse_manifest "$1" 1395 | return 0 1396 | } 1397 | rm_manifest_entry_by_db() { 1398 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$9 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1399 | (( "$3" )) || parse_manifest "$1" 1400 | return 0 1401 | } 1402 | 1403 | # parameters: manifest_file, filename, parent_id, db 1404 | add_manifest_entry() { 1405 | local md5sum 1406 | local id 1407 | local filename 1408 | local parent_id 1409 | local db 1410 | filename="$2" 1411 | parent_id="$3" 1412 | db="$4" 1413 | lock_manifest "$1" || return 1 1414 | id="${filename%.@(diff|sql)*}" 1415 | id="${id:(-8):8}" 1416 | #id="$(echo $filename | sed -re 's/.*_[0-9]{2}h[0-9]{2}m_([^\.]*)\..*/\1/')" 1417 | md5sum="$(md5sum "$filename" | awk '{print $1}')" 1418 | if [[ "x$parent_id" = 'x' ]]; then 1419 | echo -e "${filename}\tmd5sum\t${md5sum}\tdiff_id\t${id}\trel_id\t0\tdb\t${db}" >> "$1" 1420 | else 1421 | echo -e "${filename}\tmd5sum\t${md5sum}\tdiff_id\t${id}\trel_id\t${parent_id}\tdb\t${db}" >> "$1" 1422 | fi 1423 | unlock_manifest "$1" || return 1 1424 | } 1425 | 1426 | # @info: Echos number of manifest entries. 1427 | num_manifest_entries() { 1428 | echo "$((${#manifest_array[@]}/$fields))" 1429 | } 1430 | 1431 | # @info: Test if a value is in the array testarray 1432 | # @param: value 1433 | # @var in_array_index: array index of the first match 1434 | # @return 0 if a match was found, otherwise 1 1435 | in_array() { 1436 | local j 1437 | for ((j=0;j<"${#testarray[@]}";j++)); do 1438 | if [[ "x${testarray[j]}" = "x$1" ]]; then 1439 | in_array_index=$j 1440 | return 0 1441 | fi 1442 | done 1443 | return 1 1444 | } 1445 | 1446 | # @param: clear(0/1), meta_information, list_value1, list_value2, ... 1447 | extended_select() { 1448 | local a c k m i r r_number meta_information choice selection do_clear 1449 | meta_information="$2" 1450 | do_clear="$1" 1451 | shift 2 1452 | declare -a list=("$@") 1453 | selection=() 1454 | # BEGIN _select_filenames 1455 | #tput sc 1456 | while true; do 1457 | if (( $do_clear )); then 1458 | clear 1459 | else 1460 | : #tput rc 1461 | fi 1462 | declare -a testarray=("${selection[@]}") 1463 | echo "Selection for <$meta_information>" 1464 | echo "Notation: 1,2-4,-5,-6-9 or * or -* ('-' will remove selections)." 1465 | # print options 1466 | 1467 | for ((i=0;i<"${#list[@]}";i++)); do 1468 | if in_array $i; then 1469 | echo -e "$i) [+]\t${list[i]}" 1470 | else 1471 | echo -e "$i) [ ]\t${list[i]}" 1472 | fi 1473 | done 1474 | 1475 | echo -e "$i)\tDONE" 1476 | done_id=$i 1477 | 1478 | min=0 1479 | max=${#list[@]} # we have to account for the last possible number of DONE 1480 | # evaluate response 1481 | while true; do 1482 | printf '#? ' 1483 | read choice 1484 | r='^((-?[0-9]+(-[0-9]+)?,)*-?[0-9]+(-[0-9]+)?|-?\*)$' 1485 | [[ $choice =~ $r ]] || continue 1486 | if [[ "x$choice" = 'x*' ]]; then 1487 | unset m 1488 | for ((m=0;m<"${#list[@]}";m++)); do 1489 | selection=("${selection[@]}" "$m") 1490 | done 1491 | continue 2 1492 | elif [[ "x$choice" = 'x-*' ]]; then 1493 | selection=() 1494 | continue 2 1495 | else 1496 | unset string num1 num2 op op_rm 1497 | r_number='^[0-9]$' 1498 | 1499 | # BEGIN process_choice 1500 | for ((a=0;a<${#choice};a++)) 1501 | do 1502 | c="${choice:a:1}" 1503 | declare -a testarray=("${selection[@]}") 1504 | 1505 | if (( ${#string} == 0 )) && [[ "x$c" = "x-" ]] && ! (($op)); then 1506 | op_rm=1 1507 | continue 1508 | elif [[ $c =~ $r ]]; then 1509 | string=${string}"$c" 1510 | if (( $a == (${#choice}-1) )); then # last character 1511 | # we have a A-B case 1512 | if (($op)); then 1513 | num2="$string" 1514 | unset k 1515 | for ((k=$num1;k<=$num2;k++)); do 1516 | (( $k >= $min )) && (( $k <= $max )) || continue 1517 | if ! in_array $k; then 1518 | selection=("${selection[@]}" $k) 1519 | else 1520 | if (( $op_rm )); then 1521 | new_array=() 1522 | for ((m="$((${#selection[@]}-1))";m>=0;m--)); do 1523 | if [[ "x${selection[m]}" != "x$k" ]]; then 1524 | new_array=("${new_array[@]}" "${selection[m]}") 1525 | fi 1526 | done 1527 | declare -a selection=("${new_array[@]}") 1528 | fi 1529 | fi 1530 | done 1531 | unset op op_rm num1 num2 string 1532 | continue 1533 | else 1534 | (( $string >= $min )) && (( $string <= $max )) || continue 1535 | if ! in_array "$string"; then 1536 | selection=("${selection[@]}" "$string") 1537 | else 1538 | if (($op_rm)); then 1539 | unset m 1540 | new_array=() 1541 | for ((m=0;m<"${#selection[@]}";m++)); do 1542 | if [[ "x${selection[m]}" != "x$string" ]]; then 1543 | new_array=("${new_array[@]}" "${selection[m]}") 1544 | fi 1545 | done 1546 | declare -a selection=("${new_array[@]}") 1547 | fi 1548 | fi 1549 | fi 1550 | else 1551 | continue 1552 | fi 1553 | elif [[ "x$c" = "x-" ]]; then 1554 | num1="$string" 1555 | unset string 1556 | op=1 1557 | if (( $a == (${#choice}-1) )); then 1558 | break 1559 | else 1560 | continue 1561 | fi 1562 | elif [[ "x$c" = "x," ]]; then 1563 | # we have a A-B case 1564 | if (($op)); then 1565 | num2="$string" 1566 | unset k 1567 | for ((k=$num1;k<=$num2;k++)); do 1568 | (( $k >= $min )) && (( $k <= $max )) || continue 1569 | if ! in_array $k; then 1570 | selection=("${selection[@]}" $k) 1571 | else 1572 | if (( $op_rm )); then 1573 | unset m 1574 | new_array=() 1575 | for ((m=0;m<"${#selection[@]}";m++)); do 1576 | if [[ "x${selection[m]}" != "x$k" ]]; then 1577 | new_array=("${new_array[@]}" "${selection[m]}") 1578 | fi 1579 | done 1580 | declare -a selection=("${new_array[@]}") 1581 | fi 1582 | fi 1583 | done 1584 | unset op op_rm num1 num2 string 1585 | continue 1586 | else # it's just a single number 1587 | (( $string >= $min )) && (( $string <= $max )) || { unset op op_rm num1 num2 string; continue; } 1588 | if ! in_array "$string"; then 1589 | selection=("${selection[@]}" "$string") 1590 | else 1591 | if (($op_rm)); then 1592 | unset m 1593 | new_array=() 1594 | for ((m=0;m<"${#selection[@]}";m++)); do 1595 | if [[ "x${selection[m]}" != "x$string" ]]; then 1596 | new_array=("${new_array[@]}" "${selection[m]}") 1597 | fi 1598 | done 1599 | declare -a selection=("${new_array[@]}") 1600 | fi 1601 | fi 1602 | unset op op_rm num1 num2 string 1603 | continue 1604 | fi 1605 | else 1606 | continue 2; # this should not happen 1607 | fi 1608 | done 1609 | # END process_choice 1610 | 1611 | declare -a testarray=("${selection[@]}") 1612 | if in_array "$done_id"; then 1613 | break 2 1614 | else 1615 | continue 2 1616 | fi 1617 | fi 1618 | done 1619 | done 1620 | extended_select_return=() 1621 | extended_select_return_id=() 1622 | for i in "${selection[@]}"; do 1623 | [[ "x$i" != "x$done_id" ]] && { extended_select_return=("${extended_select_return[@]}" "${list[i]}"); extended_select_return_id=("${extended_select_return_id[@]}" "$i"); } 1624 | done 1625 | 1626 | } 1627 | 1628 | # END _functions 1629 | 1630 | 1631 | # BEGIN _methods 1632 | 1633 | # @info: Backup method 1634 | method_backup () { 1635 | manifest_entries_corrupted=() 1636 | manifest_entries_user_action_required=() 1637 | 1638 | 1639 | # END __FUNCTIONS 1640 | ############################################################################################################## 1641 | # BEGIN __STARTUP 1642 | 1643 | load_default_config 1644 | 1645 | trap mail_cleanup EXIT SIGHUP SIGINT SIGQUIT SIGTERM 1646 | if [[ -r "${CONFIG_configfile}" ]]; then source "${CONFIG_configfile}"; echo "Parsed config file \"${CONFIG_configfile}\""; else let "N |= $N_config_file_missing"; fi; echo 1647 | if (( $opt_flag_config_file )); then if [[ -r "${opt_config_file}" ]]; then source "${opt_config_file}"; let "N |= $N_arg_conffile_parsed"; else let "N |= $N_arg_conffile_unreadable"; fi; else let "N |= $N_too_many_args"; fi 1648 | 1649 | (( $CONFIG_dryrun )) && { 1650 | echo "NOTE: We are dry-running. That means, that the script just shows you what it would do, if it were operating normally." 1651 | echo "THE PRINTED COMMANDS CAN'T BE COPIED AND EXECUTED IF THERE ARE SPECIAL CHARACTERS, SPACES, ETC. IN THERE THAT WOULD NEED TO BE PROPERLY QUOTED IN ORDER TO WORK. THESE WERE CORRECTLY QUOTED FOR THE OUTPUT COMMAND, BUT CAN'T BE SEEN NOW." 1652 | echo 1653 | } 1654 | 1655 | export LC_ALL=C 1656 | PROGNAME=`basename $0` 1657 | PATH=${PATH}:/usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 1658 | version=3.0 1659 | fields=5 # manifest fields 1660 | 1661 | directory_checks_enable_logging 1662 | cleanup_latest 1663 | set_datetime_vars 1664 | check_dependencies # check for required programs 1665 | parse_configuration # parse configuration and set variables appropriately 1666 | 1667 | 1668 | # END __STARTUP 1669 | #-------------------------------------------------------------------------------------------------------------------------------------- 1670 | # BEGIN __PREPARE 1671 | 1672 | backupfiles=() 1673 | parse_databases 1674 | 1675 | # debug output of variables 1676 | (( $CONFIG_debug )) && { echo; echo "# DEBUG: printing all current variables"; declare -p | egrep -o '.* (CONFIG_[a-z_]*|opt|mysql_opt|opt_dbstatus|opt_fullschema)=.*'; echo; } 1677 | (( $CONFIG_debug )) && { echo "DEBUG: before pre-backup"; ( IFS=,; echo "DEBUG: CONFIG_db_names '${CONFIG_db_names[*]}'" ); ( IFS=,; echo "DEBUG: CONFIG_db_month_names '${CONFIG_db_month_names[*]}'" );} 1678 | 1679 | 1680 | # END __PREPARE 1681 | #-------------------------------------------------------------------------------------------------------------------------------------- 1682 | # BEGIN __MAIN 1683 | 1684 | ### filename formats 1685 | ## 1686 | ## example date values: 1687 | # 14'th of August (08) 2011 1688 | # week number: 32 1689 | # Sunday (date_dayno_of_week: 7) 1690 | ## 1691 | ## separate db's: 1692 | # monthly_DBNAME_2011-08-14_18h12m_August.sql(.enc).{gz,bzip2} 1693 | # weekly_DBNAME_2011-08-14_18h12m_32.sql(.enc).{gz,bzip2} 1694 | # daily_DBNAME_2011-08-14_18h12m_7.sql(.enc).{gz,bzip2} 1695 | ## all-databases: 1696 | # monthly_all-databases_DBNAME_2011-08-14_18h12m_August.sql(.enc).{gz,bzip2} 1697 | # weekly_all-databases_DBNAME_2011-08-14_18h12m_32.sql(.enc).{gz,bzip2} 1698 | # daily_all-databases_DBNAME_2011-08-14_18h12m_7.sql(.enc).{gz,bzip2} 1699 | 1700 | echo "======================================================================" 1701 | echo "AutoMySQLBackup version ${version}" 1702 | echo "http://sourceforge.net/projects/automysqlbackup/" 1703 | echo 1704 | echo "Backup of Database Server - ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host}" 1705 | ( IFS=,; echo "Databases - ${CONFIG_db_names[*]}" ) 1706 | ( IFS=,; echo "Databases (monthly) - ${CONFIG_db_month_names[*]}" ) 1707 | echo "======================================================================" 1708 | 1709 | 1710 | # -> preback commands 1711 | if [[ "${CONFIG_prebackup}" ]]; then 1712 | echo "======================================================================" 1713 | echo "Prebackup command output." 1714 | echo 1715 | source ${CONFIG_prebackup} 1716 | echo 1717 | echo "======================================================================" 1718 | echo 1719 | fi 1720 | # <- preback commands 1721 | 1722 | # -> backup local files 1723 | if [[ "${CONFIG_backup_local_files[@]}" ]] && [[ ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} = ${CONFIG_do_weekly} ]] && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/backup_local_files/bcf_weekly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_weekno}.tar${suffix}"); ((! ${#f[@]}))); then 1724 | echo "======================================================================" 1725 | echo "Backup local files. Doing this weekly on CONFIG_do_weekly." 1726 | echo 1727 | backup_local_files "${CONFIG_backup_dir}/backup_local_files/bcf_weekly_${datetimestamp}_${date_weekno}.tar" 1728 | tmp_flags=$?; var=; 1729 | if (( $? == 0 )); then 1730 | echo "success!" 1731 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/backup_local_files/bcf_weekly_${datetimestamp}_${date_weekno}.tar" ) 1732 | else 1733 | let "E |= $E_backup_local_failed" 1734 | echo "failed!" 1735 | fi 1736 | echo 1737 | echo "======================================================================" 1738 | echo 1739 | fi 1740 | # <- backup local files 1741 | 1742 | # -> dump full schema 1743 | if [[ "${CONFIG_mysql_dump_full_schema}" = 'yes' ]]; then 1744 | echo "======================================================================" 1745 | echo "Dump full schema." 1746 | echo 1747 | 1748 | # monthly 1749 | if (( ${CONFIG_do_monthly} != 0 && (${date_day_of_month} == ${CONFIG_do_monthly} || $date_day_of_month == $date_lastday_of_this_month && $date_lastday_of_this_month < ${CONFIG_do_monthly}) )) && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/fullschema/fullschema_monthly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_month}.sql${suffix}"); ((! ${#f[@]}))); then 1750 | fullschema "${CONFIG_backup_dir}/fullschema/fullschema_monthly_${datetimestamp}_${date_month}.sql" 1751 | if (( $? == 0 )); then 1752 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1753 | if (( $CONFIG_dryrun )); then 1754 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'fullschema_monthly*' -exec echo "dry-running: rm" {} \; 1755 | else 1756 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'fullschema_monthly*' -exec rm {} \; 1757 | fi 1758 | files_postprocessing "${CONFIG_backup_dir}/fullschema/fullschema_monthly_${datetimestamp}_${date_month}.sql${suffix}" 1759 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1760 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/fullschema/fullschema_monthly_${datetimestamp}_${date_month}.sql${suffix}${var}" ) 1761 | else 1762 | let "E |= $E_dump_fullschema_failed" 1763 | fi 1764 | fi 1765 | 1766 | # weekly 1767 | if [[ ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} = ${CONFIG_do_weekly} ]] && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/fullschema/fullschema_weekly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_weekno}.sql${suffix}"); ((! ${#f[@]}))); then 1768 | fullschema "${CONFIG_backup_dir}/fullschema/fullschema_weekly_${datetimestamp}_${date_weekno}.sql" 1769 | if (( $? == 0 )); then 1770 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1771 | if (( $CONFIG_dryrun )); then 1772 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'fullschema_weekly*' -exec echo "dry-running: rm" {} \; 1773 | else 1774 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'fullschema_weekly*' -exec rm {} \; 1775 | fi 1776 | files_postprocessing "${CONFIG_backup_dir}/fullschema/fullschema_weekly_${datetimestamp}_${date_weekno}.sql${suffix}" 1777 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1778 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/fullschema/fullschema_weekly_${datetimestamp}_${date_weekno}.sql${suffix}${var}" ) 1779 | else 1780 | let "E |= $E_dump_fullschema_failed" 1781 | fi 1782 | fi 1783 | 1784 | # daily 1785 | fullschema "${CONFIG_backup_dir}/fullschema/fullschema_daily_${datetimestamp}_${date_day_of_week}.sql" 1786 | if (( $? == 0 )); then 1787 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1788 | if (( $CONFIG_dryrun )); then 1789 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_daily}" -type f -name 'fullschema_daily*' -exec echo "dry-running: rm" {} \; 1790 | else 1791 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_daily}" -type f -name 'fullschema_daily*' -exec rm {} \; 1792 | fi 1793 | files_postprocessing "${CONFIG_backup_dir}/fullschema/fullschema_daily_${datetimestamp}_${date_day_of_week}.sql${suffix}" 1794 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1795 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/fullschema/fullschema_daily_${datetimestamp}_${date_day_of_week}.sql${suffix}${var}" ) 1796 | else 1797 | let "E |= $E_dump_fullschema_failed" 1798 | fi 1799 | echo 1800 | echo "======================================================================" 1801 | echo 1802 | 1803 | fi 1804 | # <- dump full schema 1805 | 1806 | # -> dump status 1807 | if [[ "${CONFIG_mysql_dump_dbstatus}" = 'yes' ]]; then 1808 | echo "======================================================================" 1809 | echo "Dump status." 1810 | echo 1811 | 1812 | # monthly 1813 | if (( ${CONFIG_do_monthly} != 0 && (${date_day_of_month} == ${CONFIG_do_monthly} || $date_day_of_month == $date_lastday_of_this_month && $date_lastday_of_this_month < ${CONFIG_do_monthly}) )) && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/status/status_monthly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_month}.txt${suffix}"); ((! ${#f[@]}))); then 1814 | dbstatus "${CONFIG_backup_dir}/status/status_monthly_${datetimestamp}_${date_month}.txt" 1815 | if (( $? == 0 )); then 1816 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1817 | if (( $CONFIG_dryrun )); then 1818 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'status_monthly*' -exec echo "dry-running: rm" {} \; 1819 | else 1820 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'status_monthly*' -exec rm {} \; 1821 | fi 1822 | files_postprocessing "${CONFIG_backup_dir}/status/status_monthly_${datetimestamp}_${date_month}.txt${suffix}" 1823 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1824 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/status/status_monthly_${datetimestamp}_${date_month}.txt${suffix}${var}" ) 1825 | else 1826 | let "E |= $E_dump_status_failed" 1827 | fi 1828 | fi 1829 | 1830 | # weekly 1831 | if [[ ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} = ${CONFIG_do_weekly} ]] && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/status/status_weekly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_weekno}.txt${suffix}"); ((! ${#f[@]}))); then 1832 | dbstatus "${CONFIG_backup_dir}/status/status_weekly_${datetimestamp}_${date_weekno}.txt" 1833 | if (( $? == 0 )); then 1834 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1835 | if (( $CONFIG_dryrun )); then 1836 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'status_weekly*' -exec echo "dry-running: rm" {} \; 1837 | else 1838 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'status_weekly*' -exec rm {} \; 1839 | fi 1840 | files_postprocessing "${CONFIG_backup_dir}/status/status_weekly_${datetimestamp}_${date_weekno}.txt${suffix}" 1841 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1842 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/status/status_weekly_${datetimestamp}_${date_weekno}.txt${suffix}${var}" ) 1843 | else 1844 | let "E |= $E_dump_status_failed" 1845 | fi 1846 | fi 1847 | 1848 | # daily 1849 | dbstatus "${CONFIG_backup_dir}/status/status_daily_${datetimestamp}_${date_day_of_week}.txt" 1850 | if (( $? == 0 )); then 1851 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1852 | if (( $CONFIG_dryrun )); then 1853 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_daily}" -type f -name 'status_daily*' -exec echo "dry-running: rm" {} \; 1854 | else 1855 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_daily}" -type f -name 'status_daily*' -exec rm {} \; 1856 | fi 1857 | files_postprocessing "${CONFIG_backup_dir}/status/status_daily_${datetimestamp}_${date_day_of_week}.txt${suffix}" 1858 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1859 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/status/status_daily_${datetimestamp}_${date_day_of_week}.txt${suffix}${var}" ) 1860 | else 1861 | let "E |= $E_dump_status_failed" 1862 | fi 1863 | echo 1864 | echo "======================================================================" 1865 | echo 1866 | 1867 | fi 1868 | # <- dump status 1869 | 1870 | 1871 | # -> BACKUP DATABASES 1872 | echo "Backup Start Time `date`" 1873 | echo "======================================================================" 1874 | 1875 | ## <- monthly backup, unique per month 1876 | if (( ${CONFIG_do_monthly} != 0 && (${date_day_of_month} == ${CONFIG_do_monthly} || $date_day_of_month == $date_lastday_of_this_month && $date_lastday_of_this_month < ${CONFIG_do_monthly}) )); then 1877 | echo "Monthly Backup ..." 1878 | echo 1879 | 1880 | subfolder="monthly" 1881 | prefix="monthly_" 1882 | midfix="_${date_month}" 1883 | extension=".sql" 1884 | rotation="${CONFIG_rotation_monthly}" 1885 | rotation_divisor="31" 1886 | rotation_string="month" 1887 | 1888 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 1889 | for db in "${CONFIG_db_month_names[@]}"; do 1890 | echo "Monthly Backup of Database ( ${db} )" 1891 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${db}/${prefix}${db}_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && continue 1892 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 0 "$db" 1893 | echo ---------------------------------------------------------------------- 1894 | done 1895 | else 1896 | echo "Monthly backup of databases ( ${CONFIG_db_month_names[@]} )." 1897 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${prefix}all-databases_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && 1898 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 1 "${CONFIG_db_month_names[@]}" 1899 | echo "----------------------------------------------------------------------" 1900 | fi 1901 | fi 1902 | ## <- monthly backup 1903 | 1904 | ## <- weekly backup, unique per week 1905 | if (( ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} == ${CONFIG_do_weekly} )); then 1906 | echo "Weekly Backup ..." 1907 | echo 1908 | 1909 | subfolder="weekly" 1910 | prefix="weekly_" 1911 | midfix="_${date_weekno}" 1912 | extension=".sql" 1913 | rotation="${CONFIG_rotation_weekly}" 1914 | rotation_divisor="7" 1915 | rotation_string="week" 1916 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 1917 | for db in "${CONFIG_db_names[@]}"; do 1918 | echo "Weekly Backup of Database ( ${db} )" 1919 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${db}/${prefix}${db}_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && continue 1920 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 0 "$db" 1921 | echo "----------------------------------------------------------------------" 1922 | done 1923 | else 1924 | echo "Weekly backup of databases ( ${CONFIG_db_names[@]} )." 1925 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${prefix}all-databases_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && 1926 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 1 "${CONFIG_db_names[@]}" 1927 | echo "----------------------------------------------------------------------" 1928 | fi 1929 | fi 1930 | ## <- weekly backup 1931 | 1932 | ## -> daily backup, test (( 1 )) is always true, just creates a grouping for Kate, which can be closed ^^ 1933 | if (( 1 )); then 1934 | echo "Daily Backup ..." 1935 | echo 1936 | 1937 | subfolder="daily" 1938 | prefix="daily_" 1939 | midfix="_${date_day_of_week}" 1940 | extension=".sql" 1941 | rotation="${CONFIG_rotation_daily}" 1942 | rotation_divisor="1" 1943 | rotation_string="day" 1944 | 1945 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 1946 | for db in "${CONFIG_db_names[@]}"; do 1947 | echo "Daily Backup of Database ( ${db} )" 1948 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 0 "$db" 1949 | echo "----------------------------------------------------------------------" 1950 | done 1951 | else 1952 | echo "Daily backup of databases ( ${CONFIG_db_names[@]} )." 1953 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 1 "${CONFIG_db_names[@]}" 1954 | echo "----------------------------------------------------------------------" 1955 | fi 1956 | fi 1957 | ## <- daily backup 1958 | 1959 | echo 1960 | echo "Backup End Time `date`" 1961 | echo "======================================================================" 1962 | # <- BACKUP DATABASES 1963 | 1964 | 1965 | # -> clean latest filenames 1966 | [[ "${CONFIG_mysql_dump_latest_clean_filenames}" = 'yes' ]] && find "${CONFIG_backup_dir}"/latest/ -type f -exec bash -c 'remove_datetimeinfo "$@"' -- {} \; 1967 | # <- clean latest filenames 1968 | 1969 | # -> finished information 1970 | echo "Total disk space used for backup storage..." 1971 | echo "Size - Location" 1972 | echo `du -hsH "${CONFIG_backup_dir}"` 1973 | echo 1974 | echo "======================================================================" 1975 | # <- finished information 1976 | 1977 | # -> postbackup commands 1978 | if [[ "${CONFIG_postbackup}" ]];then 1979 | echo "======================================================================" 1980 | echo "Postbackup command output." 1981 | echo 1982 | source ${CONFIG_postbackup} 1983 | echo 1984 | echo "======================================================================" 1985 | fi 1986 | # <- postbackup commands 1987 | 1988 | if [[ -s "$log_errfile" ]];then status=1; else status=0; fi 1989 | 1990 | exit ${status} 1991 | 1992 | } 1993 | 1994 | # @return variable method_list_manifest_entries_array 1995 | method_list_manifest_entries () { 1996 | local files files_master files_manifest file db manifest_files manifest_files_db selected_dbs i z l master_flags master to_rm actions 1997 | manifest_entries_corrupted=() 1998 | manifest_entries_user_action_required=() 1999 | files=() 2000 | files_master=() 2001 | files_manifest=() 2002 | master_flags=0 2003 | let "filename_flag_encrypted=0x01" 2004 | let "filename_flag_gz=0x02" 2005 | let "filename_flag_bz2=0x04" 2006 | let "filename_flag_diff=0x08" 2007 | 2008 | ############################################################################################################## 2009 | # BEGIN __STARTUP 2010 | 2011 | load_default_config 2012 | 2013 | if [[ -r "${CONFIG_configfile}" ]]; then source "${CONFIG_configfile}"; echo "Parsed config file \"${CONFIG_configfile}\""; else let "N |= $N_config_file_missing"; fi; echo 2014 | if (( $opt_flag_config_file )); then if [[ -r "${opt_config_file}" ]]; then source "${opt_config_file}"; let "N |= $N_arg_conffile_parsed"; else let "N |= $N_arg_conffile_unreadable"; fi; else let "N |= $N_too_many_args"; fi 2015 | 2016 | export LC_ALL=C 2017 | PROGNAME=`basename $0` 2018 | PATH=${PATH}:/usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 2019 | version=3.0 2020 | fields=5 # manifest fields 2021 | 2022 | set_datetime_vars 2023 | check_dependencies # check for required programs 2024 | parse_configuration # parse configuration and set variables appropriately 2025 | 2026 | # BEGIN __MAIN 2027 | unset manifest_files manifest_files_db i db 2028 | while IFS= read -r -d '' file; do 2029 | db="${file#/var/backup/db/@(daily|monthly|weekly|latest)/}"; 2030 | db="${db%/Manifest}"; 2031 | manifest_files_db[i]="$db" 2032 | manifest_files[i++]="$file" 2033 | done < <(find "${CONFIG_backup_dir}"/ -type f -name 'Manifest' -print0) 2034 | 2035 | extended_select 0 "Databases" "${manifest_files_db[@]}" 2036 | declare -a selected_dbs=("${extended_select_return[@]}") 2037 | 2038 | for db in "${selected_dbs[@]}"; do 2039 | selected_available_files=() 2040 | for ((i=0;i<"${#manifest_files_db[@]}";i++)); do 2041 | if [[ "x${manifest_files_db[i]}" = "x$db" ]]; then 2042 | selected_available_files[j++]="${manifest_files[i]}" 2043 | fi 2044 | done 2045 | if (( "${#selected_available_files[@]}" > 0 )); then 2046 | extended_select 1 "$db" "${selected_available_files[@]}" 2047 | declare -a selected_entries=("${extended_select_return[@]}") 2048 | if (( "${#selected_entries[@]}" > 0 )); then 2049 | for z in "${selected_entries[@]}"; do 2050 | parse_manifest "$z" 2051 | list=() 2052 | list_id=() 2053 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 2054 | if [[ "${manifest_array[i+3]}" != 0 ]]; then # only add differential backups 2055 | list=("${list[@]}" "${manifest_array[i]}") 2056 | list_id=("${list_id[@]}" "${manifest_array[i+3]}") # save rel_id, so we can retrieve the master backup file 2057 | fi 2058 | done 2059 | if (( "${#list[@]}" > 0 )); then 2060 | extended_select 1 "$z" "${list[@]}" 2061 | if (( "${#extended_select_return[@]}" > 0 )); then 2062 | for ((i=0;$i<"${#extended_select_return[@]}";i++)); do 2063 | if get_manifest_entry_by_id "${list_id[${extended_select_return_id[i]}]}"; then 2064 | files=("${files[@]}" "${extended_select_return[i]}") 2065 | files_master=("${files_master[@]}" "${manifest_entry[0]}") 2066 | files_manifest=("${files_manifest[@]}" "$z") 2067 | else 2068 | echo "no found master for id ${list_id[${extended_select_return_id[i]}]}" 2069 | fi 2070 | done 2071 | fi 2072 | fi 2073 | done 2074 | fi 2075 | fi 2076 | done 2077 | # END _select_filenames 2078 | declare -a method_list_manifest_entries_array=("${files[@]}") 2079 | declare -a method_list_manifest_entries_array_master=("${files_master[@]}") 2080 | declare -a method_list_manifest_entries_array_manifest=("${files_manifest[@]}") 2081 | 2082 | clear 2083 | echo "You have selected the following files:" 2084 | for i in "${files[@]}"; do printf '>>> %s\n' "$i"; done 2085 | echo 2086 | actions=('diff to full' 'remove files (also from Manifest)') 2087 | extended_select 0 "Actions" "${actions[@]}" 2088 | for action in "${extended_select_return[@]}"; do 2089 | case "$action" in 2090 | 'diff to full') 2091 | for ((l=0;$l<"${#files[@]}";l++)); do 2092 | # put the unpacking of the master file in here, so that in the case of multiple diffs with the same 2093 | # master file don't cause the script to unpack the same master file multiple times 2094 | master="${files_master[l]}" 2095 | diff="${files[l]}" 2096 | FileStub="${master%.@(sql|master)*}" 2097 | FileExt="${master#"$FileStub"}" 2098 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_encrypted" 2099 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_gz" 2100 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_bz2" 2101 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_diff" 2102 | if (( $master_flags & $filename_flag_gz )); then 2103 | declare -a testarray=("${to_rm[@]}") 2104 | if ! in_array "${master%.gz}"; then 2105 | gzip_compression -dc "$master" > "${master%.gz}" 2106 | to_rm=("${to_rm[@]}" "${master%.gz}") 2107 | fi 2108 | master="${master%.gz}" 2109 | elif (( $master_flags & $filename_flag_bz2 )); then 2110 | declare -a testarray=("${to_rm[@]}") 2111 | if ! in_array "${master%.bz2}"; then 2112 | bzip2_compression -dc "$master" > "${master%.bz2}" 2113 | to_rm=("${to_rm[@]}" "${master%.bz2}") 2114 | fi 2115 | master="${master%.bz2}" 2116 | else 2117 | : 2118 | fi 2119 | method_diff_to_full "$master" "$diff" 2120 | #printf '%s\n>>> master: %s\n>>> manifest: %s\n' "${files[l]}" "${files_master[l]}" "${files_manifest[l]}" 2121 | done 2122 | # cleanup all unpacked master files ... the unpacked diff files are cleaned up by method_diff_to_full 2123 | for i in "${to_rm[@]}"; do rm "$i"; done 2124 | ;; 2125 | 'remove files (also from Manifest)') 2126 | for ((l=0;$l<"${#files[@]}";l++)); do 2127 | if rm_manifest_entry_by_filename "${files_manifest[l]}" "${files[l]}" 1; then 2128 | rm "${files[l]}" 2129 | fi 2130 | done 2131 | ;; 2132 | *) 2133 | echo "Unrecognized option. This Should not happen! Error!" 2134 | ;; 2135 | esac 2136 | done 2137 | 2138 | # END __MAIN 2139 | } 2140 | 2141 | # @info: Convert a differential backup file to a full one. 2142 | # @param: master_backup_file diff_backup_file 2143 | method_diff_to_full() { 2144 | local diff full diff_flags master_flags to_rm 2145 | master="$1" 2146 | diff="$2" 2147 | diff_flags=0 2148 | master_flags=0 2149 | to_rm=() 2150 | 2151 | FileStub="${diff%.@(sql|diff)*}" 2152 | FileExt="${diff#"$FileStub"}" 2153 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_encrypted" 2154 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_gz" 2155 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_bz2" 2156 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_diff" 2157 | FileStub="${master%.@(sql|master)*}" 2158 | FileExt="${master#"$FileStub"}" 2159 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_encrypted" 2160 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_gz" 2161 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_bz2" 2162 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_diff" 2163 | 2164 | # TODO: Differential backup with encryption is not yet implemented! 2165 | if (( $diff_flags & $filename_flag_encrypted )); then 2166 | : #decrypt it 2167 | fi 2168 | 2169 | if (( $master_flags & $filename_flag_encrypted )); then 2170 | : #decrypt it 2171 | fi 2172 | 2173 | if (( $diff_flags & $filename_flag_gz )); then 2174 | gzip_compression -dc "$diff" > "${diff%.gz}" 2175 | to_rm=("${to_rm[@]}" "${diff%.gz}") 2176 | diff="${diff%.gz}" 2177 | elif (( $diff_flags & $filename_flag_bz2 )); then 2178 | bzip2_compression -dc "$diff" > "${diff%.bz2}" 2179 | to_rm=("${to_rm[@]}" "${diff%.bz2}") 2180 | diff="${diff%.bz2}" 2181 | else 2182 | : 2183 | fi 2184 | 2185 | if (( $master_flags & $filename_flag_gz )); then 2186 | gzip_compression -dc "$master" > "${master%.gz}" 2187 | to_rm=("${to_rm[@]}" "${master%.gz}") 2188 | master="${master%.gz}" 2189 | elif (( $master_flags & $filename_flag_bz2 )); then 2190 | bzip2_compression -dc "$master" > "${master%.bz2}" 2191 | to_rm=("${to_rm[@]}" "${master%.bz2}") 2192 | master="${master%.bz2}" 2193 | else 2194 | : 2195 | fi 2196 | 2197 | patch "$master" "$diff" -o "${diff/diff/sql}" 2198 | 2199 | # cleanup 2200 | for i in "${to_rm[@]}"; do rm "$i"; done 2201 | } 2202 | 2203 | # END _methods 2204 | 2205 | 2206 | # BEGIN __main 2207 | 2208 | NO_ARGS=0 2209 | E_OPTERROR=85 2210 | 2211 | if (( $# == $NO_ARGS )); then # Script invoked with no command-line args? 2212 | echo "Invoking backup method."; echo; method_backup 2213 | fi 2214 | 2215 | while getopts ":c:blh" Option 2216 | do 2217 | case $Option in 2218 | c ) echo "Using \"$OPTARG\" as optional config file."; echo; opt_config_file="$OPTARG"; opt_flag_config_file=1;; 2219 | b ) echo "MySQL backup method invoked."; echo; opt_flag_method_backup=1;; 2220 | l ) echo "List manifest entries."; echo; opt_flag_list_manifest_entries=1;; 2221 | h ) echo "Usage `basename $0` options -cblh" 2222 | echo -e "-c CONFIG_FILE\tSpecify optional config file." 2223 | echo -e "-b\tUse backup method." 2224 | echo -e "-l\tList manifest entries." 2225 | echo -e "-h\tShow this help." 2226 | exit 0;; 2227 | #n | o ) echo "Scenario #2: option -$Option- [OPTIND=${OPTIND}]";; 2228 | 2229 | #q ) echo "Scenario #4: option -q-\ 2230 | # with argument \"$OPTARG\" [OPTIND=${OPTIND}]";; 2231 | # Note that option 'q' must have an associated argument, 2232 | #+ otherwise it falls through to the default. 2233 | #r | s ) echo "Scenario #5: option -$Option-";; 2234 | * ) echo "Unimplemented option chosen.";; # Default. 2235 | esac 2236 | done 2237 | 2238 | (( $opt_flag_method_backup )) && method_backup 2239 | (( $opt_flag_list_manifest_entries )) && method_list_manifest_entries 2240 | 2241 | shift $(($OPTIND - 1)) 2242 | # Decrements the argument pointer so it points to next argument. 2243 | # $1 now references the first non-option item supplied on the command-line 2244 | #+ if one exists. 2245 | 2246 | # For backward compatibility. If no option items are present and only one non-option item is there, we expect it 2247 | # to be the optional config file and invoke the backup method. 2248 | opt_flags=( "${!opt_flag_@}" ) # array of all set variables starting with opt_flag_ 2249 | if (( $# == 1 )) && (( ${#opt_flags[@]} == 0 )); then 2250 | opt_config_file="$1"; opt_flag_config_file=1; method_backup 2251 | elif (( $# == 0 )) && (( ${#opt_flags[@]} == 0 )); then 2252 | method_backup 2253 | fi 2254 | 2255 | # END __main --------------------------------------------------------------------------------