├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── README.md ├── automysqlbackup ├── automysqlbackup.conf ├── install.sh └── run /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## [Latest] 4 | 5 | - Updated README and removed check for README changes from install script 6 | - add option to pass the '--flush-logs' flag to mysqldump from [@peterdutton](https://github.com/peterdutton) 7 | - Fixed install script after README rename [@pmontepagano](https://github.com/pmontepagano) 8 | 9 | ## [3.0_rc6] - (?) 10 | 11 | - Last update from Sourceforge 12 | 13 | ## [3.0_rc4] - (2011-11-24) 14 | 15 | - Removing mkfifo commands, thereby improving portability. 16 | - Fixing not working table exclude feature for wildcards. 17 | 18 | ## [3.0_rc3] - (2011-11-24) 19 | 20 | - Changed code to make it more portable, thereby resolving FreeBSD issue. 21 | 22 | ## [3.0_rc2] - (2011-11-21) 23 | 24 | - Added multicore support for bzip and gzip2. 25 | - Fixed error in README file. 26 | 27 | ## [3.0_rc1] - (2011-11-15) 28 | 29 | - Added differential backup method. 30 | - Added user-friendly method to recreate full backups out of differential ones. 31 | - Changed paramters, with which the script can be called, to make the new 32 | methods available. 33 | - Fixed some bugs. 34 | 35 | ## [3.0_beta2] - (2011-08-21) 36 | 37 | - Added possibility to backup local files. 38 | - Added full schema backup. 39 | - Added master-data option. 40 | - Fixed some bugs. 41 | 42 | ## [3.0_beta1] - (2011-08-15) 43 | 44 | - REMOVED: Implementation of Variables containing full path to binaries to 45 | avoid possibly confusion with aliases or builtins. (by Johannes Kolter) 46 | Inside a bash script no aliases are used! This didn't make ANY sense! 47 | Thereby resolved bug item #3074425. 48 | - Changed some variables to be arrays, i.e. lists, and removed the ugly 49 | sed stuff. 50 | Fixed bug item #3169562 51 | - Added check for weekly and monthly backups, so that they are unique per day. 52 | bug item #3185389 53 | - Changed SHEBANG to #!/usr/bin/env bash for portability reasons. 54 | bug item #3292873 55 | - Changed config file structure: read /etc/automysqlbackup/mysqlbackup.conf, 56 | if supplied read configfile parameter (no more -c or sth. like that, just 57 | the name of the file!), 58 | whatever isn't set yet, set in here to default values. 59 | - bug item #3110715: create a file in /etc/cron.d/daily and call the script 60 | from in there, i.e. place the script in /usr/local/bin 61 | - bug item #3082899: the PATH variables are different in ssh, you have to 62 | supply the complete path 63 | - Fixed bug item #3064547, suggestion accepted. 64 | - Fixed bug item #3031023, suggestion accepted. 65 | - Fixed bug item #3030604, resolved due to design correction. 66 | - Fixed bug item #3025849, as long as basename is in $PATH on your system. 67 | - Fixed bug item #3030478. 68 | - bug item #3054633: .muttrc entry save=yes will result in saving sent files! 69 | - Feature request item #1538588. 70 | - Feature request item #1538138. 71 | - Feature request item #1538142. 72 | - Feature request item #1541843 was already included. 73 | - Feature request item #2808012. 74 | - Feature request item #2831465. 75 | - Feature request item #3052484. Mysqldump already has an ssl option. 76 | - Feature request item #3190079. I hope cleaning up everything older than 24 77 | hours as a lower limit is good enough. 78 | - Feature request item #3284779 was already included. See CONFIG_mysql_dump_latest. 79 | - Feature request item #3053623. 80 | 81 | ## [2.6.0] - (2011-07-19) 82 | 83 | - Fixed bug where files would not email correctly (Fix by Jesse Vaughan) 84 | - Added section to encrypt .gz and .bz2 files using openssl (added by Jesse Vaughan) 85 | 86 | ## [2.5.1-01] - (2010-07-06) 87 | 88 | - Fixed pathname bug item #3025849 (by Johannes Kolter) 89 | 90 | ## [2.5.1] - (2010-07-04) 91 | 92 | - Added support for default and optional config file (by Johannes Kolter) 93 | - Rotating after backup was successful whith find(1) (by Johannes Kolter) 94 | - Implementation of Variables containing full path to binaries to 95 | avoid possibly confusion with aliases or builtins. (by Johannes Kolter) 96 | - Fixed bug where weekly backups were not being rotated. 97 | Added rotation of 5 monthly backups 98 | Now all old backups are deleted, not only the most recent one 99 | (inspired by oleg@bintime.com) 100 | - Use Debian special-file to access database (by Johannes Kolter) 101 | - Fixed bug ID: 1438565 102 | Moved IO redirection to a place before decicions are made and actions are taken. 103 | (inspired by Derk Bernhardt) 104 | - Fixed bug ID: #3000316 (reported by Sascha Feldhorst) 105 | - Fixed bug ID: #1529458 (reported by Natalie ( njwood )) 106 | - Fixed bug ID: #1548919 (reported by Piotr Kuczynski) 107 | 108 | ## [2.5] - (2006-01-15) 109 | 110 | - Added support for setting MAXIMUM_PACKET_SIZE and CONFIG_mysql_dump_socket parameters (suggested by Yvo van Doorn) 111 | 112 | ## [2.4] - (2006-01-23) 113 | 114 | - Fixed bug where weekly backups were not being rotated. (Fix by wolf02) 115 | - Added hour an min to backup filename for the case where backups are taken multiple 116 | - times in a day. NOTE This is not complete support for mutiple executions of the script 117 | - in a single day. 118 | - Added MAILCONTENT="quiet" option, see docs for details. (requested by snowsam) 119 | - Updated path statment for compatibility with OSX. 120 | - Added "CONFIG_mysql_dump_latest" to additionally store the last backup to a standard location. (request by Grant29) 121 | 122 | ## [2.3] - (2005-11-07) 123 | 124 | - Better error handling and notification of errors (a long time coming) 125 | - Compression on Backup server to MySQL server communications. 126 | 127 | ## [2.2] - (2004-12-05) 128 | 129 | - Changed from using depricated "-N" to "--skip-column-names". 130 | - Added ability to have compressed backup's emailed out. (code from Thomas Heiserowski) 131 | - Added maximum attachment size setting. 132 | 133 | ## [2.1] - (2004-11-04) 134 | 135 | - Fixed a bug in daily rotation when not using gzip compression. (Fix by Rob Rosenfeld) 136 | 137 | ## [2.0] - (2004-07-28) 138 | 139 | - Switched to using IO redirection instead of pipeing the output to the logfile. 140 | - Added choice of compression of backups being gzip of bzip2. 141 | - Switched to using functions to facilitate more functionality. 142 | - Added option of either gzip or bzip2 compression. 143 | 144 | ## [1.10] - (2004-07-17) 145 | 146 | - Another fix for spaces in the paths (fix by Thomas von Eyben) 147 | - Fixed bug when using PREBACKUP and POSTBACKUP commands containing many arguments. 148 | 149 | ## [1.9] - (2004-05-25) 150 | 151 | - Small bug fix to handle spaces in LOGFILE path which contains spaces (reported by Thomas von Eyben) 152 | - Updated docs to mention that Log email can be sent to multiple email addresses. 153 | 154 | ## [1.8] - (2004-05-01) 155 | 156 | - Added option to make backups restorable to alternate database names 157 | - meaning that a copy of the database can be created (Based on patch by Rene Hoffmann) 158 | - Seperated options into standard and advanced. 159 | - Removed " from single file dump DBMANES because it caused an error but 160 | - this means that if DB's have spaces in the name they will not dump when CONFIG_mysql_dump_use_separate_dirs=no. 161 | - Added -p option to mkdir commands to create multiple subdirs without error. 162 | - Added disk usage and location to the bottom of the backup report. 163 | 164 | ## [1.7] - (2004-04-22) 165 | 166 | - Fixed an issue where weelky backups would only work correctly if server 167 | - locale was set to English (issue reported by Tom Ingberg) 168 | - used "eval" for "rm" commands to try and resolve rotation issues. 169 | - Changed name of status log so multiple scripts can be run at the same time. 170 | 171 | ## [1.6] - (2004-03-14) 172 | 173 | - Added PREBACKUP and POSTBACKUP command functions. (patch by markpustjens) 174 | - Added support for backing up DB's with Spaces in the name. 175 | - (patch by markpustjens) 176 | 177 | ## [1.5] - (2004-02-24) 178 | 179 | - Added the ability to exclude DB's when the "all" option is used. 180 | - (Patch by kampftitan) 181 | 182 | ## [1.4] - (2004-02-02) 183 | 184 | - Project moved to Sourceforge.net 185 | 186 | ## [1.3] - (2003-09-25) 187 | 188 | - Added support for backing up "all" databases on the server without 189 | - having to list each one seperately in the configuration. 190 | - Added DB restore instructions. 191 | 192 | ## [1.2] - (2003-03-16) 193 | 194 | - Added server name to the backup log so logs from multiple servers 195 | - can be easily identified. 196 | 197 | ## [1.1] - (2003-03-13) 198 | 199 | - Small Bug fix in monthly report. (Thanks Stoyanski) 200 | - Added option to email log to any email address. (Inspired by Stoyanski) 201 | - Changed Standard file name to .sh extention. 202 | - Option are set using yes and no rather than 1 or 0. 203 | 204 | ## [1.0] - (2003-01-30) 205 | 206 | - Added the ability to have all databases backup to a single dump 207 | - file or seperate directory and file for each database. 208 | - Output is better for log keeping. 209 | 210 | ## [0.6] - (2003-01-22) 211 | 212 | - Bug fix for daily directory (Added in version 0.5) rotation. 213 | 214 | ## [0.5] - (2003-01-20) 215 | 216 | - Added "daily" directory for daily backups for neatness (suggestion by Jason) 217 | - Added CONFIG_mysql_dump_host option to allow backing up a remote server (Suggestion by Jason) 218 | - Added "--quote-names" option to mysqldump command. 219 | - Bug fix for handling the last and first of the year week rotation. 220 | 221 | ## [0.4] - (2002-11-06) 222 | 223 | - Added the abaility for the script to create its own directory structure. 224 | 225 | ## [0.3] - (2002-10-01) 226 | 227 | - Changed Naming of Weekly backups so they will show in order. 228 | 229 | ## [0.2] - (2002-09-27) 230 | 231 | - Corrected weekly rotation logic to handle weeks 0 - 10 232 | 233 | ## [0.1] - (2002-09-21) 234 | 235 | - Initial Release -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AutomysqlBackup 2 | 3 | - [AutomysqlBackup](#automysqlbackup) 4 | - [About](#about) 5 | - [Disclaimer](#disclaimer) 6 | - [Install](#install) 7 | - [Usage](#usage) 8 | - [Commandline Options](#commandline-options) 9 | - [Configuration Options](#configuration-options) 10 | - [Encryption](#encryption) 11 | - [Backup Rotation](#backup-rotation) 12 | - [Restoring](#restoring) 13 | 14 | ## About 15 | 16 | This script was originally developed by a team over at SourceForge: 17 | https://sourceforge.net/projects/automysqlbackup/ 18 | 19 | Since the project there seems not to be maintained anymore, I have set up a clone here for 20 | further maintenance and to simplify installation with a git repo. 21 | 22 | Pull-Requests are welcome! 23 | 24 | ## Disclaimer 25 | 26 | I take no resposibility for any data loss or corruption when using this script. 27 | This script will not help in the event of a hard drive crash. If a copy of the 28 | backup has not been stored offline or on another PC. You should copy your backups 29 | offline regularly for best protection. 30 | 31 | Happy backing up... 32 | 33 | ## Install 34 | 35 | Clone or extract the package into a directory. If you are reading this you have probably done this already. 36 | 37 | To install the Automysqlbackup the easy way: 38 | 39 | 1. Run the install.sh script. 40 | 2. Edit the /etc/automysqlbackup/myserver.conf file to customise your settings. 41 | 3. See usage section. 42 | 43 | To install it manually (the hard way): 44 | 45 | 1. Create the /etc/automysqlbackup directory. 46 | 2. Copy in the automysqlbackup.conf file. 47 | 3. Copy the automysqlbackup file to /usr/local/bin and make executable. 48 | 4. cp /etc/automysqlbackup/automysqlbackup.conf /etc/automysqlbackup/myserver.conf 49 | 5. Edit the /etc/automysqlbackup/myserver.conf file to customise your settings. 50 | 6. See usage section. 51 | 52 | ## Usage 53 | 54 | Automysqlbackup can be run a number of ways, you can choose which is best for you. 55 | 56 | 1. Create a script as below called runmysqlbackup using the lines below: 57 | ```bash 58 | #!/bin/sh 59 | 60 | /usr/local/bin/automysqlbackup /etc/automysqlbackup/myserver.conf 61 | 62 | chown root.root /var/backup/db* -R 63 | find /var/backup/db* -type f -exec chmod 400 {} \; 64 | find /var/backup/db* -type d -exec chmod 700 {} \; 65 | ``` 66 | 2. Save it to a suitable location or copy it to your /etc/cron.daily folder. 67 | 3. Make it executable, i.e. chmod +x /etc/cron.daily/runmysqlbackup. 68 | 69 | The backup can be run from the command line simply by running the following command. 70 | 71 | automysqlbackup /etc/automysqlbackup/myserver.conf 72 | 73 | If you don't supply an argument for automysqlbackup, the default configuration 74 | in the program automysqlbackup will be used unless a global file 75 | 76 | CONFIG_configfile="/etc/automysqlbackup/automysqlbackup.conf" 77 | 78 | exists. 79 | 80 | You can just copy the supplied automysqlbackup.conf as many times as you want 81 | and use for separate configurations, i.e. for example different mysql servers. 82 | 83 | ## Commandline Options 84 | 85 | As of version 3.0 we have added differential backups using the program diff. In an 86 | effort to make the reconstruction of the full archives more user friendly, we 87 | added new functionality to the script. Therefore, while preserving the old syntax, 88 | we created options for the script, so that the new functions can be accessed. 89 | 90 | Usage automysqlbackup options -cblh 91 | -c CONFIG_FILE Specify optional config file. 92 | -b Use backup method. 93 | -l List manifest entries. 94 | -h Show this help. 95 | 96 | If you use these options, you have to specify everything according to them and can't 97 | mix the old syntax with the new one. Example: 98 | 99 | before (still valid!): 100 | 101 | automysqlbackup "myconfig.conf" 102 | 103 | now: 104 | 105 | automysqlbackup -c "myconfig.conf" -b 106 | 107 | which is equivalent to 108 | 109 | automysqlbackup -bc "myconfig.conf" 110 | 111 | or in English: The order of the options doesn't matter, however those options expecting 112 | arguments, have to be placed right before the argument (as seen above). 113 | 114 | The option '-l' (List manifest entries) finds all Manifest files in your configuration 115 | directory (you need to specify your optional config file - otherwise a fallback will be 116 | used: global config file -> program internal default options). It then filters from which 117 | databases these are and presents you with a list (you can select more than one!) of them. 118 | Once you have chosen, you will be given a list of Manifest files, from which you choose 119 | again and after that from which you choose differential files. When you have completed 120 | all your selections, a list of selected differential files will be shown. You may then 121 | choose what you want to be done with/to those files. At the moment the options are: 122 | - create full backup out of differential one 123 | - remove the differential backup and its Manifest entry. 124 | 125 | ## Configuration Options 126 | 127 | !! "automysqlbackup" program contains a default configuration that should not be changed: 128 | 129 | The global config file which overwrites the default configuration is located here 130 | "/etc/automysqlbackup/automysqlbackup.conf" by default. 131 | 132 | Please take a look at the supplied "automysqlbackup.conf" for information about the configuration options. 133 | 134 | Default configuration 135 | CONFIG_configfile="/etc/automysqlbackup/automysqlbackup.conf" 136 | CONFIG_backup_dir='/var/backup/db' 137 | CONFIG_do_monthly="01" 138 | CONFIG_do_weekly="5" 139 | CONFIG_rotation_daily=6 140 | CONFIG_rotation_weekly=35 141 | CONFIG_rotation_monthly=150 142 | CONFIG_mysql_dump_usessl='yes' 143 | CONFIG_mysql_dump_username='root' 144 | CONFIG_mysql_dump_password='' 145 | CONFIG_mysql_dump_host='localhost' 146 | CONFIG_mysql_dump_socket='' 147 | CONFIG_mysql_dump_create_database='no' 148 | CONFIG_mysql_dump_use_separate_dirs='yes' 149 | CONFIG_mysql_dump_compression='gzip' 150 | CONFIG_mysql_dump_commcomp='no' 151 | CONFIG_mysql_dump_latest='no' 152 | CONFIG_mysql_dump_max_allowed_packet='' 153 | CONFIG_db_names=() 154 | CONFIG_db_month_names=() 155 | CONFIG_db_exclude=( 'information_schema' ) 156 | CONFIG_mailcontent='log' 157 | CONFIG_mail_maxattsize=4000 158 | CONFIG_mail_address='root' 159 | CONFIG_encrypt='no' 160 | CONFIG_encrypt_password='password0123' 161 | 162 | !! automysqlbackup (the shell program) accepts one parameter, the filename of a configuration file. The entries in there will supersede all others. 163 | 164 | Please take a look at the supplied "automysqlbackup.conf" for information about the configuration options. 165 | 166 | ## Encryption 167 | 168 | To decrypt run (replace bz2 with gz if using gzip): 169 | 170 | openssl enc -aes-256-cbc -d -in encrypted_file_name(ex: *.enc.bz2) -out outputfilename.bz2 -pass pass:PASSWORD-USED-TO-ENCRYPT 171 | 172 | ## Backup Rotation 173 | 174 | * Daily Backups are rotated weekly. 175 | * Weekly Backups are run on fridays, unless otherwise specified via CONFIG_do_weekly. 176 | * Weekly Backups are rotated on a 5 week cycle, unless otherwise specified via CONFIG_rotation_weekly. 177 | * Monthly Backups are run on the 1st of the month, unless otherwise specified via CONFIG_do_monthly. 178 | * Monthly Backups are rotated on a 5 month cycle, unless otherwise specified via CONFIG_rotation_monthly. 179 | 180 | Suggestion: It may be a good idea to copy backups offline or to another server. 181 | 182 | ## Restoring 183 | 184 | First you will need to uncompress the backup file and decrypt it if encryption was used (see encryption section). 185 | 186 | eg. 187 | 188 | gunzip file.gz (or bunzip2 file.bz2) 189 | 190 | Next you will need to use the mysql client to restore the DB from the sql file. 191 | 192 | eg. 193 | 194 | mysql --user=username --pass=password --host=dbserver database < /path/file.sql 195 | or 196 | 197 | mysql --user=username --pass=password --host=dbserver -e "source /path/file.sql" database 198 | 199 | NOTE: Make sure you use "<" and not ">" in the above command because you are piping the file.sql to mysql and not the other way around. 200 | -------------------------------------------------------------------------------- /automysqlbackup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | # This program is free software; you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation; either version 2 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program; if not, write to the Free Software 15 | # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 16 | # 17 | shopt -s extglob 18 | 19 | # BEGIN _flags 20 | let "filename_flag_encrypted=0x01" 21 | let "filename_flag_gz=0x02" 22 | let "filename_flag_bz2=0x04" 23 | let "filename_flag_diff=0x08" 24 | # END _flags 25 | 26 | # BEGIN _errors_notifications 27 | let "E=0x00" # no errors 28 | let "N=0x00" # no notifications 29 | 30 | let "E_dbdump_failed=0x01" 31 | let "E_backup_local_failed=0x02" 32 | let "E_mkdir_basedir_failed=0x04" 33 | let "E_mkdir_subdirs_failed=0x08" 34 | let "E_perm_basedir=0x10" 35 | let "E_enc_cleartext_delfailed=0x20" 36 | let "E_enc_failed=0x40" 37 | let "E_db_empty=0x80" 38 | let "E_create_pipe_failed=0x100" 39 | let "E_missing_deps=0x200" 40 | let "E_no_basedir=0x400" 41 | let "E_config_backupdir_not_writable=0x800" 42 | let "E_dump_status_failed=0x1000" 43 | let "E_dump_fullschema_failed=0x2000" 44 | 45 | let "N_config_file_missing=0x01" 46 | let "N_arg_conffile_parsed=0x02" 47 | let "N_arg_conffile_unreadable=0x04" 48 | let "N_too_many_args=0x08" 49 | let "N_latest_cleanup_failed=0x10" 50 | let "N_backup_local_nofiles=0x20" 51 | # END _errors_notifications 52 | 53 | # BEGIN _functions 54 | 55 | # @info: Default configuration options. 56 | # @deps: (none) 57 | load_default_config() { 58 | CONFIG_configfile="/etc/automysqlbackup/automysqlbackup.conf" 59 | CONFIG_backup_dir='/var/backup/db' 60 | CONFIG_multicore='yes' 61 | CONFIG_multicore_threads=2 62 | CONFIG_do_monthly="01" 63 | CONFIG_do_weekly="5" 64 | CONFIG_rotation_daily=6 65 | CONFIG_rotation_weekly=35 66 | CONFIG_rotation_monthly=150 67 | CONFIG_mysql_dump_port=3306 68 | CONFIG_mysql_dump_usessl='yes' 69 | CONFIG_mysql_dump_username='root' 70 | CONFIG_mysql_dump_password='' 71 | CONFIG_mysql_dump_host='localhost' 72 | CONFIG_mysql_dump_host_friendly='' 73 | CONFIG_mysql_dump_socket='' 74 | CONFIG_mysql_dump_create_database='no' 75 | CONFIG_mysql_dump_use_separate_dirs='yes' 76 | CONFIG_mysql_dump_compression='gzip' 77 | CONFIG_mysql_dump_commcomp='no' 78 | CONFIG_mysql_dump_latest='no' 79 | CONFIG_mysql_dump_latest_clean_filenames='no' 80 | CONFIG_mysql_dump_max_allowed_packet='' 81 | CONFIG_mysql_dump_single_transaction='no' 82 | CONFIG_mysql_dump_master_data= 83 | CONFIG_mysql_dump_full_schema='yes' 84 | CONFIG_mysql_dump_flush_logs='no' 85 | CONFIG_mysql_dump_dbstatus='yes' 86 | CONFIG_mysql_dump_differential='no' 87 | CONFIG_backup_local_files=() 88 | CONFIG_db_names=() 89 | CONFIG_db_month_names=() 90 | CONFIG_db_exclude=( 'information_schema' ) 91 | CONFIG_table_exclude=() 92 | CONFIG_mailcontent='stdout' 93 | CONFIG_mail_maxattsize=4000 94 | CONFIG_mail_splitandtar='yes' 95 | CONFIG_mail_use_uuencoded_attachments='no' 96 | CONFIG_mail_address='root' 97 | CONFIG_encrypt='no' 98 | CONFIG_encrypt_password='password0123' 99 | } 100 | 101 | # @return: true, if variable is set; else false 102 | isSet() { 103 | if [[ ! ${!1} && ${!1-_} ]]; then return 1; else return 0; fi 104 | } 105 | 106 | # @return: true, if variable is empty; else false 107 | isEmpty() { 108 | if [[ ${!1} ]]; then return 1; else return 0; fi 109 | } 110 | 111 | # @info: Called when one of the signals EXIT, SIGHUP, SIGINT, SIGQUIT or SIGTERM is emitted. 112 | # It removes the IO redirection, mails any log file information and cleans up any temporary files. 113 | # @args: (none) 114 | # @return: (none) 115 | mail_cleanup () { 116 | removeIO 117 | # if the variables $log_file and $log_errfile aren't set or are empty and both associated files don't exist, skip output methods. 118 | # this might happen if 'exit' occurs before they are set. 119 | if [[ ! -e "$log_file" && ! -e "$log_errfile" ]];then 120 | echo "Skipping normal output methods, since the program exited before any log files could be created." 121 | else 122 | case "${CONFIG_mailcontent}" in 123 | 'files') 124 | # Include error log if larger than zero. 125 | if [[ -s "$log_errfile" ]]; then 126 | backupfiles=( "${backupfiles[@]}" "$log_errfile" ) 127 | errornote="WARNING: Error Reported - " 128 | fi 129 | temp="$(mktemp "$CONFIG_backup_dir"/tmp/mail_content.XXXXXX)" 130 | # Get backup size 131 | attsize=`du -c "${backupfiles[@]}" | awk 'END {print $1}'` 132 | if (( ${CONFIG_mail_maxattsize} >= ${attsize} )); then 133 | if [[ "x$CONFIG_mail_use_uuencoded_attachments" = "xyes" ]]; then 134 | cat "$log_file" > "$temp" 135 | for j in "${backupfiles[@]}"; do 136 | uuencode "$j" "$j" >> "$temp" 137 | done 138 | mail -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} < "$temp" 139 | else 140 | mutt -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" -a "${backupfiles[@]}" -- ${CONFIG_mail_address} < "$log_file" 141 | fi 142 | elif (( ${CONFIG_mail_maxattsize} <= ${attsize} )) && [[ "x$CONFIG_mail_splitandtar" = "xyes" ]]; then 143 | if sPWD="$PWD"; cd "$CONFIG_backup_dir"/tmp && pax -wv "${backupfiles[@]}" | bzip2_compression | split -b $((CONFIG_mail_maxattsize*1000)) - mail_attachment_${datetimestamp}_ && cd "$sPWD"; then 144 | files=("$CONFIG_backup_dir"/tmp/mail_attachment_${datetimestamp}_*) 145 | echo -e "\n\nThe attachments have been split into multiple files.\nUse 'cat mail_attachment_2011-08-13_13h15m_* > mail_attachment_2011-08-13_13h15m.tar.bz2' to combine them and \ 146 | 'bunzip2 "$temp" 151 | uuencode "$j" "$j" >> "$temp" 152 | else 153 | uuencode "$j" "$j" > "$temp" 154 | fi 155 | mail -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} < "$temp" 156 | else 157 | mutt -s "${errornote} MySQL Backup Log and SQL Files for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}; Part $((j+1))/${#files[@]}" -a "${files[j]}" -- ${CONFIG_mail_address} < "$log_file" 158 | fi 159 | done 160 | else 161 | cat "$log_file" | mail -s "WARNING! - MySQL Backup exceeds set maximum attachment size on ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 162 | fi 163 | else 164 | cat "$log_file" | mail -s "WARNING! - MySQL Backup exceeds set maximum attachment size on ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 165 | fi 166 | rm "$temp" 167 | ;; 168 | 'log') 169 | cat "$log_file" | mail -s "MySQL Backup Log for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 170 | [[ -s "$log_errfile" ]] && cat "$log_errfile" | mail -s "ERRORS REPORTED: MySQL Backup error Log for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 171 | ;; 172 | 'quiet') 173 | [[ -s "$log_errfile" ]] && cat "$log_errfile" | mail -s "ERRORS REPORTED: MySQL Backup error Log for ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host} - ${datetimestamp}" ${CONFIG_mail_address} 174 | ;; 175 | *) 176 | if [[ -s "$log_errfile" ]]; then 177 | cat "$log_file" 178 | echo 179 | echo "###### WARNING ######" 180 | echo "Errors reported during AutoMySQLBackup execution.. Backup failed" 181 | echo "Error log below.." 182 | cat "$log_errfile" 183 | else 184 | cat "$log_file" 185 | fi 186 | ;; 187 | esac 188 | ################################################################################### 189 | # Clean up and finish 190 | [[ -e "$log_file" ]] && rm -f "$log_file" 191 | [[ -e "$log_errfile" ]] && rm -f "$log_errfile" 192 | fi 193 | } 194 | 195 | # @params: #month #year 196 | # @deps: (none) 197 | days_of_month() { 198 | m="$1"; y="$2"; a=$(( 30+(m+m/8)%2 )) 199 | (( m==2 )) && a=$((a-2)) 200 | (( m==2 && y%4==0 && ( y<100 || y%100>0 || y%400==0) )) && a=$((a+1)) 201 | printf '%d' $a 202 | } 203 | 204 | # @info: Checks if a folder is writable by creating a temporary file in it and removing it afterwards. 205 | # @args: folder to test 206 | # @return: returns false if creation of temporary file failed or it can't be removed afterwards; else true 207 | # @deps: (none) 208 | chk_folder_writable () { 209 | local temp; temp="$(mktemp "$1"/tmp.XXXXXX)" 210 | if (( $? == 0 )); then 211 | rm "${temp}" || return 1 212 | return 0 213 | else 214 | return 1 215 | fi 216 | } 217 | 218 | # @info: bzip2 compression 219 | bzip2_compression() { 220 | var=("$@") 221 | re='^[0-9]*$' 222 | if [[ "x$CONFIG_multicore" = 'xyes' ]]; then 223 | if [[ "x$CONFIG_multicore_threads" != 'xauto' ]] && [[ "x$CONFIG_multicore_threads" =~ $re ]]; then 224 | var=( "-p${CONFIG_multicore_threads}" "${var[@]}" ) 225 | fi 226 | pbzip2 "${var[@]}" 227 | else 228 | bzip2 "${var[@]}" 229 | fi 230 | } 231 | 232 | # @info: gzip compression 233 | gzip_compression() { 234 | var=("$@") 235 | re='^[0-9]*$' 236 | if [[ "x$CONFIG_multicore" = 'xyes' ]]; then 237 | if [[ "x$CONFIG_multicore_threads" != 'xauto' ]] && [[ "x$CONFIG_multicore_threads" =~ $re ]]; then 238 | var=( "-p${CONFIG_multicore_threads}" "${var[@]}" ) 239 | fi 240 | pigz "${var[@]}" 241 | else 242 | gzip "${var[@]}" 243 | fi 244 | } 245 | 246 | # @info: Remove date and time information from filename by renaming it. 247 | # @args: filename 248 | # @return: (none) 249 | # @deps: (none) 250 | remove_datetimeinfo () { 251 | mv "$1" "$(echo "$1" | sed -re 's/_[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}h[0-9]{2}m_(Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday|January|February|March|April|May|June|July|August|September|October|November|December|[0-9]{1,2})//g')" 252 | } 253 | export -f remove_datetimeinfo 254 | 255 | # @info: Set time and date variables. 256 | # @args: (none) 257 | # @deps: days_of_month 258 | set_datetime_vars() { 259 | datetimestamp=`date +%Y-%m-%d_%Hh%Mm` # Datestamp e.g 2002-09-21_18h12m 260 | date_stamp=`date +%Y-%m-%d` # Datestamp e.g 2002-09-21 261 | date_day_of_week=`date +%A` # Day of the week e.g. Monday 262 | date_dayno_of_week=`date +%u` # Day number of the week 1 to 7 where 1 represents Monday 263 | date_day_of_month=`date +%e | sed -e 's/^ //'` # Date of the Month e.g. 27 264 | date_month=`date +%B` # Month e.g January 265 | date_weekno=`date +%V | sed -e 's/^0//'` # Week Number e.g 37 266 | year=`date +%Y` 267 | month=`date +%m | sed -e 's/^0//'` 268 | date_lastday_of_last_month=$(days_of_month $(( $month==1 ? 12 : $month-1 )) $(( $month==1 ? ($year-1):$year )) ) 269 | date_lastday_of_this_month=$(days_of_month $month $year) 270 | } 271 | 272 | # @info: This function is called after data has already been saved. It performs encryption and 273 | # hardlink-copying of files to a latest folder. 274 | # @return: flags 275 | # @deps: load_default_config 276 | files_postprocessing () { 277 | local flags 278 | let "flags=0x00" 279 | let "flags_files_postprocessing_success_encrypt=0x01" 280 | 281 | # -> CONFIG_encrypt 282 | [[ "${CONFIG_encrypt}" = "yes" && "${CONFIG_encrypt_password}" ]] && { 283 | if (( $CONFIG_dryrun )); then 284 | printf 'dry-running: openssl enc -aes-256-cbc -e -in %s -out %s.enc -pass pass:%s\n' ${1} ${1} "${CONFIG_encrypt_password}" 285 | else 286 | openssl enc -aes-256-cbc -e -in ${1} -out ${1}.enc -pass pass:"${CONFIG_encrypt_password}" 287 | if (( $? == 0 )); then 288 | if rm ${1} 2>&1; then 289 | echo "Successfully encrypted archive as ${1}.enc" 290 | let "flags |= $flags_files_postprocessing_success_encrypt" 291 | else 292 | echo "Successfully encrypted archive as ${1}.enc, but could not remove cleartext file ${1}." 293 | let "E |= $E_enc_cleartext_delfailed" 294 | fi 295 | else 296 | let "E |= $E_enc_failed" 297 | fi 298 | fi 299 | } 300 | # <- CONFIG_encrypt 301 | 302 | # -> CONFIG_mysql_dump_latest 303 | [[ "${CONFIG_mysql_dump_latest}" = "yes" ]] && { 304 | if (( $flags & $flags_files_postprocessing_success_encrypt )); then 305 | if (( $CONFIG_dryrun )); then 306 | printf 'dry-running: cp -al %s.enc %s/latest/\n' "${1}" "${CONFIG_backup_dir}" 307 | else 308 | cp -al "${1}${suffix}.enc" "${CONFIG_backup_dir}"/latest/ 309 | fi 310 | else 311 | if (( $CONFIG_dryrun )); then 312 | printf 'dry-running: cp -al %s %s/latest/\n' "${1}" "${CONFIG_backup_dir}" 313 | else 314 | cp -al "${1}" "${CONFIG_backup_dir}"/latest/ 315 | fi 316 | fi 317 | } 318 | # <- CONFIG_mysql_dump_latest 319 | 320 | return $flags 321 | } 322 | 323 | # @info: When called, sets error and notify strings matching their flags. It then goes through all 324 | # collected error and notify messages and displays them. 325 | # @args: (none) 326 | # @return: true if no errors were set, otherwise false 327 | # @deps: log_base2, load_default_config 328 | error_handler () { 329 | 330 | errors=( 331 | [0x01]='dbdump() failed.' 332 | [0x02]='Backup of local files failed. This is not this scripts primary objective. Continuing anyway.' 333 | [0x04]="Could not create the backup_dir ${CONFIG_backup_dir}. Please check permissions of the higher directory." 334 | [0x08]='At least one of the subdirectories (daily, weekly, monthly, latest) failed to create.' 335 | [0x10]="The backup_dir ${CONFIG_backup_dir} is not writable AND/OR executable." 336 | [0x20]='Could not remove the cleartext file after encryption. This error did not cause an abort. Remove it manually and check permissions.' 337 | [0x40]='Encryption failed. Continuing without encryption.' 338 | [0x80]='The mysql server is empty, i.e. no databases found. Check if something is wrong. Exiting.' 339 | [0x100]='Failed to create the named pipe (fifo) for reading in all databases. Exiting.' 340 | [0x200]='Dependency programs are missing. Perhaps they are not in $PATH. Exiting.' 341 | [0x400]='No basedir found, i.e. ' 342 | [0x800]="${CONFIG_backup_dir} is not writable. Exiting." 343 | [0x1000]='Running of mysqlstatus failed.' 344 | [0x2000]='Running of mysqldump full schema failed.' 345 | ) 346 | 347 | notify=( 348 | [0x01]="${CONFIG_configfile} was not found - no global config file." 349 | [0x02]="Parsed config file ${opt_config_file}." 350 | [0x04]="Unreadable config file \"${opt_config_file}\"" 351 | [0x08]='Supplied more than one argument, ignoring ALL arguments - using default and global config file only.' 352 | [0x10]='Could not remove the files in the latest directory. Please check this.' 353 | [0x20]='No local backup files were set.' 354 | [0x40]='' 355 | [0x80]='' 356 | [0x100]='' 357 | [0x200]='' 358 | [0x400]='' 359 | [0x800]='' 360 | [0x1000]='' 361 | [0x2000]='' 362 | ) 363 | 364 | local n 365 | local e 366 | 367 | n=$((${#notify[@]}-1)) 368 | while (( N > 0 )); do 369 | e=$((2**n)) 370 | if (( N&e )); then 371 | echo "Note:" ${notify[e]} 372 | let "N-=e" 373 | fi 374 | ((n--)) 375 | done 376 | unset n; 377 | 378 | n=$((${#errors[@]}-1)) 379 | if (( E > 0 )); then 380 | while (( E > 0 )); do 381 | e=$((2**n)) 382 | if (( E&e )); then 383 | echo "Error:" ${errors[e]} 384 | let "E-=e" 385 | fi 386 | ((n--)) 387 | done 388 | exit 1 389 | else 390 | exit 0 391 | fi 392 | } 393 | 394 | # @info: Packs files in array ${#CONFIG_backup_local_files[@]} into tar file with optional compression. 395 | # @args: archive file without compression suffix, i.e. ending on .tar 396 | # @return: true in case of dry-run, otherwise the return value of tar -cvf 397 | # @deps: load_default_config 398 | backup_local_files () { 399 | if ((! ${#CONFIG_backup_local_files[@]})) ; then 400 | if (( $CONFIG_dryrun )); then 401 | case "${CONFIG_mysql_dump_compression}" in 402 | 'gzip') 403 | echo "tar -czvf ${1}${suffix} ${CONFIG_backup_local_files[@]}"; 404 | ;; 405 | 'bzip2') 406 | echo "tar -cjvf ${1}${suffix} ${CONFIG_backup_local_files[@]}"; 407 | ;; 408 | *) 409 | echo "tar -cvf ${1}${suffix} ${CONFIG_backup_local_files[@]}"; 410 | ;; 411 | esac 412 | echo "dry-running: tar -cv ${1} ${CONFIG_backup_local_files[@]}" 413 | return 0; 414 | else 415 | case "${CONFIG_mysql_dump_compression}" in 416 | 'gzip') 417 | tar -czvf "${1}${suffix}" "${CONFIG_backup_local_files[@]}"; 418 | return $? 419 | ;; 420 | 'bzip2') 421 | tar -cjvf "${1}${suffix}" "${CONFIG_backup_local_files[@]}"; 422 | return $? 423 | ;; 424 | *) 425 | tar -cvf "${1}${suffix}" "${CONFIG_backup_local_files[@]}"; 426 | return $? 427 | ;; 428 | esac 429 | fi 430 | else 431 | let "N |= $N_backup_local_nofiles" 432 | echo "No local backup files specified." 433 | fi 434 | } 435 | 436 | # @info: Parses the configuration options and sets the variables appropriately. 437 | # @args: (none) 438 | # @deps: load_default_config 439 | parse_configuration () { 440 | # OPT string for use with mysqldump ( see man mysqldump ) 441 | opt=( '--quote-names' '--opt' ) 442 | 443 | # OPT string for use with mysql (see man mysql ) 444 | mysql_opt=() 445 | 446 | # OPT string for use with mysqldump fullschema 447 | opt_fullschema=( '--all-databases' '--routines' '--no-data' ) 448 | 449 | # OPT string for use with mysqlstatus 450 | opt_dbstatus=( '--status' ) 451 | 452 | [[ "${CONFIG_mysql_dump_usessl}" = "yes" ]] && { 453 | opt=( "${opt[@]}" '--ssl' ) 454 | mysql_opt=( "${mysql_opt[@]}" '--ssl' ) 455 | opt_fullschema=( "${opt_fullschema[@]}" '--ssl' ) 456 | opt_dbstatus=( "${opt_dbstatus[@]}" '--ssl' ) 457 | } 458 | [[ "${CONFIG_mysql_dump_master_data}" ]] && (( ${CONFIG_mysql_dump_master_data} == 1 || ${CONFIG_mysql_dump_master_data} == 2 )) && { opt=( "${opt[@]}" "--master-data=${CONFIG_mysql_dump_master_data}" );} 459 | [[ "${CONFIG_mysql_dump_single_transaction}" = "yes" ]] && { 460 | opt=( "${opt[@]}" '--single-transaction' ) 461 | opt_fullschema=( "${opt_fullschema[@]}" '--single-transaction' ) 462 | } 463 | [[ "${CONFIG_mysql_dump_flush_logs}" = "yes" ]] && { 464 | opt=( "${opt[@]}" '--flush-logs' ) 465 | } 466 | 467 | [[ "${CONFIG_mysql_dump_commcomp}" = "yes" ]] && { 468 | opt=( "${opt[@]}" '--compress' ) 469 | opt_fullschema=( "${opt_fullschema[@]}" '--compress' ) 470 | opt_dbstatus=( "${opt_dbstatus[@]}" '--compress' ) 471 | } 472 | [[ "${CONFIG_mysql_dump_max_allowed_packet}" ]] && { 473 | opt=( "${opt[@]}" "--max_allowed_packet=${CONFIG_mysql_dump_max_allowed_packet}" ) 474 | opt_fullschema=( "${opt_fullschema[@]}" "--max_allowed_packet=${CONFIG_mysql_dump_max_allowed_packet}" ) 475 | } 476 | [[ "${CONFIG_mysql_dump_socket}" ]] && { 477 | opt=( "${opt[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 478 | mysql_opt=( "${mysql_opt[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 479 | opt_fullschema=( "${opt_fullschema[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 480 | opt_dbstatus=( "${opt_dbstatus[@]}" "--socket=${CONFIG_mysql_dump_socket}" ) 481 | } 482 | [[ "${CONFIG_mysql_dump_port}" ]] && { 483 | opt=( "${opt[@]}" "--port=${CONFIG_mysql_dump_port}" ) 484 | mysql_opt=( "${mysql_opt[@]}" "--port=${CONFIG_mysql_dump_port}" ) 485 | opt_fullschema=( "${opt_fullschema[@]}" "--port=${CONFIG_mysql_dump_port}" ) 486 | opt_dbstatus=( "${opt_dbstatus[@]}" "--port=${CONFIG_mysql_dump_port}" ) 487 | } 488 | 489 | # Check if CREATE DATABASE should be included in Dump 490 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 491 | if [[ "${CONFIG_mysql_dump_create_database}" = "no" ]]; then 492 | opt=( "${opt[@]}" '--no-create-db' ) 493 | else 494 | opt=( "${opt[@]}" '--databases' ) 495 | fi 496 | else 497 | opt=( "${opt[@]}" '--databases' ) 498 | fi 499 | 500 | # if differential backup is active and the specified rotation is smaller than 21 days, set it to 21 days to ensure, that 501 | # master backups aren't deleted. 502 | if [[ "x$CONFIG_mysql_dump_differential" = "xyes" ]] && (( ${CONFIG_rotation_daily} < 21 )); then 503 | CONFIG_rotation_daily=21 504 | fi 505 | 506 | # -> determine suffix 507 | case "${CONFIG_mysql_dump_compression}" in 508 | 'gzip') suffix='.gz';; 509 | 'bzip2') suffix='.bz2';; 510 | *) suffix='';; 511 | esac 512 | # <- determine suffix 513 | 514 | # -> check exclude tables for wildcards 515 | local tmp;tmp=() 516 | local z;z=0 517 | for i in "${CONFIG_table_exclude[@]}"; do 518 | r='^[^*.]+\.[^.]+$'; [[ "$i" =~ $r ]] || { printf 'The entry %s in CONFIG_table_exclude has a wrong format. Ignoring the entry.' "$i"; continue; } 519 | db=${i%.*} 520 | table=${i#"$db".} 521 | r='\*'; [[ "$i" =~ $r ]] || { tmp[z++]="$i"; continue; } 522 | while read -r; do tmp[z++]="${db}.${REPLY}"; done < <(mysql --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${mysql_opt[@]}" --batch --skip-column-names -e "select table_name from information_schema.tables where table_schema='${db}' and table_name like '${table//\*/%}';") 523 | done 524 | for l in "${tmp[@]}"; do echo "exclude $l";done 525 | CONFIG_table_exclude=("${tmp[@]}") 526 | # <- 527 | 528 | if ((${#CONFIG_table_exclude[@]})); then 529 | for i in "${CONFIG_table_exclude[@]}"; do 530 | opt=( "${opt[@]}" "--ignore-table=$i" ) 531 | done 532 | fi 533 | } 534 | 535 | # @info: Backup database status 536 | # @args: archive file without compression suffix, i.e. ending on .txt 537 | # @return: true in case of dry-run, otherwise the return value of mysqlshow 538 | # @deps: load_default_config, parse_configuration 539 | dbstatus() { 540 | if (( $CONFIG_dryrun )); then 541 | case "${CONFIG_mysql_dump_compression}" in 542 | 'gzip') 543 | echo "dry-running: mysqlshow --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_dbstatus[@]} | gzip_compression > ${1}${suffix}"; 544 | ;; 545 | 'bzip2') 546 | echo "dry-running: mysqlshow --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_dbstatus[@]} | bzip2_compression > ${1}${suffix}"; 547 | ;; 548 | *) 549 | echo "dry-running: mysqlshow --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_dbstatus[@]} > ${1}${suffix}"; 550 | ;; 551 | esac 552 | return 0; 553 | else 554 | case "${CONFIG_mysql_dump_compression}" in 555 | 'gzip') 556 | mysqlshow --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_dbstatus[@]}" | gzip_compression > "${1}${suffix}"; 557 | return $? 558 | ;; 559 | 'bzip2') 560 | mysqlshow --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_dbstatus[@]}" | bzip2_compression > "${1}${suffix}"; 561 | return $? 562 | ;; 563 | *) 564 | mysqlshow --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_dbstatus[@]}" > "${1}${suffix}"; 565 | return $? 566 | ;; 567 | esac 568 | fi 569 | } 570 | 571 | # @info: Backup of the database schema. 572 | # @args: filename to save data to 573 | # @return: true in case of dry-run, otherwise the return value of mysqldump 574 | # @deps: load_default_config, parse_configuration 575 | fullschema () { 576 | if (( $CONFIG_dryrun )); then 577 | case "${CONFIG_mysql_dump_compression}" in 578 | 'gzip') 579 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_fullschema[@]} | gzip_compression > ${1}${suffix}"; 580 | ;; 581 | 'bzip2') 582 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_fullschema[@]} | bzip2_compression > ${1}${suffix}"; 583 | ;; 584 | *) 585 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt_fullschema[@]} > ${1}${suffix}"; 586 | ;; 587 | esac 588 | return 0; 589 | else 590 | case "${CONFIG_mysql_dump_compression}" in 591 | 'gzip') 592 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_fullschema[@]}" | gzip_compression > "${1}${suffix}"; 593 | return $? 594 | ;; 595 | 'bzip2') 596 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_fullschema[@]}" | bzip2_compression > "${1}${suffix}"; 597 | return $? 598 | ;; 599 | *) 600 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt_fullschema[@]}" > "${1}${suffix}"; 601 | return $? 602 | ;; 603 | esac 604 | fi 605 | } 606 | 607 | # @info: Process a single db. 608 | # @args: subfolder, prefix, midfix, extension, rotation, rotation_divisor, rotation_string, 0/1 (db/dbs), db[, db ...] 609 | process_dbs() { 610 | local subfolder="$1" 611 | local prefix="$2" 612 | local midfix="$3" 613 | local extension="$4" 614 | local rotation="$5" 615 | local rotation_divisor="$6" 616 | local rotation_string="$7" 617 | local multipledbs="$8" 618 | shift 8 619 | 620 | local name 621 | local subsubfolder 622 | 623 | # only activate differential backup for daily backups 624 | [[ "x$subfolder" != "xdaily" ]] && activate_differential_backup=0 || activate_differential_backup=1 625 | 626 | if (( $multipledbs )); then 627 | # multiple dbs 628 | subsubfolder="" 629 | name="all-databases" 630 | else 631 | # single db 632 | subsubfolder="/$1" 633 | name="$@" 634 | fi 635 | 636 | [[ -d "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" ]] || { 637 | if (( $CONFIG_dryrun )); then 638 | printf 'dry-running: mkdir -p %s/${subfolder}%s\n' "${CONFIG_backup_dir}" "${subsubfolder}" 639 | else 640 | mkdir -p "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" 641 | fi 642 | } 643 | 644 | manifest_file="${CONFIG_backup_dir}/${subfolder}${subsubfolder}/Manifest" 645 | fname="${CONFIG_backup_dir}/${subfolder}${subsubfolder}/${prefix}${name}_${datetimestamp}${midfix}${extension}" 646 | 647 | (( $CONFIG_debug )) && echo "DEBUG: process_dbs >> Setting manifest file to: ${manifest_file}" 648 | 649 | if (( $multipledbs )); then 650 | # multiple databases 651 | db="all-databases" 652 | else 653 | # single db 654 | db="$1" 655 | fi 656 | 657 | if [[ "x$CONFIG_mysql_dump_differential" = "xyes" ]] && [[ "x${CONFIG_encrypt}" != "xyes" ]] && (( $activate_differential_backup )); then 658 | 659 | 660 | unset manifest_entry manifest_entry_to_check 661 | 662 | echo "## Reading in Manifest file" 663 | parse_manifest "$manifest_file" 664 | echo 665 | echo "Number of manifest entries: $(num_manifest_entries)" 666 | echo 667 | 668 | 669 | # -> generate diff file 670 | let "filename_flags=0x00" 671 | 672 | # ## -> get latest differential manifest entry for specified db 673 | # if get_latest_manifest_entry_for_db "$db" 1; then 674 | # pid="${manifest_entry[2]}" 675 | # # filename format: prefix_db_YYYY-MM-DD_HHhMMm_[A-Za-z0-9]{8}(.sql|.diff)(.gz|.bz2)(.enc) 676 | # FileStub=${manifest_entry[0]%.@(sql|diff)*} 677 | # FileExt=${manifest_entry[0]#"$FileStub"} 678 | # re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_encrypted" 679 | # re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_gz" 680 | # re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_bz2" 681 | # re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_diff" 682 | # manifest_latest_diff_entry=("${manifest_entry[@]}") 683 | # else # no entries in manifest 684 | # pid=0 685 | # fi 686 | # ## <- get latest differential manifest entry for specified db 687 | 688 | ## -> get latest master manifest entry for specified db 689 | # Create a differential backup if a master entry in the manifest exists, it isn't the day we do weekly master backups or the master file we fetched is already from today. 690 | if get_latest_manifest_entry_for_db "$db" 0 && ( (( ${date_dayno_of_week} != ${CONFIG_do_weekly} )) || [[ "${manifest_entry[0]}" = *_$(date +%Y-%m-%d)_* ]] ); then 691 | pid="${manifest_entry[2]}" 692 | # filename format: prefix_db_YYYY-MM-DD_HHhMMm_[A-Za-z0-9]{8}(.sql|.diff)(.gz|.bz2)(.enc) 693 | FileStub="${manifest_entry[0]%.@(sql|diff)*}" 694 | FileExt="${manifest_entry[0]#"$FileStub"}" 695 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_encrypted" 696 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_gz" 697 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_bz2" 698 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "filename_flags|=$filename_flag_diff" 699 | manifest_latest_master_entry=("${manifest_entry[@]}") 700 | else # no entries in manifest 701 | pid=0 702 | fi 703 | ## <- get latest master manifest entry for specified db 704 | 705 | fi 706 | 707 | if [[ "x$CONFIG_mysql_dump_differential" = "xyes" ]] && [[ "x${CONFIG_encrypt}" != "xyes" ]] && (( $activate_differential_backup )) && ((! ($filename_flags & $filename_flag_encrypted) )); then 708 | 709 | # the master file is encrypted ... well this just shouldn't happen ^^ not going to decrypt or stuff like that ...at least not today :) 710 | 711 | if [[ "x$pid" = "x0" ]]; then 712 | # -> create master backup 713 | cfname="$(mktemp "${fname%.sql}_"XXXXXXXX".sql${suffix}")" 714 | uid="${cfname%.@(diff|sql)*}" 715 | uid="${uid:-8:8}" 716 | case "${CONFIG_mysql_dump_compression}" in 717 | 'gzip') 718 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | gzip_compression > "$cfname"; 719 | ;; 720 | 'bzip2') 721 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | bzip2_compression > "$cfname"; 722 | ;; 723 | *) 724 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" > "$cfname"; 725 | ;; 726 | esac 727 | add_manifest_entry "$manifest_file" "$cfname" "$pid" "$db" && parse_manifest "$manifest_file" && cp -al "$cfname" "${CONFIG_backup_dir}"/latest/ && echo "Generated master backup $cfname" && return 0 || return 1 728 | # <- create master backup 729 | else 730 | cfname="$(mktemp "${fname%.sql}_"XXXXXXXX".diff${suffix}")" 731 | uid="${cfname%.@(diff|sql)*}" 732 | uid=${uid:-8:8} 733 | echo "Creating differential backup to ${manifest_entry[0]}:" 734 | case "${CONFIG_mysql_dump_compression}" in 735 | 'gzip') 736 | if (( $filename_flags & $filename_flag_gz )); then 737 | diff <(gzip_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | gzip_compression > "$cfname"; 738 | elif (( $filename_flags & $filename_flag_bz2 )); then 739 | diff <(bzip2_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | gzip_compression > "$cfname"; 740 | else 741 | diff "${manifest_latest_master_entry[0]}" <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | gzip_compression > "$cfname"; 742 | fi 743 | ;; 744 | 'bzip2') 745 | if (( $filename_flags & $filename_flag_gz )); then 746 | diff <(gzip_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | bzip2_compression > "$cfname"; 747 | elif (( $filename_flags & $filename_flag_bz2 )); then 748 | diff <(bzip2_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | bzip2_compression > "$cfname"; 749 | else 750 | diff "${manifest_latest_master_entry[0]}" <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") | bzip2_compression > "$cfname"; 751 | fi 752 | ;; 753 | *) 754 | if (( $filename_flags & $filename_flag_gz )); then 755 | diff <(gzip_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") > "$cfname"; 756 | elif (( $filename_flags & $filename_flag_bz2 )); then 757 | diff <(bzip2_compression -dc "${manifest_latest_master_entry[0]}") <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") > "$cfname"; 758 | else 759 | diff "${manifest_latest_master_entry[0]}" <(mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@") > "$cfname"; 760 | fi 761 | ;; 762 | esac 763 | add_manifest_entry "$manifest_file" "$cfname" "$pid" "$db" && parse_manifest "$manifest_file" && cp -al "$cfname" "${manifest_latest_master_entry[0]}" "${CONFIG_backup_dir}"/latest/ && echo "generated $cfname" && return 0 || return 1 764 | 765 | fi 766 | # <- generate diff filename 767 | 768 | else 769 | cfname="${fname}${suffix}" 770 | if (( $CONFIG_dryrun )); then 771 | case "${CONFIG_mysql_dump_compression}" in 772 | 'gzip') 773 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt[@]} $@ | gzip_compression > ${cfname}" 774 | ;; 775 | 'bzip2') 776 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt[@]} $@ | bzip2_compression > ${cfname}" 777 | ;; 778 | *) 779 | echo "dry-running: mysqldump --user=${CONFIG_mysql_dump_username} --password=${CONFIG_mysql_dump_password} --host=${CONFIG_mysql_dump_host} ${opt[@]} $@ > ${cfname}" 780 | ;; 781 | esac 782 | return 0; 783 | else 784 | case "${CONFIG_mysql_dump_compression}" in 785 | 'gzip') 786 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | gzip_compression > "${cfname}" 787 | ret=$? 788 | ;; 789 | 'bzip2') 790 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" | bzip2_compression > "${cfname}" 791 | ret=$? 792 | ;; 793 | *) 794 | mysqldump --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${opt[@]}" "$@" > "${cfname}" 795 | ret=$? 796 | ;; 797 | esac 798 | fi 799 | 800 | fi 801 | 802 | 803 | if (( $ret == 0 )); then 804 | echo "Rotating $(( ${rotation}/${rotation_divisor} )) ${rotation_string} backups for ${name}" 805 | if (( $CONFIG_dryrun )); then 806 | find "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" -mtime +"${rotation}" -type f -exec echo "dry-running: rm" {} \; 807 | else 808 | find "${CONFIG_backup_dir}/${subfolder}${subsubfolder}" -mtime +"${rotation}" -type f -exec rm {} \; 809 | fi 810 | files_postprocessing "$cfname" 811 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 812 | backupfiles=( "${backupfiles[@]}" "${cfname}${var}" ) 813 | else 814 | let "E |= $E_dbdump_failed" 815 | echo "dbdump with parameters \"${CONFIG_db_names[@]}\" \"${cfname}\" failed!" 816 | fi 817 | } 818 | 819 | # @info: Save stdout and stderr 820 | # @deps: (none) 821 | activateIO() { 822 | ################################################################################### 823 | # IO redirection for logging. 824 | # $1 = $log_file, $2 = $log_errfile 825 | 826 | #(( $CONFIG_debug )) || { 827 | touch "$log_file" 828 | exec 6>&1 # Link file descriptor #6 with stdout. Saves stdout. 829 | exec > "$log_file" # stdout replaced with file $log_file. 830 | 831 | touch "$log_errfile" 832 | exec 7>&2 # Link file descriptor #7 with stderr. Saves stderr. 833 | exec 2> "$log_errfile" # stderr replaced with file $log_errfile. 834 | #} 835 | } 836 | 837 | # @info: Restore stdout and stderr redirections. 838 | # @deps: (none) 839 | removeIO() { 840 | exec 1>&6 6>&- # Restore stdout and close file descriptor #6. 841 | exec 2>&7 7>&- # Restore stdout and close file descriptor #7. 842 | } 843 | 844 | # @info: Checks directories and subdirectories for existence and activates logging to either 845 | # $CONFIG_backup_dir or /tmp depending on what exists. 846 | # @args: (none) 847 | # @deps: load_default_config, activateIO, chk_folder_writable, error_handler 848 | directory_checks_enable_logging () { 849 | ################################################################################### 850 | # Check directories and do cleanup work 851 | 852 | checkdirs=( "${CONFIG_backup_dir}"/{daily,weekly,monthly,latest,tmp} ) 853 | [[ "${CONFIG_backup_local_files[@]}" ]] && { checkdirs=( "${checkdirs[@]}" "${CONFIG_backup_dir}/backup_local_files" ); } 854 | [[ "${CONFIG_mysql_dump_full_schema}" = 'yes' ]] && { checkdirs=( "${checkdirs[@]}" "${CONFIG_backup_dir}/fullschema" ); } 855 | [[ "${CONFIG_mysql_dump_dbstatus}" = 'yes' ]] && { checkdirs=( "${checkdirs[@]}" "${CONFIG_backup_dir}/status" ); } 856 | 857 | tmp_permcheck=0 858 | printf '# Checking for permissions to write to folders:\n' 859 | 860 | 861 | # "dirname ${CONFIG_backup_dir}" exists? 862 | # Y -> ${CONFIG_backup_dir} exists? 863 | # Y -> Dry-run? 864 | # Y -> log to /tmp, proceed to test subdirs 865 | # N -> check writable ${CONFIG_backup_dir}? 866 | # Y -> proceed to test subdirs 867 | # N -> error: can't write to ${CONFIG_backup_dir}. Exit. 868 | # N -> Dry-run? 869 | # N -> proceed without testing subdirs 870 | # Y -> create directory ${CONFIG_backup_dir}? 871 | # Y -> check writable ${CONFIG_backup_dir}? 872 | # Y -> proceed to test subdirs 873 | # N -> error: can't write to ${CONFIG_backup_dir}. Exit. 874 | # N -> error: ${CONFIG_backup_dir} is not writable. Exit. 875 | # N -> Dry-run? 876 | # Y -> log to /tmp, proceed without testing subdirs 877 | # N -> error: no basedir. Exit. 878 | 879 | 880 | # -> check base folder 881 | printf 'base folder %s ... ' "$(dirname "${CONFIG_backup_dir}")" 882 | if [[ -d "$(dirname "${CONFIG_backup_dir}")" ]]; then 883 | 884 | printf 'exists ... ok.\n' 885 | printf 'backup folder %s ... ' "${CONFIG_backup_dir}" 886 | 887 | if [[ -d "${CONFIG_backup_dir}" ]]; then 888 | printf 'exists ... writable? ' 889 | if (( $CONFIG_dryrun )); then 890 | printf 'dry-running. Skipping. Logging to /tmp\n' 891 | log_file="/tmp/${CONFIG_mysql_dump_host}-`date +%N`.log" 892 | log_errfile="/tmp/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 893 | activateIO "$log_file" "$log_errfile" 894 | tmp_permcheck=1 895 | else 896 | if chk_folder_writable "${CONFIG_backup_dir}"; then 897 | printf 'yes. Proceeding.\n' 898 | log_file="${CONFIG_backup_dir}/${CONFIG_mysql_dump_host}-`date +%N`.log" 899 | log_errfile="${CONFIG_backup_dir}/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 900 | activateIO "$log_file" "$log_errfile" 901 | tmp_permcheck=1 902 | else 903 | printf 'no. Exiting.\n' 904 | let "E |= $E_config_backupdir_not_writable" 905 | error_handler 906 | fi 907 | fi 908 | 909 | else 910 | 911 | printf 'creating ... ' 912 | if (( $CONFIG_dryrun )); then 913 | printf 'dry-running. Skipping.\n' 914 | else 915 | if mkdir -p "${CONFIG_backup_dir}" >/dev/null 2>&1; then 916 | printf 'success.\n' 917 | log_file="${CONFIG_backup_dir}/${CONFIG_mysql_dump_host}-`date +%N`.log" 918 | log_errfile="${CONFIG_backup_dir}/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 919 | activateIO "$log_file" "$log_errfile" 920 | tmp_permcheck=1 921 | else 922 | printf 'failed. Exiting.\n' 923 | let "E |= $E_mkdir_basedir_failed" 924 | error_handler 925 | fi 926 | fi 927 | 928 | fi 929 | 930 | else 931 | 932 | if (( $CONFIG_dryrun )); then 933 | printf 'dry-running. Skipping. Logging to /tmp\n' 934 | log_file="/tmp/${CONFIG_mysql_dump_host}-`date +%N`.log" 935 | log_errfile="/tmp/ERRORS_${CONFIG_mysql_dump_host}-`date +%N`.log" 936 | activateIO "$log_file" "$log_errfile" 937 | else 938 | printf 'does not exist. Exiting.\n' 939 | let "E |= $E_no_basedir" 940 | error_handler 941 | fi 942 | 943 | fi 944 | # <- check base folder 945 | 946 | 947 | # -> check subdirs 948 | if (( $tmp_permcheck == 1 )); then 949 | 950 | (( $CONFIG_dryrun )) || [[ -r "${CONFIG_backup_dir}" && -x "${CONFIG_backup_dir}" ]] || { let "E |= $E_perm_basedir"; error_handler; } 951 | 952 | for i in "${checkdirs[@]}"; do 953 | printf 'checking directory "%s" ... ' "$i" 954 | if [[ -d "$i" ]]; then 955 | printf 'exists.\n' 956 | else 957 | printf 'creating ... ' 958 | if (( $CONFIG_dryrun )); then 959 | printf 'dry-running. Skipping.\n' 960 | else 961 | if mkdir -p "$i" >/dev/null 2>&1; then 962 | printf 'success.\n' 963 | else 964 | printf 'failed. Exiting.\n' 965 | let "E |= $E_mkdir_subdirs_failed" 966 | error_handler 967 | fi 968 | fi 969 | fi 970 | done 971 | 972 | fi 973 | # <- check subdirs 974 | 975 | } 976 | 977 | # @info: If CONFIG_mysql_dump_latest is set to 'yes', the directory ${CONFIG_backup_dir}"/latest will 978 | # be cleaned. 979 | # @args: (none) 980 | # @deps: load_default_config 981 | cleanup_latest () { 982 | # -> latest cleanup 983 | if [[ "${CONFIG_mysql_dump_latest}" = "yes" ]]; then 984 | printf 'Cleaning up latest directory ... ' 985 | if (( $CONFIG_dryrun )); then 986 | printf 'dry-running. Skipping.\n' 987 | else 988 | if rm -f "${CONFIG_backup_dir}"/latest/* >/dev/null 2>&1; then 989 | printf 'success.\n' 990 | else 991 | printf 'failed. Continuing anyway, activating Note-Flag.\n' 992 | let "N |= $N_latest_cleanup_failed" 993 | fi 994 | fi 995 | fi 996 | # <- latest cleanup 997 | } 998 | 999 | # @info: Checks for dependencies in form of external programs, that need to be available when running 1000 | # this program. 1001 | # @args: (none) 1002 | # @deps: load_default_config 1003 | check_dependencies () { 1004 | echo 1005 | echo "# Testing for installed programs" 1006 | dependencies=( 'mysql' 'mysqldump' ) 1007 | 1008 | if [[ "x$CONFIG_multicore" = 'xyes' ]]; then 1009 | 1010 | if [[ "x$CONFIG_mysql_dump_compression" = 'xbzip2' ]]; then 1011 | if type pbzip2 &>/dev/null; then 1012 | echo "pbzip2 ... found." 1013 | else 1014 | CONFIG_multicore='no' # turn off multicore support, since the program isn't there 1015 | echo "WARNING: Turning off multicore support, since pbzip2 isn't there." 1016 | fi 1017 | elif [[ "x$CONFIG_mysql_dump_compression" = 'xgzip' ]]; then 1018 | if type pigz &>/dev/null; then 1019 | echo "pigz ... found." 1020 | else 1021 | CONFIG_multicore='no' # turn off multicore support, since the program isn't there 1022 | echo "WARNING: Turning off multicore support, since pigz isn't there." 1023 | fi 1024 | fi 1025 | 1026 | else 1027 | [[ "x$CONFIG_mysql_dump_compression" = 'xbzip2' ]] && dependencies=("${dependencies[@]}" 'bzip2' ) 1028 | [[ "x$CONFIG_mysql_dump_compression" = 'xgzip' ]] && dependencies=("${dependencies[@]}" 'gzip' ) 1029 | fi 1030 | 1031 | if [[ "x$CONFIG_mailcontent" = 'xlog' || "x$CONFIG_mailcontent" = 'xquiet' ]]; then 1032 | dependencies=( "${dependencies[@]}" 'mail' ) 1033 | elif [[ "x$CONFIG_mailcontent" = 'xfiles' ]]; then 1034 | dependencies=( "${dependencies[@]}" 'mail' ) 1035 | if [[ "x$CONFIG_mail_use_uuencoded_attachments" != 'xyes' ]]; then 1036 | dependencies=( "${dependencies[@]}" 'mutt' ) 1037 | fi 1038 | fi 1039 | 1040 | for i in "${dependencies[@]}"; do 1041 | printf '%s ... ' "$i" 1042 | if type "$i" &>/dev/null; then 1043 | printf 'found.\n' 1044 | else 1045 | printf 'not found. Aborting.\n'; 1046 | let "E |= $E_missing_deps" 1047 | error_handler 1048 | fi 1049 | done 1050 | echo 1051 | } 1052 | 1053 | # @info: Get database list and remove excluded ones. 1054 | # @args: (none) 1055 | # @deps: load_default_config, error_handler 1056 | # 1057 | # alldbnames = array of all databases 1058 | # empty? -> error 1059 | # remove excludes from array alldbnames 1060 | # CONFIG_db_names empty? -> set to alldbnames 1061 | # CONFIG_db_month_names empty? -> set to alldbnames 1062 | # 1063 | parse_databases() { 1064 | # bash 4.x version 1065 | #mapfile -t alldbnames < <(mysql --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" --batch --skip-column-names -e "show databases") 1066 | alldbnames=() 1067 | 1068 | printf "# Parsing databases ... " 1069 | # bash 3.0 1070 | local i;i=0; 1071 | while read -r; do alldbnames[i++]="$REPLY"; done < <(mysql --user="${CONFIG_mysql_dump_username}" --password="${CONFIG_mysql_dump_password}" --host="${CONFIG_mysql_dump_host}" "${mysql_opt[@]}" --batch --skip-column-names -e "show databases") 1072 | unset i 1073 | 1074 | # mkfifo foo || exit; trap 'rm -f foo' EXIT 1075 | 1076 | ((! "${#alldbnames[@]}" )) && { let "E |= $E_db_empty"; error_handler; } 1077 | 1078 | # -> remove excluded dbs from list 1079 | for exclude in "${CONFIG_db_exclude[@]}"; do 1080 | for i in "${!alldbnames[@]}"; do if [[ "x${alldbnames[$i]}" = "x${exclude}" ]]; then unset 'alldbnames[i]'; fi; done 1081 | done 1082 | # <- remove excluded dbs from list 1083 | 1084 | # check for empty array lists and copy all dbs 1085 | ((! ${#CONFIG_db_names[@]})) && CONFIG_db_names=( "${alldbnames[@]}" ) 1086 | ((! ${#CONFIG_db_month_names[@]})) && CONFIG_db_month_names=( "${alldbnames[@]}" ) 1087 | printf "done.\n" 1088 | } 1089 | 1090 | # @return: true if locked, false otherwise 1091 | # @param: manifest_file 1092 | status_manifest() { 1093 | if [[ -e "$1".lock ]]; then 1094 | return 0 1095 | else 1096 | return 1 1097 | fi 1098 | } 1099 | # @return: true if successfully created lock file, else false 1100 | # @param: manifest_file 1101 | lock_manifest() { 1102 | if status_manifest "$1"; then 1103 | return 0 1104 | else 1105 | if touch "$1".lock &>/dev/null; then 1106 | return 0 1107 | else 1108 | return 1 1109 | fi 1110 | fi 1111 | } 1112 | # @return: true if successfully removed lock file, else false 1113 | # @param: manifest_file 1114 | unlock_manifest() { 1115 | if status_manifest "$1"; then 1116 | if rm "$1".lock &>/dev/null; then 1117 | return 0 1118 | else 1119 | return 1 1120 | fi 1121 | else 1122 | return 0 1123 | fi 1124 | } 1125 | # @return: true if unlock_manifest or lock_manifest, depending on status_manifest, return true, else false 1126 | # @param: manifest_file 1127 | toggle_manifest() { 1128 | if status_manifest "$1"; then 1129 | unlock_manifest "$1" && return 0 || return 1 1130 | else 1131 | lock_manifest "$1" && return 0 || return 1 1132 | fi 1133 | } 1134 | 1135 | # expects manifest_entry_to_check to be an array with four entries 1136 | # @param: manifest_file 1137 | # return: 0, if all is okay 1138 | # 1, file doesn't exist - removed entry from manifest 1139 | # 2, file doesn't exist - tried to remove entry from manifest, but failed 1140 | check_manifest_entry() { 1141 | local entry_md5sum 1142 | [[ ! -e "${manifest_entry_to_check[0]}" ]] && { rm_manifest_entry_by_filename "${manifest_entry_to_check[0]}" 1 && return 1 || return 2; } 1143 | entry_md5sum="$(md5sum "${manifest_entry_to_check[0]}" | awk '{print $1}')" 1144 | if [[ "${entry_md5sum}" != "${manifest_entry_to_check[1]}" ]]; then 1145 | printf 'g/%s/s//%s/g\nw\nq' "${manifest_entry_to_check[1]}" "${entry_md5sum}" | ed -s "$1" 1146 | else 1147 | return 0 1148 | fi 1149 | } 1150 | 1151 | # parse manifest file and collect entries in manifest_array 1152 | # @param: manifest_file 1153 | # 1154 | # sort manifest file after first field (filename) -> read this line by line 1155 | # check if line matches regexp || add to array manifest_entries_corrupted && continue 1156 | # split lines at tab character \t and put into array line_arr 1157 | # check manifest entry 1158 | # -> file does not exist -> remove entry from manifest; continue no matter if this succeeds or not 1159 | # loop through previous entries in the manifest 1160 | # filename already in there? remove all entries with the same filename but the one that is already in the array 1161 | # md5sum has already occured? 1162 | # if size = 0 1163 | # then don't compare 1164 | # else 1165 | # request user action by adding entry to array manifest_entries_user_action_required with information, that identical files exist && continue 2 1166 | # fi 1167 | # add entry to manifest_array 1168 | # 1169 | parse_manifest() { 1170 | local i n re line line_arr check 1171 | unset manifest_array; manifest_array=() 1172 | local tmp_md5sum 1173 | # array ( filename_1, md5sum_1, id_1[, rel_id_1] ), ... ) 1174 | # reserving 4 members for each entry, thus each filename entry in the array has array key 4(n-1)+1 1175 | (( $CONFIG_debug )) && echo ">>>>>>> Parsing manifest file: $1" 1176 | n=1 1177 | [[ -s "$1" ]] && 1178 | while read line 1179 | do 1180 | # ANY CHANGES INSIDE HERE ON THE MANIFEST_FILE HAVE NO IMPACT ON THE LINES WE LOOP OVER; THE sort COMMAND READS THE FILE ENTIRELY AT THE BEGINNING AND PASSES THE OUTPUT TO THE LOOP 1181 | # check if line has expected format, i.e. check against regular expression 1182 | re=$'^[^\t]*\tmd5sum\t[^\t]*\tdiff_id\t[A-Za-z0-9]{8}\trel_id\t(0|[A-Za-z0-9]{8})\tdb\t[^\t]*$' 1183 | [[ $line =~ $re ]] || { echo "Corrupted line: $line"; manifest_entries_corrupted=( "${manifest_entries_corrupted[@]}" "$1" "$line" ); continue; } 1184 | IFS=$'\t' read -ra line_arr <<< "$line" 1185 | 1186 | # prepare array of the current line 1187 | manifest_entry_to_check=() 1188 | for ((i=0;i<${#line_arr[@]};i=$i+2)); do 1189 | manifest_entry_to_check[i/2]="${line_arr[i]}" 1190 | done 1191 | # check manifest entry, which uses the array manifest_entry_to_check 1192 | check_manifest_entry "$1" 1193 | check=$? 1194 | case $check in 1195 | 1) (( $CONFIG_debug )) && echo "File for manifest entry $line does not exist. Entry removed." 1196 | continue # file doesn't exist - removed entry from manifest 1197 | break;; 1198 | 2) (( $CONFIG_debug )) && echo "File for manifest entry $line does not exist. Failed to remove the entry." 1199 | continue # file doesn't exist - tried to remove entry from manifest, but failed 1200 | break;; 1201 | esac 1202 | 1203 | # loop through the manifest_array, as it has been filled by now and check if an entry already exists with the same values 1204 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1205 | if [[ "x${manifest_array[i]}" = "x${line_arr[0]}" ]]; then # found entry with the same filename 1206 | (( $CONFIG_debug )) && echo "Found multiple entries with the same filename. Removing all but the first-found entry from manifest." 1207 | # remove all entries with this filename and add a new one based on the values of the item already in the array 1208 | rm_manifest_entry_by_filename "$1" "${manifest_array[i]}" 1 && add_manifest_entry "$1" "${manifest_array[i]}" "${manifest_array[i+3]}" 1209 | continue 2 # the original entry, to which we compared, is already in the manifest_array; no matter if this is resolved or not, we don't 1210 | # need to add this entry to the manifest_array 1211 | elif [[ "x${manifest_array[i+1]}" = "x${line_arr[2]}" ]]; then # found entry with different filename but same md5sum - file copied and renamed?! 1212 | if [[ ! -s "${line_arr[0]}" ]]; then # empty file - don't start to compare md5sums ... 1213 | (( $CONFIG_debug )) && echo "Found empty file ${line_arr[0]}." 1214 | else 1215 | (( $CONFIG_debug )) && echo "Found multiple entries with the same md5sum but different filename." 1216 | (( $CONFIG_debug )) && echo -e ">> fname_manifest:\t${manifest_array[i]}\t${manifest_array[i+1]}\n>> fname_line:\t\t${line_arr[0]}\t${line_arr[2]}" 1217 | if [[ "x${line_arr[6]}" != "x0" ]]; then 1218 | if [[ "x${manifest_array[i+3]}" = "x${line_arr[6]}" ]]; then # parent id is the same; TODO inform user of this predicament and suggest solution 1219 | manifest_entries_user_action_required=( "${manifest_entries_user_action_required[@]}" "$1" "${manifest_array[i]}" "The file has an identical copy with the same parent id. If you don't know why it exists, it is safe to remove it." ) 1220 | continue 2 1221 | else 1222 | manifest_entries_user_action_required=( "${manifest_entries_user_action_required[@]}" "$1" "${manifest_array[i]}" "The file has an identical copy with different parent id. This should not happen. Remove the file, which is not the correct follow-up to the previous differential or master backup." ) 1223 | continue 2 1224 | fi 1225 | fi 1226 | fi 1227 | fi 1228 | done 1229 | 1230 | # add entry to manifest array 1231 | for ((i=0;i<${#line_arr[@]};i=$i+2)); do 1232 | manifest_array[(n-1)*${fields}+i/2]="${line_arr[i]}" 1233 | #echo "manifest array key $((($n-1)*4+$i/2)) with value ${line_arr[i]}" 1234 | done 1235 | 1236 | ((n++)) 1237 | done < <(sort -t $'\t' -k"1" "$1") 1238 | (( $CONFIG_debug )) && echo "<<<<<<< # manifest entries: $((${#manifest_array[@]}/$fields))" 1239 | (( $CONFIG_debug )) && echo "<<<<<<< FINISHED" 1240 | return 0 1241 | } 1242 | 1243 | # get_manifest_entry_by_* PATTERN [regexp] 1244 | # if second parameter 'regexp' (string!) is passed, PATTERN will be matched as regular expression 1245 | get_manifest_entry_by_filename() { 1246 | local i 1247 | if [[ "x$2" = "xregexp" ]]; then 1248 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1249 | if [[ "${manifest_array[i]}" =~ $1 ]]; then 1250 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1251 | return 0 1252 | break; 1253 | fi 1254 | done 1255 | else 1256 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1257 | if [[ "x${manifest_array[i]}" = "x$1" ]]; then 1258 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1259 | return 0 1260 | break; 1261 | fi 1262 | done 1263 | fi 1264 | return 1 1265 | } 1266 | get_manifest_entry_by_md5sum() { 1267 | local i 1268 | if [[ "x$2" = "xregexp" ]]; then 1269 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1270 | if [[ "${manifest_array[i+1]}" =~ $1 ]]; then 1271 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1272 | return 0 1273 | break; 1274 | fi 1275 | done 1276 | else 1277 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1278 | if [[ "x${manifest_array[i+1]}" = "x$1" ]]; then 1279 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1280 | return 0 1281 | break; 1282 | fi 1283 | done 1284 | fi 1285 | return 1 1286 | } 1287 | get_manifest_entry_by_id() { 1288 | local i 1289 | if [[ "x$2" = "xregexp" ]]; then 1290 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1291 | if [[ "${manifest_array[i+2]}" =~ $1 ]]; then 1292 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1293 | return 0 1294 | break; 1295 | fi 1296 | done 1297 | else 1298 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1299 | if [[ "x${manifest_array[i+2]}" = "x$1" ]]; then 1300 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1301 | return 0 1302 | break; 1303 | fi 1304 | done 1305 | fi 1306 | return 1 1307 | } 1308 | get_manifest_entry_by_rel_id() { 1309 | local i 1310 | if [[ "x$2" = "xregexp" ]]; then 1311 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1312 | if [[ "${manifest_array[i+3]}" =~ $1 ]]; then 1313 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1314 | return 0 1315 | break; 1316 | fi 1317 | done 1318 | else 1319 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1320 | if [[ "x${manifest_array[i+3]}" = "x$1" ]]; then 1321 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1322 | return 0 1323 | break; 1324 | fi 1325 | done 1326 | fi 1327 | return 1 1328 | } 1329 | get_manifest_entry_by_db() { 1330 | local i 1331 | if [[ "x$2" = "xregexp" ]]; then 1332 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1333 | if [[ "${manifest_array[i+4]}" =~ $1 ]]; then 1334 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1335 | return 0 1336 | break; 1337 | fi 1338 | done 1339 | else 1340 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1341 | if [[ "x${manifest_array[i+4]}" = "x$1" ]]; then 1342 | manifest_entry=( "${manifest_array[i]}" "${manifest_array[i+1]}" "${manifest_array[i+2]}" "${manifest_array[i+3]}" "${manifest_array[i+4]}" ) 1343 | return 0 1344 | break; 1345 | fi 1346 | done 1347 | fi 1348 | return 1 1349 | } 1350 | 1351 | # @params: db, master/diff (0,1) 1352 | # @return: 2: no entries in manifest for the specified database 'db' 1353 | # 1: could not get manifest element by filename 1354 | # 0: all fine, match is in array 'manifest_entry' 1355 | get_latest_manifest_entry_for_db() { 1356 | local db_array newarray i 1357 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 1358 | if (( $2 )); then # latest differential or master backup, i.e. just take the latest one! 1359 | if [[ "x${manifest_array[i+4]}" = "x$1" ]]; then 1360 | db_array=( "${db_array[@]}" "${manifest_array[i]}" ) 1361 | fi 1362 | else # latest master backup, pid=0 1363 | if [[ "x${manifest_array[i+4]}" = "x$1" && "x${manifest_array[i+3]}" = "x0" ]]; then 1364 | db_array=( "${db_array[@]}" "${manifest_array[i]}") 1365 | fi 1366 | fi 1367 | done 1368 | if (( "${#db_array[@]}" == 0 )); then return 2; 1369 | else 1370 | #newarray=(); while IFS= read -r -d '' line; do newarray+=("$line"); done < <(printf '%s\0' "${db_array[@]}" | sort -z) 1371 | get_manifest_entry_by_filename "${db_array[@]:(-1)}" # last entry of db_array, has, due to the way sort works, to be the latest one 1372 | return $? 1373 | fi 1374 | } 1375 | 1376 | # @params: manifest_file filename/md5sum/id/rel_id [1(=don't parse manifest after finished)] 1377 | # if second parameters 1378 | # 1379 | # lock manifest -> use awk, print all lines that don't have second parameter at the appropriate field -> unlock manifest 1380 | # param3=0 -> parse manifest 1381 | # 1382 | rm_manifest_entry_by_filename() { 1383 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$1 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1384 | (( "$3" )) || parse_manifest "$1" 1385 | return 0 1386 | } 1387 | rm_manifest_entry_by_md5sum() { 1388 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$3 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1389 | (( "$3" )) || parse_manifest "$1" 1390 | return 0 1391 | } 1392 | rm_manifest_entry_by_id() { 1393 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$5 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1394 | (( "$3" )) || parse_manifest "$1" 1395 | return 0 1396 | } 1397 | rm_manifest_entry_by_rel_id() { 1398 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$7 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1399 | (( "$3" )) || parse_manifest "$1" 1400 | return 0 1401 | } 1402 | rm_manifest_entry_by_db() { 1403 | lock_manifest "$1" && awk -F"\t" -v v="$2" '$9 != v' "$1" > "$1".tmp && mv "$1".tmp "$1" && unlock_manifest "$1" || return 1 1404 | (( "$3" )) || parse_manifest "$1" 1405 | return 0 1406 | } 1407 | 1408 | # parameters: manifest_file, filename, parent_id, db 1409 | add_manifest_entry() { 1410 | local md5sum 1411 | local id 1412 | local filename 1413 | local parent_id 1414 | local db 1415 | filename="$2" 1416 | parent_id="$3" 1417 | db="$4" 1418 | lock_manifest "$1" || return 1 1419 | id="${filename%.@(diff|sql)*}" 1420 | id="${id:(-8):8}" 1421 | #id="$(echo $filename | sed -re 's/.*_[0-9]{2}h[0-9]{2}m_([^\.]*)\..*/\1/')" 1422 | md5sum="$(md5sum "$filename" | awk '{print $1}')" 1423 | if [[ "x$parent_id" = 'x' ]]; then 1424 | echo -e "${filename}\tmd5sum\t${md5sum}\tdiff_id\t${id}\trel_id\t0\tdb\t${db}" >> "$1" 1425 | else 1426 | echo -e "${filename}\tmd5sum\t${md5sum}\tdiff_id\t${id}\trel_id\t${parent_id}\tdb\t${db}" >> "$1" 1427 | fi 1428 | unlock_manifest "$1" || return 1 1429 | } 1430 | 1431 | # @info: Echos number of manifest entries. 1432 | num_manifest_entries() { 1433 | echo "$((${#manifest_array[@]}/$fields))" 1434 | } 1435 | 1436 | # @info: Test if a value is in the array testarray 1437 | # @param: value 1438 | # @var in_array_index: array index of the first match 1439 | # @return 0 if a match was found, otherwise 1 1440 | in_array() { 1441 | local j 1442 | for ((j=0;j<"${#testarray[@]}";j++)); do 1443 | if [[ "x${testarray[j]}" = "x$1" ]]; then 1444 | in_array_index=$j 1445 | return 0 1446 | fi 1447 | done 1448 | return 1 1449 | } 1450 | 1451 | # @param: clear(0/1), meta_information, list_value1, list_value2, ... 1452 | extended_select() { 1453 | local a c k m i r r_number meta_information choice selection do_clear 1454 | meta_information="$2" 1455 | do_clear="$1" 1456 | shift 2 1457 | declare -a list=("$@") 1458 | selection=() 1459 | # BEGIN _select_filenames 1460 | #tput sc 1461 | while true; do 1462 | if (( $do_clear )); then 1463 | clear 1464 | else 1465 | : #tput rc 1466 | fi 1467 | declare -a testarray=("${selection[@]}") 1468 | echo "Selection for <$meta_information>" 1469 | echo "Notation: 1,2-4,-5,-6-9 or * or -* ('-' will remove selections)." 1470 | # print options 1471 | 1472 | for ((i=0;i<"${#list[@]}";i++)); do 1473 | if in_array $i; then 1474 | echo -e "$i) [+]\t${list[i]}" 1475 | else 1476 | echo -e "$i) [ ]\t${list[i]}" 1477 | fi 1478 | done 1479 | 1480 | echo -e "$i)\tDONE" 1481 | done_id=$i 1482 | 1483 | min=0 1484 | max=${#list[@]} # we have to account for the last possible number of DONE 1485 | # evaluate response 1486 | while true; do 1487 | printf '#? ' 1488 | read choice 1489 | r='^((-?[0-9]+(-[0-9]+)?,)*-?[0-9]+(-[0-9]+)?|-?\*)$' 1490 | [[ $choice =~ $r ]] || continue 1491 | if [[ "x$choice" = 'x*' ]]; then 1492 | unset m 1493 | for ((m=0;m<"${#list[@]}";m++)); do 1494 | selection=("${selection[@]}" "$m") 1495 | done 1496 | continue 2 1497 | elif [[ "x$choice" = 'x-*' ]]; then 1498 | selection=() 1499 | continue 2 1500 | else 1501 | unset string num1 num2 op op_rm 1502 | r_number='^[0-9]$' 1503 | 1504 | # BEGIN process_choice 1505 | for ((a=0;a<${#choice};a++)) 1506 | do 1507 | c="${choice:a:1}" 1508 | declare -a testarray=("${selection[@]}") 1509 | 1510 | if (( ${#string} == 0 )) && [[ "x$c" = "x-" ]] && ! (($op)); then 1511 | op_rm=1 1512 | continue 1513 | elif [[ $c =~ $r ]]; then 1514 | string=${string}"$c" 1515 | if (( $a == (${#choice}-1) )); then # last character 1516 | # we have a A-B case 1517 | if (($op)); then 1518 | num2="$string" 1519 | unset k 1520 | for ((k=$num1;k<=$num2;k++)); do 1521 | (( $k >= $min )) && (( $k <= $max )) || continue 1522 | if ! in_array $k; then 1523 | selection=("${selection[@]}" $k) 1524 | else 1525 | if (( $op_rm )); then 1526 | new_array=() 1527 | for ((m="$((${#selection[@]}-1))";m>=0;m--)); do 1528 | if [[ "x${selection[m]}" != "x$k" ]]; then 1529 | new_array=("${new_array[@]}" "${selection[m]}") 1530 | fi 1531 | done 1532 | declare -a selection=("${new_array[@]}") 1533 | fi 1534 | fi 1535 | done 1536 | unset op op_rm num1 num2 string 1537 | continue 1538 | else 1539 | (( $string >= $min )) && (( $string <= $max )) || continue 1540 | if ! in_array "$string"; then 1541 | selection=("${selection[@]}" "$string") 1542 | else 1543 | if (($op_rm)); then 1544 | unset m 1545 | new_array=() 1546 | for ((m=0;m<"${#selection[@]}";m++)); do 1547 | if [[ "x${selection[m]}" != "x$string" ]]; then 1548 | new_array=("${new_array[@]}" "${selection[m]}") 1549 | fi 1550 | done 1551 | declare -a selection=("${new_array[@]}") 1552 | fi 1553 | fi 1554 | fi 1555 | else 1556 | continue 1557 | fi 1558 | elif [[ "x$c" = "x-" ]]; then 1559 | num1="$string" 1560 | unset string 1561 | op=1 1562 | if (( $a == (${#choice}-1) )); then 1563 | break 1564 | else 1565 | continue 1566 | fi 1567 | elif [[ "x$c" = "x," ]]; then 1568 | # we have a A-B case 1569 | if (($op)); then 1570 | num2="$string" 1571 | unset k 1572 | for ((k=$num1;k<=$num2;k++)); do 1573 | (( $k >= $min )) && (( $k <= $max )) || continue 1574 | if ! in_array $k; then 1575 | selection=("${selection[@]}" $k) 1576 | else 1577 | if (( $op_rm )); then 1578 | unset m 1579 | new_array=() 1580 | for ((m=0;m<"${#selection[@]}";m++)); do 1581 | if [[ "x${selection[m]}" != "x$k" ]]; then 1582 | new_array=("${new_array[@]}" "${selection[m]}") 1583 | fi 1584 | done 1585 | declare -a selection=("${new_array[@]}") 1586 | fi 1587 | fi 1588 | done 1589 | unset op op_rm num1 num2 string 1590 | continue 1591 | else # it's just a single number 1592 | (( $string >= $min )) && (( $string <= $max )) || { unset op op_rm num1 num2 string; continue; } 1593 | if ! in_array "$string"; then 1594 | selection=("${selection[@]}" "$string") 1595 | else 1596 | if (($op_rm)); then 1597 | unset m 1598 | new_array=() 1599 | for ((m=0;m<"${#selection[@]}";m++)); do 1600 | if [[ "x${selection[m]}" != "x$string" ]]; then 1601 | new_array=("${new_array[@]}" "${selection[m]}") 1602 | fi 1603 | done 1604 | declare -a selection=("${new_array[@]}") 1605 | fi 1606 | fi 1607 | unset op op_rm num1 num2 string 1608 | continue 1609 | fi 1610 | else 1611 | continue 2; # this should not happen 1612 | fi 1613 | done 1614 | # END process_choice 1615 | 1616 | declare -a testarray=("${selection[@]}") 1617 | if in_array "$done_id"; then 1618 | break 2 1619 | else 1620 | continue 2 1621 | fi 1622 | fi 1623 | done 1624 | done 1625 | extended_select_return=() 1626 | extended_select_return_id=() 1627 | for i in "${selection[@]}"; do 1628 | [[ "x$i" != "x$done_id" ]] && { extended_select_return=("${extended_select_return[@]}" "${list[i]}"); extended_select_return_id=("${extended_select_return_id[@]}" "$i"); } 1629 | done 1630 | 1631 | } 1632 | 1633 | # END _functions 1634 | 1635 | 1636 | # BEGIN _methods 1637 | 1638 | # @info: Backup method 1639 | method_backup () { 1640 | manifest_entries_corrupted=() 1641 | manifest_entries_user_action_required=() 1642 | 1643 | 1644 | # END __FUNCTIONS 1645 | ############################################################################################################## 1646 | # BEGIN __STARTUP 1647 | 1648 | load_default_config 1649 | 1650 | trap mail_cleanup EXIT SIGHUP SIGINT SIGQUIT SIGTERM 1651 | if [[ -r "${CONFIG_configfile}" ]]; then source "${CONFIG_configfile}"; echo "Parsed config file \"${CONFIG_configfile}\""; else let "N |= $N_config_file_missing"; fi; echo 1652 | if (( $opt_flag_config_file )); then if [[ -r "${opt_config_file}" ]]; then source "${opt_config_file}"; let "N |= $N_arg_conffile_parsed"; else let "N |= $N_arg_conffile_unreadable"; fi; else let "N |= $N_too_many_args"; fi 1653 | 1654 | (( $CONFIG_dryrun )) && { 1655 | echo "NOTE: We are dry-running. That means, that the script just shows you what it would do, if it were operating normally." 1656 | echo "THE PRINTED COMMANDS CAN'T BE COPIED AND EXECUTED IF THERE ARE SPECIAL CHARACTERS, SPACES, ETC. IN THERE THAT WOULD NEED TO BE PROPERLY QUOTED IN ORDER TO WORK. THESE WERE CORRECTLY QUOTED FOR THE OUTPUT COMMAND, BUT CAN'T BE SEEN NOW." 1657 | echo 1658 | } 1659 | 1660 | export LC_ALL=C 1661 | PROGNAME=`basename $0` 1662 | PATH=${PATH}:/usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 1663 | version=3.0 1664 | fields=5 # manifest fields 1665 | 1666 | directory_checks_enable_logging 1667 | cleanup_latest 1668 | set_datetime_vars 1669 | check_dependencies # check for required programs 1670 | parse_configuration # parse configuration and set variables appropriately 1671 | 1672 | 1673 | # END __STARTUP 1674 | #-------------------------------------------------------------------------------------------------------------------------------------- 1675 | # BEGIN __PREPARE 1676 | 1677 | backupfiles=() 1678 | parse_databases 1679 | 1680 | # debug output of variables 1681 | (( $CONFIG_debug )) && { echo; echo "# DEBUG: printing all current variables"; declare -p | egrep -o '.* (CONFIG_[a-z_]*|opt|mysql_opt|opt_dbstatus|opt_fullschema)=.*'; echo; } 1682 | (( $CONFIG_debug )) && { echo "DEBUG: before pre-backup"; ( IFS=,; echo "DEBUG: CONFIG_db_names '${CONFIG_db_names[*]}'" ); ( IFS=,; echo "DEBUG: CONFIG_db_month_names '${CONFIG_db_month_names[*]}'" );} 1683 | 1684 | 1685 | # END __PREPARE 1686 | #-------------------------------------------------------------------------------------------------------------------------------------- 1687 | # BEGIN __MAIN 1688 | 1689 | ### filename formats 1690 | ## 1691 | ## example date values: 1692 | # 14'th of August (08) 2011 1693 | # week number: 32 1694 | # Sunday (date_dayno_of_week: 7) 1695 | ## 1696 | ## separate db's: 1697 | # monthly_DBNAME_2011-08-14_18h12m_August.sql(.enc).{gz,bzip2} 1698 | # weekly_DBNAME_2011-08-14_18h12m_32.sql(.enc).{gz,bzip2} 1699 | # daily_DBNAME_2011-08-14_18h12m_7.sql(.enc).{gz,bzip2} 1700 | ## all-databases: 1701 | # monthly_all-databases_DBNAME_2011-08-14_18h12m_August.sql(.enc).{gz,bzip2} 1702 | # weekly_all-databases_DBNAME_2011-08-14_18h12m_32.sql(.enc).{gz,bzip2} 1703 | # daily_all-databases_DBNAME_2011-08-14_18h12m_7.sql(.enc).{gz,bzip2} 1704 | 1705 | echo "======================================================================" 1706 | echo "AutoMySQLBackup version ${version}" 1707 | echo "http://sourceforge.net/projects/automysqlbackup/" 1708 | echo 1709 | echo "Backup of Database Server - ${CONFIG_mysql_dump_host_friendly:-$CONFIG_mysql_dump_host}" 1710 | ( IFS=,; echo "Databases - ${CONFIG_db_names[*]}" ) 1711 | ( IFS=,; echo "Databases (monthly) - ${CONFIG_db_month_names[*]}" ) 1712 | echo "======================================================================" 1713 | 1714 | 1715 | # -> preback commands 1716 | if [[ "${CONFIG_prebackup}" ]]; then 1717 | echo "======================================================================" 1718 | echo "Prebackup command output." 1719 | echo 1720 | source ${CONFIG_prebackup} 1721 | echo 1722 | echo "======================================================================" 1723 | echo 1724 | fi 1725 | # <- preback commands 1726 | 1727 | # -> backup local files 1728 | if [[ "${CONFIG_backup_local_files[@]}" ]] && [[ ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} = ${CONFIG_do_weekly} ]] && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/backup_local_files/bcf_weekly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_weekno}.tar${suffix}"); ((! ${#f[@]}))); then 1729 | echo "======================================================================" 1730 | echo "Backup local files. Doing this weekly on CONFIG_do_weekly." 1731 | echo 1732 | backup_local_files "${CONFIG_backup_dir}/backup_local_files/bcf_weekly_${datetimestamp}_${date_weekno}.tar" 1733 | tmp_flags=$?; var=; 1734 | if (( $? == 0 )); then 1735 | echo "success!" 1736 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/backup_local_files/bcf_weekly_${datetimestamp}_${date_weekno}.tar" ) 1737 | else 1738 | let "E |= $E_backup_local_failed" 1739 | echo "failed!" 1740 | fi 1741 | echo 1742 | echo "======================================================================" 1743 | echo 1744 | fi 1745 | # <- backup local files 1746 | 1747 | # -> dump full schema 1748 | if [[ "${CONFIG_mysql_dump_full_schema}" = 'yes' ]]; then 1749 | echo "======================================================================" 1750 | echo "Dump full schema." 1751 | echo 1752 | 1753 | # monthly 1754 | if (( ${CONFIG_do_monthly} != 0 && (${date_day_of_month} == ${CONFIG_do_monthly} || $date_day_of_month == $date_lastday_of_this_month && $date_lastday_of_this_month < ${CONFIG_do_monthly}) )) && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/fullschema/fullschema_monthly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_month}.sql${suffix}"); ((! ${#f[@]}))); then 1755 | fullschema "${CONFIG_backup_dir}/fullschema/fullschema_monthly_${datetimestamp}_${date_month}.sql" 1756 | if (( $? == 0 )); then 1757 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1758 | if (( $CONFIG_dryrun )); then 1759 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'fullschema_monthly*' -exec echo "dry-running: rm" {} \; 1760 | else 1761 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'fullschema_monthly*' -exec rm {} \; 1762 | fi 1763 | files_postprocessing "${CONFIG_backup_dir}/fullschema/fullschema_monthly_${datetimestamp}_${date_month}.sql${suffix}" 1764 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1765 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/fullschema/fullschema_monthly_${datetimestamp}_${date_month}.sql${suffix}${var}" ) 1766 | else 1767 | let "E |= $E_dump_fullschema_failed" 1768 | fi 1769 | fi 1770 | 1771 | # weekly 1772 | if [[ ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} = ${CONFIG_do_weekly} ]] && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/fullschema/fullschema_weekly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_weekno}.sql${suffix}"); ((! ${#f[@]}))); then 1773 | fullschema "${CONFIG_backup_dir}/fullschema/fullschema_weekly_${datetimestamp}_${date_weekno}.sql" 1774 | if (( $? == 0 )); then 1775 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1776 | if (( $CONFIG_dryrun )); then 1777 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'fullschema_weekly*' -exec echo "dry-running: rm" {} \; 1778 | else 1779 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'fullschema_weekly*' -exec rm {} \; 1780 | fi 1781 | files_postprocessing "${CONFIG_backup_dir}/fullschema/fullschema_weekly_${datetimestamp}_${date_weekno}.sql${suffix}" 1782 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1783 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/fullschema/fullschema_weekly_${datetimestamp}_${date_weekno}.sql${suffix}${var}" ) 1784 | else 1785 | let "E |= $E_dump_fullschema_failed" 1786 | fi 1787 | fi 1788 | 1789 | # daily 1790 | fullschema "${CONFIG_backup_dir}/fullschema/fullschema_daily_${datetimestamp}_${date_day_of_week}.sql" 1791 | if (( $? == 0 )); then 1792 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1793 | if (( $CONFIG_dryrun )); then 1794 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_daily}" -type f -name 'fullschema_daily*' -exec echo "dry-running: rm" {} \; 1795 | else 1796 | find "${CONFIG_backup_dir}/fullschema" -mtime +"${CONFIG_rotation_daily}" -type f -name 'fullschema_daily*' -exec rm {} \; 1797 | fi 1798 | files_postprocessing "${CONFIG_backup_dir}/fullschema/fullschema_daily_${datetimestamp}_${date_day_of_week}.sql${suffix}" 1799 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1800 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/fullschema/fullschema_daily_${datetimestamp}_${date_day_of_week}.sql${suffix}${var}" ) 1801 | else 1802 | let "E |= $E_dump_fullschema_failed" 1803 | fi 1804 | echo 1805 | echo "======================================================================" 1806 | echo 1807 | 1808 | fi 1809 | # <- dump full schema 1810 | 1811 | # -> dump status 1812 | if [[ "${CONFIG_mysql_dump_dbstatus}" = 'yes' ]]; then 1813 | echo "======================================================================" 1814 | echo "Dump status." 1815 | echo 1816 | 1817 | # monthly 1818 | if (( ${CONFIG_do_monthly} != 0 && (${date_day_of_month} == ${CONFIG_do_monthly} || $date_day_of_month == $date_lastday_of_this_month && $date_lastday_of_this_month < ${CONFIG_do_monthly}) )) && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/status/status_monthly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_month}.txt${suffix}"); ((! ${#f[@]}))); then 1819 | dbstatus "${CONFIG_backup_dir}/status/status_monthly_${datetimestamp}_${date_month}.txt" 1820 | if (( $? == 0 )); then 1821 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1822 | if (( $CONFIG_dryrun )); then 1823 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'status_monthly*' -exec echo "dry-running: rm" {} \; 1824 | else 1825 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_monthly}" -type f -name 'status_monthly*' -exec rm {} \; 1826 | fi 1827 | files_postprocessing "${CONFIG_backup_dir}/status/status_monthly_${datetimestamp}_${date_month}.txt${suffix}" 1828 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1829 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/status/status_monthly_${datetimestamp}_${date_month}.txt${suffix}${var}" ) 1830 | else 1831 | let "E |= $E_dump_status_failed" 1832 | fi 1833 | fi 1834 | 1835 | # weekly 1836 | if [[ ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} = ${CONFIG_do_weekly} ]] && (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/status/status_weekly_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m_${date_weekno}.txt${suffix}"); ((! ${#f[@]}))); then 1837 | dbstatus "${CONFIG_backup_dir}/status/status_weekly_${datetimestamp}_${date_weekno}.txt" 1838 | if (( $? == 0 )); then 1839 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1840 | if (( $CONFIG_dryrun )); then 1841 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'status_weekly*' -exec echo "dry-running: rm" {} \; 1842 | else 1843 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_weekly}" -type f -name 'status_weekly*' -exec rm {} \; 1844 | fi 1845 | files_postprocessing "${CONFIG_backup_dir}/status/status_weekly_${datetimestamp}_${date_weekno}.txt${suffix}" 1846 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1847 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/status/status_weekly_${datetimestamp}_${date_weekno}.txt${suffix}${var}" ) 1848 | else 1849 | let "E |= $E_dump_status_failed" 1850 | fi 1851 | fi 1852 | 1853 | # daily 1854 | dbstatus "${CONFIG_backup_dir}/status/status_daily_${datetimestamp}_${date_day_of_week}.txt" 1855 | if (( $? == 0 )); then 1856 | echo "Rotating $(( ${CONFIG_rotation_monthly}/31 )) month backups for ${mdb}" 1857 | if (( $CONFIG_dryrun )); then 1858 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_daily}" -type f -name 'status_daily*' -exec echo "dry-running: rm" {} \; 1859 | else 1860 | find "${CONFIG_backup_dir}/status" -mtime +"${CONFIG_rotation_daily}" -type f -name 'status_daily*' -exec rm {} \; 1861 | fi 1862 | files_postprocessing "${CONFIG_backup_dir}/status/status_daily_${datetimestamp}_${date_day_of_week}.txt${suffix}" 1863 | tmp_flags=$?; var=; (( $tmp_flags & $flags_files_postprocessing_success_encrypt )) && var=.enc 1864 | backupfiles=( "${backupfiles[@]}" "${CONFIG_backup_dir}/status/status_daily_${datetimestamp}_${date_day_of_week}.txt${suffix}${var}" ) 1865 | else 1866 | let "E |= $E_dump_status_failed" 1867 | fi 1868 | echo 1869 | echo "======================================================================" 1870 | echo 1871 | 1872 | fi 1873 | # <- dump status 1874 | 1875 | 1876 | # -> BACKUP DATABASES 1877 | echo "Backup Start Time `date`" 1878 | echo "======================================================================" 1879 | 1880 | ## <- monthly backup, unique per month 1881 | if (( ${CONFIG_do_monthly} != 0 && (${date_day_of_month} == ${CONFIG_do_monthly} || $date_day_of_month == $date_lastday_of_this_month && $date_lastday_of_this_month < ${CONFIG_do_monthly}) )); then 1882 | echo "Monthly Backup ..." 1883 | echo 1884 | 1885 | subfolder="monthly" 1886 | prefix="monthly_" 1887 | midfix="_${date_month}" 1888 | extension=".sql" 1889 | rotation="${CONFIG_rotation_monthly}" 1890 | rotation_divisor="31" 1891 | rotation_string="month" 1892 | 1893 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 1894 | for db in "${CONFIG_db_month_names[@]}"; do 1895 | echo "Monthly Backup of Database ( ${db} )" 1896 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${db}/${prefix}${db}_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && continue 1897 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 0 "$db" 1898 | echo ---------------------------------------------------------------------- 1899 | done 1900 | else 1901 | echo "Monthly backup of databases ( ${CONFIG_db_month_names[@]} )." 1902 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${prefix}all-databases_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && 1903 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 1 "${CONFIG_db_month_names[@]}" 1904 | echo "----------------------------------------------------------------------" 1905 | fi 1906 | fi 1907 | ## <- monthly backup 1908 | 1909 | ## <- weekly backup, unique per week 1910 | if (( ${CONFIG_do_weekly} != 0 && ${date_dayno_of_week} == ${CONFIG_do_weekly} )); then 1911 | echo "Weekly Backup ..." 1912 | echo 1913 | 1914 | subfolder="weekly" 1915 | prefix="weekly_" 1916 | midfix="_${date_weekno}" 1917 | extension=".sql" 1918 | rotation="${CONFIG_rotation_weekly}" 1919 | rotation_divisor="7" 1920 | rotation_string="week" 1921 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 1922 | for db in "${CONFIG_db_names[@]}"; do 1923 | echo "Weekly Backup of Database ( ${db} )" 1924 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${db}/${prefix}${db}_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && continue 1925 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 0 "$db" 1926 | echo "----------------------------------------------------------------------" 1927 | done 1928 | else 1929 | echo "Weekly backup of databases ( ${CONFIG_db_names[@]} )." 1930 | (shopt -s nullglob dotglob; f=("${CONFIG_backup_dir}/${subfolder}/${prefix}all-databases_${date_stamp}_"[0-9][0-9]"h"[0-9][0-9]"m${midfix}${extension}${suffix}"); ((${#f[@]}))) && 1931 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 1 "${CONFIG_db_names[@]}" 1932 | echo "----------------------------------------------------------------------" 1933 | fi 1934 | fi 1935 | ## <- weekly backup 1936 | 1937 | ## -> daily backup, test (( 1 )) is always true, just creates a grouping for Kate, which can be closed ^^ 1938 | if (( 1 )); then 1939 | echo "Daily Backup ..." 1940 | echo 1941 | 1942 | subfolder="daily" 1943 | prefix="daily_" 1944 | midfix="_${date_day_of_week}" 1945 | extension=".sql" 1946 | rotation="${CONFIG_rotation_daily}" 1947 | rotation_divisor="1" 1948 | rotation_string="day" 1949 | 1950 | if [[ "${CONFIG_mysql_dump_use_separate_dirs}" = "yes" ]]; then 1951 | for db in "${CONFIG_db_names[@]}"; do 1952 | echo "Daily Backup of Database ( ${db} )" 1953 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 0 "$db" 1954 | echo "----------------------------------------------------------------------" 1955 | done 1956 | else 1957 | echo "Daily backup of databases ( ${CONFIG_db_names[@]} )." 1958 | process_dbs "$subfolder" "$prefix" "$midfix" "$extension" "$rotation" "$rotation_divisor" "$rotation_string" 1 "${CONFIG_db_names[@]}" 1959 | echo "----------------------------------------------------------------------" 1960 | fi 1961 | fi 1962 | ## <- daily backup 1963 | 1964 | echo 1965 | echo "Backup End Time `date`" 1966 | echo "======================================================================" 1967 | # <- BACKUP DATABASES 1968 | 1969 | 1970 | # -> clean latest filenames 1971 | [[ "${CONFIG_mysql_dump_latest_clean_filenames}" = 'yes' ]] && find "${CONFIG_backup_dir}"/latest/ -type f -exec bash -c 'remove_datetimeinfo "$@"' -- {} \; 1972 | # <- clean latest filenames 1973 | 1974 | # -> finished information 1975 | echo "Total disk space used for backup storage..." 1976 | echo "Size - Location" 1977 | echo `du -hsH "${CONFIG_backup_dir}"` 1978 | echo 1979 | echo "======================================================================" 1980 | # <- finished information 1981 | 1982 | # -> postbackup commands 1983 | if [[ "${CONFIG_postbackup}" ]];then 1984 | echo "======================================================================" 1985 | echo "Postbackup command output." 1986 | echo 1987 | source ${CONFIG_postbackup} 1988 | echo 1989 | echo "======================================================================" 1990 | fi 1991 | # <- postbackup commands 1992 | 1993 | if [[ -s "$log_errfile" ]];then status=1; else status=0; fi 1994 | 1995 | exit ${status} 1996 | 1997 | } 1998 | 1999 | # @return variable method_list_manifest_entries_array 2000 | method_list_manifest_entries () { 2001 | local files files_master files_manifest file db manifest_files manifest_files_db selected_dbs i z l master_flags master to_rm actions 2002 | manifest_entries_corrupted=() 2003 | manifest_entries_user_action_required=() 2004 | files=() 2005 | files_master=() 2006 | files_manifest=() 2007 | master_flags=0 2008 | let "filename_flag_encrypted=0x01" 2009 | let "filename_flag_gz=0x02" 2010 | let "filename_flag_bz2=0x04" 2011 | let "filename_flag_diff=0x08" 2012 | 2013 | ############################################################################################################## 2014 | # BEGIN __STARTUP 2015 | 2016 | load_default_config 2017 | 2018 | if [[ -r "${CONFIG_configfile}" ]]; then source "${CONFIG_configfile}"; echo "Parsed config file \"${CONFIG_configfile}\""; else let "N |= $N_config_file_missing"; fi; echo 2019 | if (( $opt_flag_config_file )); then if [[ -r "${opt_config_file}" ]]; then source "${opt_config_file}"; let "N |= $N_arg_conffile_parsed"; else let "N |= $N_arg_conffile_unreadable"; fi; else let "N |= $N_too_many_args"; fi 2020 | 2021 | export LC_ALL=C 2022 | PROGNAME=`basename $0` 2023 | PATH=${PATH}:/usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 2024 | version=3.0 2025 | fields=5 # manifest fields 2026 | 2027 | set_datetime_vars 2028 | check_dependencies # check for required programs 2029 | parse_configuration # parse configuration and set variables appropriately 2030 | 2031 | # BEGIN __MAIN 2032 | unset manifest_files manifest_files_db i db 2033 | while IFS= read -r -d '' file; do 2034 | db="${file#/var/backup/db/@(daily|monthly|weekly|latest)/}"; 2035 | db="${db%/Manifest}"; 2036 | manifest_files_db[i]="$db" 2037 | manifest_files[i++]="$file" 2038 | done < <(find "${CONFIG_backup_dir}"/ -type f -name 'Manifest' -print0) 2039 | 2040 | extended_select 0 "Databases" "${manifest_files_db[@]}" 2041 | declare -a selected_dbs=("${extended_select_return[@]}") 2042 | 2043 | for db in "${selected_dbs[@]}"; do 2044 | selected_available_files=() 2045 | for ((i=0;i<"${#manifest_files_db[@]}";i++)); do 2046 | if [[ "x${manifest_files_db[i]}" = "x$db" ]]; then 2047 | selected_available_files[j++]="${manifest_files[i]}" 2048 | fi 2049 | done 2050 | if (( "${#selected_available_files[@]}" > 0 )); then 2051 | extended_select 1 "$db" "${selected_available_files[@]}" 2052 | declare -a selected_entries=("${extended_select_return[@]}") 2053 | if (( "${#selected_entries[@]}" > 0 )); then 2054 | for z in "${selected_entries[@]}"; do 2055 | parse_manifest "$z" 2056 | list=() 2057 | list_id=() 2058 | for ((i=0;$i<"${#manifest_array[@]}";i=$i+$fields)); do 2059 | if [[ "${manifest_array[i+3]}" != 0 ]]; then # only add differential backups 2060 | list=("${list[@]}" "${manifest_array[i]}") 2061 | list_id=("${list_id[@]}" "${manifest_array[i+3]}") # save rel_id, so we can retrieve the master backup file 2062 | fi 2063 | done 2064 | if (( "${#list[@]}" > 0 )); then 2065 | extended_select 1 "$z" "${list[@]}" 2066 | if (( "${#extended_select_return[@]}" > 0 )); then 2067 | for ((i=0;$i<"${#extended_select_return[@]}";i++)); do 2068 | if get_manifest_entry_by_id "${list_id[${extended_select_return_id[i]}]}"; then 2069 | files=("${files[@]}" "${extended_select_return[i]}") 2070 | files_master=("${files_master[@]}" "${manifest_entry[0]}") 2071 | files_manifest=("${files_manifest[@]}" "$z") 2072 | else 2073 | echo "no found master for id ${list_id[${extended_select_return_id[i]}]}" 2074 | fi 2075 | done 2076 | fi 2077 | fi 2078 | done 2079 | fi 2080 | fi 2081 | done 2082 | # END _select_filenames 2083 | declare -a method_list_manifest_entries_array=("${files[@]}") 2084 | declare -a method_list_manifest_entries_array_master=("${files_master[@]}") 2085 | declare -a method_list_manifest_entries_array_manifest=("${files_manifest[@]}") 2086 | 2087 | clear 2088 | echo "You have selected the following files:" 2089 | for i in "${files[@]}"; do printf '>>> %s\n' "$i"; done 2090 | echo 2091 | actions=('diff to full' 'remove files (also from Manifest)') 2092 | extended_select 0 "Actions" "${actions[@]}" 2093 | for action in "${extended_select_return[@]}"; do 2094 | case "$action" in 2095 | 'diff to full') 2096 | for ((l=0;$l<"${#files[@]}";l++)); do 2097 | # put the unpacking of the master file in here, so that in the case of multiple diffs with the same 2098 | # master file don't cause the script to unpack the same master file multiple times 2099 | master="${files_master[l]}" 2100 | diff="${files[l]}" 2101 | FileStub="${master%.@(sql|master)*}" 2102 | FileExt="${master#"$FileStub"}" 2103 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_encrypted" 2104 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_gz" 2105 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_bz2" 2106 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_diff" 2107 | if (( $master_flags & $filename_flag_gz )); then 2108 | declare -a testarray=("${to_rm[@]}") 2109 | if ! in_array "${master%.gz}"; then 2110 | gzip_compression -dc "$master" > "${master%.gz}" 2111 | to_rm=("${to_rm[@]}" "${master%.gz}") 2112 | fi 2113 | master="${master%.gz}" 2114 | elif (( $master_flags & $filename_flag_bz2 )); then 2115 | declare -a testarray=("${to_rm[@]}") 2116 | if ! in_array "${master%.bz2}"; then 2117 | bzip2_compression -dc "$master" > "${master%.bz2}" 2118 | to_rm=("${to_rm[@]}" "${master%.bz2}") 2119 | fi 2120 | master="${master%.bz2}" 2121 | else 2122 | : 2123 | fi 2124 | method_diff_to_full "$master" "$diff" 2125 | #printf '%s\n>>> master: %s\n>>> manifest: %s\n' "${files[l]}" "${files_master[l]}" "${files_manifest[l]}" 2126 | done 2127 | # cleanup all unpacked master files ... the unpacked diff files are cleaned up by method_diff_to_full 2128 | for i in "${to_rm[@]}"; do rm "$i"; done 2129 | ;; 2130 | 'remove files (also from Manifest)') 2131 | for ((l=0;$l<"${#files[@]}";l++)); do 2132 | if rm_manifest_entry_by_filename "${files_manifest[l]}" "${files[l]}" 1; then 2133 | rm "${files[l]}" 2134 | fi 2135 | done 2136 | ;; 2137 | *) 2138 | echo "Unrecognized option. This Should not happen! Error!" 2139 | ;; 2140 | esac 2141 | done 2142 | 2143 | # END __MAIN 2144 | } 2145 | 2146 | # @info: Convert a differential backup file to a full one. 2147 | # @param: master_backup_file diff_backup_file 2148 | method_diff_to_full() { 2149 | local diff full diff_flags master_flags to_rm 2150 | master="$1" 2151 | diff="$2" 2152 | diff_flags=0 2153 | master_flags=0 2154 | to_rm=() 2155 | 2156 | FileStub="${diff%.@(sql|diff)*}" 2157 | FileExt="${diff#"$FileStub"}" 2158 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_encrypted" 2159 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_gz" 2160 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_bz2" 2161 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "diff_flags|=$filename_flag_diff" 2162 | FileStub="${master%.@(sql|master)*}" 2163 | FileExt="${master#"$FileStub"}" 2164 | re=".*\.enc.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_encrypted" 2165 | re=".*\.gz.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_gz" 2166 | re=".*\.bz2.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_bz2" 2167 | re=".*\.diff.*"; [[ "$FileExt" =~ $re ]] && let "master_flags|=$filename_flag_diff" 2168 | 2169 | # TODO: Differential backup with encryption is not yet implemented! 2170 | if (( $diff_flags & $filename_flag_encrypted )); then 2171 | : #decrypt it 2172 | fi 2173 | 2174 | if (( $master_flags & $filename_flag_encrypted )); then 2175 | : #decrypt it 2176 | fi 2177 | 2178 | if (( $diff_flags & $filename_flag_gz )); then 2179 | gzip_compression -dc "$diff" > "${diff%.gz}" 2180 | to_rm=("${to_rm[@]}" "${diff%.gz}") 2181 | diff="${diff%.gz}" 2182 | elif (( $diff_flags & $filename_flag_bz2 )); then 2183 | bzip2_compression -dc "$diff" > "${diff%.bz2}" 2184 | to_rm=("${to_rm[@]}" "${diff%.bz2}") 2185 | diff="${diff%.bz2}" 2186 | else 2187 | : 2188 | fi 2189 | 2190 | if (( $master_flags & $filename_flag_gz )); then 2191 | gzip_compression -dc "$master" > "${master%.gz}" 2192 | to_rm=("${to_rm[@]}" "${master%.gz}") 2193 | master="${master%.gz}" 2194 | elif (( $master_flags & $filename_flag_bz2 )); then 2195 | bzip2_compression -dc "$master" > "${master%.bz2}" 2196 | to_rm=("${to_rm[@]}" "${master%.bz2}") 2197 | master="${master%.bz2}" 2198 | else 2199 | : 2200 | fi 2201 | 2202 | patch "$master" "$diff" -o "${diff/diff/sql}" 2203 | 2204 | # cleanup 2205 | for i in "${to_rm[@]}"; do rm "$i"; done 2206 | } 2207 | 2208 | # END _methods 2209 | 2210 | 2211 | # BEGIN __main 2212 | 2213 | NO_ARGS=0 2214 | E_OPTERROR=85 2215 | 2216 | if (( $# == $NO_ARGS )); then # Script invoked with no command-line args? 2217 | echo "Invoking backup method."; echo; method_backup 2218 | fi 2219 | 2220 | while getopts ":c:blh" Option 2221 | do 2222 | case $Option in 2223 | c ) echo "Using \"$OPTARG\" as optional config file."; echo; opt_config_file="$OPTARG"; opt_flag_config_file=1;; 2224 | b ) echo "MySQL backup method invoked."; echo; opt_flag_method_backup=1;; 2225 | l ) echo "List manifest entries."; echo; opt_flag_list_manifest_entries=1;; 2226 | h ) echo "Usage `basename $0` options -cblh" 2227 | echo -e "-c CONFIG_FILE\tSpecify optional config file." 2228 | echo -e "-b\tUse backup method." 2229 | echo -e "-l\tList manifest entries." 2230 | echo -e "-h\tShow this help." 2231 | exit 0;; 2232 | #n | o ) echo "Scenario #2: option -$Option- [OPTIND=${OPTIND}]";; 2233 | 2234 | #q ) echo "Scenario #4: option -q-\ 2235 | # with argument \"$OPTARG\" [OPTIND=${OPTIND}]";; 2236 | # Note that option 'q' must have an associated argument, 2237 | #+ otherwise it falls through to the default. 2238 | #r | s ) echo "Scenario #5: option -$Option-";; 2239 | * ) echo "Unimplemented option chosen.";; # Default. 2240 | esac 2241 | done 2242 | 2243 | (( $opt_flag_method_backup )) && method_backup 2244 | (( $opt_flag_list_manifest_entries )) && method_list_manifest_entries 2245 | 2246 | shift $(($OPTIND - 1)) 2247 | # Decrements the argument pointer so it points to next argument. 2248 | # $1 now references the first non-option item supplied on the command-line 2249 | #+ if one exists. 2250 | 2251 | # For backward compatibility. If no option items are present and only one non-option item is there, we expect it 2252 | # to be the optional config file and invoke the backup method. 2253 | opt_flags=( "${!opt_flag_@}" ) # array of all set variables starting with opt_flag_ 2254 | if (( $# == 1 )) && (( ${#opt_flags[@]} == 0 )); then 2255 | opt_config_file="$1"; opt_flag_config_file=1; method_backup 2256 | elif (( $# == 0 )) && (( ${#opt_flags[@]} == 0 )); then 2257 | method_backup 2258 | fi 2259 | 2260 | # END __main 2261 | -------------------------------------------------------------------------------- /automysqlbackup.conf: -------------------------------------------------------------------------------- 1 | #version=3.0_rc2 2 | # DONT'T REMOVE THE PREVIOUS VERSION LINE! 3 | # 4 | # Uncomment to change the default values (shown after =) 5 | # WARNING: 6 | # This is not true for UMASK, CONFIG_prebackup and CONFIG_postbackup!!! 7 | # 8 | # Default values are stored in the script itself. Declarations in 9 | # /etc/automysqlbackup/automysqlbackup.conf will overwrite them. The 10 | # declarations in here will supersede all other. 11 | 12 | # Edit $PATH if mysql and mysqldump are not located in /usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 13 | #PATH=${PATH}:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQL:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQLDUMP 14 | 15 | # Basic Settings 16 | 17 | # Username to access the MySQL server e.g. dbuser 18 | #CONFIG_mysql_dump_username='root' 19 | 20 | # Password to access the MySQL server e.g. password 21 | #CONFIG_mysql_dump_password='' 22 | 23 | # Host name (or IP address) of MySQL server e.g localhost 24 | #CONFIG_mysql_dump_host='localhost' 25 | 26 | # "Friendly" host name of MySQL server to be used in email log 27 | # if unset or empty (default) will use CONFIG_mysql_dump_host instead 28 | #CONFIG_mysql_dump_host_friendly='' 29 | 30 | # Backup directory location e.g /backups 31 | #CONFIG_backup_dir='/var/backup/db' 32 | 33 | # This is practically a moot point, since there is a fallback to the compression 34 | # functions without multicore support in the case that the multicore versions aren't 35 | # present in the system. Of course, if you have the latter installed, but don't want 36 | # to use them, just choose no here. 37 | # pigz -> gzip 38 | # pbzip2 -> bzip2 39 | #CONFIG_multicore='yes' 40 | 41 | # Number of threads (= occupied cores) you want to use. You should - for the sake 42 | # of the stability of your system - not choose more than (#number of cores - 1). 43 | # Especially if the script is run in background by cron and the rest of your system 44 | # has already heavy load, setting this too high, might crash your system. Assuming 45 | # all systems have at least some sort of HyperThreading, the default is 2 threads. 46 | # If you wish to let pigz and pbzip2 autodetect or use their standards, set it to 47 | # 'auto'. 48 | #CONFIG_multicore_threads=2 49 | 50 | # Databases to backup 51 | 52 | # List of databases for Daily/Weekly Backup e.g. ( 'DB1' 'DB2' 'DB3' ... ) 53 | # set to (), i.e. empty, if you want to backup all databases 54 | #CONFIG_db_names=() 55 | # You can use 56 | #declare -a MDBNAMES=( "${DBNAMES[@]}" 'added entry1' 'added entry2' ... ) 57 | # INSTEAD to copy the contents of $DBNAMES and add further entries (optional). 58 | 59 | # List of databases for Monthly Backups. 60 | # set to (), i.e. empty, if you want to backup all databases 61 | #CONFIG_db_month_names=() 62 | 63 | # List of DBNAMES to EXLUCDE if DBNAMES is empty, i.e. (). 64 | #CONFIG_db_exclude=( 'information_schema' ) 65 | 66 | # List of tables to exclude, in the form db_name.table_name 67 | # You may use wildcards for the table names, i.e. 'mydb.a*' selects all tables starting with an 'a'. 68 | # However we only offer the wildcard '*', matching everything that could appear, which translates to the 69 | # '%' wildcard in mysql. 70 | #CONFIG_table_exclude=() 71 | 72 | 73 | # Advanced Settings 74 | 75 | # Rotation Settings 76 | 77 | # Which day do you want monthly backups? (01 to 31) 78 | # If the chosen day is greater than the last day of the month, it will be done 79 | # on the last day of the month. 80 | # Set to 0 to disable monthly backups. 81 | #CONFIG_do_monthly="01" 82 | 83 | # Which day do you want weekly backups? (1 to 7 where 1 is Monday) 84 | # Set to 0 to disable weekly backups. 85 | #CONFIG_do_weekly="5" 86 | 87 | # Set rotation of daily backups. VALUE*24hours 88 | # If you want to keep only today's backups, you could choose 1, i.e. everything older than 24hours will be removed. 89 | #CONFIG_rotation_daily=6 90 | 91 | # Set rotation for weekly backups. VALUE*24hours 92 | #CONFIG_rotation_weekly=35 93 | 94 | # Set rotation for monthly backups. VALUE*24hours 95 | #CONFIG_rotation_monthly=150 96 | 97 | 98 | # Server Connection Settings 99 | 100 | # Set the port for the mysql connection 101 | #CONFIG_mysql_dump_port=3306 102 | 103 | # Compress communications between backup server and MySQL server? 104 | #CONFIG_mysql_dump_commcomp='no' 105 | 106 | # Use ssl encryption with mysqldump? 107 | #CONFIG_mysql_dump_usessl='yes' 108 | 109 | # For connections to localhost. Sometimes the Unix socket file must be specified. 110 | #CONFIG_mysql_dump_socket='' 111 | 112 | # The maximum size of the buffer for client/server communication. e.g. 16MB (maximum is 1GB) 113 | #CONFIG_mysql_dump_max_allowed_packet='' 114 | 115 | # This option sends a START TRANSACTION SQL statement to the server before dumping data. It is useful only with 116 | # transactional tables such as InnoDB, because then it dumps the consistent state of the database at the time 117 | # when BEGIN was issued without blocking any applications. 118 | # 119 | # When using this option, you should keep in mind that only InnoDB tables are dumped in a consistent state. For 120 | # example, any MyISAM or MEMORY tables dumped while using this option may still change state. 121 | # 122 | # While a --single-transaction dump is in process, to ensure a valid dump file (correct table contents and 123 | # binary log coordinates), no other connection should use the following statements: ALTER TABLE, CREATE TABLE, 124 | # DROP TABLE, RENAME TABLE, TRUNCATE TABLE. A consistent read is not isolated from those statements, so use of 125 | # them on a table to be dumped can cause the SELECT that is performed by mysqldump to retrieve the table 126 | # contents to obtain incorrect contents or fail. 127 | #CONFIG_mysql_dump_single_transaction='no' 128 | 129 | # http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_master-data 130 | # --master-data[=value] 131 | # Use this option to dump a master replication server to produce a dump file that can be used to set up another 132 | # server as a slave of the master. It causes the dump output to include a CHANGE MASTER TO statement that indicates 133 | # the binary log coordinates (file name and position) of the dumped server. These are the master server coordinates 134 | # from which the slave should start replicating after you load the dump file into the slave. 135 | # 136 | # If the option value is 2, the CHANGE MASTER TO statement is written as an SQL comment, and thus is informative only; 137 | # it has no effect when the dump file is reloaded. If the option value is 1, the statement is not written as a comment 138 | # and takes effect when the dump file is reloaded. If no option value is specified, the default value is 1. 139 | # 140 | # This option requires the RELOAD privilege and the binary log must be enabled. 141 | # 142 | # The --master-data option automatically turns off --lock-tables. It also turns on --lock-all-tables, unless 143 | # --single-transaction also is specified, in which case, a global read lock is acquired only for a short time at the 144 | # beginning of the dump (see the description for --single-transaction). In all cases, any action on logs happens at 145 | # the exact moment of the dump. 146 | # ================================================================================================================== 147 | # possible values are 1 and 2, which correspond with the values from mysqldump 148 | # VARIABLE= , i.e. no value, turns it off (default) 149 | # 150 | #CONFIG_mysql_dump_master_data= 151 | 152 | # Included stored routines (procedures and functions) for the dumped databases in the output. Use of this option 153 | # requires the SELECT privilege for the mysql.proc table. The output generated by using --routines contains 154 | # CREATE PROCEDURE and CREATE FUNCTION statements to re-create the routines. However, these statements do not 155 | # include attributes such as the routine creation and modification timestamps. This means that when the routines 156 | # are reloaded, they will be created with the timestamps equal to the reload time. 157 | # 158 | # If you require routines to be re-created with their original timestamp attributes, do not use --routines. Instead, 159 | # dump and reload the contents of the mysql.proc table directly, using a MySQL account that has appropriate privileges 160 | # for the mysql database. 161 | # 162 | # This option was added in MySQL 5.0.13. Before that, stored routines are not dumped. Routine DEFINER values are not 163 | # dumped until MySQL 5.0.20. This means that before 5.0.20, when routines are reloaded, they will be created with the 164 | # definer set to the reloading user. If you require routines to be re-created with their original definer, dump and 165 | # load the contents of the mysql.proc table directly as described earlier. 166 | # 167 | #CONFIG_mysql_dump_full_schema='yes' 168 | 169 | # Backup status of table(s) in textfile. This is very helpful when restoring backups, since it gives an idea, what changed 170 | # in the meantime. 171 | #CONFIG_mysql_dump_dbstatus='yes' 172 | 173 | # Backup dump settings 174 | 175 | # Include CREATE DATABASE in backup? 176 | #CONFIG_mysql_dump_create_database='no' 177 | 178 | # Separate backup directory and file for each DB? (yes or no) 179 | #CONFIG_mysql_dump_use_separate_dirs='yes' 180 | 181 | # Choose Compression type. (gzip or bzip2) 182 | #CONFIG_mysql_dump_compression='gzip' 183 | 184 | # Store an additional copy of the latest backup to a standard 185 | # location so it can be downloaded by third party scripts. 186 | #CONFIG_mysql_dump_latest='no' 187 | 188 | # Remove all date and time information from the filenames in the latest folder. 189 | # Runs, if activated, once after the backups are completed. Practically it just finds all files in the latest folder 190 | # and removes the date and time information from the filenames (if present). 191 | #CONFIG_mysql_dump_latest_clean_filenames='no' 192 | 193 | # Create differential backups. Master backups are created weekly at #$CONFIG_do_weekly weekday. Between master backups, 194 | # diff is used to create differential backups relative to the latest master backup. In the Manifest file, you find the 195 | # following structure 196 | # $filename md5sum $md5sum diff_id $diff_id rel_id $rel_id 197 | # where each field is separated by the tabular character '\t'. The entries with $ at the beginning mean the actual values, 198 | # while the others are just for readability. The diff_id is the id of the differential or master backup which is also in 199 | # the filename after the last _ and before the suffixes begin, i.e. .diff, .sql and extensions. It is used to relate 200 | # differential backups to master backups. The master backups have 0 as $rel_id and are thereby identifiable. Differential 201 | # backups have the id of the corresponding master backup as $rel_id. 202 | # 203 | # To ensure that master backups are kept long enough, the value of $CONFIG_rotation_daily is set to a minimum of 21 days. 204 | # 205 | #CONFIG_mysql_dump_differential='no' 206 | 207 | 208 | # Notification setup 209 | 210 | # What would you like to be mailed to you? 211 | # - log : send only log file 212 | # - files : send log file and sql files as attachments (see docs) 213 | # - stdout : will simply output the log to the screen if run manually. 214 | # - quiet : Only send logs if an error occurs to the MAILADDR. 215 | #CONFIG_mailcontent='stdout' 216 | 217 | # Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs]) 218 | #CONFIG_mail_maxattsize=4000 219 | 220 | # Allow packing of files with tar and splitting it in pieces of CONFIG_mail_maxattsize. 221 | #CONFIG_mail_splitandtar='yes' 222 | 223 | # Use uuencode instead of mutt. WARNING: Not all email clients work well with uuencoded attachments. 224 | #CONFIG_mail_use_uuencoded_attachments='no' 225 | 226 | # Email Address to send mail to? (user@domain.com) 227 | #CONFIG_mail_address='root' 228 | 229 | 230 | # Encryption 231 | 232 | # Do you wish to encrypt your backups using openssl? 233 | #CONFIG_encrypt='no' 234 | 235 | # Choose a password to encrypt the backups. 236 | #CONFIG_encrypt_password='password0123' 237 | 238 | # Other 239 | 240 | # Backup local files, i.e. maybe you would like to backup your my.cnf (mysql server configuration), etc. 241 | # These files will be tar'ed, depending on your compression option CONFIG_mysql_dump_compression compressed and 242 | # depending on the option CONFIG_encrypt encrypted. 243 | # 244 | # Note: This could also have been accomplished with CONFIG_prebackup or CONFIG_postbackup. 245 | #CONFIG_backup_local_files=() 246 | 247 | # Command to run before backups (uncomment to use) 248 | #CONFIG_prebackup="/etc/mysql-backup-pre" 249 | 250 | # Command run after backups (uncomment to use) 251 | #CONFIG_postbackup="/etc/mysql-backup-post" 252 | 253 | # Uncomment to activate! This will give folders rwx------ 254 | # and files rw------- permissions. 255 | #umask 0077 256 | 257 | # dry-run, i.e. show what you are gonna do without actually doing it 258 | # inactive: =0 or commented out 259 | # active: uncommented AND =1 260 | #CONFIG_dryrun=1 261 | 262 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # 3 | 4 | # return true, if variable is set; else false 5 | isSet() { 6 | if [[ ! ${!1} && ${!1-_} ]]; then return 1; else return 0; fi 7 | } 8 | 9 | activateIO() { 10 | touch "$1" 11 | exec 6>&1 12 | exec > "$1" 13 | } 14 | removeIO() { 15 | exec 1>&6 6>&- 16 | } 17 | 18 | upgrade_config_file () { 19 | ( # execute in subshell, so that sourced variables are only available inside () 20 | source "$1" 21 | #declare -p 22 | local temp 23 | temp=$(mktemp /tmp/tmp.XXXXXX) 24 | (( $? != 0 )) && return 1 25 | activateIO "$temp" 26 | echo "#version=3.0_rc2" 27 | echo "# Uncomment to change the default values (shown after =)" 28 | echo "# WARNING:" 29 | echo "# This is not true for UMASK, CONFIG_prebackup and CONFIG_postbackup!!!" 30 | echo "#" 31 | echo "# Default values are stored in the script itself. Declarations in" 32 | echo "# /etc/automysqlbackup/automysqlbackup.conf will overwrite them. The" 33 | echo "# declarations in here will supersede all other." 34 | echo "" 35 | echo "# Edit \$PATH if mysql and mysqldump are not located in /usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin" 36 | echo "#PATH=\${PATH}:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQL:FULL_PATH_TO_YOUR_DIR_CONTAINING_MYSQLDUMP" 37 | echo "" 38 | echo "# Basic Settings" 39 | echo "" 40 | echo "# Username to access the MySQL server e.g. dbuser" 41 | if isSet USERNAME; then 42 | printf '%s=%q\n' CONFIG_mysql_dump_username "${USERNAME-}" 43 | else 44 | echo "#CONFIG_mysql_dump_username='root'" 45 | fi 46 | echo "" 47 | echo "# Password to access the MySQL server e.g. password" 48 | if isSet PASSWORD; then 49 | printf '%s=%q\n' CONFIG_mysql_dump_password "${PASSWORD-}" 50 | else 51 | echo "#CONFIG_mysql_dump_password=''" 52 | fi 53 | echo "" 54 | echo "# Host name (or IP address) of MySQL server e.g localhost" 55 | if isSet DBHOST; then 56 | printf '%s=%q\n' CONFIG_mysql_dump_host "${DBHOST-}" 57 | else 58 | echo "#CONFIG_mysql_dump_host='localhost'" 59 | fi 60 | echo "" 61 | echo "# \"Friendly\" host name of MySQL server to be used in email log" 62 | echo "# if unset or empty (default) will use CONFIG_mysql_dump_host instead" 63 | if isSet CONFIG_mysql_dump_host_friendly; then 64 | printf '%s=%q\n' CONFIG_mysql_dump_host_friendly "${CONFIG_mysql_dump_host_friendly-}" 65 | else 66 | echo "#CONFIG_mysql_dump_host_friendly=''" 67 | fi 68 | echo "" 69 | echo "# Backup directory location e.g /backups" 70 | if isSet BACKUPDIR; then 71 | printf '%s=%q\n' CONFIG_backup_dir "${BACKUPDIR-}" 72 | else 73 | echo "#CONFIG_backup_dir='/var/backup/db'" 74 | fi 75 | echo "" 76 | echo "# This is practically a moot point, since there is a fallback to the compression" 77 | echo "# functions without multicore support in the case that the multicore versions aren't" 78 | echo "# present in the system. Of course, if you have the latter installed, but don't want" 79 | echo "# to use them, just choose no here." 80 | echo "# pigz -> gzip" 81 | echo "# pbzip2 -> bzip2" 82 | echo "#CONFIG_multicore='yes'" 83 | echo "" 84 | echo "# Number of threads (= occupied cores) you want to use. You should - for the sake" 85 | echo "# of the stability of your system - not choose more than (#number of cores - 1)." 86 | echo "# Especially if the script is run in background by cron and the rest of your system" 87 | echo "# has already heavy load, setting this too high, might crash your system. Assuming" 88 | echo "# all systems have at least some sort of HyperThreading, the default is 2 threads." 89 | echo "# If you wish to let pigz and pbzip2 autodetect or use their standards, set it to" 90 | echo "# 'auto'." 91 | echo "#CONFIG_multicore_threads=2" 92 | echo "" 93 | echo "# Databases to backup" 94 | echo "" 95 | echo "# List of databases for Daily/Weekly Backup e.g. ( 'DB1' 'DB2' 'DB3' ... )" 96 | echo "# set to (), i.e. empty, if you want to backup all databases" 97 | if isSet DBNAMES; then 98 | if [[ "x$DBNAMES" = "xall" ]]; then 99 | echo "#CONFIG_db_names=()" 100 | else 101 | declare -a CONFIG_db_names 102 | for i in $DBNAMES; do 103 | CONFIG_db_names=( "${CONFIG_db_names[@]}" "$i" ) 104 | done 105 | declare -p CONFIG_db_names | sed -e 's/\[[^]]*]=//g' 106 | fi 107 | else 108 | echo "#CONFIG_db_names=()" 109 | fi 110 | echo "# You can use" 111 | echo "#declare -a MDBNAMES=( \"\${DBNAMES[@]}\" 'added entry1' 'added entry2' ... )" 112 | echo "# INSTEAD to copy the contents of \$DBNAMES and add further entries (optional)." 113 | echo "" 114 | echo "# List of databases for Monthly Backups." 115 | echo "# set to (), i.e. empty, if you want to backup all databases" 116 | if isSet MDBNAMES; then 117 | if [[ "x$MDBNAMES" = "xall" ]]; then 118 | echo "#CONFIG_db_month_names=()" 119 | else 120 | declare -a CONFIG_db_month_names 121 | for i in $MDBNAMES; do 122 | CONFIG_db_month_names=( "${CONFIG_db_month_names[@]}" "$i" ) 123 | done 124 | declare -p CONFIG_db_month_names | sed -e 's/\[[^]]*]=//g' 125 | fi 126 | else 127 | echo "#CONFIG_db_month_names=()" 128 | fi 129 | echo "" 130 | echo "# List of DBNAMES to EXLUCDE if DBNAMES is empty, i.e. ()." 131 | if isSet DBEXCLUDE; then 132 | declare -a CONFIG_db_exclude 133 | for i in $DBEXCLUDE; do 134 | CONFIG_db_exclude=( "${CONFIG_db_exclude[@]}" "$i" ) 135 | done 136 | declare -p CONFIG_db_exclude | sed -e 's/\[[^]]*]=//g' 137 | else 138 | echo "#CONFIG_db_exclude=( 'information_schema' )" 139 | fi 140 | echo "" 141 | echo "# List of tables to exclude, in the form db_name.table_name" 142 | echo "#CONFIG_table_exclude=()" 143 | echo "" 144 | echo "" 145 | echo "# Advanced Settings" 146 | echo "" 147 | echo "# Rotation Settings" 148 | echo "" 149 | echo "# Which day do you want monthly backups? (01 to 31)" 150 | echo "# If the chosen day is greater than the last day of the month, it will be done" 151 | echo "# on the last day of the month." 152 | echo "# Set to 0 to disable monthly backups." 153 | echo "#CONFIG_do_monthly=\"01\"" 154 | echo "" 155 | echo "# Which day do you want weekly backups? (1 to 7 where 1 is Monday)" 156 | echo "# Set to 0 to disable weekly backups." 157 | if isSet DOWEEKLY; then 158 | printf '%s=%q\n' CONFIG_do_weekly "${DOWEEKLY-}" 159 | else 160 | echo "#CONFIG_do_weekly=\"5\"" 161 | fi 162 | echo "" 163 | echo "# Set rotation of daily backups. VALUE*24hours" 164 | echo "# If you want to keep only today's backups, you could choose 1, i.e. everything older than 24hours will be removed." 165 | echo "#CONFIG_rotation_daily=6" 166 | echo "" 167 | echo "# Set rotation for weekly backups. VALUE*24hours" 168 | echo "#CONFIG_rotation_weekly=35" 169 | echo "" 170 | echo "# Set rotation for monthly backups. VALUE*24hours" 171 | echo "#CONFIG_rotation_monthly=150" 172 | echo "" 173 | echo "" 174 | echo "# Server Connection Settings" 175 | echo "" 176 | echo "# Set the port for the mysql connection" 177 | echo "#CONFIG_mysql_dump_port=3306" 178 | echo "" 179 | echo "# Compress communications between backup server and MySQL server?" 180 | if isSet COMMCOMP; then 181 | printf '%s=%q\n' CONFIG_mysql_dump_commcomp "${COMMCOMP-}" 182 | else 183 | echo "#CONFIG_mysql_dump_commcomp='no'" 184 | fi 185 | echo "" 186 | echo "# Use ssl encryption with mysqldump?" 187 | echo "#CONFIG_mysql_dump_usessl='yes'" 188 | echo "" 189 | echo "# For connections to localhost. Sometimes the Unix socket file must be specified." 190 | if isSet SOCKET; then 191 | printf '%s=%q\n' CONFIG_mysql_dump_socket "${SOCKET-}" 192 | else 193 | echo "#CONFIG_mysql_dump_socket=''" 194 | fi 195 | echo "" 196 | echo "# The maximum size of the buffer for client/server communication. e.g. 16MB (maximum is 1GB)" 197 | if isSet MAX_ALLOWED_PACKET; then 198 | printf '%s=%q\n' CONFIG_mysql_dump_max_allowed_packet "${MAX_ALLOWED_PACKET-}" 199 | else 200 | echo "#CONFIG_mysql_dump_max_allowed_packet=''" 201 | fi 202 | echo "" 203 | echo "# This option sends a START TRANSACTION SQL statement to the server before dumping data. It is useful only with" 204 | echo "# transactional tables such as InnoDB, because then it dumps the consistent state of the database at the time" 205 | echo "# when BEGIN was issued without blocking any applications." 206 | echo "#" 207 | echo "# When using this option, you should keep in mind that only InnoDB tables are dumped in a consistent state. For" 208 | echo "# example, any MyISAM or MEMORY tables dumped while using this option may still change state." 209 | echo "#" 210 | echo "# While a --single-transaction dump is in process, to ensure a valid dump file (correct table contents and" 211 | echo "# binary log coordinates), no other connection should use the following statements: ALTER TABLE, CREATE TABLE," 212 | echo "# DROP TABLE, RENAME TABLE, TRUNCATE TABLE. A consistent read is not isolated from those statements, so use of" 213 | echo "# them on a table to be dumped can cause the SELECT that is performed by mysqldump to retrieve the table" 214 | echo "# contents to obtain incorrect contents or fail." 215 | echo "#CONFIG_mysql_dump_single_transaction='no'" 216 | echo "" 217 | echo "# http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_master-data" 218 | echo "# --master-data[=value] " 219 | echo "# Use this option to dump a master replication server to produce a dump file that can be used to set up another" 220 | echo "# server as a slave of the master. It causes the dump output to include a CHANGE MASTER TO statement that indicates" 221 | echo "# the binary log coordinates (file name and position) of the dumped server. These are the master server coordinates" 222 | echo "# from which the slave should start replicating after you load the dump file into the slave." 223 | echo "#" 224 | echo "# If the option value is 2, the CHANGE MASTER TO statement is written as an SQL comment, and thus is informative only;" 225 | echo "# it has no effect when the dump file is reloaded. If the option value is 1, the statement is not written as a comment" 226 | echo "# and takes effect when the dump file is reloaded. If no option value is specified, the default value is 1." 227 | echo "#" 228 | echo "# This option requires the RELOAD privilege and the binary log must be enabled. " 229 | echo "#" 230 | echo "# The --master-data option automatically turns off --lock-tables. It also turns on --lock-all-tables, unless" 231 | echo "# --single-transaction also is specified, in which case, a global read lock is acquired only for a short time at the" 232 | echo "# beginning of the dump (see the description for --single-transaction). In all cases, any action on logs happens at" 233 | echo "# the exact moment of the dump." 234 | echo "# ==================================================================================================================" 235 | echo "# possible values are 1 and 2, which correspond with the values from mysqldump" 236 | echo "# VARIABLE= , i.e. no value, turns it off (default)" 237 | echo "#" 238 | echo "#CONFIG_mysql_dump_master_data=" 239 | echo "" 240 | echo "# Included stored routines (procedures and functions) for the dumped databases in the output. Use of this option" 241 | echo "# requires the SELECT privilege for the mysql.proc table. The output generated by using --routines contains" 242 | echo "# CREATE PROCEDURE and CREATE FUNCTION statements to re-create the routines. However, these statements do not" 243 | echo "# include attributes such as the routine creation and modification timestamps. This means that when the routines" 244 | echo "# are reloaded, they will be created with the timestamps equal to the reload time." 245 | echo "#" 246 | echo "# If you require routines to be re-created with their original timestamp attributes, do not use --routines. Instead," 247 | echo "# dump and reload the contents of the mysql.proc table directly, using a MySQL account that has appropriate privileges" 248 | echo "# for the mysql database. " 249 | echo "#" 250 | echo "# This option was added in MySQL 5.0.13. Before that, stored routines are not dumped. Routine DEFINER values are not" 251 | echo "# dumped until MySQL 5.0.20. This means that before 5.0.20, when routines are reloaded, they will be created with the" 252 | echo "# definer set to the reloading user. If you require routines to be re-created with their original definer, dump and" 253 | echo "# load the contents of the mysql.proc table directly as described earlier." 254 | echo "#" 255 | echo "#CONFIG_mysql_dump_full_schema='yes'" 256 | echo "" 257 | echo "# Backup dump settings" 258 | echo "" 259 | echo "# Include CREATE DATABASE in backup?" 260 | if isSet CREATE_DATABASE; then 261 | printf '%s=%q\n' CONFIG_mysql_dump_create_database "${CREATE_DATABASE-}" 262 | else 263 | echo "#CONFIG_mysql_dump_create_database='no'" 264 | fi 265 | echo "" 266 | echo "# Separate backup directory and file for each DB? (yes or no)" 267 | if isSet SEPDIR; then 268 | printf '%s=%q\n' CONFIG_mysql_dump_use_separate_dirs "${SEPDIR-}" 269 | else 270 | echo "#CONFIG_mysql_dump_use_separate_dirs='yes'" 271 | fi 272 | echo "" 273 | echo "# Choose Compression type. (gzip or bzip2)" 274 | if isSet COMP; then 275 | printf '%s=%q\n' CONFIG_mysql_dump_compression "${COMP-}" 276 | else 277 | echo "#CONFIG_mysql_dump_compression='gzip'" 278 | fi 279 | echo "" 280 | echo "# Store an additional copy of the latest backup to a standard" 281 | echo "# location so it can be downloaded by third party scripts." 282 | if isSet LATEST; then 283 | printf '%s=%q\n' CONFIG_mysql_dump_latest "${LATEST-}" 284 | else 285 | echo "#CONFIG_mysql_dump_latest='no'" 286 | fi 287 | echo "" 288 | echo "# Remove all date and time information from the filenames in the latest folder." 289 | echo "# Runs, if activated, once after the backups are completed. Practically it just finds all files in the latest folder" 290 | echo "# and removes the date and time information from the filenames (if present)." 291 | echo "#CONFIG_mysql_dump_latest_clean_filenames='no'" 292 | echo "" 293 | echo "# Notification setup" 294 | echo "" 295 | echo "# What would you like to be mailed to you?" 296 | echo "# - log : send only log file" 297 | echo "# - files : send log file and sql files as attachments (see docs)" 298 | echo "# - stdout : will simply output the log to the screen if run manually." 299 | echo "# - quiet : Only send logs if an error occurs to the MAILADDR." 300 | if isSet MAILCONTENT; then 301 | printf '%s=%q\n' CONFIG_mailcontent "${MAILCONTENT-}" 302 | else 303 | echo "#CONFIG_mailcontent='stdout'" 304 | fi 305 | echo "" 306 | echo "# Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs])" 307 | if isSet MAXATTSIZE; then 308 | printf '%s=%q\n' CONFIG_mail_maxattsize "${MAXATTSIZE-}" 309 | else 310 | echo "#CONFIG_mail_maxattsize=4000" 311 | fi 312 | echo "" 313 | echo "# Email Address to send mail to? (user@domain.com)" 314 | if isSet MAILADDR; then 315 | printf '%s=%q\n' CONFIG_mail_address "${MAILADDR-}" 316 | else 317 | echo "#CONFIG_mail_address='root'" 318 | fi 319 | echo "" 320 | echo '# Create differential backups. Master backups are created weekly at #$CONFIG_do_weekly weekday. Between master backups,' 321 | echo "# diff is used to create differential backups relative to the latest master backup. In the Manifest file, you find the" 322 | echo "# following structure" 323 | echo '# $filename md5sum $md5sum diff_id $diff_id rel_id $rel_id' 324 | echo "# where each field is separated by the tabular character '\t'. The entries with $ at the beginning mean the actual values," 325 | echo "# while the others are just for readability. The diff_id is the id of the differential or master backup which is also in" 326 | echo "# the filename after the last _ and before the suffixes begin, i.e. .diff, .sql and extensions. It is used to relate" 327 | echo '# differential backups to master backups. The master backups have 0 as $rel_id and are thereby identifiable. Differential' 328 | echo '# backups have the id of the corresponding master backup as $rel_id.' 329 | echo "#" 330 | echo '# To ensure that master backups are kept long enough, the value of $CONFIG_rotation_daily is set to a minimum of 21 days.' 331 | echo "#" 332 | echo "#CONFIG_mysql_dump_differential='no'" 333 | echo "" 334 | echo "# Encryption" 335 | echo "" 336 | echo "# Do you wish to encrypt your backups using openssl?" 337 | echo "#CONFIG_encrypt='no'" 338 | echo "" 339 | echo "# Choose a password to encrypt the backups." 340 | echo "#CONFIG_encrypt_password='password0123'" 341 | echo "" 342 | echo "# Other" 343 | echo "" 344 | echo "# Backup local files, i.e. maybe you would like to backup your my.cnf (mysql server configuration), etc." 345 | echo "# These files will be tar'ed, depending on your compression option CONFIG_mysql_dump_compression compressed and" 346 | echo "# depending on the option CONFIG_encrypt encrypted." 347 | echo "#" 348 | echo "# Note: This could also have been accomplished with CONFIG_prebackup or CONFIG_postbackup." 349 | echo "#CONFIG_backup_local_files=()" 350 | echo "" 351 | echo "# Command to run before backups (uncomment to use)" 352 | if isSet PREBACKUP; then 353 | printf '%s=%q\n' CONFIG_prebackup "${PREBACKUP-}" 354 | else 355 | echo "#CONFIG_prebackup=\"/etc/mysql-backup-pre\"" 356 | fi 357 | echo "" 358 | echo "# Command run after backups (uncomment to use)" 359 | if isSet POSTBACKUP; then 360 | printf '%s=%q\n' CONFIG_postbackup "${POSTBACKUP-}" 361 | else 362 | echo "#CONFIG_postbackup=\"/etc/mysql-backup-post\"" 363 | fi 364 | echo "" 365 | echo "# Uncomment to activate! This will give folders rwx------" 366 | echo "# and files rw------- permissions." 367 | echo "#umask 0077" 368 | echo "" 369 | echo "# dry-run, i.e. show what you are gonna do without actually doing it" 370 | echo "# inactive: =0 or commented out" 371 | echo "# active: uncommented AND =1" 372 | echo "#CONFIG_dryrun=1" 373 | removeIO 374 | mv "$temp" "${1}_converted" 375 | return 0 376 | ) 377 | } 378 | 379 | 380 | parse_config_file () { 381 | printf 'Found config file %s. ' "$1" 382 | if head -n1 "$1" | egrep -o 'version=.*' >& /dev/null; then 383 | version=`head -n1 "$1" | egrep -o 'version=.*' | awk -F"=" '{print $2}'` 384 | if [[ "$version" =~ 3.* ]]; then 385 | printf 'Version 3.* determined. No conversion necessary.\n' 386 | else 387 | printf 'Unknown version. Can not convert it. You have to convert it manually.\n' 388 | fi 389 | else 390 | printf 'No version information on first line of config file. Assuming the version is <3.\n' 391 | while true; do 392 | read -p "Convert? [Y/n] " yn 393 | [[ "x$yn" = "x" ]] && { upgrade_config_file "$1" || echo "Failed to convert."; break; } 394 | case $yn in 395 | [Yy]* ) upgrade_config_file "$1" || echo "Failed to convert."; break;; 396 | [Nn]* ) break;; 397 | * ) echo "Please answer yes or no.";; 398 | esac 399 | done 400 | fi 401 | } 402 | 403 | #precheck 404 | echo "### Checking archive files for existence, readability and integrity." 405 | echo 406 | 407 | precheck_files=( automysqlbackup d6066f56200d51e869d59a0fb1c122a5 408 | automysqlbackup.conf d525efa3da15ce9fea96893e5a8ce6d5 409 | LICENSE 39bba7d2cf0ba1036f2a6e2be52fe3f0 410 | ) 411 | 412 | n=$(( ${#precheck_files[@]}/2 )) 413 | i=0 414 | while [ $i -lt $n ]; do 415 | printf "${precheck_files[$((2*$i))]} ... " 416 | if [ -r "${precheck_files[$((2*$i))]}" ]; then 417 | printf "exists and is readable ... " 418 | else 419 | printf "failed\n" 420 | exit 1 421 | fi 422 | if echo "${precheck_files[$((2*$i+1))]} ${precheck_files[$((2*$i))]}" | md5sum --check >/dev/null 2>&1; then 423 | printf "md5sum okay :)\n" 424 | else 425 | printf "md5sum failed :(\n" 426 | exit 1 427 | fi 428 | let i+=1 429 | done 430 | 431 | echo 432 | printf 'Select the global configuration directory [/etc/automysqlbackup]: ' 433 | read configdir 434 | configdir="${configdir%/}" # strip trailing slash if there 435 | [[ "x$configdir" = "x" ]] && configdir='/etc/automysqlbackup' 436 | printf 'Select directory for the executable [/usr/local/bin]: ' 437 | read bindir 438 | bindir="${bindir%/}" # strip trailing slash if there 439 | [[ "x$bindir" = "x" ]] && bindir='/usr/local/bin' 440 | 441 | #create global config directory 442 | echo "### Creating global configuration directory ${configdir}:" 443 | echo 444 | if [ -d "${configdir}" ]; then 445 | echo "exists already ... searching for config files:" 446 | for i in "${configdir}"/*.conf; do 447 | [[ "x$(basename $i)" = "xautomysqlbackup.conf" ]] && continue 448 | parse_config_file "$i" 449 | done 450 | else 451 | if mkdir "${configdir}" >/dev/null 2>&1; then 452 | #testing for permissions 453 | if [ -r "${configdir}" -a -x "${configdir}" ]; then 454 | printf "success\n" 455 | else 456 | printf "directory successfully created but has wrong permissions, trying to correct ... " 457 | if chmod +rx "${configdir}" >/dev/null 2>&1; then 458 | printf "corrected\n" 459 | else 460 | printf "failed. Aborting. Make sure you run the script with appropriate permissions.\n" 461 | fi 462 | fi 463 | else 464 | printf "failed ... check permissions.\n" 465 | fi 466 | fi 467 | 468 | echo 469 | #copying files 470 | echo "### Copying files." 471 | echo 472 | cp -i automysqlbackup.conf LICENSE README.md "${configdir}"/ 473 | cp -i automysqlbackup.conf "${configdir}"/myserver.conf 474 | cp -i automysqlbackup "${bindir}"/ 475 | [[ -f "${bindir}"/automysqlbackup ]] && [[ -x "${bindir}"/automysqlbackup ]] || chmod +x "${bindir}"/automysqlbackup || echo " failed - make sure you make the program executable, i.e. run 'chmod +x ${bindir}/automysqlbackup'" 476 | echo 477 | 478 | if echo $PATH | grep "${bindir}" >/dev/null 2>&1; then 479 | printf "if you are running automysqlbackup under the same user as you run this install script,\nyou should be able to access it by running 'automysqlbackup' from the command line.\n" 480 | printf "if not, you have to check if 'echo \$PATH' has ${bindir} in it\n" 481 | printf "\nSetup Complete!\n" 482 | else 483 | printf "if running under the current user, you have to use the full path ${bindir}/automysqlbackup since /usr/local/bin is not in 'echo \$PATH'\n" 484 | printf "\nSetup Complete!\n" 485 | fi -------------------------------------------------------------------------------- /run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | TARGET_DIR=/var/lib/automysqlbackup 3 | 4 | # Make sure the target directory exists: 5 | mkdir -p $TARGET_DIR 6 | 7 | # Run the backups: 8 | /usr/local/bin/automysqlbackup /etc/automysqlbackup/myserver.conf 9 | 10 | # Fix permissions of backups: 11 | chown root.root $TARGET_DIR* -R 12 | find $TARGET_DIR* -type f -exec chmod 400 {} \; 13 | find $TARGET_DIR* -type d -exec chmod 700 {} \; 14 | --------------------------------------------------------------------------------