├── .gitignore ├── README.markdown ├── full_backup ├── full_backup.cron ├── mysql_backup └── mysql_backup.cron /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | -------------------------------------------------------------------------------- /README.markdown: -------------------------------------------------------------------------------- 1 | # Web Server Backup Script (now with S3 sync) 2 | 3 | This is a bash script for backing up multiple web sites and MySQL databases into a specified backups directory. It's a good idea to run it every night via cron. 4 | 5 | Once configured (variables set within the script), it does this: 6 | 7 | * Creates a directory for your site (file) backups (if it doesn't exist) 8 | * Creates a directory for your MySQL dumps (if it doesn't exist) 9 | * Loops through all of your MySQL databases and dumps each one of them to a gzipped file 10 | * Deletes database dumps older than a specified number of days from the backup directory 11 | * Tars and gzips each folder within your sites directory (I keep my websites in /var/www/sites/) 12 | * Deletes site archives older than a specified number of days from the backup directory 13 | * Optionally syncs all backup files to Amazon S3 or a remote server, using s3sync.rb or rsync respectively 14 | 15 | # BETA WARNING 16 | 17 | ___This script works fine for me (using Ubuntu 8.04 on Slicehost), but servers vary greatly. USE THIS SCRIPT AT YOUR OWN RISK!! There is always risk involved with running a script. I AM NOT RESPONSIBLE FOR DAMAGE CAUSED BY THIS SCRIPT.___ 18 | 19 | You may very well know more about bash scripting and archiving than I do. If you find any flaws with this script or have any recommendations as to how this script can be improved, please fork it and send me a pull request. 20 | 21 | # Installation 22 | 23 | * __MOST IMPORTANTLY:__ Open the backup.sh file in a text editor and set the configuration variables at the top (see below). 24 | * Optionally, edit the tar command on line 91 to add some more --exclude options (e.g. --exclude="cache/*") 25 | * Place the backup.sh file somewhere on your server (something like /usr/local/web-server-backup). 26 | * Make sure the backup.sh script is owned by root: `sudo chown -R 0:0 /usr/local/web-server-backup` 27 | * Make sure the backup.sh script is executable by root: `sudo chmod 744 /usr/local/web-server-backup/backup.sh` 28 | * Set up your Amazon S3 account and bucket (or a remote account for rsync) 29 | * Set up s3sync (see below) 30 | * Preferably set up cron to run it every night (see below). 31 | 32 | # Configuration 33 | 34 | There are a bunch of variables that you can set to customize the way the script works. _Some of them __must__ be set before running the script!_ 35 | 36 | NOTE: The BACKUP\_DIR setting is preset to /backups/site-backups. If you want to use something like /var/site-backups, you'll need to create the directory first and set it to be writable by you. 37 | 38 | ## General Settings: 39 | 40 | * __BACKUP\_DIR__: The parent directory in which the backups will be placed. It's preset to: `"/backups/site-backups"` 41 | * __KEEP\_MYSQL__: How many days worth of mysql dumps to keep. It's preset to: `"14"` 42 | * __KEEP\_SITES__: How many days worth of site tarballs to keep. It's preset to: `"2"` 43 | 44 | ## MySQL Settings: 45 | 46 | * __MYSQL\_HOST__: The MySQL hostname. It's preset to the standard: `"localhost"` 47 | * __MYSQL\_USER__: The MySQL username. It's preset to the standard: `"root"` 48 | * __MYSQL\_PASS__: The MySQL password. ___You'll need to set this yourself!___ 49 | * __MYSQL\_BACKUP\_DIR__: The directory in which the dumps will be placed. It's preset to: `"$BACKUP\_DIR/mysql/"` 50 | 51 | ## Web Site Settings: 52 | 53 | * __SITES\_DIR__: This is the directory where you keep all of your web sites. It's preset to: `"/var/www/sites/"` 54 | * __SITES\_BACKUP\_DIR__: The directory in which the archived site files will be placed. It's preset to: `"$BACKUP_DIR/sites/"` 55 | 56 | ## S3sync Settings (recommended): 57 | 58 | You can sync to an Amazon S3 bucket, but you'll need to install s3sync.rb first. [Get it from here.](http://s3sync.net/) It is a Ruby script, so you'll need to make sure you have Ruby installed. 59 | 60 | The only thing tricky about getting s3sync.rb installed is the CA certificates. If you are on Ubuntu, you can install the ca-certificates package with apt-get or aptitude. You need those certificates to connect to S3 securely (SSL). 61 | 62 | * __S3SYNC_PATH__: Wherever you installed s3sync.rb. It's preset to: `"/usr/local/s3sync/s3sync.rb"` 63 | * __S3\_BUCKET__: The name of the bucket to which you wish to sync. 64 | * __AWS\_ACCESS\_KEY_ID__: Log in to your Amazon AWS account to get this. 65 | * __AWS\_SECRET\_ACCESS_KEY__: Log in to your Amazon AWS account to get this. 66 | * __USE\_SSL__: If this is set to `"true"`, s3sync will use a secure connection to S3, but you'll need to set the SSL\_CERT\_DIR or SSL\_CERT\_FILE to make it work. See the s3sync README for more info. 67 | * __SSL\_CERT\_DIR__: Where your Cert Authority keys live. It's preset to: `"/etc/ssl/certs"` 68 | * __SSL\_CERT\_FILE__: If you have just one PEM file for CA verification, you can use this instead of SSL\_CERT\_DIR. 69 | 70 | ## Rsync Settings (alternative to S3): 71 | 72 | * __RSYNC__: Whether or not you want to rsync the backups to another server. (Either "true" or "false") It's preset to: `"true"` 73 | * __RSYNC\_USER__: The user account name on the remote server. Please note that there is no password setting. It is recommended that you use an SSH key. ___You'll need to set this yourself!___ 74 | * __RSYNC\_SERVER__: The server address of the remote server. ___You'll need to set this yourself!___ It's preset to: `"other.server.com"` 75 | * __RSYNC\_DIR__: The directory on the remote server that will be synchronized with $BACKUP\_DIR. It's preset to: `"web_site_backups"` 76 | * __RSYNC\_PORT__: If you have set a custom SSH port on your remote server, you'll need to change this. It's preset to: `"22"` 77 | 78 | ## Date format: (change if you want) 79 | 80 | * __THE\_DATE__: The date that will be appended to filenames. It's preset to: `"$(date '+%Y-%m-%d')"` 81 | 82 | ## Paths to commands: (probably won't need to change these) 83 | 84 | * __MYSQL\_PATH__: Path to mysql. It's preset to: `"$(which mysql)"` 85 | * __MYSQLDUMP\_PATH__: Path to mysqldump. It's preset to: `"$(which mysqldump)"` 86 | * __FIND\_PATH__: Path to find. It's preset to: `"$(which find)"` 87 | * __TAR\_PATH__: Path to tar. It's preset to: `"$(which tar)"` 88 | * __RSYNC\_PATH__: Path to rsync. It's preset to: `"$(which rsync)"` 89 | 90 | # Running with cron (recommended) 91 | 92 | Once you've tested the script, I recommend setting it up to be run every night with cron. Here's a sample cron config: 93 | 94 | SHELL=/bin/bash 95 | PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin 96 | MAILTO=jason@example.com 97 | HOME=/root 98 | 99 | 30 4 * * * root /usr/local/web-server-backup/backup.sh 100 | 101 | That'll run the script (located in /usr/local) at 4:30 every morning and email the output to jason@example.com. 102 | 103 | If you want to only receive emails about errors, you can use: 104 | 105 | 30 4 * * * root /usr/local/web-server-backup/backup.sh > /dev/null 106 | 107 | So, take the above example, change the email address, etc., save it to a text file, and place it in `/etc/cron.d/`. That should do it. 108 | -------------------------------------------------------------------------------- /full_backup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # BEGIN CONFIGURATION ========================================================== 4 | 5 | BACKUP_DIR="/backups/site_backups" # The directory in which you want backups placed 6 | DUMP_MYSQL=true 7 | TAR_SITES=false 8 | SYNC="none" # Either 's3sync', 'rsync', or 'none' 9 | KEEP_MYSQL="14" # How many days worth of mysql dumps to keep 10 | KEEP_SITES="2" # How many days worth of site tarballs to keep 11 | 12 | MYSQL_HOST="localhost" 13 | MYSQL_USER="root" 14 | MYSQL_PASS="" 15 | MYSQL_BACKUP_DIR="$BACKUP_DIR/mysql/" 16 | 17 | SITES_DIR="/var/www/sites/" 18 | SITES_BACKUP_DIR="$BACKUP_DIR/sites/" 19 | 20 | 21 | # See s3sync info in README 22 | S3SYNC_PATH="/usr/local/s3sync/s3sync.rb" 23 | S3_BUCKET="my-fancy-bucket" 24 | AWS_ACCESS_KEY_ID="YourAWSAccessKey" # Log in to your Amazon AWS account to get this 25 | AWS_SECRET_ACCESS_KEY="YourAWSSecretAccessKey" # Log in to your Amazon AWS account to get this 26 | USE_SSL="true" 27 | SSL_CERT_DIR="/etc/ssl/certs" # Where your Cert Authority keys live; for verification 28 | SSL_CERT_FILE="" # If you have just one PEM file for CA verification 29 | 30 | # If you don't want to use S3, you can rsync to another server 31 | RSYNC_USER="user" 32 | RSYNC_SERVER="other.server.com" 33 | RSYNC_DIR="web_site_backups" 34 | RSYNC_PORT="22" # Change this if you've customized the SSH port of your backup system 35 | 36 | # You probably won't have to change these 37 | THE_DATE="$(date '+%Y-%m-%d')" 38 | 39 | MYSQL_PATH="$(which mysql)" 40 | MYSQLDUMP_PATH="$(which mysqldump)" 41 | FIND_PATH="$(which find)" 42 | TAR_PATH="$(which tar)" 43 | RSYNC_PATH="$(which rsync)" 44 | 45 | # END CONFIGURATION ============================================================ 46 | 47 | 48 | 49 | # Announce the backup time 50 | echo "Backup Started: $(date)" 51 | 52 | # Create the backup dirs if they don't exist 53 | if [[ ! -d $BACKUP_DIR ]] 54 | then 55 | mkdir -p "$BACKUP_DIR" 56 | fi 57 | if [[ ! -d $MYSQL_BACKUP_DIR ]] 58 | then 59 | mkdir -p "$MYSQL_BACKUP_DIR" 60 | fi 61 | if [[ ! -d $SITES_BACKUP_DIR ]] 62 | then 63 | mkdir -p "$SITES_BACKUP_DIR" 64 | fi 65 | 66 | if [ "$DUMP_MYSQL" = "true" ] 67 | then 68 | 69 | # Get a list of mysql databases and dump them one by one 70 | echo "------------------------------------" 71 | DBS="$($MYSQL_PATH -h $MYSQL_HOST -u$MYSQL_USER -p$MYSQL_PASS -Bse 'show databases')" 72 | for db in $DBS 73 | do 74 | if [[ $db != "information_schema" && $db != "mysql" && $db != "performance_schema" ]] 75 | then 76 | echo "Dumping: $db..." 77 | $MYSQLDUMP_PATH --opt --skip-add-locks -h $MYSQL_HOST -u$MYSQL_USER -p$MYSQL_PASS $db | gzip > $MYSQL_BACKUP_DIR$db\_$THE_DATE.sql.gz 78 | fi 79 | done 80 | 81 | # Delete old dumps 82 | echo "------------------------------------" 83 | echo "Deleting old backups..." 84 | # List dumps to be deleted to stdout (for report) 85 | $FIND_PATH $MYSQL_BACKUP_DIR*.sql.gz -mtime +$KEEP_MYSQL 86 | # Delete dumps older than specified number of days 87 | $FIND_PATH $MYSQL_BACKUP_DIR*.sql.gz -mtime +$KEEP_MYSQL -exec rm {} + 88 | 89 | fi 90 | 91 | if [ "$TAR_SITES" == "true" ] 92 | then 93 | 94 | # Get a list of files in the sites directory and tar them one by one 95 | echo "------------------------------------" 96 | cd $SITES_DIR 97 | for d in * 98 | do 99 | echo "Archiving $d..." 100 | $TAR_PATH --exclude="*/log" -C $SITES_DIR -czf $SITES_BACKUP_DIR/$d\_$THE_DATE.tgz $d 101 | done 102 | 103 | # Delete old site backups 104 | echo "------------------------------------" 105 | echo "Deleting old backups..." 106 | # List files to be deleted to stdout (for report) 107 | $FIND_PATH $SITES_BACKUP_DIR*.tgz -mtime +$KEEP_SITES 108 | # Delete files older than specified number of days 109 | $FIND_PATH $SITES_BACKUP_DIR*.tgz -mtime +$KEEP_SITES -exec rm {} + 110 | 111 | fi 112 | 113 | # Rsync everything with another server 114 | if [[ "$SYNC" == "rsync" ]] 115 | then 116 | echo "------------------------------------" 117 | echo "Sending backups to backup server..." 118 | $RSYNC_PATH --del -vaze "ssh -p $RSYNC_PORT" $BACKUP_DIR/ $RSYNC_USER@$RSYNC_SERVER:$RSYNC_DIR 119 | 120 | # OR s3sync everything with Amazon S3 121 | elif [[ "$SYNC" == "s3sync" ]] 122 | then 123 | export AWS_ACCESS_KEY_ID 124 | export AWS_SECRET_ACCESS_KEY 125 | export SSL_CERT_DIR 126 | export SSL_CERT_FILE 127 | if [[ $USE_SSL == "true" ]] 128 | then 129 | SSL_OPTION=' --ssl ' 130 | else 131 | SSL_OPTION='' 132 | fi 133 | echo "------------------------------------" 134 | echo "Sending backups to s3..." 135 | $S3SYNC_PATH --delete -v $SSL_OPTION -r $BACKUP_DIR/ $S3_BUCKET:backups 136 | fi 137 | 138 | # Announce the completion time 139 | echo "------------------------------------" 140 | echo "Backup Completed: $(date)" 141 | -------------------------------------------------------------------------------- /full_backup.cron: -------------------------------------------------------------------------------- 1 | SHELL=/bin/bash 2 | PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin 3 | MAILTO=someone@example.com 4 | HOME=/root 5 | 6 | 00 3 * * * root /usr/local/web-server-backup/full_backup > /dev/null -------------------------------------------------------------------------------- /mysql_backup: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # BEGIN CONFIGURATION ========================================================== 4 | 5 | BACKUP_DIR="/backups/" # The directory in which you want backups placed 6 | KEEP_MYSQL="14" # How many days worth of mysql dumps to keep 7 | 8 | MYSQL_HOST="localhost" 9 | MYSQL_USER="root" 10 | MYSQL_PASS="" 11 | MYSQL_BACKUP_DIR="$BACKUP_DIR/mysql/" 12 | 13 | # You probably won't have to change these 14 | THE_DATE="$(date '+%Y-%m-%d')" 15 | 16 | MYSQL_PATH="$(which mysql)" 17 | MYSQLDUMP_PATH="$(which mysqldump)" 18 | FIND_PATH="$(which find)" 19 | TAR_PATH="$(which tar)" 20 | RSYNC_PATH="$(which rsync)" 21 | 22 | # END CONFIGURATION ============================================================ 23 | 24 | 25 | 26 | # Announce the backup time 27 | echo "Backup Started: $(date)" 28 | 29 | # Create the backup dirs if they don't exist 30 | if [[ ! -d $BACKUP_DIR ]] 31 | then 32 | mkdir -p "$BACKUP_DIR" 33 | fi 34 | if [[ ! -d $MYSQL_BACKUP_DIR ]] 35 | then 36 | mkdir -p "$MYSQL_BACKUP_DIR" 37 | fi 38 | 39 | # Get a list of mysql databases and dump them one by one 40 | echo "------------------------------------" 41 | ALL_DBS=($($MYSQL_PATH -h $MYSQL_HOST -u$MYSQL_USER -p$MYSQL_PASS -Bse 'show databases')) 42 | SYSTEM_DBS=("information_schema" "mysql" "performance_schema" "test") 43 | DBS=() 44 | 45 | for i in "${ALL_DBS[@]}"; do 46 | skip= 47 | for j in "${SYSTEM_DBS[@]}"; do 48 | [[ $i == $j ]] && { skip=1; break; } 49 | done 50 | [[ -n $skip ]] || DBS+=("$i") 51 | done 52 | 53 | for db in "${DBS[@]}"; do 54 | echo "Dumping: $db..." 55 | $MYSQLDUMP_PATH --opt --skip-add-locks -h $MYSQL_HOST -u$MYSQL_USER -p$MYSQL_PASS $db | gzip > $MYSQL_BACKUP_DIR$db\_$THE_DATE.sql.gz 56 | done 57 | 58 | # Delete old dumps 59 | echo "------------------------------------" 60 | echo "Deleting old backups..." 61 | # List dumps to be deleted to stdout (for report) 62 | $FIND_PATH $MYSQL_BACKUP_DIR*.sql.gz -mtime +$KEEP_MYSQL 63 | # Delete dumps older than specified number of days 64 | $FIND_PATH $MYSQL_BACKUP_DIR*.sql.gz -mtime +$KEEP_MYSQL -exec rm {} + 65 | 66 | 67 | # Announce the completion time 68 | echo "------------------------------------" 69 | echo "Backup Completed: $(date)" 70 | -------------------------------------------------------------------------------- /mysql_backup.cron: -------------------------------------------------------------------------------- 1 | SHELL=/bin/bash 2 | PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin 3 | MAILTO=someone@example.com 4 | HOME=/root 5 | 6 | 00 3 * * * root /usr/local/web-server-backup/mysql_backup > /dev/null --------------------------------------------------------------------------------