How To Automate Amazon AWS Cloud Backups With Cron and Snapshot Script, Including Deletions

One of our partners asked me today about how to backup the EBS volumes running on their Amazon AWS instance by automating snapshots, like in this discussion: on AWS about "one approach to using Amazon EC2's Elastic Block Store (EBS) as a persistent storage mechanism for a production MySQL database server, including snapshot backup and restore."

In our case we've got to backup a MySQL database, plus some other directories, and we want to snapshot those. Since MySQL's files are not necessarily going to be in a consistent state through the snapshot window, we should also backup the database. Let's tackle that first.

Backing up the MySQL database with a cron job, and setup crontab lines for daily, monthly, and weekly snapshots:

1. use crontab -e to edit your crontab
2. Add a line to backup the database every day at 10:12am (this is usually a low load time for online backup), and a line for daily, weekly, and monthly snapshots:

   12 10 * * * /usr/bin/mysqldump --add-drop-table -uroot -pSHHHHHHHH divinsamgmtdb > /var/backup/divinsamgmdb.sql
   
   17 13 * * * /root/snapshotscripts/managesnapshots daily >/var/log/dailysnapshots 2>&1
   
   20 13 * * 0 /root/snapshotscripts/managesnapshots weekly >/var/log/weeklysnapshots 2>&1
   
   23 13 1 * * /root/snapshotscripts/managesnapshots monthly >/var/log/monthlysnapshots 2>&1

3. Make sure you have a directory called /var/backup and that the drive has enough space.
4. Verify that /var/backup is actually on your / (root) partition as that's the one we're going to backup using Amazon snapshots.

Install Amazon EC2 API Command line tools on your Ubuntu server

This is covered on the web in many places, so here is a great link to walk you through it (scroll down a bit): https://help.ubuntu.com/community/EC2StartersGuide

Deploy and customize the following script to create snapshots:

1. Create a file called /root/snapshotscripts/managesnapshots and paste the following script into it:

   #!/bin/bash
   
   		# backups and and deletes given a dailysnapshotlist (or monthly or weekly - make sure you run the cron accordingly)
   
   		
   
   		# Set env variables for Amazon as cron may run without them
   
   		export EC2_PRIVATE_KEY=/root/.ec2/YOURFILE.pem
   
   		export EC2_CERT=/root/.ec2/YOURCERT.pem
   
   		export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
   
   		
   
   		#if no $1, exit
   
   		if
   
   		then
   
   		  export runtype="$1"
   
   		else
   
   		  printf "Usage: managesnapshots daily|weekly|monthly\n"
   
   		  exit 1
   
   		fi
   
   		
   
   		#set date time stamp for temp files
   
   		export dts=$(date +"%m-%d-%Y")
   
   		
   
   		while read volume_id description daily weekly monthly
   
   		do
   
   		  #exit loop if no values - otherwise get infinite loop
   
   		  [ -z $volume_id ] && break;
   
   		  # skip lines with comments in them
   
   		  if [[ $volume_id =~ ^#.* ]]
   
   		  then
   
   		    true
   
   		  else
   
   		    #add runtype into the ec2 description so we can search for snapshot types for deletions later
   
   		    export description=$(echo "$description $runtype")
   
   		    #add 1 to the keepdays so the tail command get s right number of rows ($$ uses variable substitution)
   
   		    case $runtype in
   
   		     daily)
   
   		       export keepdays=$daily
   
   		       ;;
   
   		     monthly)
   
   		       export keepdays=$monthly
   
   		       ;;
   
   		     weekly)
   
   		       export keepdays=$weekly
   
   		       ;;
   
   		     *)
   
   		       exit 2
   
   		       ;;
   
   		    esac
   
   		    export tailnumber=$(($keepdays + 1))
   
   		    if [ $keepdays != 0 ]
   
   		      then
   
   		      # create the snapshot for today
   
   		      echo "snapshotting $volume_id -- $description"
   
   		      /usr/bin/ec2-create-snapshot "$volume_id" -d "$description"
   
   		      #delete the older shapshots now
   
   		      #get the snapshots that are older than the ones we want to keep
   
   		      printf "checking for snapshots to delete from $volume_id\n"
   
   		      /usr/bin/ec2-describe-snapshots --filter volume-id="$volume_id" | grep $runtype | sort -r -k 5 | tail -n +$ > /var/tmp/delsnapshots$dts
   
   		      while read deleteline
   
   		      do
   
   		        #get snapshot_id
   
   		        export snapshot_id=$(echo $deleteline | awk '')
   
   		        #delete the older snapshot
   
   		        printf "deleting snapshot $snapshot_id for $volume_id -- $description\n"
   
   		        /usr/bin/ec2-delete-snapshot "$snapshot_id"
   
   		      done < /var/tmp/delsnapshots$dts
   
   		      rm /var/tmp/delsnapshots$dts
   
   		    fi
   
   		  fi
   
   		done < /root/snapshotscripts/snapshotlist
   
   		echo "end"
   
   		exit 0

2. Now Create a file to list the volumes you want to snapshot, called /root/snapshotscripts/snapshotlist, and past the following contents, then modify them so that each line lists a volumeid of the volumes you want to backup, which instance they're on and which device(for tagging purposes so that you have a note attached to each snapshot), and how many daily, weekly, and monthly images you wish to keep:

   #volume_id description keepdays_daily weekly monthly [don't_use_spaces_in_values_-->_they_are_the_delimiter!]
   
   		vol-dcsdssa6 WSBU_Serv01:/dev/sda1 3 3 3
   
   		vol-32fja92r WSBU_Serv02:/dev/sda1 6 6 6
   
   		vol-89asd0fe WSBU_Serv03:/dev/sda1 3 3 3
   

3. make sure these are readable only by root, and that they're executable only by root:

   chown -R root:root /root/snapshotscripts/
   
   		chmod -R 700 /root/snapshotscripts/

OK – you're now done!

Some notes and caveats: you should check to see that the script works in your environment, as even a small typo or configuration change can wreak havoc. Verify that you have snapshots by restoring one and testing it!

There will be logfiles in /var/log/ so you can see errors if they were logged and progress notes as they were logged.

You can also manually run the commands to test them, like:

/root/snapshotscripts/managesnapshots daily