One of our partners asked me today about how to backup the EBS volumes running on their Amazon AWS instance by automating snapshots, like in this discussion: http://aws.amazon.com/articles/1663?_encoding=UTF8&jiveRedirect=1.

In our case we’ve got to backup a MySQL database, plus some other directories, and we want to snapshot those. Since MySQL’s files are not necessarily going to be in a consistent state through the snapshot window, we should also backup the database. Let’s tackle that first.

Backing up the MySQL database with a cron job, and setup crontab lines for daily, monthly, and weekly snapshots:

  1. use crontab -e to edit your crontab
  2. Add a line to backup the database every day at 10:12am (this is usually a low load time for online backup), and a line for daily, weekly, and monthly snapshots:
    12 10 * * * /usr/bin/mysqldump --add-drop-table -uroot -pSHHHHHHHH divinsamgmtdb > /var/backup/divinsamgmdb.sql
    
    17 13 * * * /root/snapshotscripts/managesnapshots daily >/var/log/dailysnapshots 2>&1
    
    20 13 * * 0 /root/snapshotscripts/managesnapshots weekly >/var/log/weeklysnapshots 2>&1
    
    23 13 1 * * /root/snapshotscripts/managesnapshots monthly >/var/log/monthlysnapshots 2>&1
  3. Make sure you have a directory called /var/backup and that the drive has enough space.
  4. Verify that /var/backup is actually on your / (root) partition as that’s the one we’re going to backup using Amazon snapshots.

Install Amazon EC2 API Command line tools on your Ubuntu server

This is covered on the web in many places, so here is a great link to walk you through it (scroll down a bit): https://help.ubuntu.com/community/EC2StartersGuide

Deploy and customize the following script to create snapshots:

  1. Create a file called /root/snapshotscripts/managesnapshots and paste the following script into it:
    #!/bin/bash
    
    		# backups and and deletes given a dailysnapshotlist (or monthly or weekly - make sure you run the cron accordingly)
    
    		
    
    		# Set env variables for Amazon as cron may run without them
    
    		export EC2_PRIVATE_KEY=/root/.ec2/YOURFILE.pem
    
    		export EC2_CERT=/root/.ec2/YOURCERT.pem
    
    		export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
    
    		
    
    		#if no $1, exit
    
    		if
    
    		then
    
    		  export runtype="$1"
    
    		else
    
    		  printf "Usage: managesnapshots daily|weekly|monthly\n"
    
    		  exit 1
    
    		fi
    
    		
    
    		#set date time stamp for temp files
    
    		export dts=$(date +"%m-%d-%Y")
    
    		
    
    		while read volume_id description daily weekly monthly
    
    		do
    
    		  #exit loop if no values - otherwise get infinite loop
    
    		  [ -z $volume_id ] && break;
    
    		  # skip lines with comments in them
    
    		  if [[ $volume_id =~ ^#.* ]]
    
    		  then
    
    		    true
    
    		  else
    
    		    #add runtype into the ec2 description so we can search for snapshot types for deletions later
    
    		    export description=$(echo "$description $runtype")
    
    		    #add 1 to the keepdays so the tail command get s right number of rows ($$ uses variable substitution)
    
    		    case $runtype in
    
    		     daily)
    
    		       export keepdays=$daily
    
    		       ;;
    
    		     monthly)
    
    		       export keepdays=$monthly
    
    		       ;;
    
    		     weekly)
    
    		       export keepdays=$weekly
    
    		       ;;
    
    		     *)
    
    		       exit 2
    
    		       ;;
    
    		    esac
    
    		    export tailnumber=$(($keepdays + 1))
    
    		    if [ $keepdays != 0 ]
    
    		      then
    
    		      # create the snapshot for today
    
    		      echo "snapshotting $volume_id -- $description"
    
    		      /usr/bin/ec2-create-snapshot "$volume_id" -d "$description"
    
    		      #delete the older shapshots now
    
    		      #get the snapshots that are older than the ones we want to keep
    
    		      printf "checking for snapshots to delete from $volume_id\n"
    
    		      /usr/bin/ec2-describe-snapshots --filter volume-id="$volume_id" | grep $runtype | sort -r -k 5 | tail -n +${tailnumber} > /var/tmp/delsnapshots$dts
    
    		      while read deleteline
    
    		      do
    
    		        #get snapshot_id
    
    		        export snapshot_id=$(echo $deleteline | awk '{print $2}')
    
    		        #delete the older snapshot
    
    		        printf "deleting snapshot $snapshot_id for $volume_id -- $description\n"
    
    		        /usr/bin/ec2-delete-snapshot "$snapshot_id"
    
    		      done < /var/tmp/delsnapshots$dts
    
    		      rm /var/tmp/delsnapshots$dts
    
    		    fi
    
    		  fi
    
    		done < /root/snapshotscripts/snapshotlist
    
    		echo "end"
    
    		exit 0
  2. Now Create a file to list the volumes you want to snapshot, called /root/snapshotscripts/snapshotlist, and past the following contents, then modify them so that each line lists a volumeid of the volumes you want to backup, which instance they’re on and which device(for tagging purposes so that you have a note attached to each snapshot), and how many daily, weekly, and monthly images you wish to keep:
    #volume_id description keepdays_daily weekly monthly [don't_use_spaces_in_values_-->_they_are_the_delimiter!]
    
    		vol-dcsdssa6 WSBU_Serv01:/dev/sda1 3 3 3
    
    		vol-32fja92r WSBU_Serv02:/dev/sda1 6 6 6
    
    		vol-89asd0fe WSBU_Serv03:/dev/sda1 3 3 3
    
  3. make sure these are readable only by root, and that they’re executable only by root:
    chown -R root:root /root/snapshotscripts/
    
    		chmod -R 700 /root/snapshotscripts/

OK – you’re now done!

Some notes and caveats: you should check to see that the script works in your environment, as even a small typo or configuration change can wreak havoc. Verify that you have snapshots by restoring one and testing it!

There will be logfiles in /var/log/ so you can see errors if they were logged and progress notes as they were logged.

You can also manually run the commands to test them, like:

/root/snapshotscripts/managesnapshots daily