Kyle M. Perkins

After many, many months (and many mistakes), we finally have our own backup server!

Let me explain: when we had our data hosted on 3rd party servers, backups were never considered because the services already provided backups (or, so we were told). We only had to use WebsiteSource's backup service once, and the other company said they were creating a backup service, but never put us on it.

When we moved our data to the primary local server back in September, my level of worrying gradually increased over time. We had all of our site data on 1 server - and if that server were (for some reason) to crap out, we'd be in a very large hole with no way out. 8 years of coding. 8 years of projects. 8 years of client work. 8 years of a full portfolio. 8 years of database records. All gone.

Luckily, that hasn't happened yet (and we hope it never will - oh, the horrors if it did). So, in order to fill in this gap, I finally got the courage to make a backup server! (finally, right?)

I was able to salvage a bunch of the spare parts that I had laying around. Some RAM, some hard drives, a few semi-barebones desktop frames, motherboards galore...and all of this stuff was just collecting dust in various areas.

I went through a bunch of combinations of RAM/Hard Drives/Motherboards/CD Drives before I found a combo that worked (a few of the frames that I had required SATA drives, and I only have ATA/IDE drives available).

I finally got some RAM that fit appropriately, a large enough hard drive to hold the information that I needed (not the same size as the main server drive, but big enough), and a CD drive that worked (I had to install the Server OS somehow, right?).

After spending all of today getting the latest Ubuntu Server ISO downloaded and burned, I popped in the CD, ran the install ISO, and had a dedicated backup server ready to go.

I then spent the latter part of the day setting up rsync, cron, and mysqldump to automate the creation and mirroring of backups.

MySQL Dump Script:

#!/bin/bash
 
BACKUP_DATE=$(date +'%F')
 
mysqldump -u [username] -p[password] --all-databases --events > \
 
   /var/www/vhosts/$BACKUP_DATE.sql

--events is here because it was returning an error saying "Skipping mysql.events table"

RSYNC Script:

#!/bin/bash
 
N=$(date +'%F %T')
 
LOG_FILE="$HOME/rsync.log"
 
echo "RSYNC Begin: $N" >> $LOG_FILE
 
rsync -r -a -v -e "ssh -l [username]" --delete \
 
   [source directory from root] [remote_server_ip]:[remote directory from root]  >> $LOG_FILE
 
echo "---------------------------------" >> $LOG_FILE

--delete is to delete any remote files that aren't on the source location. Also, if you want to only sync certain file types, add "--exclude='*' --include='*.[ext]'" after --delete (exclude everything except what is included), or you can just omit --delete altogether if you want. The rest of the script is just log information - completely optional, but makes the log easier to read

Update 2013-12-09: I've updated the script a bit to make it a little more universal after some things were called to my attention

The main part of that was spent trying to get the backup server (which I am calling "Fry" - I'm thinking of going Futurama for a server nomenclature) to automatically SSH into the main server without a password (aka: storing an SSH key). I had to learn a lot in order to do that, but I found out that the problem that eluded me for about 5 hours was that the main server's home directory had the wrong permission settings for the .ssh folder.

Next time the SSH keys aren't being stored right, run [sudo] tail -f /var/log/auth.log and see what the error is. Lesson learned.

So, the overall point of this story:

• A dedicated backup server, Fry, has been salvaged, installed, and setup
• There is an rsync script on Fry to automatically sync up the files from the main server (all 40GB+ worth)
• The Main Server has an automatic cron script to run mysqldump to make a full backup of all of the MySQL database
• Fry's rsync will keep the latest 6 (daily) copies of the MySQL Dump backups along with the main backup
• Fry's rsync script creates a custom log to record the output

Now to install a Leela (why not Zoidberg?) server for load balancing (or a production server. I don't know - I've never had more than 1 server running locally at a time).....

....and just because this whole "end of the year" thing is so chaotic, here's a comic I liked which I hope to do (just to get it all over with):


Good Night!