Backups and Sync Success!
Posted October 31st 2013 05:19 am
After many, many months (and many mistakes), we finally have our own backup server!
Let me explain: when we had our data hosted on 3rd party servers, backups were never considered because the services already provided backups (or, so we were told). We only had to use WebsiteSource's backup service once, and the other company said they were creating a backup service, but never put us on it.
When we moved our data to the primary local server back in September, my level of worrying gradually increased over time. We had all of our site data on 1 server - and if that server were (for some reason) to crap out, we'd be in a very large hole with no way out. 8 years of coding. 8 years of projects. 8 years of client work. 8 years of a full portfolio. 8 years of database records. All gone.
Luckily, that hasn't happened yet (and we hope it never will - oh, the horrors if it did). So, in order to fill in this gap, I finally got the courage to make a backup server! (finally, right?)
I was able to salvage a bunch of the spare parts that I had laying around. Some RAM, some hard drives, a few semi-barebones desktop frames, motherboards galore...and all of this stuff was just collecting dust in various areas.
I went through a bunch of combinations of RAM/Hard Drives/Motherboards/CD Drives before I found a combo that worked (a few of the frames that I had required SATA drives, and I only have ATA/IDE drives available).
I finally got some RAM that fit appropriately, a large enough hard drive to hold the information that I needed (not the same size as the main server drive, but big enough), and a CD drive that worked (I had to install the Server OS somehow, right?).
After spending all of today getting the latest Ubuntu Server ISO downloaded and burned, I popped in the CD, ran the install ISO, and had a dedicated backup server ready to go.
I then spent the latter part of the day setting up
mysqldumpto automate the creation and mirroring of backups.
MySQL Dump Script:
#!/bin/bash BACKUP_DATE=$(date +'%F') mysqldump -u [username] -p[password] --all-databases --events > \ /var/www/vhosts/$BACKUP_DATE.sql
--eventsis here because it was returning an error saying "Skipping mysql.events table"
#!/bin/bash N=$(date +'%F %T') LOG_FILE="$HOME/rsync.log" echo "RSYNC Begin: $N" >> $LOG_FILE rsync -r -a -v -e "ssh -l [username]" --delete \ [source directory from root] [remote_server_ip]:[remote directory from root] >> $LOG_FILE echo "---------------------------------" >> $LOG_FILE
--deleteis to delete any remote files that aren't on the source location. Also, if you want to only sync certain file types, add "
--exclude='*' --include='*.[ext]'" after
--delete(exclude everything except what is included), or you can just omit
--deletealtogether if you want. The rest of the script is just log information - completely optional, but makes the log easier to read
Update 2013-12-09: I've updated the script a bit to make it a little more universal after some things were called to my attention
The main part of that was spent trying to get the backup server (which I am calling "Fry" - I'm thinking of going Futurama for a server nomenclature) to automatically SSH into the main server without a password (aka: storing an SSH key). I had to learn a lot in order to do that, but I found out that the problem that eluded me for about 5 hours was that the main server's home directory had the wrong permission settings for the
Next time the SSH keys aren't being stored right, run
[sudo] tail -f /var/log/auth.logand see what the error is. Lesson learned.
So, the overall point of this story:
- A dedicated backup server, Fry, has been salvaged, installed, and setup
- There is an
rsyncscript on Fry to automatically sync up the files from the main server (all 40GB+ worth)
- The Main Server has an automatic
cronscript to run
mysqldumpto make a full backup of all of the MySQL database
rsyncwill keep the latest 6 (daily) copies of the MySQL Dump backups along with the main backup
rsyncscript creates a custom log to record the output
Now to install a Leela (why not Zoidberg?) server for load balancing (or a production server. I don't know - I've never had more than 1 server running locally at a time).....
....and just because this whole "end of the year" thing is so chaotic, here's a comic I liked which I hope to do (just to get it all over with):