Author - Web Developer - Educator
Found 2 results for tag "ubuntu"
RSS Feed

Backups and Sync Success!

Backups are important

After many, many months (and many mistakes), we finally have our own backup server!

Let me explain: when we had our data hosted on 3rd party servers, backups were never considered because the services already provided backups (or, so we were told). We only had to use WebsiteSource's backup service once, and the other company said they were creating a backup service, but never put us on it.

When we moved our data to the primary local server back in September, my level of worrying gradually increased over time. We had all of our site data on 1 server - and if that server were (for some reason) to crap out, we'd be in a very large hole with no way out. 8 years of coding. 8 years of projects. 8 years of client work. 8 years of a full portfolio. 8 years of database records. All gone.

Luckily, that hasn't happened yet (and we hope it never will - oh, the horrors if it did). So, in order to fill in this gap, I finally got the courage to make a backup server! (finally, right?)

I was able to salvage a bunch of the spare parts that I had laying around. Some RAM, some hard drives, a few semi-barebones desktop frames, motherboards galore...and all of this stuff was just collecting dust in various areas.

I went through a bunch of combinations of RAM/Hard Drives/Motherboards/CD Drives before I found a combo that worked (a few of the frames that I had required SATA drives, and I only have ATA/IDE drives available).

I finally got some RAM that fit appropriately, a large enough hard drive to hold the information that I needed (not the same size as the main server drive, but big enough), and a CD drive that worked (I had to install the Server OS somehow, right?).

After spending all of today getting the latest Ubuntu Server ISO downloaded and burned, I popped in the CD, ran the install ISO, and had a dedicated backup server ready to go.

I then spent the latter part of the day setting up rsync, cron, and mysqldump to automate the creation and mirroring of backups.

MySQL Dump Script:
#!/bin/bash
 
BACKUP_DATE=$(date +'%F')
 
mysqldump -u [username] -p[password] --all-databases --events > \
 
   /var/www/vhosts/$BACKUP_DATE.sql
--events is here because it was returning an error saying "Skipping mysql.events table"

RSYNC Script:
#!/bin/bash
 
N=$(date +'%F %T')
 
LOG_FILE="$HOME/rsync.log"
 
echo "RSYNC Begin: $N" >> $LOG_FILE
 
rsync -r -a -v -e "ssh -l [username]" --delete \
 
   [source directory from root] [remote_server_ip]:[remote directory from root]  >> $LOG_FILE
 
echo "---------------------------------" >> $LOG_FILE
--delete is to delete any remote files that aren't on the source location. Also, if you want to only sync certain file types, add "--exclude='*' --include='*.[ext]'" after --delete (exclude everything except what is included), or you can just omit --delete altogether if you want. The rest of the script is just log information - completely optional, but makes the log easier to read

Update 2013-12-09: I've updated the script a bit to make it a little more universal after some things were called to my attention

The main part of that was spent trying to get the backup server (which I am calling "Fry" - I'm thinking of going Futurama for a server nomenclature) to automatically SSH into the main server without a password (aka: storing an SSH key). I had to learn a lot in order to do that, but I found out that the problem that eluded me for about 5 hours was that the main server's home directory had the wrong permission settings for the .ssh folder.

Next time the SSH keys aren't being stored right, run [sudo] tail -f /var/log/auth.log and see what the error is. Lesson learned.

So, the overall point of this story:
  • A dedicated backup server, Fry, has been salvaged, installed, and setup
  • There is an rsync script on Fry to automatically sync up the files from the main server (all 40GB+ worth)
  • The Main Server has an automatic cron script to run mysqldump to make a full backup of all of the MySQL database
  • Fry's rsync will keep the latest 6 (daily) copies of the MySQL Dump backups along with the main backup
  • Fry's rsync script creates a custom log to record the output


Now to install a Leela (why not Zoidberg?) server for load balancing (or a production server. I don't know - I've never had more than 1 server running locally at a time).....

....and just because this whole "end of the year" thing is so chaotic, here's a comic I liked which I hope to do (just to get it all over with):
BC


Good Night!


Tags:#bc #xkcd #backup #ubuntu #futurama #rsync #sudo #zoidberg #bash #tutorials

RSS Feed

Summer Review

This has certainly been a crazy summer. Since my last post in May, I'll sum up what has happened, but first, this:
Dilbert

Business:

  • We have decided to evolve the business: after 7 years of creating client websites, we are taking a different direction focus to the business. The plan is as follows: we will be offering "quick to create" CMS websites at a low-end price for the businesses and organizations that don't have $1000 to spend for a website, and require the ability to update their own website (although we have been pushing and emphasizing our "Website Management" service), and focusing more on developing "Software as a Service" (SaaS) applications. These applications include the following:

    • CityFix

    • OmniCard

    • WNC Local (which is actually coming along quite nicely since I am developing a primary framework for it to work off of, which I can use for other projects to help reduce the amount of coding required)

    • ScrumIt

    • ...and a few others


    Let's just say that based on what our developers want, they would like the ability to code, develop, design, test, and innovate in-house applications instead of having to conform to deadlines or produce less-than-quality websites.

  • Based on the next upcoming months, we should be moving out of our current office space. All details aside, things just aren't working in our favor to keep it. The move out should happen within the next 60 days.

  • We did receive 2 new clients during the away-time, but we also lost about 5 clients. One client we gained wanted a full-fledge basic website, while the other wanted some custom work done to their current site. Either way, it's good money. However, we did lose about 5 clients, and (to say what I've told others) I have no specific idea why. So there was some bad to go with the good, but I do believe that this was a sign telling us to get out of the "design client websites" business. Therefore, "message received."

  • (This can be classified as either business or personal, but either way) I signed up for CoFoundersLab.com, which is basically a Match.com for Entrepreneurs. Overall, I like the concept, and within the first few days of signing up, I already had a meeting with a potential business partner in Asheville. (I put on there that "[I am] looking for someone to join [me]" to help bring all of these SaaS services up to speed. We can produce the code if they can manage the business) After the meeting, things went well, but not as well as expected. I was later contacted by many people who were looking for me to join them. When I explained that I already had prototypes of my services and were looking for someone to join me, they backed off. However, I did receive one good contact from Atlanta who (after a lengthy phone conversation) will send me investor connections he has. So, things look good so far.



Personal:

  • I beat Tales of the Abyss (finally). When I last played it, it took me over 70 hours to play fully, and when I reached the final boss, I could never beat him. Using a walkthrough I found (supercheats.com/playstation/walkthroughs/talesoftheabyss-walkthrough01.txt), I was able to level up appropriately and beat the final boss on the first try. I got to see the end of the game, and I'm playing through again with 2x the experience, which should allow me to beat the stronger optional boss at a later time. Yay!

  • Keat and I have started (seriously) began planning for the wedding. I'm saying "seriously" because her parents are finally getting serious. We have pretty much everything planned out, including: the locations, the downpayment schedules, the overall budget, the payment logs, the tuxes, the reception menu, the program design, invitation design, Save the Date designs, groomsmen, bridesmaids, preacher plans, limo scheduled, flowers, etc, etc etc. The only things that we have left are to finalize everything, test/decide on the cake, and order the printing material. Just over a year to go.

  • I was interviewed for the Magnet Program for Wake County Public Schools. I wasn't quite sure what to expect (since I had been through the magnet program since 1st grade and didn't have anything to compare it to), but I went down last Tuesday, went through the interview on Wednesday, and got it over.

  • I finished my Python program through Coursera and received a 90.6% with distinction (I didn't do the first week's quizzes). Not bad for my first online course. I tried to enroll in a "Programming with Design" course, but that seemed to be more of a "how to emphasize design in programming" geared more towards designers with little to no coding, so not really my kind of course. I also tried enrolling in an Algorithm course from Stanford, but I did not have the time during my week to watch 20 hours of videos each week, so I had to unenroll myself out of that course.

  • I've tried to install Ubuntu on my home server and failed miserably. After multiple, multiple attempts at trying to install it, I believe the issue was that my hard drive had officially died. I don't mean "dead as a doornail" kind of dead, but it could not have anything else written to it, so I cannabilized the spare office computer and inserted the hard drive from it into my machine, and I'm using that version of Ubuntu on the old mainframe. Not pretty, but it works.

  • Keat and I have picked walking back up. Based on the past few weeks, we are doing an average of 2.5 miles a night, at a pace of 2.2 mph, which isn't bad. I'm sure I can go a lot faster, but I'm going Keat's speed, which is fine with me. I'll get up to running around the block again....one of these days.

  • I finally saw "Doctor Who: Planet of the Dead". This is the one Tennant movie that I could not get via Netflix or any other online streaming source (aside from BitTorrent), and while I was in Raleigh for my WCPSS interview, it was on BBC America! So I got to watch it! YAY!

  • One of my favorite animes, Sword Art Online, got an American adaptation with dubbing. Now, I know that I'm probably being prejudicial by saying "The American version won't be as good as the Japanese version", but in this case, I can freely admit that. Based on my previous blog posting, I said that I had a fan-crush on Asuna, and I meant that. However, when I heard that Toonami was picking up SAO when ThunderCats ended, I was excited....and scared. When I heard Episode 2 (that's the first one that we get to hear Asuna speak), I will admit that they didn't do anything to capture Asuna's spirit or personality. In addition, the same actress that played the daugter (Natsumi) in Sgt. Frog voices Asuna. I was semi-okay with the voice-over work, until the tail end of Episode 2 when Asuna said "I'm so stupid". At that point, I didn't see Asuna anymore, but the pink haired main female character from Sgt. Frog: Natsumi Hinata (don't get me wrong: Sgt Frog is a great, yet stupid, series). And...that ruined it for me. I still prefer the original Japanese version any day of the week.

  • I got a haircut. Yes, I know. "What a shocker!". I thought "It was time for me to get rid of the ponytail and get with a more realistic haircut....how about a 'David Tennant' inspired style?", and that's pretty much what I was aiming for. Not perfect, but I have a year to get it right before the wedding.
    My haircut

  • Wow, has it really been over 2 months since ConCarolinas? Keat and I attended Con during the first weekend in June and it was really fun. We saw the Team Tardis Cosplayers which did an amazing job cosplaying Doctors #9, #10, and #11, with an awesome real life Dalek, Keat entered into a Costume Contest as Katniss and she should have won (personal arguments against the actual winner aside, and the judges for making the contestants and audience wait 4 hours for a result), we met up with a lot of friends, attended a few parties, got a bunch of stuff, then drove back. I didn't have much of a costume, so I threw together some khaki's and a button up white shirt, trying to be Jamie Hyneman (I can't be Adam Savage), but it turned out more "Han Solo" than "Mythbuster" (see below image). Oh well, better luck next time.
    Wannabe Mythbuster turned Star Wars

That's it for now. I'll try to get back in to posting soon, as soon as I can get back into it.


Tags:#talesoftheabyss #saas #coursera #ps2 #wedding #ubuntu #debian #concarolinas #mythbuster