Author - Web Developer - Educator
Found 3 results for tag "rsync"
RSS Feed

Farewell to BlueIce

It has taken me a while to get over this, but I am officially saying "Goodbye" to a faithful, but frustrating, partner for the past (at least) 10 years.

My BlueIce server, which has been a desktop/server, then a desktop, then a dedicated web server for 3 years, then network storage, then a dedicated web server for another 3 years, has finally reached its end.

As of October 1st, it was the primary hosting server for all of my Q5 Industries services, along with my primary email server, NAS (before I got a NAS), and database server. Then at 7:28PM, it slowed to a crawl, then became unresponsive, then would not restart into an OS. Recovery efforts proved futile. (Good thing I learned my lesson and had rsync'ed a backup of the files and databases, but missed out on the last 2 days of work, which wasn't too bad to recreate.)

The hard drive lasted me a very good long time, as you can see on its manufacturing date: 13 Dec 2001. Almost 14 years it has given its all, including being formatted again and again and again for various purposes.

It is nearly impossible to find an ATA hard drive nowadays, and even so, upgrading the processor and RAM for this thing will cost an arm and a leg. The hardware has lasted a good ~10 years since I ordered it from Newegg back in 2005. (It still works, but modern hardware has just moved on too far)

I've had other servers that I've networked with, including some that were to replace my BlueIce server, but BlueIce kept going and going - like the Energizer bunny, and it always came out on top over the others.

So it is with a heavy heart that I bid it farewell. After putting up with me and my frustrating command line errors, along with creating a calming "hum" throughout the house (it's eerily quiet without it), it has finally called it quits.

You will be missed, BlueIce. I hope that you find harmony among your code.
The 120GB Hard drive that lasted 14 years
The 120GB Hard Drive last lasted 14 years
The inside of BlueIce - the hardware that was faithful
The inside of BlueIce - the hardware that was faithful
The reason for the name: the blue eye - also for its nickname: Pluto
The reason for the name: the blue ie - also for its nickname: Pluto

The BlueIce server it in entirety


Tags:#blueice #farewell #rsync

RSS Feed

Pre Xmas Status

It's been a busy end-of-November, and the holiday chaos is just beginning.

The good news: I finally got to work on the Pi some. After reconnecting my Pi and finding out that it had been dormant for about 7 months, I decided to kick things back into gear. First, the server setup:
Server Setup

With the Pi set up in the window, I set up the Cron system to automatically delete all files older than 20 minutes and automatically RSYNC over to the main server for remote management. Having all of the images on the Pi over the course of a day creates over 90,000 files and fills up the tiny 8GB device within hours. So I had to find another method. Luckily, I already knew how to automatically back up files over RSYNC with Password-less SSH sessions. Yay.

Cron commands:
*/2 * * * * root find /var/www/motion/* -mmin +30 -exec rm -f {} \;
 
*/5 * * * * pi /home/pi/rsync.sh
See my rsync script at my other post

Other good news: Night Blade has been published! Yes, we are out of the Proofing stage and officially selling copies!

Luckily, I got my Proof copy right before we left for Thanksgiving, and I spent the majority of Thanksgiving break proofing the edition (all of 6 days). After I got home, I made the modifications to the manuscript, reformatted it, and re-uploaded it to CreateSpace, Smashwords, and Leanpub. Personally, I'm proud of the fact that I wrote a book. Now on to #2: Day of Reckoning!

Back to Thanksgiving for a bit. Keat and I did the turkey, which turned out well, as expected. A little drama happened over the break that I really am glad that I wasn't a part of, although I wish I was in order to defend Keat a bit. As expected, as usual, we went on a hike. This time to Stone Mountain.
Keat on Stone Mountain

Overall, it was "fun" with the expectation of someone questioning your every decision. Ah, the holidays.

Also, since Shadow Dev is going through a bit of a revamp (and I spent some time writing instead of coding), I've been looking for some jobs with very little success. My requirements are as follows: non-sales job and non-fast food. Aside from that, I'm pretty much open to anything, but 90% of the jobs are "Part Time Sales", if you know what I mean.
Our Generation on Jobs
(I did not say this - it is a screenshot from Failbook)

I mean, I don't mean to be picky, and I have very good reasons to why I said what I did. As per my previous experience (and what many others have told me), I suck at selling. I'll say it again: I suck at selling. I'll admit it. Anyone who hires me to sell will never get their ROI back. They'll pay me to stand there and do many other things (stock, checkout, manage, etc), but not to sell. I'm a "people pleaser", and if I know of something that you can do to save you money and not hand it over in an "upsell" tactic, I will let you know. Although I'm thinking I need to move away from the "people pleaser" state as it seems to be doing more harm than good for me.

Anyway, here's what has been happening so far:
- Haywood Community College: I applied for the Small Business Director position. Personally, I thought I was a great fit. I met all of their qualifications (including a Degree in Entrepreneurship, which I have), so I thought I was a shoe-in. I sent in my application, my research paper, and my transcript copy to them in September as the due date for the application was October 30th. After a month, I didn't hear back, but it was still before the due date, so I sent in a message via USPS contacting them about the status of my application.

A few days before the due date, I was contacted by the HR Department saying "We received your letter, but not your application." Confused about the situation, I hand delivered in the other copy of the application, my research paper, and another transcript request copy to them an hour afterwards.

Not hearing anything back, I emailed the HR person about a followup. They said that they never received my transcript, and I had to go on a wild goose chase to find out where it went. Western said that they sent out both requests that I sent in, with the matching dates and addresses as well, so the problem was with HCC. Apparently, the problem was that my transcripts (and probably my first application) went to Student Services, not HR. I called Student Services and asked them to deliver my transcript to HR (across the hall).

Afterwards, I contacted the HR person again and asked for a status update. She said that the committee was making their decisions and that interviews were to be held on December 5th and 6th. Well, here it is, December 9th, and no callback. Which means, I didn't even get considered.

- Thinking that the HCC job was a deadend (and looking for backups), I applied to Target on December 3rd for "Backroom Stock" (not Sales and not Fast Food - perfect!). The application process wasn't that hard, but I have yet to hear back.

- I also applied to Biltmore a while back (October), but was turned down for 3 different positions: IT Staff, Security Guard, and Dishwasher. Yes, Dishwasher. I was passed up for a Dishwasher. Their message:
Thank you for your recent application for our Dishwasher position 
 
with The Biltmore Company.  We regret to inform you that we have filled this 
 
position. After considering each application carefully, our hiring team felt 
 
that there were other candidates that more closely matched the specific needs
 
 of the position.
Seriously? WTF?

- Looking elsewhere, I also applied to Regal Grand Theaters hoping to get some low-line cleanup crew position (hey, it's money). After visiting the theater with Keat at the Catching Fire premiere, the whole building said that they were hiring. So, I applied on December 3rd and have yet to hear back (I was hoping my LMP experience would mean something. I guess not)

If "overqualification" is a thing (it was when I applied to Best Buy back in 2004), then maybe I should do this:
Overqualification solved


Onto a different topic, I visited the NCCAT center last night to celebrate the "Hour of Code" (an initiative program to increase programming skills in schools) and talked to Dr. Wade with the NCCAT. After briefly meeting with him, I think that we may be on our way to start our own CoderDojo in the WNC area (the closest one is in Missouri). I also got an "Hour of Java" which was interesting. I contacted Dr. Andrew Scott for more information. Although, this has resparked my love for programming.

Meanwhile, Keat is working with a few "clients" on a Consulting business. Knowing that, we had a discussion and may be opening a "Web Technologies Consulting Business". Since "Web Design" is such a flooded market, and there aren't really any upsides to DIY Websites (and people want the moon, but only want to pay $1), we explored WordPress (yes, I know - stop badgering me) as a viable option. I installed a test environment and it does seem quite easy to use, so we may be able to offer that as a "Small Website Option" and make the clients manage it themselves. Self-managed websites with client decisions and actions, without being yelled at for things out of our control or expectations that we can never meet? I think it's definitely a considerable option, but not just for the "Web Design" factor - a consultant should be referenced.

And just because my week had been going like a wild rollercoaster, I saw/heard this, and it really helped me along:

And a reference:
Tso - get it?

And with that, I probably won't post again until after Christmas, as the holiday chaos is catching up and my sleep pattern is completely messed up.


Tags:#thanksgiving #nightblade #rsync #cron #stonemountain #jobs #consulting #target #haywood #regal #bc #nccat #coderdojo #hourofcode #picard #startrek #tng

RSS Feed

Backups and Sync Success!

Backups are important

After many, many months (and many mistakes), we finally have our own backup server!

Let me explain: when we had our data hosted on 3rd party servers, backups were never considered because the services already provided backups (or, so we were told). We only had to use WebsiteSource's backup service once, and the other company said they were creating a backup service, but never put us on it.

When we moved our data to the primary local server back in September, my level of worrying gradually increased over time. We had all of our site data on 1 server - and if that server were (for some reason) to crap out, we'd be in a very large hole with no way out. 8 years of coding. 8 years of projects. 8 years of client work. 8 years of a full portfolio. 8 years of database records. All gone.

Luckily, that hasn't happened yet (and we hope it never will - oh, the horrors if it did). So, in order to fill in this gap, I finally got the courage to make a backup server! (finally, right?)

I was able to salvage a bunch of the spare parts that I had laying around. Some RAM, some hard drives, a few semi-barebones desktop frames, motherboards galore...and all of this stuff was just collecting dust in various areas.

I went through a bunch of combinations of RAM/Hard Drives/Motherboards/CD Drives before I found a combo that worked (a few of the frames that I had required SATA drives, and I only have ATA/IDE drives available).

I finally got some RAM that fit appropriately, a large enough hard drive to hold the information that I needed (not the same size as the main server drive, but big enough), and a CD drive that worked (I had to install the Server OS somehow, right?).

After spending all of today getting the latest Ubuntu Server ISO downloaded and burned, I popped in the CD, ran the install ISO, and had a dedicated backup server ready to go.

I then spent the latter part of the day setting up rsync, cron, and mysqldump to automate the creation and mirroring of backups.

MySQL Dump Script:
#!/bin/bash
 
BACKUP_DATE=$(date +'%F')
 
mysqldump -u [username] -p[password] --all-databases --events > \
 
   /var/www/vhosts/$BACKUP_DATE.sql
--events is here because it was returning an error saying "Skipping mysql.events table"

RSYNC Script:
#!/bin/bash
 
N=$(date +'%F %T')
 
LOG_FILE="$HOME/rsync.log"
 
echo "RSYNC Begin: $N" >> $LOG_FILE
 
rsync -r -a -v -e "ssh -l [username]" --delete \
 
   [source directory from root] [remote_server_ip]:[remote directory from root]  >> $LOG_FILE
 
echo "---------------------------------" >> $LOG_FILE
--delete is to delete any remote files that aren't on the source location. Also, if you want to only sync certain file types, add "--exclude='*' --include='*.[ext]'" after --delete (exclude everything except what is included), or you can just omit --delete altogether if you want. The rest of the script is just log information - completely optional, but makes the log easier to read

Update 2013-12-09: I've updated the script a bit to make it a little more universal after some things were called to my attention

The main part of that was spent trying to get the backup server (which I am calling "Fry" - I'm thinking of going Futurama for a server nomenclature) to automatically SSH into the main server without a password (aka: storing an SSH key). I had to learn a lot in order to do that, but I found out that the problem that eluded me for about 5 hours was that the main server's home directory had the wrong permission settings for the .ssh folder.

Next time the SSH keys aren't being stored right, run [sudo] tail -f /var/log/auth.log and see what the error is. Lesson learned.

So, the overall point of this story:
  • A dedicated backup server, Fry, has been salvaged, installed, and setup
  • There is an rsync script on Fry to automatically sync up the files from the main server (all 40GB+ worth)
  • The Main Server has an automatic cron script to run mysqldump to make a full backup of all of the MySQL database
  • Fry's rsync will keep the latest 6 (daily) copies of the MySQL Dump backups along with the main backup
  • Fry's rsync script creates a custom log to record the output


Now to install a Leela (why not Zoidberg?) server for load balancing (or a production server. I don't know - I've never had more than 1 server running locally at a time).....

....and just because this whole "end of the year" thing is so chaotic, here's a comic I liked which I hope to do (just to get it all over with):
BC


Good Night!


Tags:#bc #xkcd #backup #ubuntu #futurama #rsync #sudo #zoidberg #bash #tutorials