Backup, Backup, Backup!!!

No, I’m not a police officer, asking for backup.  But, you know what, for any of us who have important or critical data on our computers, it is almost as important.  The sad part is that sometimes as our technology gets familiar and seems trustworthy, it can reach out and grab us by the throat.

This virtually happened to me last fall when I had a computer and a hard drive die on me within a couple weeks.  Now, I didn’t loose anything of real value, because I did have a backup, but I hadn’t run it in a week or two.  So, I did lose a few things of minimal importance.  But, it got me thinking.  I tell my clients that I will keep their photos for years.  This is partially because we had some photos done of our family when our family was younger, but they disappeared during a recent move.  How that happened, I have no clue, but they aren’t to be found.  Now if the photographer had them in his archive, we could be saved from that disappointment.  So, this prompted me to go to significant measures to assure that all of my data is backed up.

Let me talk about the pitfalls of potential backup systems.

  1. Don’t rely on manual systems, it is always easier to forget and not do it
  2. Hard drives very seldom give users much if any warning that they are about to die. My hard drive failure gave me no warning.
  3. SSD drives fail as well, and they always give no warning
  4. One extra copy isn’t enough, if your data is critical

Basics of backup

  1. Your data isn’t really safe, unless you have three copies of the data
  2. One of these copies should be off site, if your office is destroyed by fire, tornado, hurricane, etc., can your business survive?
  3. If you do have a major disaster or a simple hard drive failure, how long can you afford to be out of commission while backups are restored?

Previous Computer Setup

I had a MacMini connected to a LaCie 6 TerraByte (dual 3 TB) hard drive connected through Thunderbolt 1 in a RAID 0 configuration.  I also had a 3TB USB3 connected hard drive as my backup.  Some of you may be thinking RAID hard drive therefore very secure.  Well, RAID 0 will lose all of your data when a drive fails.  However, I had become very addicted to the speed of RAID 0.  After a drive in the LaCie TB1 enclosure died, I replaced the drive and restored from my backup.  Some data was lost, but nothing very significant and it took me nearly 2 days to do this.  Well, I survived, but I realized that I was a bit lucky that it wasn’t worse.  So, I evaluated better backup options using the principles described above.

Backup Options

  1. Use my RAID 1 hard drive as my main data source and buy an extra USB 3 hard drive to backup my data for off site storage.  This could certainly work and meets the backup requirements, but I was finding the RAID 1 to be slower than I like for many of the operations that I do.
  2. Configure the Thunderbolt drive back to RAID 0 and run 2 backups.  It gave me the speed that I desire, but if one of those two RAID drives dies, I still have a couple days of reconfiguring the system again.
  3. There is also a RAID 4 or RAID 5 that requires at least 3 drives.  With this configuration, I can get the speed of the RAID 0, but if one drive dies, I can still use the RAID data.  You just simply replace the bad hard drive and the data rebuilds in the background.

My final solution

Click her for PDF Backup Diagram.

I didn’t expect to end up where I did, because it can be argued that it is overkill.  But, I sleep well every night when I think about my valuable business and personal data.  While researching RAID on the Mac, I discovered a really cool program called SoftRaid.  It allows you to configure RAID 0, 1, 4, 5, and 1+0 configurations.  It also allows you to setup a 3 or more disks in a RAID 1 mirror.  Software RAID is much more flexible than hardware based solutions and there is minimal overhead on the computer’s CPU.  RAID 5 has a little more overhead on the connection bus for similar data flows, and is capable of near SSD data flows, so I wanted to keep my main data on the Thunderbolt bus.  Hardware RAID solutions require replacement disks of equal specifications.  If the hardware fails, the disks probably would only work if put into an identical enclosure.

With SoftRaid, when a hard drive fails, you simply put in a new drive of equal or larger capacity and the software automatically rebuilds the RAID data.  This rebuild process happens in the background and you may continue to work on your data.  If an enclosure fails, I can simply move the drives to another enclosure and the RAID will continue to work.  Another advantage of SoftRAID, is that it can warn you when a drive is showing signs of crashing.  It isn’t 100% accurate in it’s predictions, but about 75% of the time it can predict a failure.

LaCie Thunderbolt 2 solution on the left and older Thunderbolt 1 solution on the right.  This provides me data throughput of up to 600MB/sec.

For backup software I decided on using software called Carbon Copy Cloner.  It is a wonderful piece of software that can be scheduled to wake up your computer, backup, and then shutdown the computer.  It also does other things, but it met my automated requirements.  Apple has a wonderful piece of software called Time Machine and I know many people who have been saved by it, including me.  I am not a huge fan of it, because of how Lightroom works.  There is a significant backup task that happens every hour that does slow my computer down and it can fill up the backup drive very quickly.

I ended up with a 4 drive RAID 5 configuration for my working data.  Around Christmas, LaCie had a great deal on 6TB dual drive TB2 hard drive that I couldn’t resist.

3 single USB3 drives are mirrored in a RAID 0 configuration.  One drive is always off site.

My backup drive is actually a 3 drive mirror managed by SoftRaid.  Each of the drives is connected through a USB 3 interface.  One of the drives always lives at my off site backup location.  Periodically, every week or so, I disconnect one of the drives from the mirror, take it to the off site location, bring back the drive that was at the off site location and connect it to my computer.  SoftRaid automatically re-synchs this drive with the mirror backup in about 6 minutes.

With this system, I can handle up to 4 hard drives failing at about the same time without significant data loss.  One drive could fail in the RAID 5 array and 1 of the mirror drives at the same time, and my downtime would be limited to the time that it takes to replace the drives.  If 2 or more of the 4 RAID 5 drives fail at the same time, I would have to go through the 1 to 2 days process to rebuild from the backup.

There are many ways to manage your backup system.  But I would always recommend looking at the pitfalls of what you are doing and following the basics of backup which I outlined earlier in this blog.  Believe me, when you think something can’t happen to you, it does with computer data.