Backup, Backup, Backup!!!

No, I’m not a police officer, asking for backup.  But, you know what, for any of us who have important or critical data on our computers, it is almost as important.  The sad part is that sometimes as our technology gets familiar and seems trustworthy, it can reach out and grab us by the throat.

This virtually happened to me last fall when I had a computer and a hard drive die on me within a couple weeks.  Now, I didn’t loose anything of real value, because I did have a backup, but I hadn’t run it in a week or two.  So, I did lose a few things of minimal importance.  But, it got me thinking.  I tell my clients that I will keep their photos for years.  This is partially because we had some photos done of our family when our family was younger, but they disappeared during a recent move.  How that happened, I have no clue, but they aren’t to be found.  Now if the photographer had them in his archive, we could be saved from that disappointment.  So, this prompted me to go to significant measures to assure that all of my data is backed up.

Let me talk about the pitfalls of potential backup systems.

1. Don’t rely on manual systems, it is always easier to forget and not do it
2. Hard drives very seldom give users much if any warning that they are about to die. My hard drive failure gave me no warning.
3. SSD drives fail as well, and they always give no warning
4. One extra copy isn’t enough, if your data is critical

Basics of backup

1. Your data isn’t really safe, unless you have three copies of the data
2. One of these copies should be off site, if your office is destroyed by fire, tornado, hurricane, etc., can your business survive?
3. If you do have a major disaster or a simple hard drive failure, how long can you afford to be out of commission while backups are restored?

Previous Computer Setup

I had a MacMini connected to a LaCie 6 TerraByte (dual 3 TB) hard drive connected through Thunderbolt 1 in a RAID 0 configuration.  I also had a 3TB USB3 connected hard drive as my backup.  Some of you may be thinking RAID hard drive therefore very secure.  Well, RAID 0 will lose all of your data when a drive fails.  However, I had become very addicted to the speed of RAID 0.  After a drive in the LaCie TB1 enclosure died, I replaced the drive and restored from my backup.  Some data was lost, but nothing very significant and it took me nearly 2 days to do this.  Well, I survived, but I realized that I was a bit lucky that it wasn’t worse.  So, I evaluated better backup options using the principles described above.

Backup Options

1. Use my RAID 1 hard drive as my main data source and buy an extra USB 3 hard drive to backup my data for off site storage.  This could certainly work and meets the backup requirements, but I was finding the RAID 1 to be slower than I like for many of the operations that I do.
2. Configure the Thunderbolt drive back to RAID 0 and run 2 backups.  It gave me the speed that I desire, but if one of those two RAID drives dies, I still have a couple days of reconfiguring the system again.
3. There is also a RAID 4 or RAID 5 that requires at least 3 drives.  With this configuration, I can get the speed of the RAID 0, but if one drive dies, I can still use the RAID data.  You just simply replace the bad hard drive and the data rebuilds in the background.

My final solution

Click her for PDF Backup Diagram.

I didn’t expect to end up where I did, because it can be argued that it is overkill.  But, I sleep well every night when I think about my valuable business and personal data.  While researching RAID on the Mac, I discovered a really cool program called SoftRaid.  It allows you to configure RAID 0, 1, 4, 5, and 1+0 configurations.  It also allows you to setup a 3 or more disks in a RAID 1 mirror.  Software RAID is much more flexible than hardware based solutions and there is minimal overhead on the computer’s CPU.  RAID 5 has a little more overhead on the connection bus for similar data flows, and is capable of near SSD data flows, so I wanted to keep my main data on the Thunderbolt bus.  Hardware RAID solutions require replacement disks of equal specifications.  If the hardware fails, the disks probably would only work if put into an identical enclosure.

With SoftRaid, when a hard drive fails, you simply put in a new drive of equal or larger capacity and the software automatically rebuilds the RAID data.  This rebuild process happens in the background and you may continue to work on your data.  If an enclosure fails, I can simply move the drives to another enclosure and the RAID will continue to work.  Another advantage of SoftRAID, is that it can warn you when a drive is showing signs of crashing.  It isn’t 100% accurate in it’s predictions, but about 75% of the time it can predict a failure.

For backup software I decided on using software called Carbon Copy Cloner.  It is a wonderful piece of software that can be scheduled to wake up your computer, backup, and then shutdown the computer.  It also does other things, but it met my automated requirements.  Apple has a wonderful piece of software called Time Machine and I know many people who have been saved by it, including me.  I am not a huge fan of it, because of how Lightroom works.  There is a significant backup task that happens every hour that does slow my computer down and it can fill up the backup drive very quickly.

I ended up with a 4 drive RAID 5 configuration for my working data.  Around Christmas, LaCie had a great deal on 6TB dual drive TB2 hard drive that I couldn’t resist.

My backup drive is actually a 3 drive mirror managed by SoftRaid.  Each of the drives is connected through a USB 3 interface.  One of the drives always lives at my off site backup location.  Periodically, every week or so, I disconnect one of the drives from the mirror, take it to the off site location, bring back the drive that was at the off site location and connect it to my computer.  SoftRaid automatically re-synchs this drive with the mirror backup in about 6 minutes.

With this system, I can handle up to 4 hard drives failing at about the same time without significant data loss.  One drive could fail in the RAID 5 array and 1 of the mirror drives at the same time, and my downtime would be limited to the time that it takes to replace the drives.  If 2 or more of the 4 RAID 5 drives fail at the same time, I would have to go through the 1 to 2 days process to rebuild from the backup.

There are many ways to manage your backup system.  But I would always recommend looking at the pitfalls of what you are doing and following the basics of backup which I outlined earlier in this blog.  Believe me, when you think something can’t happen to you, it does with computer data.

 

While we were in vacation, we came across an interesting situation, that a non-photographer probably wouldn’t even notice.

We were in Santa Cruz, California walking near the beach.  The first thing that I noticed when getting out of the car, a photographer with 2 soft boxes and a couple cameras heading toward the ocean.

We then stopped to watch some surfers for a while.  Once we got past the surfers I noticed a photographer taking pictures of a dancer.  She was striking various poses that aren’t normally held in dance, but they make a good photograph.  This photographer was using natural light without flash.

Then as we moved on we watched another photographer that was getting a picture that to me looked like either an engagement picture or a set the date type of picture.  The hard part about this was that they had a small dog with them that they obviously wanted in the picture.  But, every time the photographer crouched to get the proper angle, the dog would go running to the photographer.  This photographer was using an on camera flash, but holding it away from the camera with his free arm.

I apologize for not getting pictures of these photographers, but at the time, it didn’t occur to me that this would turn into a blog post.

The question that many people might ask, which method is better?

Using the soft boxes creates as the name implies, a soft diffused light, with very gradual shadows.  This is the type of lighting that is usually used in a studio or indoors.  It is bulky and this photographer had several people with him carrying equipment.  They were on the beach for a considerable time, but we didn’t see them, so I assume they were below us hidden by the cliffs that we were standing on.  It is difficult or impossible to overpower the sun with soft boxes, so it makes perfect sense that they were taking pictures in the shade of the cliffs.  When we did see them come up from the beach, they started taking pictures in a shady area a ways from the beach.  The pictures that they took on the beach probably resulted in some very nice soft faces with a fairly contrasty background.  The pictures in the shady area probably allowed the photographer to make his subjects considerably better illuminated with a darker but blurred and non-distracting background.

The dancer pictures without flash most likely resulted in a nice environmental portrait showing cliffs, and/or breaking waves and the dancer.  Contrast in the bright, although evening, sun may have been more harsh than some would like.  But, for photographs of sport, gymnastics, and dance can effectively use this higher contrast.

The pictures of the couple and their dog where the photographer was holding a flash at arms length from the camera probably came out nice.  The extra flash probably softened the photo some.  But, if you want to have a picture of a couple walking on the beach, you really don’t have any other option (soft boxes just don’t work in full sun, until almost sunset).  Holding it arms length would have a more softening effect than mounting the flash on camera.

So which is better?  It depends upon what you want for an effect.  The photographer with the soft boxes should have been able to create some nice pictures in the shade of the trees near the beach.  The other two photographers probably created some nice pictures with beautiful ocean and cliffs in the background.  Adding the flash should have softened the couples pictures a bit, which would be a nice improvement.

Customers in this category will have a very broad range of choices and desires.  But, remember they are interested in controlling most every part of photography.  This includes Field Of View (FOV) and Depth of Field (DOF).

Cameras with 3 different sensor sizes fall into this discussion.  1″, 4/3″, and APS-C.  There are interchangeable lens cameras and fixed lens cameras available in all three categories.  I put the 1″ sensor in this category, because you can start to do a good job in controlling depth of field with this sensor and low light shooting starts to improve significantly.  But you will also see that costs start to escalate significantly in this category.

Nikon was the first company to attack this segment with their One series.  Just to put a line in the sand, the camera sell for about $900 today.  Lots of good features, but my problem with this camera is the f4.0 lens.  Since it is a smaller camera, I would have liked to see a f2.8 lens.  For an ILC, switching lenses and other hassles of this type of camera, there needs to be more reward.  Faster Prime lenses can be purchased.  Lenses run about $150 to $900 for this camera.

Recently Nikon added fixed lens cameras to their 1″ or CX line.  Actually these are called DL cameras for $650 to $900 depending on the lens attached.  These cameras begin with a f-stop of 1.8 and the longest zoom has wide angle f-stop of 2.8.  The Camera with the longest zoom 24-500mm has an f-stop range of 2.8 to 5.6.  The least expensive camera has a 24-85mm zoom and an f-stop range of 1.8 to 2.8 for $650.  The mid-range has a 18-50mm zoom and the same 1.8 to 2.8 f-stop range for $850 (wide angles are generally more expensive).

While Sony wasn’t the first to market with a 1″ sensor, they do make all the 1″ sensors for Nikon and of course for themselves.  But, Sony, is about the only camera maker that is seeing profits go up, and I think to a large degree it is because of cameras in this category.  Sony was the first to offer the 1″ sensor in a fixed lens camera.  They offer 7 different cameras in this segment.  Prices range from $500 to $1,500 for these cameras.  The $500 model, RX-100, doesn’t have the BSI, but it has a good 28-100mm lens which is very functional and a maximum f-stop of 1.8.  The most expensive (RX10-III) has the BSI sensor, 24-600mm lens, and a 2.4 to 4.0 f-stop range on the zoom.  Not to mention 4k video with 960 fps video at 4k for slow motion effects.  While these cameras are expensive, they are loaded with features that make them very capable.

These cameras will provide much better low light capability than cell phones and much better zoom performance than the 1/2.3″ sensors.  Plus, what they can do with video makes them very attractive.

Then there are the micro 4/3rds format.  This is a larger sensor, and I think all cameras made in this format are ILCs.  Panasonic and Olympus sell this format.  There is a large installed base and lots of lenses that can be purchased to custom fit your needs.  If a person wants an ILC system, this sensor size is the smallest that I would recommend.  Bodies without lenses start at about $400 and go up to about $1300.  Lenses cost about $150 to $2500, although the vast majority are $200 to $600.  But, it is a huge selection which is nice.

The next sensor size is the APS-C.  About every major camera maker is in this category.  Although there is a divide between Mirrorless and DSLR.  I won’t touch that controversy today, but I do have my opinion.  But, ultimately they are capable of producing the same quality pictures.  Body only packages begin at around $300 and basic lens kit packages start at around $400.  A high end body will be around $1600. Lenses start at around $150 and go up to a little over $2000.  So due to the fact they are a little larger they are a bit more expensive than the 4/3rds lenses.  These camera lens combinations can provide excellent results in a variety of situations.  They will generally provide better low light capability than their 4/3rds brethren.

Remember I talked about some advantages to being small to level the playing field.  One of them is that the 1″ cameras are capable to doing faster frame bursts than the APS-C cameras.  They also might be able to provide better FPS in the video mode as well, which will result in better slow motion effects.  Basically, since the sensors are smaller and usually a few less pixels, they can read the data off the sensor faster, and move on to the next frame or picture quicker.

As you can see there is a big range with what you can do here and spend.  $500 can get you into a good system.  If, being small and taking very good pictures in a variety of situations, the 1″ sensor with a fixed lens is the way to go.  If you want more flexibility and don’t mind carrying around a little more weight, the MFT and APS-C are good choices.