My Drobo Story: How Our Photos Survived Three Hard Drive Failures


Hard drive failure. Three words that strike fear into the heart of every photographer — the thought of losing every photo you’ve ever taken is quite sobering indeed. Over the last four months we’ve suffered a total of three failed hard drives — including two at the exact same time — yet all our photos and data is safe, thanks in big part due to Drobo technology.

Four years ago, we moved away from a tangle of external hard disk drives, and consolidated onto a 5-disk Drobo network-attached storage device. (Current models go for about $499, available at B&H.) There’s something very convenient about having all of your images accessible on a single large disk volume — no more searching for the right external hard drive to power and connect; all your images are available, indexed, and instantly available in Lightroom (or your image management software of choice). We lovingly named our device “Toasty” as it quite closely resembled an oversized five-slice toaster.

Drobo uses a custom “BeyondRaid” technology to help protect your data. In traditional RAID-5 setups, one of the disks is a ‘parity’ disk used to preserve your files even if one of your other hard drives has failed. BeyondRaid also allows seamless upgrading of your storage capacity — we really liked the idea of being able to replace our 1.5TB disks with larger drives as our storage requirements grew, and still seamlessly keep everything as a single large, logical partition. BeyondRaid also allows you to seamlessly set a “double redundancy” feature to tolerate up to two disks failing at the same time. That choice made the most sense for us as we weren’t needing the full 6TB storage array at the time.

And boy, are we ever glad we chose to enable double redundancy. Read on for our story.

First Drive Failure

mmm, Toasty.

mmm, Toasty.

In December of 2013 we experienced our first hard drive failure. “Toasty” (perhaps not the best name in retrospect) reported issues with one of the 1.5TB disks in the array, then went into “data protection mode” where it moved blocks around to ensure my photos were still protected in case another disk failed. After 20 tense hours of “relayout” it completed successfully. I pulled the drive out, sent it back to the drive manufacturer (under warranty), and a few days later, re-inserted the refurbished replacement drive. No problems, no downtime!

Second and Third Drive Failures

We were literally on the other side of the world when the next drive failure happened in April. While we were traveling, Toasty e-mailed us a critical alert to let us know that one of the drives had failed — it happened to be the refurbished replacement drive — and relayout was now underway. No problem. But less than 12 hours later, during this intensive relayout process, our device e-mailed again to let us know a second drive failed. Only three working disks remained, and we worried about the safety of our images and files. We returned home to find all was safe — even though we had two drive failures, no data or precious photos were lost, thanks to double redundancy. Our little Drobo had successfully defended us against three hard drive failures out of six in the course of a few months. (Apparently, only 80% of hard drives make it successfully to their fifth birthday, so having three out of six drives fail within five years is not as rare as you’d think: about 1 in 183.)

Oops and DroboCare

drobocare_240x368_a5756c2d-9c7f-4202-adce-aa087b62b468 Before inserting new hard drives I made the unwise decision of trying to upgrade the never-updated four-year old firmware on our Drobo — jumping four years of updates in one go. After the upgrade, our device failed during boot up (all red lights), and my files were inaccessible. Oops. The one downside of Drobo is their system is quite “closed” — only Drobo engineers have access to the debug info and diagnostics of your system. Unlike conventional RAID systems where you can reassemble your array yourself if the original hardware fails, with Drobo you must deal with the manufacturer, Data Robotics. Problem was — my one year warranty was long expired, and I hadn’t purchased the DroboCare extended warranty. My product was so old it wasn’t even featured on their website anymore. What to do?

I called their support line on a Sunday evening — and live humans actually picked up. Impressive. Even though I was well out of warranty, Drobo was able to extend a one-time courtesy to help resolve my issue — over the next week, I was led through a few diagnostics and a reflash of the firmware. Toasty was up and running again. Drobo support staff were responsive, courteous and efficient, and overall a pleasant experience. I’m normally not one for extended warranties, but all things considered I’d buy a DroboCare extended warranty with any new Drobo given what I know now about the close-endedness of their systems and the quality of the support that they provide.

The Happy Ending

We’ve now replaced all the failed drives with new WD 3TB Red drives. Apparently it’s not just us — the 1.5TB drives have proven to be unreliable. With the new drives we also get a nice capacity bump. As more 1.5TB drives drop out we’ll slowly replace them with larger drives.

Note, a RAID or BeyondRaid system doesn’t completely remove the need for backups. You can still be affected by physical damage (fire, flood) to the entire unit, or file deletion (either accidental or malicious/hackers). In our case we still had a offsite backup of our Drobo but it was several months out of date, and we would have permanently lost a few months of photos. I’m glad we didn’t lose anything in the end, but after this close call I’ll be sure to keep our offsite backup more up-to-date.

All in all, with this experience, I am now a believer. Fancy redundant disk arrays aren’t just for the paranoid, but are now essential tools for every photographer, especially with hard drive reliability being what it is these days. Drobo very elegantly simplifies this technology and makes it much easier to use, and when it works seamlessly it is quite a marvel. We are now proud Drobo supporters, and now watch prices on Drobo products on the accessories page of our site.

And please, don’t forget to enable double redundancy!



23 comments on “My Drobo Story: How Our Photos Survived Three Hard Drive Failures

  1. tcw
    May 7, 2014 at 5:59 pm EDT

    Hopefully they’ve gotten better since this post; they picked up quite a bit of notoriety after failing a famous photographer:

    http://scottkelby.com/2012/im-done-with-drobo/

    1. CanonPriceWatch
      May 7, 2014 at 6:08 pm EDT

      Yes. I was fearing that as well, and was very pleasantly surprised when they provided a one-time free support incident for my four-year old Drobo which was out of warranty for three whole years.

      It sounds like their new commitment to supporting customers now is showing.

  2. Tdragone
    May 7, 2014 at 9:59 pm EDT

    I have had 2 interactions with Drobo Support; and both have been extremely positive; but both of my questions were preventative. I have experienced two separate drive failures since I purchased my original drobo 4 bay enclosure, and am saving for the 5 bay. I support them 100%; but I DO have 2 separate offsite backups of all my data (Mostly raw files) since a fire would wipe out the drobo and everything on it.

  3. aaron
    May 7, 2014 at 11:32 pm EDT

    I have no financial interest in Carbonite except that I’ve been a happy customer. Offsite enterprise level backup with access from anywhere in the world beats Drobo hands down. This story just makes me happier I’ve never messed with NAS boxes.

    1. CanonPriceWatch
      May 8, 2014 at 9:25 am EDT

      We also looked into Carbonite and other online backup tools (such as Amazon Glacier). My only concern with them is seeding the initial backup is difficult when you have many terabytes of data; I notice a few of them have an option where you can mail in hard drives that they’ll copy.

      To upload 10 TB of seed data from would conservatively take 4 months on a 10 Mbit upload connection (1 MB/s). Every time you go out and fill a 32GB card that’d be a solid 8 hours of uploading. And that’s assuming 10 Mbit up, which is probably not the norm.

      I’m paranoid enough to backup my CF cards in the field (onto my laptop) and I don’t wipe the cards until my photos are backed up again somewhere (i.e. always ensuring two copies).

  4. May 8, 2014 at 9:08 am EDT

    The problem with carbonite is that it oy backs up what is on your hard drive so unless you have a 10gb hard drive you’re out of luck.

  5. May 8, 2014 at 9:25 am EDT

    I have all my photos on a simple RAID mirror in my PC, separate from my main (SSD) drive.
    I also export my RAW photos to relatively high resolution JPEGs and have those sent to an offsite backup. That way, in the case of a catastrophic 2-drive failure, at least I have something.
    Mind you, these are all personal/hobby photos and not my profession, so I wouldn’t be financially affected if something did go wrong.

  6. May 8, 2014 at 12:22 pm EDT

    SK’s experiences put me off the Drobo in a big way. I’m now in the process of installing FreeNAS for local backup (roll your own or commercial solutions available), and have started testing Blackblaze ($5/mo unlimited) for offsite backup until I can install an offsite FreeNAS.

  7. John Hellyer
    May 8, 2014 at 4:05 pm EDT

    A little known feature in Windows 8 (along with Server 2012) is Storage Pools and Storage Spaces, which allows you to “pool” multiple drives together and then partition out “spaces” to store files. My personal server is running 12 hard drives, ranging in size from 750GB to 3TB. Out of the 23TB pool, I’ve partitioned off a triple redundancy space for all my pictures. All files are stored “as is” on the drive and the system ensures that there a copies on 3 different physical drives. This is different than a RAID which stores parity information on different drives. The advantage here is that these are still NTFS drives; I can pull a single drive and put it into another machine and access all the files if some truly drastic crashes the server.

    Also, I found Carbonite terrible. I cancelled my subscription after a year when I almost lost data b/c the “backup” hadn’t finished yet (the upload speed REALLY slows down after you hit a certain data threshold). Since then I’ve been using Crashplan (Code 42 Software) and couldn’t be happier.

    1. CanonPriceWatch
      May 8, 2014 at 4:06 pm EDT

      That’s useful John. Thanks for sharing that.

      1. John Hellyer
        May 8, 2014 at 4:43 pm EDT

        And BTW – thank you for your site! You saved me a bunch of money on a 5D couple weeks back. Now my RAW photos will be that much larger. 🙂

    2. May 8, 2014 at 6:36 pm EDT

      Does that still use NTFS though? If so, then the drive BER almost guarantees uncaught errors with current disk sizes. That’s why I’m moving to zfs. I believe MS have a new filesystem in preparation as well for this same reason.

      1. John Hellyer
        May 9, 2014 at 10:54 am EDT

        I take it back. It’s not NTFS – or at least not standard NTFS – although I’m not exactly sure how the drive is formatted at the physical level. What I said is true though, you can take an individual drive out of the pool and put it into any machine running windows 8 (or Server 2012) or higher and be able to read the data on that disk. Since it has to be Win8 or higher there’s definitely something different at the file level.

        1. May 9, 2014 at 12:29 pm EDT

          Looks like SS can use ReFS, MS’s long-anticipated and still-not-quite-there-yet replacement for NTFS, as long as you configure it a certain way. Everything I read about ReFS though says that even when its complete its not going to be as robust as ZFS. But maybe it will be better than a Drobo.

        2. John Hellyer
          May 9, 2014 at 12:36 pm EDT

          Personally I like SS better than Drobo just so I’m not locked into any specific hardware. I have a friend who runs a similar solution to SS on Linux (don’t remember what it’s called though).
          The real lesson is to have redundancy, at multiple levels, especially on something as important as pictures. How you get there, and how much you spend, all depends on your budget and your comfort level.

  8. May 9, 2014 at 12:46 pm EDT

    Agree with that.

    I prefer commercial solution whenever I can but I’m happy to roll my own when it makes sense. For those who don’t I’d recommend looking at these boxes

    http://www.ixsystems.com/storage/freenas/

    before considering a Drobo.

  9. J packer
    May 10, 2014 at 3:00 am EDT

    +1 on Crashplan. I also run a “mirror” RAID with two 3TB drives for data and have the OS and software on an SSD. Crashplan is always running in the background. Cloud backup is slow and I could theoretically lose whatever is not yet uploaded should my entire PC sizzle and burn. I guess I could do an external backup and keep those drives offsite, as well but I know I am not obsessive enough to do so.

    And, I also want to thank CPW for saving me some serious $ on some purchases.

Leave a Reply