Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | General Gear-talk | Join Upload & Sell

  

Archive 2016 · Advice on image backup procedure

  
 
mitesh
Offline
• • • • •
Upload & Sell: On
p.1 #1 · p.1 #1 · Advice on image backup procedure


As part of my plan to develop a strategy for managing my growing image library, I have read several past threads here on FM as well as articles online related to image backup and data recoverability. Using what I've learned, I have formulated a procedure that I believe will achieve my goals of protecting against drive failures and accidental deletions, and offering basic versioning while keeping files synched across storage devices. I would appreciate input from members on anything that I may have failed to consider, or should do differently.


Step 1: Shoot to dual cards.

Step 2: Copy from cards to external drive and second copy to DAS (Drobo 5D with dual disk redundancy).

Step 3: Import images to LR, cull, and add keywords (edits saved as XML sidecar files). LR library and one copy of images are on the external drive.

Step 4: Use Carbon Copy Cloner to sync external drive and Drobo 5D (DAS). "SafetyNet" feature on destination to preserve older versions of modified files and files deleted from source.

Step 5: Use Carbon Copy Cloner to sync Drobo 5D with NAS (Drobo 5N with dual disk redundancy). Unmount NAS when finished.

Step 6: Take Drobo 5D offline, unplug from power.

Step 7: Upload only selected RAW files and sidecar files to cloud storage.


Might sound like a lot, but steps 4 through 7 are automated, so once I've culled and keyworded images, Carbon Copy Cloner does the bulk of the work without me. I also have a simple JBOD enclosure that I can sync and keep at a different location.


Anything I have overlooked, or potential pitfalls that I have missed? Thanks in advance!




Oct 30, 2016 at 10:22 AM
dgdg
Offline
• • • • •
Upload & Sell: Off
p.1 #2 · p.1 #2 · Advice on image backup procedure


Getting ready for Haines, huh?
Some on FM have not been a fan of the Drobo. I've read that it can be slow and quirky. Maybe that has changed. I simply use JBOD via thunderbolt/usb3, or the Western Digital My Book Duos.
Your reason for NAS is to easily upload to a cloud?
You don't have a single massive database or anything, but can the Carbon Copy Cloner periodically check the backups for data integrity (ex md5 sum)?
Can Carbon Copy Cloner email you reports with backup success and failures?
Can you automatically sync to your offsite drive? Manually offsite backs do not happen as often as we initially hope.
You back up your entire Mac with Carbon Copy Cloner too?
Does the A7R2 have dual slots?



Oct 30, 2016 at 02:44 PM
mitesh
Offline
• • • • •
Upload & Sell: On
p.1 #3 · p.1 #3 · Advice on image backup procedure


dgdg wrote:
Getting ready for Haines, huh?
Some on FM have not been a fan of the Drobo. I've read that it can be slow and quirky. Maybe that has changed. I simply use JBOD via thunderbolt/usb3, or the Western Digital My Book Duos.
Your reason for NAS is to easily upload to a cloud?
You don't have a single massive database or anything, but can the Carbon Copy Cloner periodically check the backups for data integrity (ex md5 sum)?
Can Carbon Copy Cloner email you reports with backup success and failures?
Can you automatically sync to your offsite drive? Manually offsite backs do not happen
...Show more

Hey David,

I've had the Drobo 5N for about 3.5 years now, and it's been flawless for me. I know some people don't care for them, but it seems to be a good solution for those of us who can barely comprehend the simple solutions . I use the NAS so that as a family, we can easily access the media on it from our different computers. In addition to images, we have our family videos on it as well.

Checking data integrity was the one thing I'm not too sure about. I don't believe CCC does that. It can, however, automatically synch to an offsite drive and also send email alerts. I have a bootable clone of my laptop's internal drive that was made using CCC, so yes, it can backup the Mac.

Wish the a7rii had dual slots, but luckily the 1DX does .



Oct 30, 2016 at 04:43 PM
leethecam
Offline
• • •
Upload & Sell: Off
p.1 #4 · p.1 #4 · Advice on image backup procedure


I ditched my Drobos when I discovered how slow the rebuild times are. Days for my system...!

I changed to G-Tech G-Speed RAID drives which can rebuild very fast.

I tend to transfer card data initally to drives with Shotput Pro. This software verifies the transfer with accuracy and virtually no decrease in transfer speed. AFTER that I import to my RAW processing. It also has the advantage of being able to transfer to mutiple drives at the same time if that is useful.



Oct 30, 2016 at 05:29 PM
dgdg
Offline
• • • • •
Upload & Sell: Off
p.1 #5 · p.1 #5 · Advice on image backup procedure


Sounds like a plan
As you have it, automation and offsite backups are the key.
Maybe a sanho colorspace hyperdrive for your a7r2 if you use it heavily on a trip. If you buy your own hdd, it is not too expensive.



Oct 30, 2016 at 08:36 PM
Andrew J
Offline
• • • • •
Upload & Sell: On
p.1 #6 · p.1 #6 · Advice on image backup procedure


321 backup system.
3 copies.
2 different types of media.
1 copy off site.



Oct 31, 2016 at 01:35 AM
arbitrage
Offline
• • • • • • •
Upload & Sell: On
p.1 #7 · p.1 #7 · Advice on image backup procedure


So in the end are you having 3 full copies. One on an external drive (what drive is this?), one on the 5D and one on the 5N? Plus your best stuff in the cloud?

I don't see any problems with the plan but just seems like one more copy than you need at home and not everything backed up fully offsite.

I was running two RAID 5 boxes at home (ThunderBay enclosures and their software) and some portable HDs kept at work but only manually backed up every few weeks or so. Then when I had a few drives fail and the rebuild times were sooo long and then one time it couldn't rebuild that I decided to move it to RAID 1 and keep things a little simpler in case it can't rebuild. So I have my two enclosures and each in RAID 1 so really I guess I have 4 copies at home. My solution isn't perfect as my offsite isn't always up to date




Oct 31, 2016 at 10:56 AM
mitesh
Offline
• • • • •
Upload & Sell: On
p.1 #8 · p.1 #8 · Advice on image backup procedure


leethecam wrote:
I tend to transfer card data initally to drives with Shotput Pro. This software verifies the transfer with accuracy and virtually no decrease in transfer speed. AFTER that I import to my RAW processing. It also has the advantage of being able to transfer to mutiple drives at the same time if that is useful.


Thanks for the suggestion of Shotput pro. I'll check it out.
---------------------------------------------

dgdg wrote:
Sounds like a plan
As you have it, automation and offsite backups are the key.
Maybe a sanho colorspace hyperdrive for your a7r2 if you use it heavily on a trip. If you buy your own hdd, it is not too expensive.


I do have a Nexto, which is a similar device that I use to copy cards to when on the road.
---------------------------------------------

arbitrage wrote:
So in the end are you having 3 full copies. One on an external drive (what drive is this?), one on the 5D and one on the 5N? Plus your best stuff in the cloud?

I don't see any problems with the plan but just seems like one more copy than you need at home and not everything backed up fully offsite.

I was running two RAID 5 boxes at home (ThunderBay enclosures and their software) and some portable HDs kept at work but only manually backed up every few weeks or so. Then when I had a few drives fail and
...Show more

Hi Geoff,

Good points. Keeping the offsite copy current is a challenge. Maybe I should look into putting the Drobo 5N offsite, since I could theoretically set up remote access and back up to it remotely.



Oct 31, 2016 at 11:40 AM
JBPhotog
Offline
• • •
Upload & Sell: Off
p.1 #9 · p.1 #9 · Advice on image backup procedure


I used to run my data redundancy via RAID but I gave up on RAID since replacing a defective drive of the same like and model can be problematic after a year or so when HD models get replaced. Identical firmware on each drive can also become a reliability issue.

Instead, I went with an external Thunderbay 4 enclosure, and loaded it with drives with a pair of drives copying themselves via Carbon Copy Cloner on an automatic daily schedule. When I return from a shoot, I load the images onto my computer then run CCC immediately. I also have a Newertech Voyager S3 I use to copy the data onto a drive which resides off-site.



Oct 31, 2016 at 12:07 PM
dgdg
Offline
• • • • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · Advice on image backup procedure


JBPhotog wrote:
I used to run my data redundancy via RAID but I gave up on RAID since replacing a defective drive of the same like and model can be problematic after a year or so when HD models get replaced.


From my limited experience, if I were to put the same capacity hard drive in my Raid 1, it should rebuild just fine. I use Windows (not Mac like Mitesh).
Have one JBOD Thunderbolt drive managed by Windows Storage Spaces as effectively Raid 1. And also some Western Digital My Book Duo's (has WD red drives) some in Raid 1, others Raid 0. .




Oct 31, 2016 at 01:54 PM
Michael White
Offline
• • • •
Upload & Sell: Off
p.1 #11 · p.1 #11 · Advice on image backup procedure


The source for data asset management is a book called the DAM book.


Oct 31, 2016 at 02:30 PM
15Bit
Offline
• • • • •
Upload & Sell: Off
p.1 #12 · p.1 #12 · Advice on image backup procedure


One thing i do is to keep a copy of all the RAW files as they come off the cards. They don't get culled, edited or anything - they just sit there as a final safety net in case a RAW in my main directory gets corrupted and that corruption propagates through the backup system.

Don't forget off-site copies, just in case the house burns down or someone breaks in and steals everything.



Oct 31, 2016 at 05:30 PM
mitesh
Offline
• • • • •
Upload & Sell: On
p.1 #13 · p.1 #13 · Advice on image backup procedure


JBPhotog wrote:
I used to run my data redundancy via RAID but I gave up on RAID since replacing a defective drive of the same like and model can be problematic after a year or so when HD models get replaced. Identical firmware on each drive can also become a reliability issue.

Instead, I went with an external Thunderbay 4 enclosure, and loaded it with drives with a pair of drives copying themselves via Carbon Copy Cloner on an automatic daily schedule. When I return from a shoot, I load the images onto my computer then run CCC immediately. I also have a
...Show more

Thanks for the input. One of the reasons I chose the Drobo system is that I can mix and match drives without worry. I guess I just like knowing that with dual disk redundancy, one drive failure won't wipe out my data. Rebuilding the array successfully is, I understand, not necessarily a given, nor is it necessarily a quick process.



Nov 01, 2016 at 04:22 PM
mitesh
Offline
• • • • •
Upload & Sell: On
p.1 #14 · p.1 #14 · Advice on image backup procedure


15Bit wrote:
One thing i do is to keep a copy of all the RAW files as they come off the cards. They don't get culled, edited or anything - they just sit there as a final safety net in case a RAW in my main directory gets corrupted and that corruption propagates through the backup system.

Don't forget off-site copies, just in case the house burns down or someone breaks in and steals everything.


Keeping one set of untouched RAW files is a good idea. I can easily make that the offsite set. Thanks for the idea!



Nov 01, 2016 at 04:24 PM
arbitrage
Offline
• • • • • • •
Upload & Sell: On
p.1 #15 · p.1 #15 · Advice on image backup procedure


mitesh wrote:
Keeping one set of untouched RAW files is a good idea. I can easily make that the offsite set. Thanks for the idea!


At 14FPS keeping all my RAWs just isn't feasible to me



Nov 02, 2016 at 10:25 PM
15Bit
Offline
• • • • •
Upload & Sell: Off
p.1 #16 · p.1 #16 · Advice on image backup procedure


arbitrage wrote:
At 14FPS keeping all my RAWs just isn't feasible to me


I can see that being a problem. Indeed, for all high volume shooters it is a problem even though hard drives are historically cheap - If you are shooting tens of thousands of images a year, you are generating TB of data, and that adds up on storage cost (especially if you have drive arrays).

I only shoot a few thousand a year and my total drive usage is around a TB, so this works for me.



Nov 03, 2016 at 01:28 AM
arbitrage
Offline
• • • • • • •
Upload & Sell: On
p.1 #17 · p.1 #17 · Advice on image backup procedure


15Bit wrote:
I can see that being a problem. Indeed, for all high volume shooters it is a problem even though hard drives are historically cheap - If you are shooting tens of thousands of images a year, you are generating TB of data, and that adds up on storage cost (especially if you have drive arrays).

I only shoot a few thousand a year and my total drive usage is around a TB, so this works for me.


I often come home with 1000-2000 per day and usually quickly cull that down to 100-200. Usually I keep about 10,000 up to 40,000 per year. Thats enough to back up on its own. But for people like yourself it makes total sense.



Nov 03, 2016 at 06:56 PM
mitesh
Offline
• • • • •
Upload & Sell: On
p.1 #18 · p.1 #18 · Advice on image backup procedure


arbitrage wrote:
I often come home with 1000-2000 per day and usually quickly cull that down to 100-200. Usually I keep about 10,000 up to 40,000 per year.


I ask again... exactly when do you work??




Nov 03, 2016 at 06:59 PM
15Bit
Offline
• • • • •
Upload & Sell: Off
p.1 #19 · p.1 #19 · Advice on image backup procedure


I like the phrase "quickly cull". It takes me 2 months to sort and edit 2000 pictures. I'm always in awe at Pro photographers who can cover an event and have the images ready in a week or two.


Nov 04, 2016 at 03:59 AM





FM Forums | General Gear-talk | Join Upload & Sell

    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.