Home · Register · Software · Software · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Nikon Forum | Join Upload & Sell

1       2       3       4      
5
       end
  

HD capacity: Your storage strategy?
  
 
kosin
Offline
• • • •
Upload & Sell: On
p.5 #1 · p.5 #1 · HD capacity: Your storage strategy?


Even though external 8TB drives are cheap these days: http://amzn.to/2jYtCB2 I stopped backing up this way long time ago and ended up going with the NAS/RAID option



Sep 26, 2017 at 01:09 AM
tdlavigne
Offline
• •
Upload & Sell: Off
p.5 #2 · p.5 #2 · HD capacity: Your storage strategy?


I've got 3tb, of which about 500gb are photos from the past decade. I've never seen the point in keeping every single RAW image I've ever shot. I edit my shoots down to the final selects, retouch those to completion...then delete everything else that didn't make the cut. Makes storage a non-issue thankfully.


Sep 26, 2017 at 01:15 AM
timbyers
Offline

Upload & Sell: Off
p.5 #3 · p.5 #3 · HD capacity: Your storage strategy?


I use ZFS on FreeNAS with 5 10TB NAS drives. 4 drives are active. Any two can fail. As soon as one fails, the fifth drive is used to recreate the failed drive.

This gives me a bit over 18TB of usable storage.

I have a second 4 10TB FreeNAS server in a similar configuration (minus the 5th drive) as on online backup.

I also keep external bare offline backup drive(s) for every year's images.



Sep 26, 2017 at 04:56 AM
15Bit
Offline
• • • • •
Upload & Sell: Off
p.5 #4 · p.5 #4 · HD capacity: Your storage strategy?


timbyers wrote:
I use ZFS on FreeNAS with 5 10TB NAS drives. 4 drives are active. Any two can fail. As soon as one fails, the fifth drive is used to recreate the failed drive.

This gives me a bit over 18TB of usable storage.

I have a second 4 10TB FreeNAS server in a similar configuration (minus the 5th drive) as on online backup.

I also keep external bare offline backup drive(s) for every year's images.


Erring on the side of safety with Raidz2 on a 4 drive array. A bit too much so for me - given that you have a backup, i would have taken the option to use more of the drive capacity and reduced redundancy to 1.

Are you using snapshots on the backup array to secure against accidental deletions propagating through the backup system?



Sep 26, 2017 at 05:40 AM
15Bit
Offline
• • • • •
Upload & Sell: Off
p.5 #5 · p.5 #5 · HD capacity: Your storage strategy?


MintMar wrote:
ZFS? Did ZFS make it already to the Linux kernel?


Not in the kernel i think, but it's been available for several years as a package. It's now fully supported by Canonical i think.



Sep 26, 2017 at 05:43 AM
markshelby
Offline
• • •
Upload & Sell: Off
p.5 #6 · p.5 #6 · HD capacity: Your storage strategy?


I highly recommend Synology or similar-caliber NAS with a cloud backup. A real NAS is much more robust than just storage on a PC--hard drive monitoring, RAID scrubbing, versioning (recover old versions or deleted versions) etc. etc. (Additionally you can provide your own cloud services from a Synology NAS, for example my kids' notebooks automagically back up to it from college.)

I have a Synology server with seven disks and 2 disk redundancy (I have 20TB of data) that backs itself up to Amazon cloud (unlimited photo storage).

My Synology NAS has been so robust that when my current primary storage (RAID5 on my primary PC) becomes inadequate, I will probably upgrade my home network to 10GBe and buy a second Synology NAS for my primary storage.

BTW if cloud services charged for photo storage I would probably set up another Synology NAS and mail it to my brother across the country to use as my own personal "cloud".

PPS--one thing people sometimes forget is that at least one of your backup methods should keep old deleted files, so that unintended file deletion doesn't delete all the backups as well.



Sep 26, 2017 at 03:33 PM
markshelby
Offline
• • •
Upload & Sell: Off
p.5 #7 · p.5 #7 · HD capacity: Your storage strategy?


I highly recommend Synology or similar-caliber NAS with a cloud backup. A real NAS is much more robust than just storage on a PC--hard drive monitoring, RAID scrubbing, versioning (recover old versions or deleted versions) etc. etc. (Additionally you can provide your own cloud services from a Synology NAS, for example my kids' notebooks automagically back up to it from college.)

I have a Synology server with seven disks and 2 disk redundancy (I have 20TB of data) that backs itself up to Amazon cloud (unlimited photo storage).

My Synology NAS has been so robust that when my current primary storage (RAID5 on my primary PC) becomes inadequate, I will probably upgrade my home network to 10GBe and buy a second Synology NAS for my primary storage.

BTW if cloud services charged for photo storage I would probably set up another Synology NAS and mail it to my brother across the country to use as my own personal "cloud".

PPS--one thing people sometimes forget is that at least one of your backup methods should keep old deleted files, so that unintended file deletion doesn't delete all the backups as well.



Sep 26, 2017 at 03:33 PM
dalite
Offline
• • • •
Upload & Sell: On
p.5 #8 · p.5 #8 · HD capacity: Your storage strategy?


Thank goodness external hard drive prices are so much lower than years ago. Think terabytes.


Sep 26, 2017 at 09:59 PM
 

Search in Used Dept. 

        


Jan Brittenson
Offline
• • • •
Upload & Sell: On
p.5 #9 · p.5 #9 · HD capacity: Your storage strategy?


Mike_5D wrote:
Everything possible lives on a 5 bay Synology NAS. The array can be expanded as needed without even taking it off-line.

Sounds similar to my setup: a Synology DS1515+ with five WD Red 3TB drives, for about 12TB effectively. 6GB memory. I absolutely love it.

I back it up using external USB-3 drives. They get an incremental backup each night, but only one is ever plugged in while the other is offsite. The one not plugged in will keep failing. I then swap the offsite and active backup drives, at which point the backup set that kept failing will catch up and add incrementals each night, while the previously active one will fail. And so I go back and forth between them. The fact that the NAS makes it so easy to use an external USB drive to back it up makes this very straightforward.

The interface is great, too.





Sep 27, 2017 at 12:50 AM
Jan Brittenson
Offline
• • • •
Upload & Sell: On
p.5 #10 · p.5 #10 · HD capacity: Your storage strategy?


markshelby wrote:
I highly recommend Synology or similar-caliber NAS with a cloud backup.

I've been considering backing up to an AWS S3 bucket using a glacier policy since it's typically write-only and accessing it would be rather exceptional. This costs about $4/TB per month (region us-west-2). It's a bandwidth hog though, and even though I have 150M/30M service a large photo set would soak the uplink for 4-6 hours. The initial set would also take quite a while...



Sep 27, 2017 at 01:02 AM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.5 #11 · p.5 #11 · HD capacity: Your storage strategy?


Jan Brittenson wrote:
Sounds similar to my setup: a Synology DS1515+ with five WD Red 3TB drives, for about 12TB effectively. 6GB memory. I absolutely love it.



Many of those DS1x515+ NAS have the defective Intel C2538 CPUs.
https://www.anandtech.com/show/11110/semi-critical-intel-atom-c2000-flaw-discovered
The defect was corrected in the later stepping, so you should check.

EBH



Sep 27, 2017 at 02:28 AM
FrancisAlberto
Offline

Upload & Sell: Off
p.5 #12 · p.5 #12 · HD capacity: Your storage strategy?


I have a 750GB HD installed in an external enclosure. I keep my backup stuff there, it is for archival and retrieval use only. My old camera files are small from my 30D. I may need to get an bigger hard drive in the future to use since I moved to a different camera.
I have considered when they go into sale getting one of those BlueRay burners with BDXL. Unfortunately when there's a sale, it's usually the LG barebone ones that aren't fully compatible with Windows 10.



Sep 27, 2017 at 03:23 AM
technic
Offline
• • •
Upload & Sell: Off
p.5 #13 · p.5 #13 · HD capacity: Your storage strategy?


I use a very modest mini-PC with 500GB internal HD and 1 TB external HD that holds all my image files (large number, many from long ago so relatively small sizes); it's now 95% full so time for a bigger one. There's a backup of all image files (except most recent ones) and other important stuff on another external drive. For the most important processed images (couple 10.000's) I have a reduced size copy on microSD card as well, as a very portable personal archive that is easy to use on a tablet etc. Older images have DVD backups as well but I gave up several years ago because it became impractical. I keep RAW files but only those that survive the culling process.

I worked in IT/imaging for many years and at a time rescuing professional image/animation people from disaster was part of the job; it's not something I like to delve in too much in my private life. I have seen the most weird kinds of storage (and computer ...) problems over the years, difficult to protect against everything especially user stupidity/laziness. Photography is just a hobby for me so I make backups to books and accept the possibility of catastrophe ;-)

I also have a storage question for other (I lost touch with the IT world years ago), sorry if this hijacks the thread a bit ...

I'm considering buying a Microsoft Surface Pro4, the ones with i5/8GB/256GB are cheap now and that sounds OK for my photography needs. It's mostly for home use, better battery capacity of the new Surface model is not important to me. The 1 TB SSD version sounds nice (internal Lightroom archive etc.) but is way more expensive so not very attractive.

How does one handle storage and backup with such a system? From what I understand it has only one USB port available, so when one uses an external drive for image (archive) storage, even making backups would be a problem. I know there are external devices that automatically copy a complete disk to another disk, but I have no idea how reliable that is. Maybe someone using a Surface Pro4 or similar system has some tips for a good configuration?



Sep 28, 2017 at 10:50 AM
R.H. Johnson
Offline
• • • •
Upload & Sell: Off
p.5 #14 · p.5 #14 · HD capacity: Your storage strategy?


just filled 4TB @ 99.0% capacity. waiting on delivery of WD 16TB My Book Duo Desktop RAID External Hard Drive - USB 3.1. will configure RAID1 @ 8TB backup the 4TB and have 4TB+ remaining in the array. don't know what i'll do with the 4TB book once it's backed up. also use 3TB WD My Passport wireless Pro in the field. won't go into details but, i've lost over 2 terabytes of images in the past when my Raid array failed. the idiot i paid to recover drives screwed me. the one thing about image storage: backup backup and more backup.


Sep 28, 2017 at 04:51 PM
wesley lee
Offline

Upload & Sell: Off
p.5 #15 · p.5 #15 · HD capacity: Your storage strategy?


Here's my strategy. I strove to have system that's most cost-effective and fast. The components are :

Primary drives - where the photos are stored, processed, etc (1st instance of photos)

Backup drives - copies of the "primary" drives and physically separate (2nd instance of photos)

Software - Syncback which is scheduled everyday. The frequency is good enough for me as I would rarely shoot more that one time per day. Worst case, I would lose a day's photos in this automatic mode. If I'm paranoid, then I simply manually back up the files in Syncback.

I just use external USB drives or internal drives for primary and backups. I do have a NAS but it's slow compared to the performance of an USB 3.0 drive (5Gbs vs 1Gbs for NAS). And each dollar spent on an external drive is storage (vs the h/w for the NAS cage).

For the backups, I like those "portable" 2.5in drives which are powered by the USB itself. If the drive gets too full, I can simply disconnect and insert another. If were really paranoid, I would make an additional backup on a portable drive (3rd instance of photos), then store the drive at another location (such as parents house or at work).

To get more USB 3.0 ports, I bought a couple of 4 port PCIe cards at about $25/each. I considered a Drobo but I consider a multi-port USB PCIe card + external drives just a good, probably faster, and definitely cheaper.

The above obviously doesn't have RAID. For just photos, I don't think RAID is really necessary since photos (at least in my case) are stored really infrequently and are not touched afterwards (editing in Lightroom does not touch the actual photos; just keeps a database of edits). I think the periodic daily backup which Syncback which is configured for is good enough for me. (Syncback could be configured for more frequent backup if desired)



Sep 28, 2017 at 09:11 PM
1       2       3       4      
5
       end






FM Forums | Nikon Forum | Join Upload & Sell

1       2       3       4      
5
       end
    
 

You are not logged in. Login or Register

Username     Reset password