Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | General Gear-talk | Join Upload & Sell
1      
2
       end
   

Archive 2020 · external hard drive storage advice
  
 
jcolwell
Online
• • • • • • •
Upload & Sell: On
p.2 #1 · p.2 #1 · external hard drive storage advice


docsmiles17 wrote:
is your desktop an iMac i7? ...

It's a big, gamer PC maxed-out for PhotoShop.
Feb 07, 2020 at 12:52 PM
mjm6
Offline
• • •
Upload & Sell: Off
p.2 #2 · p.2 #2 · external hard drive storage advice


If you are using an iMac of previous years (back to 2014 I think) you can get external drive boxes from OWC that will hold 2, 4, or I think six drives in them.

Those can be set up as JBOD through the drive utility in OS10 without any additional effort. You can also combine both SSD and HDD in the boxes with no trouble. I have a 4-bay that I use a single small SSD in one bay as a swap disk for cache from PS and LR. The other three are HHDs of various sizes. It would certainly be possible to set one SSD up for cache, one SSD for recent and working files, and two other HDDs for longer term storage.

Or, you could build a RAID with a four-drive box using RAID 5 and HDDs. You will need SoftRaid for this, but in my experience, it works well and is easy to set up. I have that running now on my network through the server (Mac Mini) and a 10GbE switch. I'm getting read and write speeds that are 2.5x to 3x the speed of a single drive connected locally, but probably only about 90% of the speed of the same RAID if it were connected locally. I could have kept them locally, but decided that I wanted them in the other room for noise reasons.

Anyway, if you are thinking about buying this for a machine you may replace soon, you may want to look at the TB3 versions of the drives from OWC, and then get a Apple TB3 ---> TB2 adapter. As far as I know, they are the only ones that make these adapters that are bi-directional, but you should check with OWC to confirm it will work in the 'other' direction (from a TB2 computer to a TB3 device). If that works, you will be set up for your next computer, which won't have TB2, but certainly will have TB3/USB3.2.
Feb 07, 2020 at 07:00 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.2 #3 · p.2 #3 · external hard drive storage advice


docsmiles17 wrote:
is your desktop an iMac i7? ...

jcolwell wrote:
It's a big, gamer PC maxed-out for PhotoShop.


I recently did a new build and decided to use my old large mid-tower case with 140mm and 200mm quiet fans since I don't find the new ones to be as good.
For years I had nine 3.5" drives, three 2.5" SSDs and an optical drive in it. Most of the hard drives were on an 8-port SAS controller that supports hot swap. I can turn off power to some of the drives when not in use.

EBH
Feb 09, 2020 at 09:01 PM
codename
Offline
•
Upload & Sell: Off
p.2 #4 · p.2 #4 · external hard drive storage advice


SSD vs HDD - I still prefer HDD (Spinning disks) for bulk storage because they are cheaper per TB. The finite read/write count for SSDs is also a consideration.

My recommendation is to also consider the quantity of the disks you have for storage. I like to follow the mantra of one is none, two is one. If you are considering an external storage device at least look at 2-bay NAS. You can go with a NAS that has more bays if you want to spend the money, but a 2-bay will do just fine to start. My key considerations for modern NAS platforms are:
- RAID & automatic protection from a single drive failure (The more bays you have to more drive failures you can tolerate at once)
- Automation to copy/backup your files to an external hard drive that is connected to the NAS
- Cloud integration to copy/backup your files to an offsite location

Many years ago I lost all of my RAW files (Probably 5+ years worth) because of a hardware failure in a home built file server. Even though I had RAID I was not following my mantra because all of my photos lived only on that system. So make sure your files are replicated to multiple devices and locations. Cloud storage makes that easy and affordable.
Feb 16, 2020 at 09:15 AM
AmbientMike
Offline
• • • • • •
Upload & Sell: Off
p.2 #5 · p.2 #5 · external hard drive storage advice


It may have improved recently, but SSD's generally used to not be considered good for long term storage, since you could lose data over a short period of time. Like if you didn't start it up for months. Also RAID can have issues, from what I have read resulting in lost data.

I haven't done it yet, but it looks like the best idea is SSD for the OS drive, then maybe hdd's for the others. SSD is supposed to be a lot faster hdd less expensive and lasts longer unplugged, apparently.

I was working on this in Dec and Jan, and decided to go with enclosures for internal (mostly 3.5") drives. I may just use a cord, leaving the drive open during use, improving cooling. I just use a small fan and leave the enclosure open, pointing the fan at it. I'll also point the fan at older MyBook type drives if the software says they're heating up. I downloaded Crystal Disk info to monitor drive temperature, other programs available too. Been storing the drives in the silver envelope and bubble wrap etc they came in when not in use. I'll probably continue to use the WD passport, etc, but it is nice to expand the availability of different drives, and Idk if anyone us putting the NAS, enterprise, etc higher quality drives in the basic USB drives you can get.


Feb 23, 2020 at 09:13 PM
RDKirk
Offline
• • • • •
Upload & Sell: Off
p.2 #6 · p.2 #6 · external hard drive storage advice


AmbientMike wrote:
It may have improved recently, but SSD's generally used to not be considered good for long term storage, since you could lose data over a short period of time. Like if you didn't start it up for months.

About 120 months, maybe.

Also RAID can have issues, from what I have read resulting in lost data.

Use the RAID schemes that tolerate drive failures. There's nothing that can't have an "issue" but there are things that can tolerate "issues."

There's no corporate data storage entity that isn't using RAID types that tolerate drive failures.
Feb 23, 2020 at 10:25 PM
AmbientMike
Offline
• • • • • •
Upload & Sell: Off
p.2 #7 · p.2 #7 · external hard drive storage advice




RDKirk wrote:
About 120 months, maybe.

Use the RAID schemes that tolerate drive failures. There's nothing that can't have an "issue" but there are things that can tolerate "issues."

There's no corporate data storage entity that isn't using RAID types that tolerate drive failures.


120 months? Maybe but I wouldn't depend on it. Read more like potentially 6 if you don't power it up. Also:

https://superuser.com/questions/1334494/lifespan-of-an-ssd-nand-flash-for-minimal-write-use-archive-purposes-write-on

Yeah, if you use the correct RAID configuration. But there I've read that the entire recovery (if that's the correct term) can fail if one sector fails. Odds of that something like 50% on 4tb. So it's important to use the correct configuration.



Feb 23, 2020 at 10:51 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.2 #8 · p.2 #8 · external hard drive storage advice


AmbientMike wrote:
It may have improved recently, but SSD's generally used to not be considered good for long term storage, since you could lose data over a short period of time. Like if you didn't start it up for months. Also RAID can have issues, from what I have read resulting in lost data.

I haven't done it yet, but it looks like the best idea is SSD for the OS drive, then maybe hdd's for the others. SSD is supposed to be a lot faster hdd less expensive and lasts longer unplugged, apparently.

I was working on this in Dec and Jan,...Show more →

MLC SSDs are quite reliable and very durable. Samsung Pro SSDs (860 Pro or 970 Pro series) are an example of such drives readily available on the consumer market.
Even TLC Enterprise grade SSDs exceed the vast majority of bulk storage needs for durability. Any good drive will have a five year warranty. So for example, if you get an Intel DC P4610 the durability is >3 DWPD for five years. That is with overprovisioned TLC NAND flash, but still impressive. How many users will exceed three drive writes per day for five years of warranty? After five years you will be looking for new drives anyway. I have SSDs over 10 years old that work fine, though they sat in a drawer for 6 years, and I recently used some 1TB drives from 2013 that also still work and have been running only 500 something hours total.

RAID should not have issues if it is implemented with sufficient redundancy (ideally RAID 6/ZFS RAID-Z2 or maybe a RAID 1 variant for small data sets) and good hardware. A lot of really cheap firmware-based RAID enclosures exist that give RAID a bad name.

One other thing to consider is that the cheap hard drives have an UBER of 10^-14 vs. 10^-15 for the enterprise hard drives and some of the better NAS drives. Most SSDs have a 10^-17 UBER; that is 100 times less likely to have an unrecoverable byte than a good drive and 1000 times less likely than a cheap drive like whatever is in the external WD enclosures. If SSDs were more economical I'd run RAID 5 and have no concerns about storing them in a fire box for five years. OTOH, I don't trust the cheap portable 2.5" hard drives farther than I could throw them.

EBH
Feb 23, 2020 at 11:27 PM
AmbientMike
Offline
• • • • • •
Upload & Sell: Off
p.2 #9 · p.2 #9 · external hard drive storage advice




EB-1 wrote:
MLC SSDs are quite reliable and very durable. Samsung Pro SSDs (860 Pro or 970 Pro series) are an example of such drives readily available on the consumer market.
Even TLC Enterprise grade SSDs exceed the vast majority of bulk storage needs for durability. Any good drive will have a five year warranty. So for example, if you get an Intel DC P4610 the durability is >3 DWPD for five years. That is with overprovisioned TLC NAND flash, but still impressive. How many users will exceed three drive writes per day for five years of warranty? After five years you...Show more →

I'd be curious where you get this info from, although I can't find an article about mine. I looked at RAID several years ago, and after reading about 1 sector crashing the recovery of the drive, I didn't go that route. I think it was RAID 5?

And I started a thread about what to buy, and you mentioned the WD easyshare, and sounded like you'd bought them. Not sure what is going on here. I doubt they're better than the 2.5", which I suspect have wd blue or similar, which I suspect are probably about the same durability as we've been using for years.

https://www.fredmiranda.com/forum/topic/1629185/0#15099572
Feb 23, 2020 at 11:51 PM
runamuck
Offline
• • • • • •
[X]
p.2 #10 · p.2 #10 · external hard drive storage advice


I have been using SSD's for a couple yers now. Never will go back to spinners.
Feb 24, 2020 at 03:02 AM
AmbientMike
Offline
• • • • • •
Upload & Sell: Off
p.2 #11 · p.2 #11 · external hard drive storage advice


I think this guy had some good discussion of ssd vs hdd, upgrading an old computer. Looks like an ssd is a definite advantage for the main OS drive. But he uses hdd's, too

https://youtu.be/9zGMi7HZ2AA
Feb 24, 2020 at 12:11 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.2 #12 · p.2 #12 · external hard drive storage advice


I've been using SSDs for over 11 years, since the Intel X25-E. It remains in the top three most life-altering computer experience ever, and that was coming from 15,000 RPM SCSI hard drives.

EBH
Feb 25, 2020 at 12:42 AM
retrofocus
Offline
• • • • • •
Upload & Sell: Off
p.2 #13 · p.2 #13 · external hard drive storage advice


docsmiles17 wrote:
I have been storing my image files on internal HD but I have almost 2 TB of images and time to move to external for storage. My plan is to change the import location in LR to external drive. I already back up with external hdd

I am considering a SSD vs HDD but it seems like the price is $400+ for a 3tb ssd drive

Am I being silly not to seriously consider the ssd vs hdd?

I know the ssd is more durable but is there any noticable speed difference between ssd and hdd?

For those who have switched like I am,...Show more →

That's what I started doing about a year ago. I bought a 20 TB LaCie drive with two HDD discs which is connected via USB to my desktop PC. I set up the external drive with mirror image meaning that I have it set as 2x10 TB platforms. Programs like LR and PS easily take large photo files from the external HDD drive without lag. Big advantage: if I ever change my PC, I can simply reconnect the external drives.


Feb 25, 2020 at 10:34 AM
mjm6
Offline
• • •
Upload & Sell: Off
p.2 #14 · p.2 #14 · external hard drive storage advice


The point of RAID isn't that it is fault impervious, it's that it is faster to write and read and there is some ability to rebuild if you have a failed drive. IT IS NOT A SECURE FAILSAFE. The only way to make a secure failsafe is to make duplicates, and one of them must be off-site.

If you are running a backup cloning action every night, you won't lose data beyond that day's work (or week, etc., depending on how long you go between cloning actions). For most of us, this is completely sufficient and you can safely rely on a RAID (0, 5, 6, or other configurations) drive to be fast and cost efficient. You could even set up a redundant RAID like a RAID 1+0 and actually have a duplicate for fast rebuilds but that is unnecessarily duplicative for non-business applications where down-time is unacceptable.

It all comes down to your needs, of course, but there is no reason to fear RAID if you have a proper set of expectations and you set up the system properly to meet your needs, in particular to recognize that RAID isn't a proper backup approach.

For example, I am running a four-disc RAID 5 array of 4T Ironwolf Pro drives for about 12T of storage. I have CCC running a backup action from the RAID to a single large 8TB drive as a backup. I do it once a week, but I figure I'll trigger it after every large change to my files (like after I upload a trip's worth of images). I also have a second 8T drive backup that I make and keep off site at the office. This will again mostly be triggered by large changes to the files, but I'm thinking about doing it weekly. These 8T drives will be replaced with larger ones as I get mote data to back up. I'm currently at about 5T, so I have a bit of time.

If a drive in the RAID goes down, I have two options; 1. Replace the drive and rebuild. 2. Replace the drive and copy the data back from the on-site backup.

I'm certain the on-site backup will be a faster way to go, so the only way I would do the other approach is if there is something crucial that hadn't made it onto the most decent disk image on the backup, but that is really unlikely.

Lastly, the RAID is pretty fast. It's about 2.5x the speed of a HD connected locally and that is when running through a 10GbE network, not directly plugged into the computer, so it completely satisfies my need for access speed and also allows me to run it remote to keep the noise of spinning HDs out of the room entirely. The server (Mac Mini) performs all the backup tasks for it using CCC. The external drive that I take off-site can be plugged in at my workstation for the duplication when that occurs.
Feb 25, 2020 at 02:00 PM
docsmiles17
Offline
• • •
Upload & Sell: Off
p.2 #15 · p.2 #15 · external hard drive storage advice


mjm6 wrote:
The point of RAID isn't that it is fault impervious, it's that it is faster to write and read and there is some ability to rebuild if you have a failed drive. IT IS NOT A SECURE FAILSAFE. The only way to make a secure failsafe is to make duplicates, and one of them must be off-site.

If you are running a backup cloning action every night, you won't lose data beyond that day's work (or week, etc., depending on how long you go between cloning actions). For most of us, this is completely sufficient and you can safely rely on...Show more →

good info. thanks. whats the reason for a 4 disc RAID vs a 2 disc?
Feb 26, 2020 at 01:18 PM
mjm6
Offline
• • •
Upload & Sell: Off
p.2 #16 · p.2 #16 · external hard drive storage advice


docsmiles17 wrote:
good info. thanks. whats the reason for a 4 disc RAID vs a 2 disc?

It depends on what kind of RAID you want. You could do a RAID0 with two drives and have it be about 1.9x the speed of a single drive, but with absolutely no redundancy. It will be faster, but your risk of failure is 2x the risk of a single drive. If you did a RAID0 with 4 drives, it would be very fast, probably 3.5x the speed of a single drive, but again, no redundancy and your risk of failure is now 4x that of a single drive.

I decided to go for RAID5, which has a parity portion (1 out of every four segments, but it is spread out over all four drives). It is faster, but less prone to a complete failure. You have to have 3 drives minimum for RAID5.

If you are going for speed, then a RAID0 is the way to go, but you have to be very good about backup copies, etc., because there is no chance to reconstruct the data if you have a drive failure.

I also had the external drive box, so I figured I'd utilize all four drive bays for the RAID.

Considering I was using the computer with a single drive (locally before the upgrade) as my image drive, this change has been a substantial speed improvement when saving files, loading for use in PS or LR, and moving files around. While a SSD is great, the cost of 12TB of storage on SSD makes no sense to me. I happen to be running this through a network, but you could easily do this with a directly-connected external drive box (or internal drives if you have a big server-style computer case). This approach meets my needs and I feel that it is a better use of my resources.
Feb 26, 2020 at 02:54 PM
1      
2
       end
FM Forums | General Gear-talk | Join Upload & Sell

1      
2
       end
    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.