Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Post-processing & Printing | Join Upload & Sell

       2       3       4       end
  

SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBo...

  
 
eyal
Offline
• • •
Upload & Sell: Off
p.1 #1 · p.1 #1 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


My current setup is an M1 Pro 14" MacBook Pro connected via single USB-C cable to a 5 port OWC Thunderbolt hub then to:
2 externals monitors (one USB-C, other DisplayPort to TB2 adapter)
RAID 5 array (4x8 TB drives for 24TB usable - drives purchased 5/2021)
2 TB External SSD (partitioned as 1TB Time Machine and 1 TB bonus storage)

The RAID holds 11 TB of data including my photos (6.5 TB)

Blackmagic Speed
2 TB external - 865 write/700 read
Thunderbay RAID5 - 430 write/500 read

Current dislikes of setup:
Annoying/noisy whenever HDDs spin up
Slow to search for photos in LRc (I use a single catalogue on the Mac internal drive) but do so infrequently

I had thought the OWC 4M2 with NVMe drives would be a no-brainer and could go with JBOD as I wouldn't need to boost speeds beyond the baseline NVMe but it only reaches full speeds at RAID0 and otherwise is severely hampered by being limited to 750 MB/s per drive as each is connected via PCIe 3.0 x 1 (found after some searching).

The alternative then is the Thunderbay 4 mini with 4 2.5" SSDs in RAID4 which Lloyd Chambers clocked at 1152 read and 931 write back in 2014 (?!?!?). Further big benefit of the Thunderbay mini is it is apparently far quieter. BUT I lose the bonus display port. Also 2.5" SSDs are half the cost of the NVMe at 4 to 8 TB sizes.

Also, despite the name, the Thunderbay 4 mini and 4M2 are both thunderbolt 3 devices.

So here are some questions/thoughts I would appreciate help with:

1. Would use of Thunderbolt 4 in either device make a difference with a modern M1/M2 Mac? Would this fix the slow speeds of the 4M2 in JBOD mode? I have to believe OWC will move to TB4 soon-ish.

2. Any way to know what max speed I can get in my current setup where everything flows through the OWC Thunderbolt Hub which is TB4?

I love my single cable connection which, when using the OWC dock app, allows me to disconnect with one-click. I don't want to give that up but also don't want to spend $1-2k on fast external storage if it will be bottlenecked by this setup. I guess I could run a single cable from the new external and have 2 cables to the MacBook and still benefit from the dock unmount option. Just a little less clean.

Thanks!

Clarifying with diagram:
Screenshot 2023-06-15 at 1.34.10 PM by Eyal Oren, on Flickr

Edited on Jun 15, 2023 at 12:36 PM · View previous versions



Jun 15, 2023 at 10:50 AM
mcbroomf
Online
• • • • •
Upload & Sell: Off
p.1 #2 · p.1 #2 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


I'm not following the exact path of all your connections but I wanted to ponit out that the Thunderbay 4 Mini has a TB passthrough, so you could run a monitor from it with a TB 3 or 4 to Displayport interconnect.

I can't answer other questions but I didn't think it's a good idea to run everything off 1 TB4 port as essentially all the data then shares that pipe. Perhaps others can comment.



Jun 15, 2023 at 11:53 AM
ShotByTom
Offline
• • •
Upload & Sell: Off
p.1 #3 · p.1 #3 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


There's a lot going on there! I just switched from a 16" MBP to an M1 Max Studio. I was using a Pluggable dock to connect to 3 external monitors, 2 USB3 enclosures and 1 TB4 drive. The usb 3 enclosures had read/write speeds around 140mbs and was really slow!

I now use a 4tb TB3 NVMe enclosure for my images and LR catalog, and I archive my old images from more than a year to the usb3 external enclosures, which I almost never access.

I would love to have more than 4TB, but that gets pricey, and with archiving I'm able to make it work. My read/write speeds for the TB3 are ~ 2600/1300 mbs



Jun 15, 2023 at 03:54 PM
mcbroomf
Online
• • • • •
Upload & Sell: Off
p.1 #4 · p.1 #4 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


Thanks for the diagram, that helps.

I can confirm that if you switch out the Raid 5 box with a Thunderbay 4 mini you'll be able to drive the monitor through it. I have one external monitor, a Ben Q 2700 (2.5k monitor), hooked up to my MBP M1 Max through the HDMI port, so I disconnected it and reconnected it from the Thunderbay 4 mini to the monitor DP port with a TB/DP cable. Works fine.



Jun 15, 2023 at 05:46 PM
schlotz
Offline
• • • • •
Upload & Sell: On
p.1 #5 · p.1 #5 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


As you already know the TB2 Raid5 is the slower link but if it's used for mainly archived photos, i.e. those you don't do much editing to, it serves its purpose. Sound wise when spinning up, well that's different issue. I have a very similar setup but made the decision some time back to incorporate a Thunderblade TB3/4 drive where I store current year photos. The Blade has a second TB port which I hook my monitor to. Speeds on it are approx 1500 MB/s read & write. All other photos are on a OWC Raid5 with 4-4TB drives running TB2.

Don't know your annual volume of photos but a NVMe enclosure configured to support that volume sounds like a good solution. At the end of the year move the photos over to the Raid5.



Jun 18, 2023 at 06:26 AM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #6 · p.1 #6 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


schlotz wrote:
As you already know the TB2 Raid5 is the slower link but if it's used for mainly archived photos, i.e. those you don't do much editing to, it serves its purpose. Sound wise when spinning up, well that's different issue. I have a very similar setup but made the decision some time back to incorporate a Thunderblade TB3/4 drive where I store current year photos. The Blade has a second TB port which I hook my monitor to. Speeds on it are approx 1500 MB/s read & write. All other photos are on a OWC Raid5 with 4-4TB drives running
...Show more

Thanks! Sounds like a very similar situation.

How does it 'feel' to run through the photos on the thunderblade vs the slower RAID you had/have?

Can't decide if I'm really trying to throw money at a very occasional 'problem' with spinning up drives as I rarely access files on the RAID5. I ingest all to the internal MacBook drive, process, post and then move to what really is an archive. I'll go to the archive for print products but that is more occasional than anything.



Jun 18, 2023 at 06:49 AM
schlotz
Offline
• • • • •
Upload & Sell: On
p.1 #7 · p.1 #7 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


Very perky. LR flies through any of the photos stored there. Maybe you might consider using the partitioned 1Tb SSD and see how that does for current photos?


Jun 18, 2023 at 06:53 AM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #8 · p.1 #8 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


Hmmm.... A bit of Google'ing led me to a MacRumors post then another and discovered the Sonnettech Echo Dual NVMe Thunderbolt Dock which has 2 NVMe slots at $250 and seems FAR improved internally vs the OWC equivalents.

It seems to offer 1500 speeds at JBOD so, if coupled with a relatively 'slow' Crucial P3 would get me 3x current speeds and 8TB of space with the extra dock/hub slots all for about $700.

Not bad...



Jun 18, 2023 at 07:36 AM
schlotz
Offline
• • • • •
Upload & Sell: On
p.1 #9 · p.1 #9 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


If I read the specs correctly, the TB4 port to the computer only supports 15W, believe the MBP M1 requires a lot more than this. Not up on whether you can daisy chain 2 docks or not. If its possible, I would connect it to the OWC doc where the RAID5 is now. Then connect the RAID5 to the second dock BUT.... there is a limit on how many devices (I believe it's 6) can be chained to one port. Relooking at your diagram you'll probably have to reconfigure and use a second port off of the MBP.


Jun 18, 2023 at 09:23 AM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


schlotz wrote:
If I read the specs correctly, the TB4 port to the computer only supports 15W, believe the MBP M1 requires a lot more than this. Not up on whether you can daisy chain 2 docks or not. If its possible, I would connect it to the OWC doc where the RAID5 is now. Then connect the RAID5 to the second dock BUT.... there is a limit on how many devices (I believe it's 6) can be chained to one port. Relooking at your diagram you'll probably have to reconfigure and use a second port off of the MBP.


Good catch! Sort of curious what speeds would do if ran this through the OWC hub since that would allow me to keep using their dock app.

But likely would need to use a second port off the MBP

Appreciate the back-and-forth on this. May end up being more of an exercise than actually pulling trigger as I realize all the little things I'll have to do with minimal gains for the change.



Jun 18, 2023 at 02:33 PM
 


Search in Used Dept. 

Scott Stoness
Offline
• • • • • •
Upload & Sell: On
p.1 #11 · p.1 #11 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


eyal wrote:
Hmmm.... A bit of Google'ing led me to a MacRumors post then another and discovered the Sonnettech Echo Dual NVMe Thunderbolt Dock which has 2 NVMe slots at $250 and seems FAR improved internally vs the OWC equivalents.

It seems to offer 1500 speeds at JBOD so, if coupled with a relatively 'slow' Crucial P3 would get me 3x current speeds and 8TB of space with the extra dock/hub slots all for about $700.

Not bad...


Be careful with your assumptions. I don't know there crucial P3 NVME but some (many) NVME's have a cache and will throttle down from 1500 to say 150mb/s when they hit it. Even in raid mode. So if you are going this way I suggest Kingston 3000's which have good cached and price.. Maybe the crucial is okay, and it is okay if you are just doing gaming (but maybe not big drive transfers) , but make sure you don't spend lots of money for slower than a raided 2.5" ssd (1000).

https://www.tomshardware.com/reviews/crucial-p3-ssd-review/2

(for 2tb) The P3, like the P3 Plus, is capable of using all of its native QLC in a single-bit pSLC mode. This allows for a cache up to about 550GB. Once it runs out of cache the P3 is forced to free up space by copying data from pSLC to QLC which slows it down greatly; the write speed plummets from 3.2 GBps to 100 MBps.

so it is slower than a 2tb 2.5" ssd after 550gb



Jun 18, 2023 at 08:08 PM
rscheffler
Offline
• • • • • •
Upload & Sell: Off
p.1 #12 · p.1 #12 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


Definitely read up on NVMEs. There are so many options it can be confusing and overwhelming. I was also looking at the P3 and similar due to attractive price points, but yes, those are QLC NAND and will definitely slow down once the cache fills. I opted for the WD Black SN850X 2TB (Tom's Hardware review). It's also available in 4TB for $300 currently at B&H. I only have it connected with a $25 Sabrent USB-C 10Gbs enclosure, but it sustained 10Gbs (1GB/s) for more than 1TB of continuous transfer the day I installed it. I've since filled it to about 90% before clearing off archived content, and even with that much on it, it still maintained ~10Gbs transfer speed. It's possible in a faster enclosure, or if installed internally in a PC, that it would have slowed down, but in the USB enclosure it could still maintain 10Gbs. In this enclosure, the SN850X is nowhere near reaching its full speed capability (supposedly over 7000MB/s). Not sure how much better it would be in a TB3/4 enclosure (both are 40Gbs). In any case, 1GB/s sustained read/write is much better than my 8TB 7200rpm HDDs connected via USB at around 200MB/s, and twice as fast or more than the best I can get from 2.5" SSDs connected by SATA-USB adapter cables.

Based on the Tom's Hardware SN850X review linked above, the 2TB version appears to have a fast cache of about 585GB at ~6.5GB/s before it drops down to ~1.5GB/s, which is still faster than typical 10Gbs USB-3. It looks like the 2TB model's cache is about 2x larger than the 1TB model. Unfortunately they haven't tested the 4TB model, but it's possible it in turn will have a cache 2x larger than the 2TB model. And it's possible its sustained write after filling the cache will be higher than 1.5GB/s. See below.



If you can fit most of a year on 4TB, I'd just go dual 4TB NVMEs, with the second a back up of the first. I likely would not do a RAID 1 mirror, rather, would run daily/hourly automated backups with Carbon Copy Cloner (or similar) to the second SSD so that if something was accidentally deleted, it would still be on the backup. Whatever you decide to go for, at least get something that will maintain a decent sustained write speed, in the 10Gbs range.

Also, RAID 5 is not really an archival solution. It's a great way to create a large volume on which to store everything with some redundancy, but ultimately it's still only a single copy of your files. Personally, I'm not comfortable if I can't take a single HDD, put it in an enclosure or bay, and read files off it. Of course this archiving approach requires multiple copies of files across multiple HDDs.




Jun 18, 2023 at 10:03 PM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #13 · p.1 #13 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


That is some great info Scott and Ron! Appreciate it and doesn't seem so easy/relatively cheap anymore...


Jun 19, 2023 at 08:31 AM
rscheffler
Offline
• • • • • •
Upload & Sell: Off
p.1 #14 · p.1 #14 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


I think it depends on what kind of SSD performance you want. If it's sheer storage at the best price and you're not writing more than the pseudo-SLC cache size on a given SSD, QLC-based SSDs will still perform well. In an enclosure, read/write speeds while within the QLC's cache, should be similar to TLC-based SSDs, since the enclosure will usually be the bottleneck.

For example, Tom's Hardware test of the Crucial P3 Plus shows that it has a huge pseudo-SLC cache, which will maintain extended fast write speeds (~550GB for the 2TB model), but after that, write speed drops off the cliff to only 100MB/s:



The P3 Plus appears to have one of the larger caches for an SSD in its category tested by Tom's Hardware. If it's unlikely you'll ever transfer more data in one shot than the cache size, it's likely a very good option. And the assumption is the 4TB model will have double the cache of the 2TB model.

From the Tom's Hardware review:

The good news is, the (2TB) P3 Plus can absorb up to about 550GB of writes within its pSLC cache. This indicates that all of the QLC is capable of acting in single-bit mode for a total cache capacity thatís one-fourth of the flash. This cache will diminish in size proportionately as the drive is filled based on how much space is left free. A large, dynamic cache is a good way to hide weak native performance. The P3 Plusís cache is ample to handle typical, bursty workloads.

The bad news is that the native performance is extremely poor. Speeds drop down from 4.4 GBps to 100 MBps. We know that this flash is about as fast as Intelís 144-layer QLC, that is up to 40 MBps per die, but the use of a massive cache forces the drive to bottleneck as it must free up capacity by moving data over to QLC. The 670p has some static cache and also DRAM, so it doesnít suffer quite as much. Within the compared drives we do see weak post-cache performance from the SN770, Rocket NVMe 4.0, and SN770 as well.

On the bright side, the P3 Plus is very good at recovering its pSLC cache given sufficient idle time. Leaving some data in the cache can be beneficial for future reads on newly-written data. In fact, this is a tactic utilized by the upcoming P41 Plus when itís less full, although that discussion is to be left for another time. With the P3 Plus it does seem that Crucial tries to strike a reasonable balance but itís definitely not as consistent as drives with TLC.


But... the 4TB P3 Plus is only $75 less than the WD Black SN850X 4TB ($225 vs. $300 currently at B&H)...



Jun 19, 2023 at 09:09 AM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #15 · p.1 #15 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


My particular use case would be to move all 6.5TB of images (all raw with sidecars accessed by LRc) to a faster external than the current RAID5 spinning drive array and hopefully gain far faster views when quickly scrolling, using keywords to find images and/or reprocessing images. I have no idea how quickly that might saturate the cache and slow down the performance of various types of SSDs.

Would be great to have a real-world guide to true speeds for external solutions for photographers:
Spinning drives in various RAID formats
2.5" SSDs as JBOD vs RAID0, 4
NVMe as JBOD, RAID0, 4




Jun 19, 2023 at 12:02 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.1 #16 · p.1 #16 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


I have not seen RAID 4 since the 20th century, but will defer to the IT folks if anybody still uses it. Distributed parity (RAID 5, 6, 50, 60) replaced it long ago for most purposes.

I'm not a fan of external SSDs. Since one TB port has four PCIe lanes, ideally it should be for only one NVMe SSD. In your diagram the bandwidth is extremely bottlenecked. That may or may not matter for your application, but it's conceptually suboptimal.

Much of the performance advantages of SSDs are from the several orders of magnitude higher IOPs than HDDs rather than simply the sustained transfer rates. I have plenty of 10GbE NAS units for bulk storage but of course they are slower than an single SSD for indexing or previewing large numbers of files. Any HDD arrangement will have those limitations. In a NAS the CPU and RAM are also important, more or less, depending on RAID mode and file system.

Since you have only 6.5 TB, then 2x8TB or 3x4TB NVMe SSDs either striped (RAID 0) or individual would be fine for primary storage and then back them up to another drive or array. I separate laptop from desktop computer use, but I don't have a mandate to process in the field. I have 7 internal SSDs and typically work off of a 3x4TB striped array (3 SN850Xs yield over 12,000 MB/sec.). That's what I consider a working space and will move the data back to NAS when finished with it and sync at least daily or as needed.

I don't bother with RAID 5 on small sets of SSDs. Keep in mind that the UBER for a decent SSD is about 10^-17 compared to 10^-14 for most consumer hard drives and 10^-15 for enterprise and top hard drives. A frequent backup is more important than RAID 5 SSD IME. That probably doesn't fly in a regulated environment.

EBH



Jun 19, 2023 at 01:09 PM
rscheffler
Offline
• • • • • •
Upload & Sell: Off
p.1 #17 · p.1 #17 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


I'd be inclined to roughly follow what EB-1 suggests - put everything on a couple 4TB NVMEs and back that up to a few higher capacity HDDs, like 12 or 16TB. Those could either be JBOD in a multi-bay enclosure, or through use of a HDD dock that mostly sit offline until needed. It'd be up to you how you configure the NVMEs, whether RAID or JBOD. Your photos only consume 6.5TB, which easily fits on a single HDD for archiving. I just don't see a point to the RAID 5 for this purpose.

What about building full resolution previews for your entire LR catalog so you don't have to have the RAID array online whenever you search/browse LR? I have no idea how large the catalog will end up if you do that, but if storage on your MBP is a concern, you could move it to the 'spare' 1TB external SSD partition you currently have available. This could be a stopgap while you continue to use the RAID until you figure out whether or not you will replace it.



Jun 19, 2023 at 01:43 PM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #18 · p.1 #18 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


My admittedly naive/limited read was that RAID4 was the RAID5 equivalent but for SSDs. I thought RAID 4 was preferable for those.

Here is a real-world view with iStat Menus showing you the speed of accessing the RAID5 while scrolling through images.




Jun 19, 2023 at 04:44 PM
rscheffler
Offline
• • • • • •
Upload & Sell: Off
p.1 #19 · p.1 #19 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


Where is your LR catalog currently located? If there are scrolling lags and delays waiting for thumbnails to load, I would think it's more database related than because of the original files residing on the RAID. My understanding of LR is that it only accesses the original files if required to build previews, or larger previews than already exist in the database. You could consider building Smart Previews for everything so that you can do virtually everything offline and sync changes back to the original files when you bring the RAID online. Smart Previews are only 2MP, so you're not going to be able to do critical detail evaluation.

With your system the fastest location for the LR catalog should be the MBP's internal SSD. Anything external, even over TB, will be slower. But if the catalog is huge, it might be worth testing it on your external SSD to see if there is any difference. IMO you definitely don't want the catalog on anything HDD based.



Jun 19, 2023 at 08:17 PM
eyal
Offline
• • •
Upload & Sell: Off
p.1 #20 · p.1 #20 · SSD vs NVMe and RAID4 vs JBOD config for LRc photo storage with M1 MacBook Pro?


rscheffler wrote:
Where is your LR catalog currently located? If there are scrolling lags and delays waiting for thumbnails to load, I would think it's more database related than because of the original files residing on the RAID. My understanding of LR is that it only accesses the original files if required to build previews, or larger previews than already exist in the database. You could consider building Smart Previews for everything so that you can do virtually everything offline and sync changes back to the original files when you bring the RAID online. Smart Previews are only 2MP, so you're not
...Show more

Hey. Catalog is on the internal MacBook drive. It is 2.4 GB in size with the previews at 23 GB currently.

That scroll included old images which should not have any previews loaded which is what I think I'm seeing - LR Catalog 'knows' about the images but doesn't have screen renders for thumbnails so is pulling the image from the RAID5 to generate a quick thumbnail and, as you watch, refines it using the sidecar info.

At least I think that's what is going on.

I will add that I got my MacBook when my iMac was rapidly dying so settled for an in-store model with only 16gb RAM and plan to update that to a proper 64gb with the M3 model but not sure low ram would cause this. I could be wrong.



Jun 20, 2023 at 05:12 AM
       2       3       4       end






FM Forums | Post-processing & Printing | Join Upload & Sell

       2       3       4       end
    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.