Home · Register · Join Upload & Sell

Moderated by: Fred Miranda

New fredmiranda.com Mobile Site

New Feature: SMS Notification alert

New Feature: Buy & Sell Watchlist

FM Forums | Post-processing & Printing | Join Upload & Sell

All SSD NAS storage?

oguruma
Offline
• • •
Upload & Sell: On

p.1 #1 · p.1 #1 · All SSD NAS storage?

I have a TrueNAS box with 8 8TB drives in RAIDZ2 (48TB useable). The drives are getting a bit long in the tooth, and will likely start failing soon. The box has enough PCIe lanes to support 8+ NVMe drives.

8TB HDDs are about $150-180ish.

7.68TB Kioxia CM6 drives are $1000ish.

So, replacing the 8 drives with spinning hard drives would be about $1200ish, and replacing with enterprise-grade NVMe SSDs would be $8000ish (though I could probably be safe enough with only using RAIDZ1 with the SSD drives).

The upfront cost would be substantial, but the drives are significantly faster, and since the house is wired with short runs of CAT6, I actually could get 10Gbps to/from the NAS. The SSDs also draw less power, generate less heat (I think) and generate virtually zero noise.

Has anybody else switched to SSDs for NAS storage?

Jul 27, 2023 at 11:31 AM

chrisjhood
Offline
•
Upload & Sell: Off

p.1 #2 · p.1 #2 · All SSD NAS storage?

I haven’t but I’ve been looking into options. So I’m very interested should you do this to hear results. Currently have a 2019 Mac Pro with internal raid and external raid. Recently upgraded to a 12 bay Synology filled with 18tb dives. Has 10gbe so speeds are better but not super fast to work off of all the time. Debating hard on the new mac studio or Mac Pro obviously the main diff would be able to do an internal ssd raid like the card from OWC. My Lightroom catalog is a couple TB could be shrunk down if I discarded some previews but still have some editing to get done. Have had an issue with my current Mac Pro and trying to edit off 2 monitors since I bought the machine. There’s massive lag moving sliders and adjustments being applied once I turn on the secondary display. Turn it off and it works great.

Realistically I need 8TB internal and another 8TB preferably more to work from and obviously speed is the most important factor. Obviously the fastest is gonna be the internal card in the new Mac Pro, same card in an external enclosure or an external thunderbolt ssd or a network ssd raid.

oguruma wrote:
I have a TrueNAS box with 8 8TB drives in RAIDZ2 (48TB useable). The drives are getting a bit long in the tooth, and will likely start failing soon. The box has enough PCIe lanes to support 8+ NVMe drives.

8TB HDDs are about $150-180ish.

7.68TB Kioxia CM6 drives are $1000ish.

So, replacing the 8 drives with spinning hard drives would be about $1200ish, and replacing with enterprise-grade NVMe SSDs would be $8000ish (though I could probably be safe enough with only using RAIDZ1 with the SSD drives).

The upfront cost would be substantial, but the drives are significantly faster, and...Show more →

Jul 28, 2023 at 12:30 PM

gordec
Offline
• • •
Upload & Sell: On

p.1 #3 · p.1 #3 · All SSD NAS storage?

I assume you have a 10G card for you desktop and your TrueNas box as a 10G otherwise, you won’t be getting that kind of speed. Honestly, I use Synology DS920 with Western Digital reds. Even with larger Q2 files, it’s pretty fast loading previews etc.

Aug 05, 2023 at 11:36 AM

oguruma
Offline
• • •
Upload & Sell: On

p.1 #4 · p.1 #4 · All SSD NAS storage?

gordec wrote:
I assume you have a 10G card for you desktop and your TrueNas box as a 10G otherwise, you won’t be getting that kind of speed. Honestly, I use Synology DS920 with Western Digital reds. Even with larger Q2 files, it’s pretty fast loading previews etc.

Yepp. I have a few 10Gbps NICs in most of the workstations and the NAS and the main workstation have 25Gbps NICs.

What I'm really after is:

1) Fast writes to the NAS - ideally as fast as they can be read from the card.
2) Faster loading of metadata for browsing the files with SMB

I could solve 2) if I added some SSD drives to store the metadata. ZFS has a feature in which you can store the metadata on faster SSD drives, and the files themselves on spinning hard drives, but it doesn't do anything to increase write performance.

I could theoretically get pretty close to 10Gbps with spinning hard drives if I added multiple pools, but that would equate to a lot more power consumption and some more noise. It would be cheaper up front, but it might actually cost more in the long run as th drives start to die.

Aug 05, 2023 at 01:29 PM

EB-1
Offline
• • • • • • •
Upload & Sell: Off

p.1 #5 · p.1 #5 · All SSD NAS storage?

oguruma wrote:
I have a TrueNAS box with 8 8TB drives in RAIDZ2 (48TB useable). The drives are getting a bit long in the tooth, and will likely start failing soon. The box has enough PCIe lanes to support 8+ NVMe drives.

8TB HDDs are about $150-180ish.

7.68TB Kioxia CM6 drives are $1000ish.

So, replacing the 8 drives with spinning hard drives would be about $1200ish, and replacing with enterprise-grade NVMe SSDs would be $8000ish (though I could probably be safe enough with only using RAIDZ1 with the SSD drives).

The upfront cost would be substantial, but the drives are significantly faster, and...Show more →

I just use SSDs internally and have all of my NAS as secondary storage and for rarely accessed files.
You don't get anywhere near the capability of SSDs with SMB over 10GbE. You will also need a good CPU in your NAS for Z2 , but a TrueNAS box probably does. Are you not already saturating a single 10GbE link with the 8x in Z2? Will you be happy with ~41TB of storage in the future (though I would be fine with Z1)? Do you have another NAS or some good full backup when the SSD NAS fails or is corrupted?

I'm planning to eventually trash all my SSD arrays and get a few of the 30TB NVMe 4x4 TLC ION SSDs (~$2700) to simplify my life. That won't help you if it is necessary to have 10GbE everywhere, but my NAS are fast enough for that limited use.
I always have two NAS locally and some in other locations.

EBH

Aug 05, 2023 at 03:12 PM

johnvanr
Offline
• • • • •
Upload & Sell: On

p.1 #6 · p.1 #6 · All SSD NAS storage?

I’m no expert at all, but someone who claimed to be told me a while back that it’s not smart to only have SSDs because unlike spinning drives they can’t be restored as easily in case of problems.

As I said, I have no clue, but he sounded convincing enough for me to automatically back up my main external SSD to a regular hard drive.

Aug 05, 2023 at 03:24 PM

oguruma
Offline
• • •
Upload & Sell: On

p.1 #7 · p.1 #7 · All SSD NAS storage?

EB-1 wrote:
I just use SSDs internally and have all of my NAS as secondary storage and for rarely accessed files.
You don't get anywhere near the capability of SSDs with SMB over 10GbE. You will also need a good CPU in your NAS for Z2 , but a TrueNAS box probably does. Are you not already saturating a single 10GbE link with the 8x in Z2? Will you be happy with ~41TB of storage in the future (though I would be fine with Z1)? Do you have another NAS or some good full backup when the SSD NAS fails or is...Show more →

My 8 drives are in a single VDEV. With ZFS, the read/write performance is a function of the number of vdevs, not the number of total disks. So a single vdev will perform as the slowest disk in the vdev, regardless of how many disks there are.

For me with a single vdev, that means I could theoretically get the same performance of as a single drive, which would be 250ishMBps (2Gbps).

Now, if I added 3 additional vdevs, I could theoretically arrive at 8Gbps. (2Gbps x 4 vdevs). But, that would mean a lot more spinning rust, and thus a lot more power consumption, heat, and noise.

I have a secondary NAS with fewer, larger drives that serve as a backup. I also backup important stuff to Amazon S3, and to another NAS across town at my office.

Aug 05, 2023 at 03:55 PM

skid00skid00
Offline
• • •
Upload & Sell: Off

p.1 #8 · p.1 #8 · All SSD NAS storage?

I use HDD and SSD for backups. Some of my SSD's have a MUCH lower thruput than the HDD's. Backups take longer.

Be sure to check reviews that test long runs.

Aug 06, 2023 at 12:10 AM

EB-1
Offline
• • • • • • •
Upload & Sell: Off

p.1 #9 · p.1 #9 · All SSD NAS storage?

EB-1 wrote:
I just use SSDs internally and have all of my NAS as secondary storage and for rarely accessed files.
You don't get anywhere near the capability of SSDs with SMB over 10GbE. You will also need a good CPU in your NAS for Z2 , but a TrueNAS box probably does. Are you not already saturating a single 10GbE link with the 8x in Z2? Will you be happy with ~41TB of storage in the future (though I would be fine with Z1)? Do you have another NAS or some good full backup when the SSD NAS fails or is...Show more →
oguruma wrote:
My 8 drives are in a single VDEV. With ZFS, the read/write performance is a function of the number of vdevs, not the number of total disks. So a single vdev will perform as the slowest disk in the vdev, regardless of how many disks there are.

For me with a single vdev, that means I could theoretically get the same performance of as a single drive, which would be 250ishMBps (2Gbps).

Now, if I added 3 additional vdevs, I could theoretically arrive at 8Gbps. (2Gbps x 4 vdevs). But, that would mean a lot more spinning rust, and thus a...Show more →

It sounds like a plan, but not really performance oriented considering all those fast SSDs. https://americas.kioxia.com/en-us/business/ssd/oem/hpe/cm6-enterprise-nvme.html

EBH

Aug 06, 2023 at 06:48 AM

jeffbuzz
Offline
• • • •
Upload & Sell: On

p.1 #10 · p.1 #10 · All SSD NAS storage?

We had a discussion on this topic a few months back. I ended up building a newer spinning disk NAS because I simply could not justify the cost of using SSD's for the storage size I need.

Frankly, my WLAN speed maxes out at ~500Mbps so I'd need to address that bottleneck to get any appreciable benefit from faster NAS disks. If you can take advantage of real 10Gbps LAN speeds, you could see some obvious gains using SSD NAS.

When I was shopping, I didn't find any appealing personal sized NAS devices specific to M.2/NVMe. Synology had one for 2.5" disks that you could load with adapted M.2 drives. Purpose built all flash devices were all rack mount units. Now the QNAP TS-h1290FX and Asus 6 and 12 bay are both desktop(-ish) M.2 options.

Even with 12x 4TB SSD's, you'll only get 40TB of storage at your current level of redundancy.

Edited on Aug 08, 2023 at 07:41 PM · View previous versions

Aug 07, 2023 at 06:12 PM

EB-1
Offline
• • • • • • •
Upload & Sell: Off

p.1 #11 · p.1 #11 · All SSD NAS storage?

The problem with the ASUS is that it ony supports 2.5GbE.

The CPU is wimpy so I doubt it will perform well for other than RAID 0, 1, 10. (Your QNAP link is wrong, but we know what you mean.)

EBH

Aug 08, 2023 at 04:58 PM

jeffbuzz
Offline
• • • •
Upload & Sell: On

p.1 #12 · p.1 #12 · All SSD NAS storage?

EB-1 wrote:
The problem with the ASUS is that it ony supports 2.5GbE.

The CPU is wimpy so I doubt it will perform well for other than RAID 0, 1, 10. (Your QNAP link is wrong, but we know what you mean.)

EBH

The 12 bay Asus FS6712X is 10GbE. The smaller 6 bay unit is 2.5GbE. He'd need the 12 bay unit to get anywhere near the storage he was looking for.

Asustor Flashstor 12 10GbE = $800
Sabrent 8GB NVMe 4 $1000 (x8) = $8,000 (ouch)
$8,800 total for same 48TB SHR-2/RAID6 setup

Asus Lockerstor 8 10GbE = $1,100
WD Red 12TB HDD $215 (x6) = $1290 -or- WD Red 8TB HDD $150 (x8) = $1200
~$2,300 total for 48TB RAID6 spinners

So ~3.5x price differential for SSD vs HDD

Aug 08, 2023 at 07:48 PM

Scott Stoness
Offline
• • • • • •
Upload & Sell: On

p.1 #13 · p.1 #13 · All SSD NAS storage?

oguruma wrote:
I have a TrueNAS box with 8 8TB drives in RAIDZ2 (48TB useable). The drives are getting a bit long in the tooth, and will likely start failing soon. The box has enough PCIe lanes to support 8+ NVMe drives.

8TB HDDs are about $150-180ish.

7.68TB Kioxia CM6 drives are $1000ish.

So, replacing the 8 drives with spinning hard drives would be about $1200ish, and replacing with enterprise-grade NVMe SSDs would be $8000ish (though I could probably be safe enough with only using RAIDZ1 with the SSD drives).

The upfront cost would be substantial, but the drives are significantly faster, and...Show more →

I know I am late to this discussion but:
- You can buy 20TB HDD spinning drives now for about $250usd
- the fastest practical interface has a theoretical speed of 5,000mb / s - thunderbolt 4 with 40gb/s
- the fastest wireless (NAS) interface has a theoretical limit of 10gbs implying 1250mb/s
- the fastest spinning HDD can achieve about 200mb/s
- the fastest HDD raid 0 can achieve about 500mb/s
- the fastest 2.5" SSD is about is about 500mb/s (4tb is about $250usd)
- the fastest raid 0 2.5" SSD is likely close to the 1250mb/s range
- the fastest reaonable priced nvme (eg Kingston kc3000 4tb $350usd) is about 3,000mb/s exceeding the available interface
- going cheaper on NVME does not work because they typically have limited mb/s above their cache so for example my WD 1tb is fast for 70gb (1200 and then drops to 100 for remaining on a sequential write).

My point:
- Going to nvme externally will be costly and far exceed your wireless NAS capability. NAS is limited by your 10gb wireless speed to about 1,000mb/s.
- Going to 2.5" SSD,s will~double the speed (1GB/s vs 0.5gb/s), but cost you about $5000 for 48tb with redundancy.
- I think you would be better to leave the existing NAS as is and connect a separate single 4TB fast NVME NAS for faster work off NAS - it still will not be overly fast because of wireless speed limitations, but it will be 2x as fast. And save you $5000.

Aug 24, 2023 at 09:53 AM

amv8
Offline
• •
Upload & Sell: On

p.1 #14 · p.1 #14 · All SSD NAS storage?

Scott Stoness wrote:
I know I am late to this discussion but:
- You can buy 20TB HDD spinning drives now for about $250usd
- the fastest practical interface has a theoretical speed of 5,000mb / s - thunderbolt 4 with 40gb/s
- the fastest wireless (NAS) interface has a theoretical limit of 10gbs implying 1250mb/s
- the fastest spinning HDD can achieve about 200mb/s
- the fastest HDD raid 0 can achieve about 500mb/s
- the fastest 2.5" SSD is about is about 500mb/s (4tb is about $250usd)
- the fastest raid 0 2.5" SSD is likely close to the 1250mb/s range
- the fastest reaonable priced nvme (eg...Show more →

My apologies if this sounds nitpicking, but the OP's network connection is "wired" at 10 Gb/s, not "wireless." In your post, I think you are using b/s and B/s (bits per second vs. bytes per second) interchangeably which inadvertently misrepresents the throughput from these drives. For example, a 2.5 inch SSD is spec'd around 500 MB/s, not 500 Mb/s. Depending on his NAS and the rest of his network hardware he could potentially trunk his NAS interface to get faster than 10Gb/s (or switch to a non-NAS external RAID unit with a Thunderbolt 3/4 interface). The economics may still suggest a similar conclusion, but just wanted to clarify terminology to avoid confusion. Thx.

Aug 24, 2023 at 02:08 PM

EB-1
Offline
• • • • • • •
Upload & Sell: Off

p.1 #15 · p.1 #15 · All SSD NAS storage?

oguruma wrote:
My 8 drives are in a single VDEV. With ZFS, the read/write performance is a function of the number of vdevs, not the number of total disks. So a single vdev will perform as the slowest disk in the vdev, regardless of how many disks there are.

For me with a single vdev, that means I could theoretically get the same performance of as a single drive, which would be 250ishMBps (2Gbps).

Now, if I added 3 additional vdevs, I could theoretically arrive at 8Gbps. (2Gbps x 4 vdevs). But, that would mean a lot more spinning rust, and thus a...Show more →
So what did you end up doing?
I received another 12TB (to have a total of 36TB internal SSDs) but have not decided on the configurations.

EBH

Aug 24, 2023 at 09:05 PM

chrisjhood
Offline
•
Upload & Sell: Off

p.1 #16 · p.1 #16 · All SSD NAS storage?

Synology made a 6 bay slim for 2.5ssd a few years ago and you can still buy it problem is tho its only 1GBe so you don't get to take full advantage of the speed. They make some rack mounted SSD only systems, they aren't cheap and you have to use their ssd drives which are crazy expensive for the size. Wish sinology or someone would make an updated slim that works with 10GBe or a NAS that's for nvme drives. I think we are still a year or two away from really having SSD NAS options that aren't meant for enterprise.

Aug 25, 2023 at 08:51 AM

oguruma
Offline
• • •
Upload & Sell: On

p.1 #17 · p.1 #17 · All SSD NAS storage?

chrisjhood wrote:
Synology made a 6 bay slim for 2.5ssd a few years ago and you can still buy it problem is tho its only 1GBe so you don't get to take full advantage of the speed. They make some rack mounted SSD only systems, they aren't cheap and you have to use their ssd drives which are crazy expensive for the size. Wish sinology or someone would make an updated slim that works with 10GBe or a NAS that's for nvme drives. I think we are still a year or two away from really having SSD NAS options that aren't meant...Show more →

Well, actually there are currently 8TB SSDs available for about $300-350. Namely the Samsung 870QVO, which is definitely a lower-endurance consumer drive. There are also consumer-grade 2TB NVMe drives available for $150-200.

For some applications, such as long-term archival storage, the SSDs will end up with a lower cost of ownership than spinning rust. SSDs basically only "wear out" when they're written to. So if you have a use case where you're writing the data once, and just leaving it there (like with a media server, for example), consumer-grade SSDs are a good solution. Of course, if you're writing several TB of data per day, then you're going to go through the consumer-grade SSDs pretty quickly.

Whether something is "meant for enterprise" means basically nothing, the only thing that really matters is if the solution makes sense for the user, whether they're a Fortune 100 company, or a guy in his mom's basement.

A challenge with inexpensive hardware (like the cheaper Synology's), is that CPUs start to get expensive, generate more heat, and necessitate more noise, if you want things like lots of PCIe lanes (which you need if you want several NVMe drives), and faster than 1Gbps interfaces.

If you're looking for an alternative to Synology that bridges the gap between "consumer" and "enterprise" take a look at the iX Systems TrueNAS Mini. You can get a total of 7 Bays and dual 10Gbps interfaces for under $2,000.

Also, you referenced 10GBe. That might have been a typo, but if not, it's import to understand the difference between bits and bytes. There are 8 bits in a byte. A bit is represented with a lowercase "b" (as in 10Gbps), and a byte is represented with an uppercase "B" as in (500MBps). A 10GBe interface would actually be an 80 gigabit interface.

Aug 25, 2023 at 01:08 PM

chrisjhood
Offline
•
Upload & Sell: Off

p.1 #18 · p.1 #18 · All SSD NAS storage?

oguruma wrote:
Well, actually there are currently 8TB SSDs available for about $300-350. Namely the Samsung 870QVO, which is definitely a lower-endurance consumer drive. There are also consumer-grade 2TB NVMe drives available for $150-200.

For some applications, such as long-term archival storage, the SSDs will end up with a lower cost of ownership than spinning rust. SSDs basically only "wear out" when they're written to. So if you have a use case where you're writing the data once, and just leaving it there (like with a media server, for example), consumer-grade SSDs are a good solution. Of course, if you're writing several...Show more →

Sorry yes I meant 10Gbps.

Ill take a look at those systems. My use case is exactly that, data backup won't need to be written too often but accessed here and there. That could change tho if a system could offer fast enough transfer and read and write speed to function more as a external drive to work off of. But thats the problem, I have 70TB of just images not including video files or plex media. Someday we will get to the point where the cost and ability to build a big raid out of nvme will be there.

Aug 25, 2023 at 06:01 PM

GOVA
Offline
• • •
Upload & Sell: On

p.1 #19 · p.1 #19 · All SSD NAS storage?

Trim capability, file system type and SSD memory type as well as their controllers do play a role.

That SSD you specified uses TLC type, per their page, which is just okay IMO.

Now, there are plenty of TLC type SSDs. You might even look into getting a bunch of MLC drives which technically should last much longer.

Finally, there are plenty of aspects that will contribute to the speed increase you are expecting.

Sep 27, 2023 at 11:52 AM

Bruce n Philly
Offline
• • •
Upload & Sell: Off

p.1 #20 · p.1 #20 · All SSD NAS storage?

Do you have an empty bay to try out an SSD? SSDs promise great speed but I think your NAS processor will be the weak link. An empty bay with a single drive will get you the truth you need to make a good decision.

I've been using two Netgear NASs for many years now...four drives each. When a spinner dies, I plunk in a new one... I have the unit spin everything down after two hours of non-use... I believe this extends the life in a big way. I don't need the performance.... transfer of big files is not a common thing for me... when I do, I wish it was faster... but spinners are so dang cheap!

Peace
Bruce in Philly

Sep 29, 2023 at 03:35 PM


FM Forums \| Post-processing & Printing \| Join Upload & Sell

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.