I have a 4 bay Thunderbolt enclosure I bought to use as a RAID drive. The OWC SoftRaid software is terrible and the read/write speeds were so bad I abandoned the idea and use it to house 4 discs that can be accessed separately(JBOD).
NAS is basically a computer that you access its drive via your network. A couple reasons to go with a NAS:
1. Allows multiple computers to access the data on the NAS
2. You can put the NAS anywhere you want physically and just connect it to your network.
3. Data protection
RAID means it uses the disks in different fashion. For you needs you would use one of two types:
1. RAID 5 - One disk is used as a "backup" in case one of the other drives fail. eg. 4 bay NAS with 4 10TB drives would give you 30TB of storage and protection if one drive fails. The NAS will warn you when a drive is failing.
2. RAID - Two disks are used as "backup" for data. So a 4 bay NAS with 4 10TB drives would give you 20TB of storage. Data is protected from two drives failing.
Setting up a NAS is pretty simple. I have had both Synology and QNAP. Both have served me well. In my home I have a server rack in the basement. One of the items in the rack is an 8 bay NAS with eight 16TB drives. I can access those files from any computer in the house.
When you set up a NAS, you pretty much just check a box... That is it. Which RAID? RAID 5.
What it does... if you install two or three or four drives, it treats them all as one drive. If one of the drives fail, the system continues to run, you can access your files, and you order a new drive. With the system still running, you snap out the old baddie, and snap in the new. The system will automatically make it all work again although it may take a few days.
That's it.
How does RAID 5 work? It smears data and redundant data across all the drives. Prettty cool.
How do you know a drive failed if it continues to operate? When you set up your NAS, you give it your email address and it shoots you an email. Or, you fire up your browser and connect... it will have a large display of status telling you something went wrong... but doing this requires you to check. A friend of mine set up a NAS in RAID 5 at his house and lost everything... he had a drive failure and didn't know it then a 2nd drive failed and that was game over... lost all data. He told me he had not checked it in over two years... oops.
Do you need to think about other RAID options? No. If you are a geek and have some special needs, then yes, RAID options can be to your benefit, but if you are an average user who wants centralized storage, access to storage from many devices, and protection from drive failure, then a NAS with RAID 5 is it.
When you set up a NAS, you pretty much just check a box... That is it. Which RAID? RAID 5.
What it does... if you install two or three or four drives, it treats them all as one drive. If one of the drives fail, the system continues to run, you can access your files, and you order a new drive. With the system still running, you snap out the old baddie, and snap in the new. The system will automatically make it all work again although it may take a few days.
That's it.
How does RAID 5 work? It smears data and redundant data across all the drives. Prettty cool.
How do you know a drive failed if it continues to operate? When you set up your NAS, you give it your email address and it shoots you an email. Or, you fire up your browser and connect... it will have a large display of status telling you something went wrong... but doing this requires you to check. A friend of mine set up a NAS in RAID 5 at his house and lost everything... he had a drive failure and didn't know it then a 2nd drive failed and that was game over... lost all data. He told me he had not checked it in over two years... oops.
Do you need to think about other RAID options? No. If you are a geek and have some special needs, then yes, RAID options can be to your benefit, but if you are an average user who wants centralized storage, access to storage from many devices, and protection from drive failure, then a NAS with RAID 5 is it.
I think it's also important to understand how your scale raid volumes. Synology has multiple methods dealing with raid 5 and 6 creating volume pools and # of drives needed, but how you scale these pools with more storage is different. Synology has a calculator synology.com/en-us/support/RAID calculator to compare storage scaling. Buying the right size drives is important upfront, so let's say you create a raid 5 pool with 5 drives, you can only increase drives in the pool, never shrink so that drive bay is part of the pool unless. So whether you use Synology raid pools or shr pools, understand how these work but both are super easy to setup.
Bruce n Philly wrote:
How do you know a drive failed if it continues to operate? When you set up your NAS, you give it your email address and it shoots you an email. Or, you fire up your browser and connect... it will have a large display of status telling you something went wrong... but doing this requires you to check. A friend of mine set up a NAS in RAID 5 at his house and lost everything... he had a drive failure and didn't know it then a 2nd drive failed and that was game over... lost all data. He told me he had not checked it in over two years... oops. ...Show more →
He should have set it up as RAID 6! But not all hardware will support 6 if they support 5.
The lesson though, with any storage, is you need to keep on top of it. Use it, check/test it, migrate it to new HDDs/storage technology as the existing ones get long in the tooth. Once it's in a 'shoebox' and forgotten about, there's no guarantee. I have dozens of old HDDs... 120, 180, 320, 500GB, etc., in a storage case and would be surprised if some aren't dead (they've all been migrated to current larger HDDs). Same with the readability of CD-Rs and DVD-Rs from 15-25 years ago... (I did migrate all of those too...).
graham_martin wrote:
I have about 5 external hard drives totaling 10 TBs. They create a lot of clutter on my desk, and there are cables everywhere. There are probably close to 100,000 images on them. I have started looking into NAS devices, and it seems that the two big players are Qnap and Synology. One of the downsides is the initial cost. It looks like I would have to get either a 2 or 4 bay system plus 2 to 4 hard drives. This is strictly for personal hobby use.
What sort of advice can you give me?
Thanks
I was in your boat a few years ago. I actually tried out a 4 bay synology along with a QNAP and thought they were both very good. I went with synology because the software seemed less buggy, but they were both very good. Base your decision on features and perhaps if you want to login to the unit from outside your home, go with Synology as I think they update their security patches more regularly. They also have tech support staff you can reach, but I wouldn't call them knowledgeable, but at least they exist.
As for if you need a NAS, the N stands for network and you can get the identical features for less money if you if you don't mind plugging into a USB port using something like this https://www.qnap.com/en-us/product/tr-004.
These gives you all the features of a NAS for storage such as being able as the flexibility to expand using multiple drives and not lose data if one drive happens to fail (that's what RAID does) and the cost is much lower. That being said, internal drives like WD REDs are pretty expensive so saving two hundred on the bays when you still have to pay perhaps 1000 for the drives isn't perhaps that big a deal. The drives are expensive and internal drives that work with these systems cost more per TB than external USB drives.
If you do decide you don't mind slower transfer speeds over a network and like the idea of being able to access files even if you're roaming around your house on a network or outside your house, the NAS gives you the same storage capacity, with some added features. Wirecutter I think suggests a qnap and a synology that are both very good.
In addition to the drives and the unit though, I ended up buying a lot more crap. This includes...
- two external 14TB drive that backs up the NAS which I put in storage away from my home and rotate every few months
- for another 60 to 120, I probably should get a battery backup.
- a 20 dollar plug which is set to a timer which turns on once a week when I do my backup so if my system does get hacked, there's nothing physically attached to the drive.
So while I think it's strangely complicated what I have setup, I finally have a backup that works that I don't have to think about. So if you have a lot of photos and the total is well under 18tb, you could easily back that up with a one BAY NAS and an external 18tb drive for under 1000 total. If you have more data than that, you'd probably need more drives and more bays which could easily be 1500 to 2000. I think backups like this are decent value compared to a monthly service over a few years, but storage just isn't cheap. And maybe the last idea is shoot compact RAW?
graham_martin wrote:
I have about 5 external hard drives totaling 10 TBs. They create a lot of clutter on my desk, and there are cables everywhere. There are probably close to 100,000 images on them. I have started looking into NAS devices, and it seems that the two big players are Qnap and Synology. One of the downsides is the initial cost. It looks like I would have to get either a 2 or 4 bay system plus 2 to 4 hard drives. This is strictly for personal hobby use.
What sort of advice can you give me?
Thanks
I used a system like that for several years with a pair of OWC 4 bay enclosures. Had large capacity drives (6gb, 4gb), plus the USBC box had 4 early SSDs which I ran in a RAID protocol. This became my main catalog/pic file work drive. Photo collection had grown too large for the internal one. Smooth and almost as fast as the interval drive.
(Don't know if I'm breaking protocol here. If I am, let me know. That entire setup is for sale. Finally moved to new machine and external SSDs. PM if interested.)
Thanks, but in the meantime I bought an 18TB external drive and will gradually copy my important photos from me old external drives, and leave them as backups.
Glad you got it rigged up. The single disk as backup conundrum bounces me back and forth. Yes, the files are on 2 different drives… I've additionally considered some sort of 'cloud' storage but, 1) am dependent on some unknown other to make them secure, 2) it would require a monthly rental, 3) given the size of and number of my images it would take a hell of a lot bandwidth to work. I live on the backside of nowhere. We say our connection comes via a barbwire fence. I'd probably have to rent a T-1 line for it to work. So, I hold with what I've got. Honestly, the odds of double failure are minuscule and nothing can ever be 'absolutely' secure.
I try to build desktops with capacity for >=8 hard disks in the chassis and motherboard (Fractal Design has been good for this) and with ECC memory (AMD has been good for ECC)
The old desktop becomes the NAS and the new one becomes the editing machine. When something dies it’s time for a new editing machine
If you can run some cables it’s helpful to have a direct lan connection from the NAS to the editing machine. This way data transfer to the NAS does not affect your internet speed. This requires 2x lan ports on each computer
If you have SSDs on the NAS you want your lan ports to be 2.5G or 10G, but routers that support this tend to be expensive. Direct connection between 2.5G ports works well and costs nothing
Backblaze is good way to backup your entire NAS to the cloud
A NAS has other uses, for example you can run next cloud and have a Dropbox like tool.
Plex can host your images but it only displays jpegs in the resolution of the device you the streaming to
Netgear had a sale for a 5 port 2.5 GB ethernet hub for $100 (MS105)... I purchased one and it did bump the machine-to-machine transfers by quite a bit.
I just checked and the sale is over and the unit is back to $150. But keep checking...
You can add this hub to any network, you just plug one of the ethernet hub ports into one of your router's ports and all will just work. But of course, your PCs must be connected via cat5 cable back to this hub... no wireless here. And... both of your motherboards must have a 2.5gb ports. At this speed, you can share that cable/port with internet activity, the internet activity is usualy chatty and not that heavy for most users.
Oh, and you really don't need cat 6 or any of the other expensive cables as distances in your home won't call for it. I am using Cat5 cables from 10 years ago and all is good.
OntheRez wrote:
Glad you got it rigged up. The single disk as backup conundrum bounces me back and forth. Yes, the files are on 2 different drives… I've additionally considered some sort of 'cloud' storage but, 1) am dependent on some unknown other to make them secure, 2) it would require a monthly rental, 3) given the size of and number of my images it would take a hell of a lot bandwidth to work. I live on the backside of nowhere. We say our connection comes via a barbwire fence. I'd probably have to rent a T-1 line for it to work. So, I hold with what I've got. Honestly, the odds of double failure are minuscule and nothing can ever be 'absolutely' secure....Show more →
Then maybe do three separate back up HDDs and store one remotely and rotate them every so often? For cloud, consider uploading only Jpeg exports of your RAW selects. This way you at least have an image available if everything else is lost. But more likely, that you can access remotely if there is such a need and you don't have immediate access to your HDDs.
IMO dependence on some unknown other is one of the key benefits of using the cloud. That unknown other is likely a team of IT experts operating a high quality server facility with oodles of redundancy that I would not be able to remotely duplicate.
Bruce n Philly wrote:
Netgear had a sale for a 5 port 2.5 GB ethernet hub for $100 (MS105)... I purchased one and it did bump the machine-to-machine transfers by quite a bit.
I just checked and the sale is over and the unit is back to $150. But keep checking...
You can add this hub to any network, you just plug one of the ethernet hub ports into one of your router's ports and all will just work. But of course, your PCs must be connected via cat5 cable back to this hub... no wireless here. And... both of your motherboards must have a 2.5gb ports. At this speed, you can share that cable/port with internet activity, the internet activity is usualy chatty and not that heavy for most users.
Oh, and you really don't need cat 6 or any of the other expensive cables as distances in your home won't call for it. I am using Cat5 cables from 10 years ago and all is good.
You have good luck with the cables, sadly I have not had the same experience.
Some of my Cat 5a and 6a had the habit of dropping to 10/100. Ditto for the Cat 5a cables. I am now running as much cat8 as possible but the stuff in the wall is still cat 5a
Imo cat 8 cables are dirt cheap compared to the labor cost of installing it in the walls. I wouldn’t cheap out on the cables when redoing electrical
aCuria wrote:
You have good luck with the cables, sadly I have not had the same experience.
Some of my Cat 5a and 6a had the habit of dropping to 10/100. Ditto for the Cat 5a cables. I am now running as much cat8 as possible but the stuff in the wall is still cat 5a
Imo cat 8 cables are dirt cheap compared to the labor cost of installing it in the walls. I wouldn’t cheap out on the cables when redoing electrical
Yea, I guess I have been lucky. I agree that if I were to install stuff in the walls, I would do the latest/best.
I wired my last home myself with Cat5 to almost every room... put in both Cat5 and coax to a face plate. Put a wall-mounted jack panel... real pro and I was very proud of it. Did my own terminations etc.... what a royal pain fishing cables through walls... drilling through studs etc. I purchased one of those 6' long drill bits from Home Depot and hit a live electrical cable. WOW!!!! Code dictates that cables, any cables, installed with straight runs... someone coiled a few feet of electrical cable in the wall and of course, I hit it. Ended up having to cut a hole in a ceiling to do the repair. But.... once done, it is fabulous. I really don't like wireless for a ton of reasons.
rscheffler wrote:
Then maybe do three separate back up HDDs and store one remotely and rotate them every so often? For cloud, consider uploading only Jpeg exports of your RAW selects. This way you at least have an image available if everything else is lost. But more likely, that you can access remotely if there is such a need and you don't have immediate access to your HDDs.
IMO dependence on some unknown other is one of the key benefits of using the cloud. That unknown other is likely a team of IT experts operating a high quality server facility with oodles of redundancy that I would not be able to remotely duplicate....Show more →
Using an old PC as a server is a great idea. I kinda do that myself although I already had a NAS. Issue is when you want to take spinner drives offsite or put on a shelf.... you need to have the drive external to the machine. For that, I have one of those two-drive caddies where you just snap in a drive. Performance is not tops, but it works and frankly, does not need to be fast.
Bruce n Philly wrote:
Yea, I guess I have been lucky. I agree that if I were to install stuff in the walls, I would do the latest/best.
I wired my last home myself with Cat5 to almost every room... put in both Cat5 and coax to a face plate. Put a wall-mounted jack panel... real pro and I was very proud of it. Did my own terminations etc.... what a royal pain fishing cables through walls... drilling through studs etc. I purchased one of those 6' long drill bits from Home Depot and hit a live electrical cable. WOW!!!! Code dictates that cables, any cables, installed with straight runs... someone coiled a few feet of electrical cable in the wall and of course, I hit it. Ended up having to cut a hole in a ceiling to do the repair. But.... once done, it is fabulous. I really don't like wireless for a ton of reasons.
Yeah have the same setup, Ethernet to every room and an access point or router in access point mode in every room to provide WiFi and more lan ports
This is the correct way to do it, WiFi doesn’t pass through walls
The trouble I had is that it’s too easy to saturate a 1G link… if I had infinite resources I would run optical or multiple cat 8 links to the room with my desktop
graham_martin wrote:
I have about 5 external hard drives totaling 10 TBs. They create a lot of clutter on my desk, and there are cables everywhere. There are probably close to 100,000 images on them. I have started looking into NAS devices, and it seems that the two big players are Qnap and Synology. One of the downsides is the initial cost. It looks like I would have to get either a 2 or 4 bay system plus 2 to 4 hard drives. This is strictly for personal hobby use.
What sort of advice can you give me?
Thanks
A NAS is not for you because you are clearly using a single computer at a single location (which is your desk).
As you are dealing with many external individual HDDs it would seem that top performance is also not a primary requirement.
However, a DAS (Direct Accessed Storage) could be useful because the connection interface will be faster than a Gbps LAN. That could be thunderbolt 3 or 4 or USB 4. The big downside is considerable cost just for the case, let alone the drives to go in it. Another is that an always-connected box full of drives is just as exposed as your computer is to anything that could go wrong. i.e. you get more performance but no more security.
You'll need backups too, and your existing HDDs will become a single backup. You should have more than one backup.
I would normally suggest many-HDD RAID 6 cases for speedy HDD storage, or a not-so-many-SSD RAID 5 or 6 for even faster storage, but a single case failure could trash the lot and you probably cannot afford two or three.
If top speed is not the ultimate requirement, then a number of 4TB or 2TB SATA SSDs will max out the SATA interface of your computer for big-file transfers and greatly enhance small-file response times compared with HDDs. This makes them seem faster than they really are. They can be inside your computer or in external case(s). They're cheaper and more portable than the very speedy NVMe M.2 SSDs that are inside modern high performance PCs.
Also get a USB 3 open-top SATA III drive cloning dock to connect external drives for backing up your SSD content, and for cloning any external SATA drive to another at any time, even while not connected to the PC. Great for overnight backups of backups. Cloning is done sector by sector instead of file by file. The time taken depends on the capacity of the drives rather than total size of the files, because every sector is copied without involving the computer operating system.
You'll need to label your loose HDDs so that you know which ones are backups and which ones are primary storage. I date mine too. With the aid of some appropriate software such as Macrium Reflect you can even clone your system drive to an external drive and be able to boot from it.
NAS is great if you have many devices, I can get the files from PCs, Mac, laptops, iPad, phone ....
I have a 8 Bay NAS (with 4x 1Gbe and added extra 2x 10Gbe ports, directly connected to 10Gbe ports of PC and Mac Studio without a 10gbe hub), 2 x RAID10 Disk groups so if I want extra redundancy I can place files into another group. I still have lots of hard disk (from 3TB to 18TB and I keep the older one as I upgrade the drives) with several USB-SATA docks.
Personally I don't trust RAID5. It takes quite some time to rebuild a RAID5 (if you drives are like 18TB) if you need to replace a drive, count to the fact that RAID5 allow only 1 disk fail, I'd really worry that another drive might fail during that time.
My issues with DAS is I can't guarantee I can get the data in RAID if the "box" failed and the company is gone.
I *THINK* some QNAP models offer NAS with TB3 connections but they are quite expensive.
WIth most NAS, they are essentially a Linux box so it's not difficult to get the data inside with computer.
Recently I have been thinking about getting a low-power, mini-pc with 2(or more) NVME slots and at least 2.5gbe LAN to build a portable and silent SSD based NAS.
johnvanr wrote:
I just have two external HDDs and they’re copies. The original HD also gets backed up in the cloud via BackBlaze. Seems simpler than RAID.
Similar here. I have this QNAP TR-002 which take 2 drives, copy of each other. I also have separate USB drives and then backed online. I only have 1 work computer for photo editing so no need for my NAS. That's why I prefer DAS like the TR002. Simple, cheap at $150, USB3.2.
ytwong wrote:
...My issues with DAS is I can't guarantee I can get the data in RAID if the "box" failed and the company is gone...
That's certainly a fair concern, which I share, and is partly why I have my data backed up to other devices as well. The risk and cost are balanced somewhat by the speed boost I routinely get by having enough drives striped in the RAID, and by the likelihood that the my data will still be on my PC and/or other backups if the RAID is unavailable. A key point worth adding is that a single DAS or NAS should never be the sole backup device, because it could be destroyed by whatever destroys the PC it is connected to at the same time.
Another advantage of having a relatively large capacity DAS is the potential for it to hold multi-generational backups. Only one is ever current, but the older backups may have a valid version of files that had been corrupted quietly without me realising it at the time.
...Recently I have been thinking about getting a low-power, mini-pc with 2(or more) NVME slots and at least 2.5gbe LAN to build a portable and silent SSD based NAS.
I considered that too but the link speed was a drawback. So was having another copy of Windows or Mac OS running, because deep down I just know from experience that like software bugs and hardware glitches they're all out to get me and have been for decades
Not so long ago a 2.5Gbps LAN speed would have been the stuff of dreams, and even more so with 10Gbps. However, these days it is still slow enough to seemingly put the brakes on transfers to and from a computer system like mine that is based on speedy NVMe M.2 SSDs. 10Gbps is half of what a single SSD is capable of. And less than half for PCIe 4/5 systems. Sure, it may be tolerable but apart from price nothing causes people to avoid making backups quite like "slow" speed does.
Consider too that a speedy system allows quicker subsequent routine verification of backup file integrity by testing stored md5 checksums. The quicker such tests are completed the quicker the backup can be taken off-line again to keep it safe.
To be honest, recovering from any major system failure is a major hassle no matter how fast the gear is, but not having backups is potentially disastrous. Anything that deters us from making backups will eventually contribute to that disaster. If you can afford it then speedy is generally better because you will be more likely use it more often. Slower is always better than nothing if it is all you can afford.
My DAS has six drives, with two being redundant (RAID 6). Full recovery from a DAS failure would take me several days plus however long it takes to diagnose the problem and fix it or buy and receive a replacement, but at least it is feasible. To have lost all of my backups I'd probably also be needing a new home too. Short of that, I would initially be making fresh slower backups to other backup devices that are not as fast as the RAID 6 but will get the job done.
But not all hardware will support 6 if they support 5.