Home · Register · Search · View Winners · Software · Hosting · Software · Join Upload & Sell

Moderated by: Fred Miranda
Username   Password

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | General Gear-talk | Join Upload & Sell

1
       2       end
  

Dealing with storage issues and a growing library.
  
 
whiskey tango
Offline
• •
Upload & Sell: On
p.1 #1 · p.1 #1 · Dealing with storage issues and a growing library.


Currently I'm on an i3 iMac 3.2 with 16GB of RAM (mid 2010 model). It has a 1TB HD that's almost slap full. I use Aperture 3 for my photo management software and over 700BG of my HD space is my A3 library. About a year or so ago I moved my library off of an external hard drive onto my main hard drive because the USB 2.0 transfer speed for editing photos was horrible. Additionally, I found that accessing the external hard drive was murdering RAM.

I also have an i7 MBP with a 500GB HD and 8GB RAM. I use this for tethering, traveling, editing in the field, etc.

So, here's where I'm at: I've just about pegged out my i3 machine. It does me well, but my library size is forcing me out of this computer. I want to be able to have all of my photos in one library and not have to close out A3 and re-open another library somewhere on another peripheral. I've considered getting a faster external hard drive, but I only have FW800 on the iMac. Why should I spend the money on old technology when there is much better available, and I have the capability on my MBP. Which brings me to my next point- Thunderbolt. I have Thunderbolt on the MBP but not on the iMac. What I'm considering is getting rid of the iMac and purchasing an external Thunderbolt display and daisy chaining a larger Thunderbolt external hard drive to it to operate off of as far as photo library access goes. If I had to travel with my laptop I'd just unplug the Thunderbolt cable and be on my way while the photo library stays at home.

I'm sure someone else out there has been in this situation and I'd love to hear your opinions and thoughts as I'm quickly approaching a decision point. How stable is your photo management program while accessing through Thunderbolt? I think it goes without saying that I don't want to go and spend 4k on a new hopped up iMac. I'm trying to keep cash out of pocket to a minimum and I'm prepared to part ways with my iMac to help fund a display/ hard drive purchase.

Thanks for your input!



Oct 09, 2013 at 01:46 AM
Hardcore
Offline
• • •
Upload & Sell: On
p.1 #2 · p.1 #2 · Dealing with storage issues and a growing library.


Thunderbolt external drives are very pricey. I looked into one and decided against it. I now back up all my photos on a synology NAS box. This stores all my photos in a RAID array and allows me to keep them in my lightroom library for later access. The nas box is amazing and functions as more than a backup. I have an Itunes server running off it, along with unlimited cloud backup for mine and my wifes computer. As well, I have a security camera that works with it and I can access the live feed through my phone to check on my house.

These are just a few of the things you can do other than have a solid backup system for your photos!



Oct 09, 2013 at 02:30 AM
whiskey tango
Offline
• •
Upload & Sell: On
p.1 #3 · p.1 #3 · Dealing with storage issues and a growing library.


Backing up is not my issue. I can plug in a USB 2.0 external hard drive for that. My problem is storing my library on an external device and accessing that library where it's speed is not diminished. I need to store that library on my operating hard drive in order to access that photo and be able to edit it with a pretty high degree of efficiency. Are you editing on a library that you've got stored on a NAS, or are you just using the NAS for a backup?


Oct 09, 2013 at 02:53 AM
Hawkan
Offline
• •
Upload & Sell: Off
p.1 #4 · p.1 #4 · Dealing with storage issues and a growing library.


whiskey tango wrote:
Backing up is not my issue. I can plug in a USB 2.0 external hard drive for that.

What happens if there is a fire or someone breaks into your house?

whiskey tango wrote:
My problem is storing my library on an external device and accessing that library where it's speed is not diminished.

Is it critical to have all of your stuff locally? Then you're looking at a workstation and file server in one. It's going to be big, noisy and - if you want a current model - not running Mac OS X. I use two laptops and an iMac. All of these clients back up to a little cheap headless file server in my closet. The file server has mirrored drives for local redundancy, but I don't have to keep the noisy disks on my desk as I work.

whiskey tango wrote:
I need to store that library on my operating hard drive in order to access that photo and be able to edit it with a pretty high degree of efficiency.

Are you pulling out files from storage for editing so often you need to keep all files local? I keep files from the latest twelve months or so on my laptops and desktop. Everything older then that gets purged - the files are on the file server (and backed up to Crashplan) anyhow so I don't need to worry about that part. If I want to edit an older set of files it's easy to pull it from the file server for editing.

If you want to have quick NAS access, buy a quality NAS that supports NFS mounts. Your Macs have NFS support built in. Do not use SMB (Windows type) shares, those are inherently slower but most NAS units will default to using SMB. Use 1Gbit Ethernet or faster and forget about wireless.



Oct 09, 2013 at 10:35 AM
Paul Mo
Offline
• • • •
Upload & Sell: Off
p.1 #5 · p.1 #5 · Dealing with storage issues and a growing library.


whiskey tango wrote:
Currently I'm on an i3 iMac 3.2 with 16GB of RAM (mid 2010 model). It has a 1TB HD that's almost slap full. I use Aperture 3 for my photo management software and over 700BG of my HD space is my A3 library.

It does me well, but my library size is forcing me out of this computer. I want to be able to have all of my photos in one library and not have to close out A3 and re-open another library somewhere on another peripheral.


This is, IMHO, so completely wrong.

You obviously want some advice so here's a little.

An i3 imac with a 1TB drive is going to be pretty awful for editing anyway. Add to that the fact that you are filling up the drive with images and it's a recipe for dull computing.

I would consider ditching the iMac for anything photo related - use it for surfing the 'net and word processing - and using the i7 MBP with images on external drives for Aperture to work from.

If you want to view your images on the iMac and to continue using it then you HAVE to move your images onto external drives and get rid of all the clutter on that slow ass 1TB internal drive.

Look into digital asset management. What capacity drives do you have to buy and how many? Do you have USB 3? Are you prepared to back up to the cloud?

For starters.



Oct 09, 2013 at 10:53 AM
mhayes5254
Offline
• • •
Upload & Sell: On
p.1 #6 · p.1 #6 · Dealing with storage issues and a growing library.


Your basic idea of using the MBP sounds fine but does your MBP have USB 3? If yes go with that for the external and just get a monitor. If you only have thunderbolt, it is a harder choice, due to the price.


Oct 09, 2013 at 11:16 AM
jimmy462
Offline
• • •
Upload & Sell: Off
p.1 #7 · p.1 #7 · Dealing with storage issues and a growing library.


Hi whiskey tango,

Just a few thoughts for you...IMHO, for desktop solutions for image editing, anything in the i7 lineup from Sandy Bridge (2600k) forward should more than suffice for anyone's needs. As nice as a maxxed-out new iMac sounds (and I'm not discouraging anyone from going that route) methinks that looking at either the late-2011 i7-2600k or late-2012 i7-3770k iMac models as alternatives should not be overlooked. What you'll reap in savings over buying shiny-new should help fund either a nice RAID or NAS setup for your needs. Oh, and 16GB RAM is what I consider "minimum requirements" these days.

When it comes time for me to purchase new computer gear I usually wait on the arrival of the latest models and then go shopping for the deals on the previous-generation closeout models or refurbs. A recent exception to that rule was my purchase of a brandie-new 2011 iMac which I jumped on for no other reason than to have a Thunderbolt port...well that port is still vacant as I have yet to decide on my own RAID and/or NAS solutions!

Anyhoo, however you decide to go you will notice a huge improvement with your external storage responsiveness once you migrate up from that straw-thin bottleneck you've got going on with that USB 2.0 setup! As for FW800, it's what I currently use for my externals and it certainly suffices...is it eSATA or Thunderbolt fast? Heck, no. But at this point I can't justify the expense of going Thunderbolt, that'll have to be next years budget...maybe.

Best of luck with your research...keep us posted as to how you decided!


Jimmy G



Oct 09, 2013 at 02:09 PM
whiskey tango
Offline
• •
Upload & Sell: On
p.1 #8 · p.1 #8 · Dealing with storage issues and a growing library.


Thanks for the responses guys! I guess I should address a few things:

1- I have a local redundant backup via USB2.0 as well as an offsite backup. It's USB2.0 because I'm not concerned with my backup speed. My storage there is free because those hard drives were already paid for!

2- My MBP has thunderbolt and USB 3.0.

3- I'm not sure a NAS is going to work for me as I'm looking for something to store my library on and operate off of. I don't want to store files elsewhere, have to import or switch libraries, I want to have them all in one place.

So, to that, I ask... Is anyone using LR or A3 on a MBP or iMac and storing their library on an external drive? If so, how is the responsiveness? When I put my library on my internal drive of the iMac, I thought I turbo boosted the speed. From the sound of it, I'll have much better responsiveness on an external drive with thunderbolt or USB3.0. I'm leaning more towards the Thunderbolt just because of the speed difference.



Oct 09, 2013 at 11:04 PM
Paul Mo
Offline
• • • •
Upload & Sell: Off
p.1 #9 · p.1 #9 · Dealing with storage issues and a growing library.


whiskey tango wrote:
Is anyone using LR or A3 on a MBP or iMac and storing their library on an external drive? If so, how is the responsiveness?

I'm leaning more towards the Thunderbolt just because of the speed difference.


If you can afford Thunderbolt you'll not notice any lag - even with USB3 you shouldn't notice any.

The trick is to find the Thunderbolt device that is right for you.

If the search becomes tedious then, IMHO, go with a quality USB3 enclosure.

You could use a:

http://www.ebay.com/itm/ORICO-2588US3-External-2-5-SATA-9-5mm-SDD-HDD-Enclosure-USB-3-0-NS1066-Chipset-/251345594466?pt=US_Drive_Enclosures_Docks&hash=item3a855d6862

With a:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822236348

Or a http://www.ebay.com/itm/ORICO-7618US3-1-Bay-3-5-SATA-HDD-External-Enclosure-with-USB-3-0-/310558290595?pt=US_Drive_Enclosures_Docks&hash=item484eb772a3

With a:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822136792

Or whatever combination suits.

Then back up your data!



Oct 10, 2013 at 01:28 AM
EB-1
Online
• • • • • • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · Dealing with storage issues and a growing library.


Paul Mo wrote:
You could use a:

http://www.ebay.com/itm/ORICO-2588US3-External-2-5-SATA-9-5mm-SDD-HDD-Enclosure-USB-3-0-NS1066-Chipset-/251345594466?pt=US_Drive_Enclosures_Docks&hash=item3a855d6862

With a:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822236348


That 2.5" drive is a 15mm format not typical for consumer use. It's more the type for DVR/surveillance or a cheap nearline storage replacement filling in for 15mm SAS drives.

EBH



Oct 10, 2013 at 01:35 AM
 

Search in Used Dept. 



Gochugogi
Offline
• • • • •
Upload & Sell: Off
p.1 #11 · p.1 #11 · Dealing with storage issues and a growing library.


I keep a huge stash of RAW images stored on a two-disk FW800 RAID on my old Mac Pro (24GB RAM with SSD system) and Aperture 3 is pretty darn fast even with 2TB of images on a 4TB array. Sure, not Thunderbolt fast but it's pretty tough for a single external HD to max out FW800. You'd need a SSD RAID to to really take advantage of the speed of Thunderbolt. Hooking up a single external HD to Thunderbolt won't be much faster than FW800.

I'd toss that stock 1TB HD and replace it with a 512GB SSD and keep your system and apps on it. Setup up a two-disk RAID of two 2 or 3 TB disks and use it for your image library. Unless you're willing to spend really big bucks for a SSD RAID, Thunderbolt ain't gonna speed things up much over FW.



Oct 10, 2013 at 05:09 AM
Paul Mo
Offline
• • • •
Upload & Sell: Off
p.1 #12 · p.1 #12 · Dealing with storage issues and a growing library.


EB-1 wrote:
That 2.5" drive is a 15mm format not typical for consumer use. It's more the type for DVR/surveillance or a cheap nearline storage replacement filling in for 15mm SAS drives.

EBH


Well spotted, thanks. I was rushing out the door. Here's an alternative - lower capacity:

http://www.amazon.com/WD-Blue-Mobile-Hard-Drive/dp/B005DVJJWQ



Oct 10, 2013 at 07:03 AM
talexander
Offline
• • •
Upload & Sell: On
p.1 #13 · p.1 #13 · Dealing with storage issues and a growing library.


I'm wondering why you feel a network access drive wouldn't work for you?

I'm currently using a Synology 2 bay system (2 TB x2 drives in Raid 1 mirroring) may upgrade to a 4 bay unit and go with 4x4 (8 GB total space mirrored) and works fine over gigabit ethernet.

Tim



Oct 12, 2013 at 12:05 AM
Hardcore
Offline
• • •
Upload & Sell: On
p.1 #14 · p.1 #14 · Dealing with storage issues and a growing library.


talexander wrote:
I'm wondering why you feel a network access drive wouldn't work for you?

I'm currently using a Synology 2 bay system (2 TB x2 drives in Raid 1 mirroring) may upgrade to a 4 bay unit and go with 4x4 (8 GB total space mirrored) and works fine over gigabit ethernet.

Tim


Agreed. I've used to use external drives to edit, citing speed for a reason. Now I'm done with it. Gigabit ethernet, anywhere in the house and practically unlimited storage with so many other benefits. Gone are the days of finding my photos buried somewhere on an external and toting them around. Wireless is an amazing technology.



Oct 12, 2013 at 12:33 AM
jimmy462
Offline
• • •
Upload & Sell: Off
p.1 #15 · p.1 #15 · Dealing with storage issues and a growing library.


whiskey tango wrote:
...

So, to that, I ask... Is anyone using LR or A3 on a MBP or iMac and storing their library on an external drive? If so, how is the responsiveness? When I put my library on my internal drive of the iMac, I thought I turbo boosted the speed. From the sound of it, I'll have much better responsiveness on an external drive with thunderbolt or USB3.0. I'm leaning more towards the Thunderbolt just because of the speed difference.


Hi whiskey tango,

I went with the 2GD HDD option on my 2011 i7 iMac, this allowed mt to partition my internal drive into 4 separate 500GB partitions. 2 of these partitions are bootable...one bootable drive section for video editing with FCP, etc. installed there, the other bootable drive section for image editing with Aperture 3, CS5.5, etc. The two other non-bootable partitions are where I have either my scratch disks located or, in the case of A3, its Library.

On the desk are several 1TB or 2TB FW800 external drives. These contain backups of the projects I am currently working on, a backup of my A3 Library, along with backups of all my original files. By having these externals I can access, say, the A3 backuo library with my 15" MBP...the Library is just a disguised storage folder, afterall. Should I need a Library subfolder or original for on-the-go editing I can just grab a copy for the MBP.

The MBP has a duplicate install of all my image-editing software which generally only finds use if I'm in the video boot on the iMac and need a quick turnaround for an image off the MBP...I've got only so many eyes and fingers!

Now, granted, some folks here will probably be looking at my setup and methods and will be giving their head a good scratch as to the inelegance of it all (and probably a good laugh), but this dual computer arrangement has been working just fine for me for the past decade or so (obviously with earlier machines). NAS and or RAID solutions have gotten me curious, and I've been promising myself that I'll be putting aside some of next years budget for just that...that is, I'm not anticipating any new shiny lenses coming along. New camera bodies the following year (yeah, we all have budgets to keep).

FWIW, the folks over at OtherWorld Computing do offer a variety of iMac upgrades suitable for your 2010 i5...extra internal HDDs, SSDs and even an eSATA port for the machine...I had considered hiring them for an SSD and an eSATA port for when the 2010 iMac was a front contender, and I haven't ruled an SSD out, yet, for what I have. I have no connection with them whatsoever, just a friendly "maybe you want to take a look"...

OWC Turnkey Program for 2010 & 2011 Apple iMac:
http://eshop.macsales.com/shop/turnkey/iMac

Anyhoo, those are my methods...however primitive they may be.

Best of luck,
Jimmy G



Oct 12, 2013 at 08:33 PM
Gochugogi
Offline
• • • • •
Upload & Sell: Off
p.1 #16 · p.1 #16 · Dealing with storage issues and a growing library.


Hardcore wrote:
Agreed. I've used to use external drives to edit, citing speed for a reason. Now I'm done with it. Gigabit ethernet, anywhere in the house and practically unlimited storage with so many other benefits. Gone are the days of finding my photos buried somewhere on an external and toting them around. Wireless is an amazing technology.


Sounds like the ideal solution for a well-heeled dude in a big house with a fleet of computers. Maybe not so much for Joe Average in a one bedroom condo with an iMac.

I think Jimmy's ideal of adding an esata port to the iMac is a very cost effective method of adding fast storage.



Oct 12, 2013 at 11:22 PM
whiskey tango
Offline
• •
Upload & Sell: On
p.1 #17 · p.1 #17 · Dealing with storage issues and a growing library.


So a lot of good options I've heard, but it doesn't sound liken anyone is operating under the method that I'm considering.

While upgrading the i3 iMac would be financially the easiest choice, I'm not sure if it's worth the funds in the long run investing in a rather slow machine.

Network storage sounds interesting, though I've never considered it as an option. I guess it's something outside of my comfort zone. Maybe I'll explore that as an option. I think at the end of the day, I'm going to make the MBP my main editing machine, sell my iMac to fund a cinema display, and then decide on storage options.

Thanks for all the options guys, it's generating some good ideas.



Oct 13, 2013 at 02:06 PM
secondclaw
Offline
• •
Upload & Sell: On
p.1 #18 · p.1 #18 · Dealing with storage issues and a growing library.


I'd second the Synology NAS option. I'm currently on a mirrored 2TB setup, gigabit internet, and manage library directly over the network. I am thinking of placing Lightroom's catalog on the NAS as well, so i can access it remotely if needed. Will try that one day.

Access is definitely slower than a local drive, but its tolerable.



Oct 20, 2013 at 09:33 PM
wilt
Offline
• • •
Upload & Sell: Off
p.1 #19 · p.1 #19 · Dealing with storage issues and a growing library.


whiskey tango wrote:
Thanks for the responses guys! I guess I should address a few things:

1- I have a local redundant backup via USB2.0 as well as an offsite backup. It's USB2.0 because I'm not concerned with my backup speed. My storage there is free because those hard drives were already paid for!

2- My MBP has thunderbolt and USB 3.0.

3- I'm not sure a NAS is going to work for me as I'm looking for something to store my library on and operate off of. I don't want to store files elsewhere, have to import or switch libraries, I want
...Show more


I think you are confusing the issue of two libraries vs. one library but files in two different drives.
Your program should be able to support a single Library, but within that one Library the pointers tell it to go to two different locations for the actual RAW files.
Using that approach with Lightroom on a Windows7 PC, I have files on the internal harddrive of the PC for the current year, while files are on my network access storage (NAS) for anything from prior years. But the data for mods to the RAW files are stored within the Lightroom catalog file, which is always resident on the internal harddrive, even for RAW files which are actually on the NAS.
One Library, one catalog file, yet RAW files in two different harddrives (one of them a NAS, or could be a USB-connected harddrive)



Oct 26, 2013 at 05:09 PM
arbitrage
Offline
• • • • •
Upload & Sell: On
p.1 #20 · p.1 #20 · Dealing with storage issues and a growing library.


whiskey tango wrote:
Thanks for the responses guys! I guess I should address a few things:

1- I have a local redundant backup via USB2.0 as well as an offsite backup. It's USB2.0 because I'm not concerned with my backup speed. My storage there is free because those hard drives were already paid for!

2- My MBP has thunderbolt and USB 3.0.

3- I'm not sure a NAS is going to work for me as I'm looking for something to store my library on and operate off of. I don't want to store files elsewhere, have to import or switch libraries, I want
...Show more

I use an external USB3 hard drive to store my Raw files and use that with my RetinaMacbookPro and LR5 to edit from. I find the access speeds to be acceptable, I usually have LR generate 1:1 previews on import so going through a new import, I will actually be accessing those previews from the internal SSD of the RMBP but once I'm editing it is accessing the external HD and the speed is fine.

As far as considering Thunderbolt vs USB3, there is no difference in speed unless your external drive is an SSD. Even then, the speed differences aren't that much. And if you are looking for a 1TB or larger external then obviously SSD is not an option. So I would just stick with the much more affordable USB3 drives.



Oct 26, 2013 at 09:40 PM
1
       2       end




FM Forums | General Gear-talk | Join Upload & Sell

1
       2       end
    
 

You are not logged in. Login or Register

Username   Password    Retrive password