Home · Register · Search · Software · Hosting · Software · Join Upload & Sell

	Moderated by: Hendrik, Fred Miranda
	Username   Password FM Actions & Plugins support Workflow Guidelines FM Forums | Post-processing & Printing | Join Upload & Sell   end    LR4 Hard disk I/O testing

15Bit Offline • • • • Upload & Sell: Off p.1 #1 · LR4 Hard disk I/O testing We see fairly frequent threads relating to the best arrangement of hard drives for LR performance, but no actual numbers to support peoples suggestions. As i'm just about to upgrade my PC to an i5 (mainly due to LR4 performance issues) i thought it might be an idea to do some testing to hopefully get an idea of how to lay out the disks in the new build. The test setup is LR 4.1 on a Core 2 E6300 running at 1.86Ghz, 6Gb RAM, Win 7 running off an Intel X25 SSD (C: drive), photos and LR catalogue on a Western Digital 640Gb SATA (H: drive). The ACR cache is on the C: drive, the catalogue and raw files are on the H: drive. Specfically the raw files are in "H:\temp_cat\incoming raw" and the previews in "H:\temp_cat Previews.lrdata". In order to simplify things i created a new catalogue with 387 RAW files (from a canon 5D) for the tests. I have 4 tests, hopefully representative of how many of us use LR: 1. Building thumbnail cache 2. Browsing and tagging in the Library module 3. Browsing and tagging in the Develop Module 4. Editing a simple picture in the Develop module File I/O was monitored using the very funky Process Monitor from Sysinternals. Results 1. Building preview cache. The test is as simple as it sounds - i set LR to build previews for all the images in the folder and left for a while to do it. I've highlighted the interesting bits. We see 4.5Gb of reads from the RAW file directory, along with 1.1Gb of writes to the Previews directory. There is also 760Mb of reads from the Previews directory, which strikes me as a little strange. There are a lot of file I/O operations going on here - 71k reads from the raw directory and 35k writes to the previews directory. Notice also there is some activity on C drive - 200Mb of writes to the Camera Raw cache and 80Mb to a temp directory. 47k total writes to C: drive, most in the adobe temp directory. In short, there is a lot of disk I/O reading the raw files and generating the preview cache, along with a significant bit of I/O to a temp directory on C: 2. Browsing and tagging in the Library module For this i went through the file folder, tagged some pictures as rejected, zoomed and panned around on a lot of them. As might be expected, the majority of file I/O is from the preview cache. The raw files are still being accessed quite a lot though. And again there is quite a lot of action in the temp directory on C: 3. Browsing and tagging in the Develop module Basically the same test as above, but in the Develop module. I did this because i tend to tag and edit as i go, and LR4 is noticably slower zooming and panning in the Develop module than in the Library module. Now the file I/O is split between the RAW files and the preview cache, with more reading of the RAW's. Again notice 18k writes to the adobe temp directory on C: 4. Editing in the Develop module For this test i took one image and edited it as i would a normal image. So the usual white balance, exposure etc, some minor curves changes and then sharpening and noise reduction. I also did some editing with the local adjustments brush. Again, both the preview cache and the RAW file are accessed, and you can probably see each individual change reflected in the I/O to the .xmp sidecar file. Notice there was no I/O to the Camera Raw cache on C:, but the adobe temp directory was again used. It is also interesting to see that the slowest part of the file I/O for this test was actually to the LR catalogue itself. Comments It should be remembered that file I/O is only one part of the performance equation, and that CPU is not really being tested here. We have all noticed that LR4 is a lot slower than LR3, especially in the develop module where zooming and panning now involves visible re-drawing/re-rendering on the image on my computer. I would expect that this should mostly be a CPU problem, not a disk I/O problem, but there is a lot of file accessing going on during editing so it might well be that the choice of disk and file layouts impacts editing as well as previewing. So what is the best disk layout for LR4? Well, its obvious that the majority of reads come from the RAW files and the preview cache. Also, for some operations there is a lot of reading from RAW files and writing to cache. Logically, the RAW files and the catalogue/preview cache should then be held on separate disks. The high level of I/O to the adobe temp directory on C drive is odd, and its effect on performance i wouldn't like to guess at. On the basis of these results i would definitely choose to put the LR catalogue and preview cache on an SSD if available, simply due to the high level of file I/O. In the (likely) event that only one SSD is available, putting the LR cat on it and running windows from a separate spinning disk is probably the fastest choice for LR performance. With a decent SSD though, having both Windows and the LR catalogue on the C Drive together might be the best compromise for most. For my new PC i will have 2 SSD's, so i will be booting from one SSD, hosting the LR catalogue on a second and holding the RAW files on a traditional spinning disk. I will experiment with holding the RAW files on SSD whilst editing just to see if it makes any difference, but i'm not sure that it will. As a final thought, i wonder what the point of the Camera Raw cache is? Jun 17, 2012 at 04:20 PM

Alan321 Online • • • • • Upload & Sell: On p.1 #2 · LR4 Hard disk I/O testing A decent SSD is pretty fast and will happily handle the OS as well as the Lr catalogue and preview cache and the ACR cache. Access times are so quick compared with a hard drive that it will seem to fly even with all of these things on - provided it is big enough. That would not be so clever for a HDD system. My understanding is that in the Develop module Lr will always reload the raw data from the original files or the ACR cache and the preview cache will offer little benefit. In the Library module Lr will make full use of previews in the preview cache if they are there. If they are not there then it needs to load data from the raw files or ACR cache. Having both the ACR cache and Lr preview cache on the SSD will speed things up unless your HDD system is already very fast. How much you appreciate the performance boost varies with how you operate. e.g. if you slowly work through your images one by one, tweaking as you go, then instant recall from cache hardly matters. Ditto if the files are not in cache or if you zoom in more than the preview that is in cache. If you whiz through them at standard preview size then a speedy cache will help a lot. For whatever reason I have found that opening files in the develop module is significantly faster than in the Library module if I want to see 100% size and the preview cache only has standard (smaller than 100%) previews. It's a few seconds per image. The ACR cache holds partially-decoded raw file data. Lr always wants to reload raw data when photos are opened in the Develop module but to have them all in the ACR cache requires a huge cache that may not be worthwhile, especially on high-cost SSDs. If you know which files you will be working on then perhaps having Lr rebuild their previews to 100% the night before will provide maximum benefit in Library and Develop modules by ensuring that the Lr previews are in the cache and the ACR cache holds the relevant photos too. Then you can get away with a smaller ACR cache. In my computer I have one SSD and one HDD. The raw files live on the HDD except for the auto-import folder that is on the SSD. This arrangement speeds up transfers from card reader to computer (a bit) and allows relatively fast tweaking in Lr. Once the photos have been given the once over most of them are less far likely to be tweaked again in a hurry or in bulk and they are transferred to the HDD (using Lr to move them so that no metadata is lost and the catalogue always knows where they all are). - Alan Jun 18, 2012 at 09:05 AM

15Bit Offline • • • • Upload & Sell: Off p.1 #3 · LR4 Hard disk I/O testing Alan321 wrote: My understanding is that in the Develop module Lr will always reload the raw data from the original files or the ACR cache and the preview cache will offer little benefit. From the numbers it looks like this is only half true - the ACR cache is not really touched in the Develop module, but the previews are. In the Library module Lr will make full use of previews in the preview cache if they are there. If they are not there then it needs to load data from the raw files or ACR cache. Having both the ACR cache and Lr preview cache on the SSD will speed things up unless your HDD system is already very fast. Again the ACR cache seems to see a lot less activity in the Library module than you might expect. For whatever reason I have found that opening files in the develop module is significantly faster than in the Library module if I want to see 100% size and the preview cache only has standard (smaller than 100%) previews. It's a few seconds per image. Perceptually i've found the Develop module to be slower than the Library. I've not investigated closely though. The ACR cache holds partially-decoded raw file data. Lr always wants to reload raw data when photos are opened in the Develop module but to have them all in the ACR cache requires a huge cache that may not be worthwhile, especially on high-cost SSDs. If you know which files you will be working on then perhaps having Lr rebuild their previews to 100% the night before will provide maximum benefit in Library and Develop modules by ensuring that the Lr previews are in the cache and the ACR cache holds the relevant photos too. Then you can get away with a smaller ACR cache. It is generally commented that LR doesn't use the pre-generated Previews in the Develop module, and whilst my numbers above certainly suggest that it does, they also show it relies more heavily on reading the RAW data. The question i find myself having is what role does the ACR cache actually play? As LR is reading the RAW files anyway (rather than the ACR cache) when working in the develop module. In my computer I have one SSD and one HDD. The raw files live on the HDD except for the auto-import folder that is on the SSD. This arrangement speeds up transfers from card reader to computer (a bit) and allows relatively fast tweaking in Lr. Once the photos have been given the once over most of them are less far likely to be tweaked again in a hurry or in bulk and they are transferred to the HDD (using Lr to move them so that no metadata is lost and the catalogue always knows where they all are). The results above would suggest this to be a good arrangement for a computer with 2 disks. Jun 18, 2012 at 06:56 PM

theSuede Offline • • • Upload & Sell: Off p.1 #4 · LR4 Hard disk I/O testing I've put the PS scratchdisk, the system pagefile, FastViewer cache and LR library on a separate 90GB SSD, not because that is necessarily faster - but in case I need to go over the main system drive I won't have to move that stuff around. Well, the pagefile would need to be rebuilt anyway with a system reinstall, but anyway. Not that I rebuild the system that often, when you've done it about ten or fifteen times (since 1994) some stuff gets tedious. Might as well keep stuff where it can stay for a while. (and also, 128GB + 90GB was a lot cheaper than buying a 240/256GB SSD when I built the system). Jun 19, 2012 at 03:17 PM

15Bit Offline • • • • Upload & Sell: Off p.1 #5 · LR4 Hard disk I/O testing You raise a good point there - buying 2 smaller SSD's is probably better value for this sort of application than buying one big one. Similarly, adding something like a 40 or 64Gb SSD to a machine just for LR related files should really speed things up and isn't going to break the bank. Jun 19, 2012 at 06:10 PM

Alan321 Online • • • • • Upload & Sell: On p.1 #6 · LR4 Hard disk I/O testing 15Bit, how big is your ACR cache ? What I said may be true but not so obvious if the ACR cache is too small and does not have the right images in it at the right time. The Lr preview cache is generally as big as it needs to be but the ACR cache is allocated a set maximum size in preferences. Also, the ACR cache holds some partially converted raw file data for each raw file that is "in" the cache but I have no idea how big that data compared with the original raw file. It's just something that is faster to read from cache than completely re-reading and re-converting the original raw file. Maybe it's more of a time advantage than a data transfer advantage. Your own data for the browsing and tagging in the two modules clearly shows that in Develop module it read more data from raw files and less from the Lr preview cache but it still spent much less time reading the data. It might be interesting to use your trace program to edit one or more files in Library module and compare that with editing the same files in Develop module. This would preclude curves adjustments but many of the sliders could be used. You'd also have to be careful that you are not seeing effects of the OS drive caching. I very recently upgraded to Lr 4 and I now find little speed difference between opening files in the Develop module and zooming in to 100% (which is not my standard preview size) in the Library module. However, there was always a significant speed advantage to the Develop module when I was using Lr 3. Perhaps my caches are not yet working optimally because I haven't used Lr 4 enough since upgrading and scrapping the old caches. When it comes to the value of buying multiple SSDs there are a few points that I would like to make: 1. The price value is decreasing as the price per GB in the size range you are considering is now pretty even. The relatively expensive sizes are no longer say 128GB or 240GB but more likely 480GB and bigger. 2. Using multiple drives takes up an extra drive bay. You may have plenty of room in your computer but it would be less feasible in a cramped laptop. 3. Using multiple drives ought to be faster but because they are already very speedy there will be not much in it - unless you striped two SSDs and had a SATA system that could utilize the maximum data transfer rates. 4. SSDs are far more expensive than HDDs and using two smaller drives instead of one bigger drive may prove to be an advantage if one drive dies. In theory the SSDs are very reliable but I have now had several die. I've also had many HDDs die on me over the years but never so many so quickly. - Alan Jun 20, 2012 at 10:48 AM

15Bit Offline • • • • Upload & Sell: Off p.1 #7 · LR4 Hard disk I/O testing Alan, The ACR cache is set to 4Gb, so should be big enough to cache all the files in the catalogue i created. I also cleared it before starting the tests. The reason for the speed difference between ACR cache and Previews is that the ACR cache is on my SSD (with the OS) and the Previews are on a spinning disk. The editing in the Library module might be interesting to try. Avoiding OS caching is not so easy i think, but i would just generate a new catalogue like for these tests. You are correct in your comments on SSD's i think. Your SSD reliability problems intrigue me - i've not seen any problems yet, but my X25 is only 2 years old. I generally get 3-5 years out of spinning disks, and if the SSDs last that long i'll be happy. Jun 20, 2012 at 11:35 AM

Alan321 Online • • • • • Upload & Sell: On p.1 #8 · LR4 Hard disk I/O testing My SSDs have died within months or even weeks. I'm grateful when they die completely and suddenly but two of them played up intermittently not long after installation and then died after several months, causing me a lot of grief and a bit of data loss (between my ever more frequent backups). Regarding the data transfer times, the develop module test read about 30% of the preview cache data that the library module test did, but took only 8% of the time to do it. Not sure how to account for that but maybe it spent less time reading different levels of previews when in develop module. Or maybe there was some OS file caching going on. Another factor not directly covered by your tests is the impact of a larger catalogue of images. Maybe that would increase the difference between the performance in library and develop modules. Jun 21, 2012 at 01:03 AM

	end FM Forums | Post-processing & Printing | Join Upload & Sell end

You are not logged in. Login or Register

Username   Password    Lost your password?