Upload & Sell: Off
| p.1 #1 · Lightroom 5 Performance Testing: Pt.2 - SSD's |
Continuing my latest series of LR5 performance testing, I am looking at the impact of SSD’s on how fast LR5 runs. I was going to do this as Pt.3, but i have a strong suspicion that the Develop module testing will show exactly the same results as for LR4.3 last year, and i reckon folk are more interested in the SSD question anyway: Historically this seems to be a pretty confused issue so I hope to bring a bit of fact-based clarity to the table here.
In case you have missed it, the first part of the current performance testing series is at:
The system on test is again:
Intel i5-3570K clocked at various speeds from 1.6Ghz to 4.3Ghz, with turbo boost off.
Asus P8Z77-V-Pro motherboard
16GB of DDR3-1600 RAM
128GB Samsung 830 SSD - Boot drive (mounted as C-Drive)
80Gb Intel X25 G2 SSD – Mounted as G-Drive
320GB Seagate Barracuda 7200.10 – Mounted as F-Drive
4GB RAMDISK – Mounted as O-Drive
Assorted other hard disks which don’t come into play here.
A word on the disks:
- The Samsung 830 is about 1 generation old. On paper it should read at around 520MB/Sec and write at around 320MB/sec. In this test it is hosting the OS (Win7), which will influence performance a little. Unfortunately I can’t change that without a lot of work.
- The Intel X25 G2 is an old SSD now. On paper it should read at around 250MB/sec and write at around 80MB/sec. These numbers may seem low, but the model has a reputation for displaying very low latency, which really helps performance.
- The Seagate Barracuda was a good 7200rpm drive in its day, but that day was some years ago now. On paper it should give around 75MB/sec reading and perhaps a little less in writing (I can’t find exact numbers).
- The RAMDISK is a mounted “drive” built from physical RAM. I used the freeware version of the DataRAM software to build it. In terms of speed, nothing comes close – I measure 6GB/sec read speed with HDTach. Write speed is probably the same.
All the drives are SATA interface (except the RAMDISK, obviously)
System monitoring is done using the nifty utils called “process explorer” and “process monitor”, formally from systinternals, but now from Microsoft. Timing is done by hand using the stopwatch on my phone.
For testing, I’ve used the same catalogue and images as for the other tests in this series. So all tests are performed on an empty catalogue specifically made for this testing, and a directory of 200 images (100 from a Canon 5D and 100 from a Fuji X10). Some additional testing was done with a second catalogue containing 1 image only – a 454MB, 13147 x 4532 pixel panorama chosen to really slow things down.
LR has 3 particularly active directories: The image directory (where the images are kept), the catalogue directory (which houses the LR catalogue and Previews) and the Camera Raw cache directory. There is additionally some traffic to a temp directory in the users home directory on C-drive (c:\users\username\AppData\Local\Temp for those interested). This last one is fixed, but you can move the others around as you wish. The only thing you can’t do is separate the catalogue from the previews.
So the testing method should simply be to locate the catalogue, cache and image directories on different drives and see how fast things go in the different configurations. As this gives an enormous amount of testing I cut the variables down a little to “read” and “write”, with the image directory being “read” and the catalogue, previews and ACR cache being “write”. The catalogue, previews and ACR cache were then located on the same drive for most tests (all except the RAMDISK tests in fact). As well reducing my testing, this also makes data presentation easier as the results can be given in a simple easy to read matrix/table.
Test 1 – Library module: Importing
A simple test, see how long it takes to import the directory of 200 images and render 1:1 previews. The CPU is set to 4 cores running at 4.3Ghz for this test.
In case it is confusing, the left axis is presenting where the image directory is located (data reading) and the top axis is where the Catalogue and ACR Cache are located (data writing).To read, just pick the value where column and row cross each other. So for example, the import time when the catalogue is on the X25 and the images are on the HDD would be 4m 11s. Similarly, the import time when both catalogue and images are on the SSD 830 would be 4m 13s. Hopefully you get it.
I also note that I did a couple more tests using the RAMDISK. With these tests I took the additional step of separating the ACR cache, catalogue and images to separate “drives”, which is why the results aren’t in the table above. The results for these were:
1. ACR cache on SSD 830, Catalogue on X25, images on RAMDISK – import time 4m 12s
2. ACR cache on X25, Catalogue on RAMDISK, images on SSD 830 – import time 4m 18s
The conclusion here is pretty obvious I think – there is no life-changing speedup available by moving to a superfast disk subsystem. In fact, the import times seemed remarkably independent of the speed of the storage media – putting everything on a slow HDD gives pretty much the same result as distributing the load across SSDs and RAMDISK.
I admit I am surprised: I expected to see some benefit to faster storage, but the numbers are clear.
As a sort of addendum, I can say that the CPU traces for these imports look identical, irrespective of where the files are located. This is the case where the files were located on the SSD 830 and the Catalogue and ACR cache were on the Seagate HDD:
And the corresponding filesystem activity. Again, the numbers are similar for all tests, just the drive letters change:
Test 2 – Library module – 1:1 render of one large image
This is a repeat of the test done earlier in the Library Module testing: zooming in on a large file and seeing how fast LR can render it (i.e. when it goes “sharp” at 1:1 view), deleting the previews cache and repeating.
I have had to make a slight modification to the test though, as when looking at the file access traces I noticed that LR was not reading anything from disk during the testing – i.e. it was caching the raw data in RAM and rendering from there. In order to make the test work it is thus necessary to clear the preview cache AND restart LR between tests.
You can see clearly in the file access traces:
Caching in RAM:
Loading from disk:
I’ve given it quite some thought and I don’t think this finding affects the results in the earlier Library module testing, as those tests were performed quite consistently and were focused on CPU performance rather than disk performance. Indeed, by caching in RAM the test perhaps even better isolates the CPU performance metric.
I’m presenting the results in the same matrix as before. Note that because of the changed methodology the numbers are not comparable with the numbers from the Library mode testing in the earlier thread (you’ll note it takes longer here). CPU is set to run 4 cores at 1.6Ghz.
It became pretty obvious that there was going to be nothing to see here so I didn’t bother filling the test matrix. In a sense we should really see the same general result as for Test 1, as we are doing pretty much the same thing – rendering a 1:1 preview. The only differences are that the image here is not a RAW (and so ACR cache is not touched) and the image is very large (which gives all 4 cores time to spin up fully). Anyway, again we see no benefit for SSD’s.
Testing in Develop Module
The big question – what is different in the develop module that would be pertinent to the SSD vs HDD debate? Not much I suspected, as we have seen that LR is caching the big pano image in RAM. Still that was a TIF not a RAW, and it is conceivable that the two are treated differently.
So I checked, and an interesting thing occurs – the Develop module appears to ignore the already generated previews, or at least generates some new data to go with them.
So this is the file system trace immediately after generating previews and zooming on 3 raw files:
You can see each file is loaded from disk rather than from Previews (which is in the catalogue directory at the bottom), and some data is written to the ACR cache also.
If I then return to those 3 images and start to edit them in the Develop module (without restarting LR or purging the cache), you can see there is almost no further file access – nothing from the raw files, ACR Cache and only a little from the previews directory.
So LR *is* caching images in RAM while you edit them, though how many images it will cache I don’t know.
Based on that I’ve chosen not to test in the develop module, as I think the Library module testing is more than representative: The rendering of previews almost certainly uses the same code as the Develop module (as it includes any edits you have done), and in the Develop module files are cached after initial load, so the impact of the disk subsystem is going to negligible after the first click.
I would also argue that Exporting is most likely going to show the same results as importing, so I’m going to be a little lazy and not bother to test that either.
I think you’ve pretty much guessed it – within the scope of this testing at least, SSD’s offer no real benefit for LR, and I must conclude that LR is limited primarily by CPU horsepower and not disk I/O. I admit i expected that as a general result, but i am surprised to see *no* measurable benefit for having a faster disk subsystem. Still, i can't argue with the numbers. I would note that the results suggest faster memory would also be beneficial, but without testing it I can’t say how great the impact would be: Intel CPU’s have extremely good pre-fetching algorithms and large caches nowadays, so i wouldn’t expect too much from superfast RAM.
I do have to be honest and throw in the caveat that this testing was done with a small catalogue that doesn’t have keywords etc in it. So it might well be that with a really large catalogue there is some advantage to a low latency, fast disk subsystem. At startup, for instance, quite a lot of the database may be read into memory (a quick test shows 60MB of my 600MB full lrcat being read on startup). However, in general editing there is not much traffic to and from the database file, so I am sceptical that an SSD would help much even for bigger catalogues unless they have become horrifically fragmented, and then the better solution would be to defrag them. If someone wants to test it though, I am quite happy to be proven wrong….
Edited on Jun 16, 2013 at 02:14 PM · View previous versions