p.1 #1 · Lightroom Exporting - Any ways to speed it up
Does anybody have tips on how to speed up JPEG exporting out of Lightroom? I don't believe LR properly takes advantage of multi-core processors, so I'm guessing hardware upgrades are pretty much out of the question. I'm already running on 64GB RAM on an Intel i7-3660 CPU.
I'm looking for software setting recommendations so that exporting 2,000 jpegs doesn't take a couple hours.
p.1 #3 · Lightroom Exporting - Any ways to speed it up
While old, the article does a good job explaining the quality aspect but does not really mention it's relationship to time required to generate each jpeg. The inference most will assume is that a lesser quality setting takes less time to generate. So I took a few minutes to see for my self using a MacBook Air mid 2012, 2 GHz - i7, 8Gb ram.
56 photo Raw files (30.1Mb avg size), LR 6.7 export setting sRGB, resize 990 pixels long edge, 120 resolution and std sharpening for screen. Times are mm:ss
THEN: tested the same files on my iMac late 2012 3.4 Ghz - i7, 32Gb ram with same settings
#4 2:16 avg jpg size 259k <same as #3
Conclusion: #1-3 tests show essentially the same results thus indicating it's the math being done by LR for each file to be converted regardless of output settings in quality or resolution. Test #4 shows that applying brute force (more powerful equipment) to the task makes a big difference, approx 50% improvement.
So, if you have a lot of files to export, a faster machine will do it quicker.
p.1 #4 · Lightroom Exporting - Any ways to speed it up
while LR doesn't use the 'hyper thread' cores of my i7 iMac it does use all 4 actual cores . . but throwing faster GHZ at LR has the most effect of anything .
my guess is that having LR write to the fastest available drive like an internal SSD rather than a slower external drive may help a bit , but in reality large exports like that are going to take a while.
if your not exporting at full res then maybe having LR use smart previews as the export source could help as well .
but size needs to be under 2560 on the long edge .
p.1 #5 · Lightroom Exporting - Any ways to speed it up
Until I rebuild my system I probably won't have an SSD in there (I need a primary SSD hard drive of 1TB, but that costs too much right now), but didn't even think about adding a secondary hard drive just for LR.
I'm exporting at Quality 80 right now, but I guess I could drop it down and test the speeds to see if it improves.
p.1 #6 · Lightroom Exporting - Any ways to speed it up
Running two concurrent exports at once will speed things up a little, as LR doesn't make make complete use of multicore CPU's with a single export process.
p.1 #8 · Lightroom Exporting - Any ways to speed it up
15Bit wrote:
Running two concurrent exports at once will speed things up a little, as LR doesn't make make complete use of multicore CPU's with a single export process.
thats an interesting thought . . I may have to test that out . if it works out faster I wonder if running 2 of the same process for other things is faster than 1 . i.e. setting LR to build 1:1 previews in 2 batches at the same time .
p.1 #10 · Lightroom Exporting - Any ways to speed it up
Ian.Dobinson wrote:
thats an interesting thought . . I may have to test that out . if it works out faster I wonder if running 2 of the same process for other things is faster than 1 . i.e. setting LR to build 1:1 previews in 2 batches at the same time .
The export trick is quite well known i think, and something i quantified in one of my old LR performance testing threads. It's not a huge effect, but every little helps. I haven't tested with the new GPU accelerated LR though.
It is an interesting idea to also test out the preview build this way
p.1 #11 · Lightroom Exporting - Any ways to speed it up
hi schlotz - hmm.. very interesting test results showing the same amount of time as compression time increases.
from my limited readings jpeg compression algorithms, i have understood compression time is a function of # of pixel. thus lowering the pixel counts should have reduce the compression time, but your test results showed the opposite
i would also expect a time reduction if exif (and any gps) data are not being saved.
schlotz wrote:
While old, the article does a good job explaining the quality aspect but does not really mention it's relationship to time required to generate each jpeg. The inference most will assume is that a lesser quality setting takes less time to generate. So I took a few minutes to see for my self using a MacBook Air mid 2012, 2 GHz - i7, 8Gb ram.
56 photo Raw files (30.1Mb avg size), LR 6.7 export setting sRGB, resize 990 pixels long edge, 120 resolution and std sharpening for screen. Times are mm:ss
THEN: tested the same files on my iMac late 2012 3.4 Ghz - i7, 32Gb ram with same settings
#4 2:16 avg jpg size 259k <same as #3
Conclusion: #1-3 tests show essentially the same results thus indicating it's the math being done by LR for each file to be converted regardless of output settings in quality or resolution. Test #4 shows that applying brute force (more powerful equipment) to the task makes a big difference, approx 50% improvement.
So, if you have a lot of files to export, a faster machine will do it quicker....Show more →
p.1 #12 · Lightroom Exporting - Any ways to speed it up
15Bit wrote:
Running two concurrent exports at once will speed things up a little, as LR doesn't make make complete use of multicore CPU's with a single export process.
Yes ... and it's by design that Lr does not make use of multiple cores ... with the concept being that the export would not hog resources so users could continue other work while an export is being processed in the background.
It would be nice if they offered an option to turn on multiple cores for exports as needed to address larger exports, though.
To the OP:
You may see an improvement in overall speed of large exports by sending smaller multiple simultaneous exports in smaller groups. Keep in mind, that it isn't just about overall processor performance but throughput from reading and writing the data as well. YMMV.
p.1 #13 · Lightroom Exporting - Any ways to speed it up
I suspect there is a minor effect but the basic process of converting 30Mb files to sRGB jpgs takes the overwhelming portion of the time. Since the exif & gps data are strictly text I rather doubt they have much effect. JMTC
Matt
tntcorp wrote:
hi schlotz - hmm.. very interesting test results showing the same amount of time as compression time increases.
from my limited readings jpeg compression algorithms, i have understood compression time is a function of # of pixel. thus lowering the pixel counts should have reduce the compression time, but your test results showed the opposite
i would also expect a time reduction if exif (and any gps) data are not being saved.
p.1 #14 · Lightroom Exporting - Any ways to speed it up
goobers wrote:
... I don't believe LR properly takes advantage of multi-core processors, so I'm guessing hardware upgrades are pretty much out of the question. ...
p.1 #15 · Lightroom Exporting - Any ways to speed it up
Things that might help...
1. Crank up the CPU speed. That could entail reducing the CPU load or the number of active CPU cores in order to let turbo boost kick in. Everything happens roughly in proportion to CPU speed.
2. Avoid using .xmp files.
3. Have the ACR cache and Lr catalog on a speedy SSD.
4. Output the converted jpegs to a speedy SSD.
5. Increase the size of your OS storage cache if you can. You've got RAM to burn and it's grossly underutilised by Lr.
If you had used .xmp files then it might be possible to use multiple instances of other software to do the edits and exports on say a quarter of the files each, but Lr won't do it. I don't know if any other software uses the CPU cores efficiently and also has compatibility with Lr edits.
If you had the time (in which case this thread would not exist) you could simplify the edit list for any images that have been seriously hacked about. It seems that Lr will follow each edit command in sequence and not aggregate multiple commands of the same type into one equivalent command. e.g. how many tweaks do you make to sharpening or exposure before arriving at the final settings ? Only the final settings matter.
I recently rebuilt previews for a whole bunch of photos and Lr was taking nine seconds for each photo. For many of them it was quicker to remove them completely from the catalog and start again importing clean image files that had no edits. However, that was only really viable because I wanted to use new features in Lr that had not been available in the original edits, and much of the early work would have been wasted anyway. It would be nice of Adobe to give us edit command aggregation that applies between any external file references such as import, export, virtual copy, etc.
p.1 #16 · Lightroom Exporting - Any ways to speed it up
Alan321 - I currently work with .xmp files with my NEF files. Is the only way to eliminate .xmp files to convert my imports into DNG files, or is there a method that I don't know about?
If I do go the DNG route, I've read suggestions that I should always keep my original NEF files. Seems like I'm going to be using up a ton of hard drive space to speed up my post processing efforts if this is the case. Any thoughts on this?
p.1 #17 · Lightroom Exporting - Any ways to speed it up
you can turn off XMP sidecars in the LR preferences . normal raw files (ie NEF and CR2 ) dont need them in LR .
if you need to open the files in some other software (not using the EDIT IN option in LR) then the XMP file will carry the edits done to the file in LR as long as the software supports XMP .
if you do most of your work in LR and only send files to Photoshop (or other software) then you dont really need the XMP file as the LR catalogue contains all your edits .
p.1 #18 · Lightroom Exporting - Any ways to speed it up
Ian.Dobinson wrote:
you can turn off XMP sidecars in the LR preferences . normal raw files (ie NEF and CR2 ) dont need them in LR .
Yes and no. Technically you don't need the sidecars if you don't perform any work in anything but Lightroom and Ps using 'Edit in" ...
But ... you can have the best of both worlds. The preference you turn off doesn't eliminate XMP sidecar files completely, it turns off 'automatically write changes to XMP' ...
You can have the best of both worlds with very little effort. Turning off the auto update does help in some performance situations, but you can manually write updated info to the sidecars by simply selecting the images in the Library module and hitting CMD+S on Mac keyboard (not sure of the shortcut on Windows) and it will write out the sidecars. Or you can use from the menu: Metadata > Save Metadata to File and it can write out the updates at your leisure.
That way, in case of catastrophe, it is another level of backup to save your RAW processing efforts because you could rebuild a new catalog based upon that XMP data ... of course it would be sans any History states or Develop snapshots. etc.
Back in the early days of Lr performance was even worse than today on many machines even with much smaller files. Many of us developed the habit to to a sort in Library to show all the images from a work session then select all and save metadata to files. Easy peasy.
p.1 #19 · Lightroom Exporting - Any ways to speed it up
Just remember that if you start with xmp sidecars then have it turned off in LR, the latest adjustments are not recorded so a third party app reading them won't have the up to date changes. You'll have to 'save the metadata to files' in LR to bring the sidecar current before running the third party app.
p.1 #20 · Lightroom Exporting - Any ways to speed it up
I've turned off automatic writing .XMP files and did a trial run on exports. It's sped up a little bit, but nothing miraculous. The more and more I think about it, the more and more I want a reason to go upgrade some hardware components, like the motherboard, cpu, ram, ssd drives
I'll keep tinkering around. Thanks for the feedback so far!
