Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Canon Forum | Join Upload & Sell

1
       2       end
  

Archive 2015 · 5DsR and LR CC.. Poor Perf with RAW

  
 
Ron Hu
Offline

Upload & Sell: Off
p.1 #1 · p.1 #1 · 5DsR and LR CC.. Poor Perf with RAW


Its SLOOOWWWW AND I have one of the fastest machine you can put together and overclock.

It constantly regenerates images if you enlarge into them and move from photo to photo. Even moving from photos to photos in the Develop module is a pig at 'fill' mode. Yes I have OpenCL Enabled, even turned it off to see if the algorithms were bad for the sensor layout data.

Also the rendition of the colors suck somewhat. I find it hard the camera profiles are even close.

Now I am trying Capture One 8.3.1 (8.3) was support for 5Ds/5DsR and it handles these sized files better. I don't really know how to use it. I've been just playing with it to see.

This is not a sales pitch for Capture One.. Other than trying it because I was curious on its Perf.



Jul 03, 2015 at 07:17 PM
ben egbert
Offline
• • • • • •
Upload & Sell: Off
p.1 #2 · p.1 #2 · 5DsR and LR CC.. Poor Perf with RAW


DPP is also slow and Topaz plug-ins are glacial. Of course a 50mp file is pretty big. Just opening in ACR and saving TIFF without going into Photoshop is slow.

I agree on the colors but I made a profile so I am happy with that.



Jul 03, 2015 at 07:46 PM
snapsy
Online
• • • • • •
Upload & Sell: On
p.1 #3 · p.1 #3 · 5DsR and LR CC.. Poor Perf with RAW


Try Smart Previews.


Jul 03, 2015 at 08:09 PM
johnctharp
Offline
• • • • •
Upload & Sell: Off
p.1 #4 · p.1 #4 · 5DsR and LR CC.. Poor Perf with RAW


snapsy wrote:
Try Smart Previews.


Basically this. Front-load your processing.



Jul 03, 2015 at 09:01 PM
snapsy
Online
• • • • • •
Upload & Sell: On
p.1 #5 · p.1 #5 · 5DsR and LR CC.. Poor Perf with RAW


johnctharp wrote:
Basically this. Front-load your processing.


Yep, it front-loads the preview generation and back-loads the application of the edits when the original raws are made available to LR again. All the time-intensive interactive editing work in the middle is done on previews that are only 5% the size of the original raws and thus very fast.



Jul 03, 2015 at 09:18 PM
Paul Gardner
Offline
• • •
Upload & Sell: Off
p.1 #6 · p.1 #6 · 5DsR and LR CC.. Poor Perf with RAW


You did not say if you were using HDs or SSDs. My 2015 LR seems as fast now as in the 10D days. My catalog is on the OS 256 SSD "C" drive with the Images on a separate 1TB SSD.


Jul 03, 2015 at 09:35 PM
Ron Hu
Offline

Upload & Sell: Off
p.1 #7 · p.1 #7 · 5DsR and LR CC.. Poor Perf with RAW


My OS drive, along where camera raw cache is, on a 512 NVME SSD (you almost can't buy faster), the catalog is on a SATA SSD and I've watched the IO patterns and it doesn't saturate the 500MB/Sec R/W of that. The RAW are on 4TB drive that can stream (read seq. >200MB/sec)


Jul 03, 2015 at 09:41 PM
snapsy
Online
• • • • • •
Upload & Sell: On
p.1 #8 · p.1 #8 · 5DsR and LR CC.. Poor Perf with RAW


Yeah it's not an I/O issue esp since the slowdowns occur even after images have been loaded into memory. The problem is serial coding of Lightroom's logic. I've analyzed its parallelism using Microsoft's SDK tools and it's rather poor. I suspect it'll get progressively faster in each generation of LR as Adobe improves its parallelism and also moves more work into the GPU.


Jul 03, 2015 at 09:49 PM
Paul Gardner
Offline
• • •
Upload & Sell: Off
p.1 #9 · p.1 #9 · 5DsR and LR CC.. Poor Perf with RAW


That's where I plan to go next board update.


Jul 03, 2015 at 09:50 PM
stanj
Offline
• • • • • •
Upload & Sell: On
p.1 #10 · p.1 #10 · 5DsR and LR CC.. Poor Perf with RAW


snapsy wrote:
suspect it'll get progressively faster in each generation of LR as Adobe improves its parallelism and also moves more work into the GPU.


It hasn't actually changed the level of parallelism, ever. The only exception is LR6 export which now uses 30% more CPU, but that's not something to brag about. Indeed, take the example of importing: my CF card goes at 100MB/s plus, as does my target drive, but nope, just a trickle. So is it processing? Nope, the CPU is only fractionally loaded. Thrashing, maybe? Nope, there's no RAM pressure either. DB contention? No that's not it either (I once put the catalog into a ramdisk for no avail). I know from a reliable source that ACR by itself is thread safe, so even without any changes to ACR itself, you could brute force it and run several decoder instances at the same time. Nope, that's not happening either.

I make my money with writing highly parallel systems and LR is a head scratcher to me. It has been clear for a decade (for longer than LR has existed) that we won't get much faster clocks but rather more cores, and yet there's no sign of improvement. Some generations are actually slower than the previous ones. So I'm not holding my breath.



Jul 03, 2015 at 11:31 PM
snapsy
Online
• • • • • •
Upload & Sell: On
p.1 #11 · p.1 #11 · 5DsR and LR CC.. Poor Perf with RAW


stanj wrote:
It hasn't actually changed the level of parallelism, ever. The only exception is LR6 export which now uses 30% more CPU, but that's not something to brag about. Indeed, take the example of importing: my CF card goes at 100MB/s plus, as does my target drive, but nope, just a trickle. So is it processing? Nope, the CPU is only fractionally loaded. Thrashing, maybe? Nope, there's no RAM pressure either. DB contention? No that's not it either (I once put the catalog into a ramdisk for no avail). I know from a reliable source that ACR by itself is thread
...Show more

Thread safety is relatively easy esp if designed into the architecture from the beginning - finding a way to parallelize ACR/LR's long image processing pipeline is a lot more challenging - based on hints Eric Chan (Adobe ACR engineer) has dropped, many of those steps must be performed in sequence and they're not presently coded in a way to easily divide the work even within individual steps. A lesser challenge even with just the demosaicer is dividing the data-intensive work across threads in a manner that wont thrash L1/L2. I've done a lot of work in that latter area in my embedded firmware designs, on processors with much smaller caches and thus more challenging. I think Adobe will get there but it'll be a lot of small incremental steps.

Btw if you're curious here are some perf measurements I did on LR6:

The Export module is making better use of multiprocessing. Here is a comparison of CPU utilization between LR5.7 and LR6 for multi-file D800 export:
LR5.7 vs LR6 CPU Utilization
LR5.7 vs LR6 Thread Utilization

And a measurement of GPU use in the develop module:
GPU utilization using an adjustment brush with only the exposure slider set and GPU HW queue



Jul 04, 2015 at 12:02 AM
stanj
Offline
• • • • • •
Upload & Sell: On
p.1 #12 · p.1 #12 · 5DsR and LR CC.. Poor Perf with RAW


Yes, LR6 is faster than its predecessor - which is sadly unprecedented.

I understand thread safety pretty well Things may be iffy with processing and jumping between images, where you want one thing to be done really quickly. For that you may need to rearchitect it (some would argue that that was doable within the last 10 years when it became clear that threads and not GHz will rule). But stuff like import and export, where you really can brute force it even as a black box, the current molasses are inexcusable since the actual black box is thread safe. Proof is that you can do multiple exports manually - why doesn't the app do it by itself, in an optimal fashion?



Jul 04, 2015 at 12:12 AM
Ron Hu
Offline

Upload & Sell: Off
p.1 #13 · p.1 #13 · 5DsR and LR CC.. Poor Perf with RAW


Imports I don't know, but I know from watching the export to jpgs, they are done (a few) at a time as the tmp files will shows in the export directory as they are generated, then renamed.

I've also watched the process with process-explorer and there are active threads running through the routines generating the jpgs. It is a no-win having to many threads running and waiting for the CPU, but you can get away with more than the ThreadCnt = (cores + ( Cores* HTEnabled [ 0 or 1] ) / 2), Since I don't run w/o HT enabled I can't *really* say it would use 6 threads as I see now with it enabled or it would use 3 of the 6 when HTEn = 0.

Running to many threads with dis-similar data with thrash the caches and make them 1/2 the size as the 2 processors (HTEn = 1) share the compute and L2 caches and L3 I think is shared across all. But this changes all the time with which Intel CPU model you have. AMD doesn't use Hyper-Threaded CPU's and not all Intel CPU's have it either.

But the above it really paramount dealing with highly parallel code as the other gentlemen would say. LR isn't such and there are other things it is doing and in my experience with apps that are doing work+IO, you can do #cpu's*.75 or say 9 out of my 12.



Jul 04, 2015 at 12:28 AM
stanj
Offline
• • • • • •
Upload & Sell: On
p.1 #14 · p.1 #14 · 5DsR and LR CC.. Poor Perf with RAW


Ron Hu wrote:
Imports I don't know, but I know from watching the export to jpgs, they are done (a few) at a time as the tmp files will shows in the export directory as they are generated, then renamed.


Yet you can start a different export with one already running, and the two processes will finish faster than if you had run just one with all the images. So this shows pretty clearly that we aren't at the point of thrashing yet, and that this could be done more efficiently by the machine.

I will leave it at this point, as I don't think I want this to turn into a show of who has a bigger one in software engineering.



Jul 04, 2015 at 12:38 AM
Ian.Dobinson
Offline
• • • • • •
Upload & Sell: Off
p.1 #15 · p.1 #15 · 5DsR and LR CC.. Poor Perf with RAW


OK let me first say I DONT have the 5Ds/r but I did download a few raw files .

my machine is a late 2012 iMac (i7 3.4 , 680mx GPU 2gig , 1tb Fusion Drive and 8gig of ram) . I imported the raw files into my complete catalogue (i.e. I didn't start a new cat) .
the images were imported onto my external USB3 drive which benches around the 120mbs read/write using the black magic speed tester . my cat file and preview files are on the fusion drive .

I can't see any slow down playing with these files compared to my standard 5D3 files . using fill in the develop mode and even zooming in past 1:1 I can move around the image without problem and can move from image to image without noticing any 'redraw'

OP: could it be that even though your machine is high spec there is something that isn't playing well with LR (gpu for example) ?


EDIT:

OK I just pushed a ton of edits on a few of the files and then started to move around the image at 1:1 and then 3:1 and also from image to image and yes I see a bit of a slow down and redraw but I say its far from 'glacial' .



Jul 04, 2015 at 02:28 AM
justruss
Offline
• • • • •
Upload & Sell: Off
p.1 #16 · p.1 #16 · 5DsR and LR CC.. Poor Perf with RAW


LR performance is one of the reasons (the bigger one was workflow/layout) I far preferred Aperture to LR over the years. But Apple took away that option, so here I am, taking big steps backwards in the actual adjustments/develop speeds. It's going to be interesting what happens when I get the A7rII.

That said, the non-standard Bayer pattern of my Fuji 16mp XTrans files take far longer to redraw than my 5D2 files.




Jul 04, 2015 at 02:57 AM
Ron Hu
Offline

Upload & Sell: Off
p.1 #17 · p.1 #17 · 5DsR and LR CC.. Poor Perf with RAW


I played with a few Nvidia setting in the control panel. I am talking on the (lack of) caching of the data when moving between files in a zoomed state or 1:1.

The renderings are dumped immediately and if you go back say to balance photos not of same exposure or you've skewed color temp and need to match that up as the source light changed ever so slightly.

Its the rendering, even non-zoomed, but cropped as it seems it renders the full frame then shows you the cropped portion, either full sized if it fits for then reduced to fit in the window for "fit" obviously.

Just the UI and the rendering are painful now as it never was. More so than just waiting 2 times longer for 2 times the data.

I do only have a Ti680, at 970Mhz, maybe its getting long on the tooth.

I will thank you for who mentioned Smart Previews I never used them, but I won't go back and forth with removing/adding my main tree of pictures.



Jul 04, 2015 at 04:04 AM
Gochugogi
Offline
• • • • • •
Upload & Sell: Off
p.1 #18 · p.1 #18 · 5DsR and LR CC.. Poor Perf with RAW


justruss wrote:
LR performance is one of the reasons (the bigger one was workflow/layout) I far preferred Aperture to LR over the years. But Apple took away that option, so here I am, taking big steps backwards in the actual adjustments/develop speeds. It's going to be interesting what happens when I get the A7rII.

That said, the non-standard Bayer pattern of my Fuji 16mp XTrans files take far longer to redraw than my 5D2 files.



I'm still limping along with Aperture 3.5 and CS5.1 under Mavericks. Everything works amazingly well so no reason to change unless I buy a new camera. My 6D and 70D are good for another year or two but I'm dreading the switch to LR...



Jul 04, 2015 at 01:42 PM
amirm
Offline
• • •
Upload & Sell: Off
p.1 #19 · p.1 #19 · 5DsR and LR CC.. Poor Perf with RAW


No problem here with my Core i7 laptop with 1 Tbyte Samsung 850 SSD. Moving from image to image takes one second or so before sharp preview shows up. Adjustments to the image are all real-time with no lag (I have an Nvidia GPU). The only odd delay was the one time I zoomed in and the hour glass came up for 10 to 20 seconds before correct preview appeared. Can't duplicate it now.

On multi-threading, when I import, CPU usage shoots up to 80% or so. Parallelization is easy because each thread can work on a different image. And with SSD, you don't have the problem with the hard disk thrashing.

This was one of my big worries but with whatever optimization Adobe has made, it seems quite performant although one can always use more speed.



Jul 04, 2015 at 03:41 PM
DAphoto77
Offline
• •
Upload & Sell: Off
p.1 #20 · p.1 #20 · 5DsR and LR CC.. Poor Perf with RAW


You guys are way the hell over my head. I have a 12GB Ram Dell desktop. I am not processing large MB files but I do 500 or so picture files after a big swim meet. I am able to process those pictures in a reasonable period of time like two or three hours. Because of low light and ISOs up to 6400 I have to process these pictures for noise, exposure and crop when I have to as I am not a perfectly precise photographer and swimmers don't stay still very long with their heads up. I have LR cc. If I don't scew up, which I do occasionally, I don't have trouble with speed of processing even though there are things I cannot sync because of variability. I have been using Adobe PS and LR since they came out. i have to convert raw to jpeg for the web. That seems to go ok. I think LR, imo, is a perfectly good consumer product for me. It seems, in my ignorance, that what you expect is commercial grade production processing but what the hell do I know I am just one of the unwashed who doesn't understand that my LR processing is slow. I mean no hostility rather I say this with a smile as I am way in over my head.


Jul 04, 2015 at 04:14 PM
1
       2       end




FM Forums | Canon Forum | Join Upload & Sell

1
       2       end
    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.