Home · Register · Join Upload & Sell

Moderated by: Fred Miranda

New fredmiranda.com Mobile Site

New Feature: SMS Notification alert

New Feature: Buy & Sell Watchlist

FM Forums | Post-processing & Printing | Join Upload & Sell

How long does it take to import ...

RustyBug
Online
• • • • • • •
Upload & Sell: On

p.1 #1 · p.1 #1 · How long does it take to import ...

So, I'm new to using LR.

Got the M1 with a pretty nice configuration.

Seems to me like the import process is kinda slow. I suspect it is just my unfamiliarity with what all is happening during the import process in LR, that makes it seem like that.

Which just makes me curious what others are experiencing (files / size / time).

I haven't timed anything yet, just seat of the pants waiting stuff. Wondering if import preference settings, config, etc. come in to play. So, like the other stuff with the new rig, would be welcome to hear what others are experiencing / and how they are achieving it ... to get a sense of normal / realistic expectation.

For discussion sake, let's say my SD cards are 170 MB/s and my files are DNG (24MP or 40 MP). What is a reasonable time to expect for importing say 100 or 500? Is the card a bottleneck, and would it make sense to get a faster card (even though my camera can't harness the extra speed). It's not like I'm shooting events, or am in a massive hurry to meet a journalistic deadline, etc. Just wondering if the card is an issue, or is it just some things that LR is doing behind the scenes.

Also, while I see the import status bars (two of them), I don't see a file count of how many have been imported, the same way I used to with Bridge. I expect it is there somewhere, just not sure where to see it, or how to find it.

Seems like it imports in "chunks" or "batches" off the card ... rather than a steady state, the way I see it when importing with Bridge. With Bridge, I can see the steady rate of files being imported, and the rate speeds up / down with file size (of course, jpg vs. dng, 24MP vs 40MP, etc.). With LR, I don't really see the correlation presented numerically. It's not bad (nor lightning quick), just unfamiliar with how to project how long it'll take with remaining time to import kind of thing.

Thanks.

Nov 16, 2022 at 06:48 AM

Majestictone
Offline
• • •
Upload & Sell: On

p.1 #2 · p.1 #2 · How long does it take to import ...

Good Morning,

When I go to import, I set a folder in external hard drive and import those files through LR into that folder. I don't want to actually store those files in LR but as long as I hooked up to the drive I am good to go. I have a new 14" MBP1 that I was to keep as little as possible on the hard drive to preserve its longevity. On the average I am importing about 50-100 uncompressed RAW files from a A7RII and I don't have any lag time whatsoever.

I also use On1 but I do not import on that software because it labels the files on On1 sidecars so when I going through my externals looking for images to process, I want to keep as vanilla as possible if that makes sense.

Now with downloading the LR 12, I have most definitely see a 30 second lag time when exporting a finished file where as before LR12, it was 3-5 seconds. Not sure what the issue is but every time I see the beach ball of death, it freaks me. Interesting enough, when using On1, when exporting a finished file, it takes about 1-2 seconds which I do enjoy

Nov 16, 2022 at 10:49 AM

Majestictone
Offline
• • •
Upload & Sell: On

p.1 #3 · p.1 #3 · How long does it take to import ...

Rustybug,
FYI
I am using MPB1 Max Chip 32GB Unified RAM - 2TB SSD16-Core GPU - 16-Core Neural Engine

Nov 16, 2022 at 10:55 AM

dordek
Offline
• •
Upload & Sell: On

p.1 #4 · p.1 #4 · How long does it take to import ...

I haven't done specific timings, but there are many factors to take into account with importing:
Card speed
Card reader speed
File location write speed
Lightroom read speed (of file location device)
Lightroom preview size and build speed
and there are probably others not coming to mind at the moment.

I'm using a Mac Studio with lots of memory. My workflow is to put the card into a card reader and use Bridge to rename my files into my yymmdd_nnnn format while moving them from the card to storage. If I'm temporarily going to my SSD this is much faster than putting them onto a spinning hard drive (which is what happens eventually). I then run DNG Converter on those raw files to get the DNGs I'm going to have in Lightroom. When I then go into Lightroom and ADD those DNGs on the SSD it goes very quickly - I see thumbnails for all almost immediately. There is still the bar at the top showing the 1:1 previews being made which takes a little longer, but not too long. I think you can mouse over those bars to see what is happening (or maybe click on the right side of them - haven't done it in some time).

In the above process, Bridge does the renaming very quickly when going to SSD and quite a bit slower when going to my permanent hard disk storage. I have the same lag when I take those RAW files from the SSD to the hard disk drive - that takes time. DNG Converter (working on the SSD) does as many files as I have processors at a time and goes quite quickly (basically a file copy on SSD). The Lightroom import, reading from the SSD, seems almost instantaneous except for the preview building which gives me the bar and the preview building doesn't take too much time and I'm able to look at the shots although there is sometimes a delay if I've gotten ahead of where the previews have been built.

So, I see delays if my RAW files are going to hard disk and a much smaller delay if I'm using the SSD for temporary storage of them. Creating DNG files on the SSD takes a little less time than the Bridge process going to the hard drive, but a little longer than Bridge creating the same files on the SSD. And then Lightroom doing the import of the DNGs is the fastest part of the whole operation.

I use this process to keep the original RAW files as backups on hard disk drives and the DNGs are the smaller, working files that Lightroom processes on SSD. Every few months my Lightroom DNGs get moved out onto a different hard disk, too. That can take a while.

So a big part of the delay in my processing is getting a file from card to hard disk. If you're using Lightroom to do this at the time of import, it is going to be a major slowdown in the import process. That is why I often take the temporary step of staging through the SSD with the RAW files. I can always do the SSD to hard disk copy in the background later when I'm not waiting to look at my shots in Lightroom.

When I come back from shooting I start this process and then head from breakfast if I have hundreds of shots. But I would say the whole process isn't more than 30 minutes for 400 to 600 pictures. Once I have the DNG files and start up Lightroom I had better be done with breakfast because the pictures are going to be imported and available for me to see almost immediately.

I'm talking 60MB average ARWs from an A1 and 50-55MB average DNGs. I think the "chunks" you are seeing are the number of files equal to your number of processors (1 file per processor) being processed at a time. So, 10 processors then ten files being processed at a time and then onto the next 10.

By the time I start Lightroom and do the Import I'm expecting to be able to see my pictures and get right to work and I can, almost immediately. The previews can take a little while, but I usually don't overrun that process.

Nov 16, 2022 at 03:07 PM

dordek
Offline
• •
Upload & Sell: On

p.1 #5 · p.1 #5 · How long does it take to import ...

I will also point out that when I move the Lightroom known files from SSD to hard disk I use the operating system copy or drag-and-drop to do it. Then when Lightroom tells me it can't find the files I tell it where they are. I once lost all my editing for several batches of pictures trying to have Lightroom do the move from SSD to hard disk.

Nov 16, 2022 at 03:22 PM

RustyBug
Online
• • • • • • •
Upload & Sell: On

p.1 #6 · p.1 #6 · How long does it take to import ...

Thanks guys.

Not very many pics to import on today's card. 16 DNG's in the 85MB range.

24 seconds.

As it just happens to turn out 85MB is 1/2 of card speed 170MB/s ... so my math suggests it should be about .5 sec / file. 1.5 sec / file is triple that.

Nov 16, 2022 at 05:42 PM

chez
Offline
• • • • • • •
Upload & Sell: Off

p.1 #7 · p.1 #7 · How long does it take to import ...

When importing into LR, I usually start the import than go pour myself a nice cup of coffee or a rum and coke depending on what part of the day it is.

Nov 16, 2022 at 05:46 PM

johnvanr
Offline
• • • •
Upload & Sell: On

p.1 #8 · p.1 #8 · How long does it take to import ...

So many variables to take into account. My MFT files, I just sit and wait (rendering the full files is another matter, however, for some bizarre reason). My R5 files, I leave the computer to do its thing.

Nov 16, 2022 at 06:05 PM

bobby350z
Offline
• • • •
Upload & Sell: On

p.1 #9 · p.1 #9 · How long does it take to import ...

Watch these:
https://www.youtube.com/results?search_query=artisright%20M1%20Mac%20import

My files are 110-120MB each. Reasonable speed SD card, not the latest gen V90, only cheaper V60. I say roughly 1 second 1 file. I use C1 which I know is slow in import/exports. And I can start working as file is imported by like Chez, I take a coffee break if I have a lot of files to import. What's the hurry.

Nov 17, 2022 at 10:22 PM

Alan321
Offline
• • • • • •
Upload & Sell: Off

p.1 #10 · p.1 #10 · How long does it take to import ...

Try copying your image files onto your internal SSD and then import into LrC from there as a separate step. This offers two advantages:
1. It lets you eject the card and keep it safe asap
2. It lets LrC operate as quickly as possible.
If the LrC import process speeds up then you'll know that the SD card was the bottleneck. Otherwise, you'll know it is LrC.

What sort of import are you doing ? I generally put my files where I want them to live and import/add from there. I avoid import/move and import/copy. I also do any file renaming before LrC is involved.

Modern SD cards can be reasonably efficient but older cards were pretty slow when getting a new file was involved. i.e. the file system overheads caused a significant reduction in overall transfer speed even if actual file transfers were speedy. The smaller the files the slower the overall speed, though 85GB is clearly not small. Anything that involved writing to SD cards, including renaming files, can be relatively slow too.

Depending on your LrC configuration, you may have LrC building full size previews and other automated processing as the files are imported. I generally have LrC import with minimal processing and minimum previews. Then I cull the obvious rubbish, apply "standard" changes such as exposure, WB, etc., and then I have LrC do a bulk full-size preview generation (select the images and then build the previews). This is heavily multi-threaded and very fast on SSDs. CPU utilization will be very high so long as the SSD can transfer the data fast enough both ways (reading and writing). My PCIe 3 NVMe M.2 SSDs work at about 2000GB/s. Very nice

IMO full size previews are largely wasted on the initial import because they'll soon need to be rebuilt. They're more useful when the culling and bulk corrections have already been done.

Nov 17, 2022 at 11:37 PM

RustyBug
Online
• • • • • • •
Upload & Sell: On

p.1 #11 · p.1 #11 · How long does it take to import ...

bobby350z wrote:
Watch these:
https://www.youtube.com/results?search_query=artisright%20M1%20Mac%20import

My files are 110-120MB each. Reasonable speed SD card, not the latest gen V90, only cheaper V60. I say roughly 1 second 1 file. I use C1 which I know is slow in import/exports. And I can start working as file is imported by like Chez, I take a coffee break if I have a lot of files to import. What's the hurry.

It's not so much, "the hurry" ... just wanting to have a proper understanding of what to expect.

It seems like 1.5 sec per file is kinda slow. Back when I first rented the older Macbook, it seemed much faster to get files off the cards. I'll have to search for the post to see those results for comparison. I mean, part of the reason I made the move to the Mac was for the import speeds on that older MBP rental than my PC. Now, this new Mac is importing at the SAME speed as my PC. Seems kinda

to be right back where I started for import times. I mean, you spend $$$ to buy performance ... right? If I just want to do the "get a cup of coffee" routine, I already have that in my TP for the same import speed (via Bridge).

I guess the short question is "What is the fastest way to get files off the card?"

In the Art is More video, the 1000 45MP files took around 25 minutes (1500 sec), so that works out to the same 1.5 sec / file. That was on the 2019 MBP. This is a new MBP, so hmmm

at roughly same.

Just wanting to understand what the expectation is. I hear anecdotal reports of how blazingly quick things are with the new M1, so I'm wondering if folks have a some different type of setup / configuration that produces those blazingly quick results. Imo, 1.5 sec / file is not "blazingly quick". That, and I experienced (recall) something better in the older MBP. If that is the norm to be expected, then so be it ... I'll adjust accordingly. Just wanting to set up the new rig to get the most out of it.

Nov 18, 2022 at 06:50 AM

RustyBug
Online
• • • • • • •
Upload & Sell: On

p.1 #12 · p.1 #12 · How long does it take to import ...

Alan321 wrote:
Try copying your image files onto your internal SSD .

Is there a preferred / fastest way for doing this on the MBP?

Should that be via the SD reader, or via a dongle and the USB port, etc. to get the fastest x-fer rate?

Nov 18, 2022 at 06:52 AM

RustyBug
Online
• • • • • • •
Upload & Sell: On

p.1 #13 · p.1 #13 · How long does it take to import ...

RustyBug wrote:
So, I just ran a couple more tests.

1000 files on the SD card ... completed previews in Bridge.

Mac: 3 minutes
TB: 6 minutes

Found this from back in the summer. 2X faster, sounds good.

Now, I'm at 1X, same speed. There's the rub. Why am I getting the same results with the new MBP vs. the old PC?

Nov 18, 2022 at 07:07 AM

bobby350z
Offline
• • • •
Upload & Sell: On

p.1 #14 · p.1 #14 · How long does it take to import ...

RustyBug wrote:
It's not so much, "the hurry" ... just wanting to have a proper understanding of what to expect.

It seems like 1.5 sec per file is kinda slow. Back when I first rented the older Macbook, it seemed much faster to get files off the cards. I'll have to search for the post to see those results for comparison. I mean, part of the reason I made the move to the Mac was for the import speeds on that older MBP rental than my PC. Now, this new Mac is importing at the SAME speed as my PC. Seems kinda

...Show more →

C1 is importing my 100MP+ files at 1 sec. I am using Mac Studio with 64GB RAM not the Ultra. And I know C1 is not faster. I am using SD card straight (V60 ProGrade cards, nothing fancy.) in the front reader on my Mac, importing to internal SSD. I would think separate card reader would be faster, but I haven't tested it. I would suggest that you also try better cards maybe. Read speed matters, isn't it. Try speed test on your card to see how it performs.

Nov 18, 2022 at 08:47 AM

dordek
Offline
• •
Upload & Sell: On

p.1 #15 · p.1 #15 · How long does it take to import ...

Yesterday I got curious and decided to put some numbers to my perceptions.

With 404 pictures on my card I timed various paths in my workflow. Bridge took 58 sec to rename them from the card to SSD and 5:44 to rename to HDD. DNG Converter took 50 sec to rename to SSD (from SSD) and 7:22 to rename from HDD to SSD. This is on a Mac Studio using 20 processors and converting 60MB A1 ARW files. I'll probably always use the SSD for a temporary repository for my ARW files and then do mass copies to HDD at a later time when I can just start a copy and go on to other things.

Oh, and it is a left click on the Lightroom progress bar that brings up what is going on and a couple of menu options (on my machine).

I would also suspect card speed if you weren't seeing the performance increase you were expecting with a new machine and you were using the same old cards.

Nov 18, 2022 at 10:09 AM

mcbroomf
Offline
• • • • •
Upload & Sell: Off

p.1 #16 · p.1 #16 · How long does it take to import ...

dordek wrote:
Yesterday I got curious and decided to put some numbers to my perceptions.

With 404 pictures on my card I timed various paths in my workflow. Bridge took 58 sec to rename them from the card to SSD and 5:44 to rename to HDD. DNG Converter took 50 sec to rename to SSD (from SSD) and 7:22 to rename from HDD to SSD. This is on a Mac Studio using 20 processors and converting 60MB A1 ARW files. I'll probably always use the SSD for a temporary repository for my ARW files and then do mass copies to HDD at a...Show more →

What kind of card?

Thx

Nov 18, 2022 at 10:34 AM

dordek
Offline
• •
Upload & Sell: On

p.1 #17 · p.1 #17 · How long does it take to import ...

I was using a ProGrade 160 CFExpress (A, 800MB/s Read) and a Sony MRW-G2 card reader plugged directly to one of the USB/c ports on the front of the Studio box.

Nov 18, 2022 at 11:07 AM

mcbroomf
Offline
• • • • •
Upload & Sell: Off

p.1 #18 · p.1 #18 · How long does it take to import ...

Thanks. I thought that was a bit quick

Nov 18, 2022 at 12:43 PM

RustyBug
Online
• • • • • • •
Upload & Sell: On

p.1 #19 · p.1 #19 · How long does it take to import ...

Just found this on a google search.

The SD card slot on the new MacBook Pros supports up to 250MB/s of data transfer with UHS-II SD cards and up to 90MB/s with UHS-I SD cards, according to information shared by Apple after this story was published.

The math on the 90MB/s works out to be what I'm experiencing, when I transfer directly (not through LR). Just did a test importing 400 files and at the 10GB point, it was 1:50 seconds (i.e. 110 seconds). 110 * 90 = 9,900

Which, even though my card is 170MB/s (Class 1) ... I'm not sure I understand WHY the reader is slower than the card. Seems pretty lame to have a reader that can't keep up with the cards in the class. Sure, I can go get Class 2 cards to get the 250MB/s ... and likely will. Never needed it for my camera bodies, so Class I was sufficient. You'd think for the $$$ ... you'd get "class leading" performance, rather than bottlenecked performance ... particularly when you realize that the older rental unit transferred at 2X (i.e. harnessing the 170MB/s).

Odd, but at least I feel as though I understand what's happening.

Nov 18, 2022 at 02:28 PM


FM Forums \| Post-processing & Printing \| Join Upload & Sell