Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Post-processing & Printing | Join Upload & Sell

  

Archive 2016 · Lr Catalogue size concerns....time for a new one?

  
 
arbitrage
Offline
• • • • • • •
Upload & Sell: On
p.1 #1 · p.1 #1 · Lr Catalogue size concerns....time for a new one?


My Lr Catalogue is getting very large....basically it covers 2008-present with the majority from 2012 onwards and is 1.53GBs in size with about 100,000 RAW files referenced from it. A lot of that is wasted stuff from days of poor culling practices and laziness in never going back to cull properly (something I'm much more diligent about over the past year or so).

My question is if it is beneficial to start a new Catalogue say from 2017 onwards or even have the new one be 2016 and onwards so I can still go back without opening a new Cat for 2016?

Or is there no real performance benefit. I do practice good backup strategies for my catalogues on multiple drives (of course my RAW files also) including an offsite at my office.

I'm not worried about a corrupt catalogue because of the backups but I was wondering if performance is taking a hit with the large size of the Catalogue file?

If not I will just continue on with my current main Catalogue into 2017....

I would appreciate any comments, suggestions or otherwise....

Thanks



Dec 29, 2016 at 11:52 AM
howardm4
Offline
• • • •
Upload & Sell: Off
p.1 #2 · p.1 #2 · Lr Catalogue size concerns....time for a new one?


I'd start a new one and see how it goes. If you dont like it, can always just export from the new one and import that into the older/huge catalog and use that.


Dec 29, 2016 at 01:36 PM
kwilliam8
Offline
• • • • •
Upload & Sell: On
p.1 #3 · p.1 #3 · Lr Catalogue size concerns....time for a new one?


I have wondered the same thing. All of my images are in one LR catalog that is almost 3GBs in size. I do not notice any speed problems, but I have wondered about this...
Keith W.



Dec 29, 2016 at 02:19 PM
butchM
Offline
• • • • •
Upload & Sell: Off
p.1 #4 · p.1 #4 · Lr Catalogue size concerns....time for a new one?


I have two catalogs. I keep the current year and the previous year in one catalog (over 100,000 images, 4GB cat file) and I have the rest (1999-2014) in a master archive catalog (over 400,000 images, over 6GB cat file).

If you are having problems with performance try optimizing your catalog. From the Menu: File>Optimize Catalog

The function goes through and housecleans the cat file and can help in some situation. Another consideration for larger catalogs is to allocate a larger Camera RAW cache. I think it defaults to 10GB. Making this cache larger allows for more storage of preview files. A smaller cache can cause Lr to rebuild previews as you go back and visit older images that may have had the previews deleted in order to keep the cache size specified.

The main 'speed' enhancement you may experience with a smaller catalog is in the initial startup and the time it takes to confirm your images are available to your system. Other areas don't seem to show any improvement regardless of cat file size.



Dec 29, 2016 at 03:22 PM
Michael White
Offline
• • • •
Upload & Sell: Off
p.1 #5 · p.1 #5 · Lr Catalogue size concerns....time for a new one?


Here's what I'm planning on doing now I've got a dedicated had for the photos and another had for backups photos and lr plus the had in the pc for the library.

Creat a master catalog that has everything in it for a database then have a catalog for each year for actual work. How I do this and sync everything is like this when I need to work on photos I export that year do the modifications to the images then reimport the catalog back in. Just like if I'm on location I create a new catalog for the shoot the when I return home import that info in the master catalog. Catalog files can not be on network drives even if they are mapped but the actual files can be.



Dec 30, 2016 at 12:08 AM
wlpelzmann
Offline
• • •
Upload & Sell: Off
p.1 #6 · p.1 #6 · Lr Catalogue size concerns....time for a new one?


butchM wrote:
... The main 'speed' enhancement you may experience with a smaller catalog is in the initial startup and the time it takes to confirm your images are available to your system. Other areas don't seem to show any improvement regardless of cat file size.


That is the main issue I have with a 100,000 image database. Also, it takes a while to clean-up the database and create a catalog backup. However, at the moment, I can live with the startup and back-up speeds for the benefit of having everything in one place. I organize my landscape images geographically, so being able to search a single database is a big benefit.



Dec 30, 2016 at 12:19 AM
Alan321
Offline
• • • • • •
Upload & Sell: Off
p.1 #7 · p.1 #7 · Lr Catalogue size concerns....time for a new one?


Generally stick with a single catalog because...

1. it's a database, so everything is indexed and found quickly. That's what databases do. In normal use there is no "searching" in the sense of the program going from file to file looking for what you wanted.

2. Only a single catalog gives you best access to all metadata - both what came from the camera and anything you have added - to help you find whatever you want. This includes keywords, ratings, labels, collections, and camera/lens/exposure information. You won't want to do the same search in multiple catalogs just because you forgot which one it is in.

3. Lr can only handle one catalog at a time. There is no sharing data between catalogs except via import/export, and no retaining "user data" such as selection criteria while switching to a different catalog.

4. you'll never eliminate crap if it isn't right there in the catalog you are using. With the right selection techniques it is easy to find the junk along with the good stuff, and then you can spend a little while removing some of the junk or perhaps just flag it in some way so that you can avoid it next time. And you can do it knowing that there is in fact a better image to keep. You may not know that unless they are all there together.

5. You need to be careful when dealing with extra catalogs; you don't want to edit an image in two separate catalogs because you'll lose something if/when you try to recombine the catalogs.


Exceptions exist, such as for photographers who want to keep clients' images and associated data completely separate from all others. Likewise, you may want to keep very private stuff separate from what can be shown to anyone. I use Lr to manage photos and also scans of documents and so on; they're not all for "public" viewing.

A good use for multiple catalogs is when you have one in your home-base computer and a duplicate in your laptop for when you are traveling. Again, just be sure to edit only the original or only the duplicate until you copy the edited one to the other computer, and then you won't lose any data.



Edited on Dec 30, 2016 at 11:03 AM · View previous versions



Dec 30, 2016 at 10:40 AM
Alan321
Offline
• • • • • •
Upload & Sell: Off
p.1 #8 · p.1 #8 · Lr Catalogue size concerns....time for a new one?


butchM wrote:
I have two catalogs. I keep the current year and the previous year in one catalog (over 100,000 images, 4GB cat file) and I have the rest (1999-2014) in a master archive catalog (over 400,000 images, over 6GB cat file).

If you are having problems with performance try optimizing your catalog. From the Menu: File>Optimize Catalog

The function goes through and housecleans the cat file and can help in some situation. Another consideration for larger catalogs is to allocate a larger Camera RAW cache. I think it defaults to 10GB. Making this cache larger allows for more storage of preview files. A
...Show more

I fully agree with the intent of this post, but the "previews" in the camera raw cache (or "ACR cache") are actually partially converted raw data rather than image previews. The Lr preview cache has the image previews but it will just keep on growing as needed. The ACR cache data is much more compact than it used to be, so that 10GB might now be plenty whereas years ago it was easily too little. In the scheme of things the Lr preview cache will dominate the ACR cache for large photo libraries.

ACR cache size affects performance because if the images you are looking for in Lr are not in the ACR cache then Lr will have to rebuild that data from the original raw file instead, and this can happen any time you show an image in the develop module with or without actually doing any editing.

It pays to keep the ACR cache on speedy storage rather than on slow HDDs where the image files may live.



Dec 30, 2016 at 10:53 AM
butchM
Offline
• • • • •
Upload & Sell: Off
p.1 #9 · p.1 #9 · Lr Catalogue size concerns....time for a new one?


Alan321 wrote:
I fully agree with the intent of this post, but the "previews" in the camera raw cache (or "ACR cache") are actually partially converted raw data rather than image previews. The Lr preview cache has the image previews but it will just keep on growing as needed. The ACR cache data is much more compact than it used to be, so that 10GB might now be plenty whereas years ago it was easily too little. In the scheme of things the Lr preview cache will dominate the ACR cache for large photo libraries.

ACR cache size affects performance because if the
...Show more

I agree I may have not used the most accurate semantics to describe the Camera RAW cache ... however, with larger catalogs (or users with many smaller catalogs) the cache serves any and all catalogs regardless of the space allocated for the cache (along with any work performed in ACR using Bridge and/or Ps). Even if the current cache is more efficient on space than in previous iterations, eventually you will run out of space for the cache as your catalog grows. Indeed, the default setting for the cache is fine for low volume and infrequent catalog wide perusing of images ... but ... offering a bit more resources to the cache can add to overall efficiency.



Dec 30, 2016 at 11:10 AM
dalite
Offline
• • • • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · Lr Catalogue size concerns....time for a new one?


Seems like many of us have this file storage problem using Lightroom. My question is this: are the LR catalog files "compressed" files or larger files (such as native .dng non-lossy files)? I only have over 8440 files in my catalog so the collection is a lot smaller than others'

And if the first question is clarified, can I just delete the .dng files from "My Lightroom Photos" folder? I have them backed up in an external drive.



Jan 02, 2017 at 07:06 PM
butchM
Offline
• • • • •
Upload & Sell: Off
p.1 #11 · p.1 #11 · Lr Catalogue size concerns....time for a new one?


dalite wrote:
Seems like many of us have this file storage problem using Lightroom. My question is this: are the LR catalog files "compressed" files or larger files (such as native .dng non-lossy files)? I only have over 8440 files in my catalog so the collection is a lot smaller than others'

And if the first question is clarified, can I just delete the .dng files from "My Lightroom Photos" folder? I have them backed up in an external drive.



NO ... YOU CAN'T DELETE YOUR IMAGE FILES.

Sorry for yelling, but need to make that perfectly clear. Lightroom doesn't actually 'import' the images ... it only imports dat about the image and the image file's current location at the time you did add it to your catalog. If you delete the file it would be gone, though the related information about the image Lr gathered (along with any changes/additions you made after import) would still be there, but if you delete your DNG files, they won't jeavailable for Lightroom to access. Though you could point Lr to the backup copies or restore the images from the backup copies.

As to the first part of your question, yes, .dng files are compressed to some extent. How much depends upon your preferences for you camera (if it saves in DNG format) and/or the setting you chose in the Lr Import or Adobe DNG converter preferences if you convert from your camera maker's RAW file type after capture.



Jan 02, 2017 at 10:05 PM
arbitrage
Offline
• • • • • • •
Upload & Sell: On
p.1 #12 · p.1 #12 · Lr Catalogue size concerns....time for a new one?


Thanks for all the replies as I just remembered today that I posted this thread....to much 'nog I guess

I currently make a new Cat when I travel to just use for the trip and then combine when home. From reading all above it looks like the only speed increase I might see is opening the cat and backing up. The backup process is getting longer and longer and is now compressing (LR pops up and tells me this) because of the Cat size.

But other than that doesn't seem to be any reason to split the Cat. I may try for a month or so and see if there is any difference. All my files are personal use so no business reason/privacy reason to split for me.



Jan 05, 2017 at 09:12 AM





FM Forums | Post-processing & Printing | Join Upload & Sell

    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.