Home · Register · Search · View Winners · Software · Hosting · Software · Join Upload & Sell

Moderated by: Fred Miranda
Username   Password

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Post-processing & Printing | Join Upload & Sell

1
       2       end
  

Archive 2013 · File management help
  
 
Squirrely Eyed
Offline
• •
Upload & Sell: Off
p.1 #1 · p.1 #1 · File management help


Preface:
Please lend me your experience and expertise to assist with my goal of having a simple, efficient file management system that can be created from my existing ad-hoc system. I know there are myriad ways to do this and I have researched it a bit, but much of the advice is for people starting out fresh versus coming from an existing management system.

Much thanks in advance for your time to read my description, consider my goals & dependencies, and provide your expert insight as to what might be good for my situation and why it is a good system for me.

Purpose:

  1. Organization of multiple years of existing and future family & vacation photos.
  2. This is not for a professional or client-based workflow.
  3. Need a system that is portable to other software in the future, i.e. not tying myself to an ecosystem that will cause great pain if I choose not to upgrade later.
  4. Need a system that can reasonably be setup from my existing structure with minimal risk to data loss or corruption.


Current Setup:

  1. RAW files are stored on a NAS that is only sometimes connected to my laptop.
  2. Processed JPG files are in a similar folder structure on the laptop.
  3. Currently I only work from a laptop using external calibrated displays, but this may change in the future so the system needs to be robust for future scaling. I use different display configurations depending on whether I'm at home or at work.
  4. File names are the original ones from the camera, due to multiple Canon cameras there are duplicate file names that could even overlap from the same date & event.
  5. Folder structure is something like this:
    ┬─<Camera A>

    └─► <year>.<month>.<day>.<project name>
    ┬─<Camera B>

    └─►<year>.<month>.etc...
  6. LR5 is in the mail and I hope to switch from using Canon's DPP to LR5 for my RAW processing.
  7. No tags or other management systems are in place.
  8. My existing structure is backed up to both a network backup (in the same NAS) and external backups. I'm willing to pay the storage penalty in my backup such that it will contain duplicates with the old & new folder structures. The live data will need to migrate to only 1 structure.


So...I feel the need to migrate my existing structure to something more simple and efficient. Probably highest on my list of choices is to move to a purely date-based folder hierarchy and work on some form of file names.

Concerns:

  1. Getting tied to LR and being stuck with its management, i.e. if I rely on some sort of tags/metadata, can I migrate to some other management software in the future without having to redo all of that?
  2. Amount of effort required to migrate the existing data into a new structure, does the cost outweigh the benefit?
  3. Risk of losing something during migration due to file name collisions, etc.


Sep 04, 2013 at 02:00 PM
cwebster
Offline
• • • •
Upload & Sell: Off
p.1 #2 · p.1 #2 · File management help


I suggest you get a copy of Peter Krough's book "The DAM Book - Digital Asset Management for Photographers" It talks a lot about the basic storage organization system (buckets) that exists outside the DAM software.

<Chas>



Sep 04, 2013 at 02:51 PM
15Bit
Online
• • • •
Upload & Sell: Off
p.1 #3 · p.1 #3 · File management help


All my family stuff is sorted "Year -> Season/Event". So general shots, like say junior outdoors in the woods, are then in a folder "20xx -> Summer", whilst specific events like a birthday are in a folder "20xx -> Junior's Birthday". Further fine-graining i do with keywords in LR. I have a very long keyword list...

I wish you luck with this. I went through the same process earlier this year to get what i have now, and it's a lot of work. I would suggest that you make sure whatever you come up with is simple, as a complicated system will completely collapse within a few weeks. Also, an organisational system is not a fire and forget exercise - it requires considerable ongoing input to keep it working.



Sep 04, 2013 at 03:17 PM
Squirrely Eyed
Offline
• •
Upload & Sell: Off
p.1 #4 · p.1 #4 · File management help


Chas, thanks for the recommendation on the book.

15Bit, thanks for your insight.



Sep 05, 2013 at 03:40 PM
GCasey
Offline
• • •
Upload & Sell: Off
p.1 #5 · p.1 #5 · File management help


Guess I'm old-fashioned, but I like to keep mine as simple as possible. I'm the only one using it.

All photos go in the "My Pictures" directory. No surprises.

All photos taken in 2013 go in a "2013" directory.

In the 2013 directory, I set up a sub-directory for each event, and store all photos of that event in it. The number that the camera puts on the image remains with it.

After I process the best images I store them in a sub-dub-directory, labeled either "Selections" or "best." Again, the camera's image number remains with it.
I can usually remember which year a photo was taken. I my first guess is not accurate, it is easy to use the Explorer file structure and look at other years.

Two cameras show different number sequences, that is not an issue.My files are too voluminous to switch systems.

I don't claim perfection, but this works for me.



Sep 05, 2013 at 04:15 PM
Alan321
Offline
• • • • •
Upload & Sell: Off
p.1 #6 · p.1 #6 · File management help


You will enter an ecosystem and there is no guarantee of an easy way to transfer to a different one later on, but it will surely be easier than fully utilising your first one. Even if you had to transfer some settings manually it will be easy to locate the relevant files because the hard work will already have been done.

My system had a few fundamental goals:
1. Allow easy use of the camera-brand-specific software without it being cluttered up by irrelevant images from other camera brands. (This remains important to me because I need to use that s/w to find things like which AF sensor I used and neither Aperture nor Lr will do everything. Also, I use different brands of camera gear.)
2. Allow any software including the operating system finder / explorer to easily find my better images.
3. Have unique names for every image file (to prevent confusion when doing backups or restoring from them).
4. Include the camera, date, time, and a unique image id in each filename
5. Allow me to locate the current version of any image file by having an original archived image with the camera/folder/image number, in case I ever had to restore some files from archived originals rather than a more recent backup.
6. Allow easy location of files based on technical data within the files (exif data), applied ratings/labels, and/or applied keywords.
7. Allow non-destructive editing and viewing of images at any reasonable magnification.
8. Work with the existing or any other folder system that I want to use.
9. Allow portability across operating systems (Windows and Mac).

I started with iMatch for keywording, BreezeBrowser for editing and viewing, and Downloader Pro for renaming. When I switched to Mac I lost all of those.

I went to Aperture 2 but back then it would not let me see sharpening effects except at 100% viewing size, and its internal structure could not duplicate my folder structure, and so I switched to Lr 2. All the while I still had DPP on the go.

My files are renamed by Lr as they are imported but I have to pick the right preset according to the camera. If I was using Windows then I would still much prefer Downloader Pro for file renaming. I developed Lr filename presets that suited each camera/folder combination but Downloader Pro could read the folder value and apply my preferred camera nickname automatically with only a single preset.

My folders are organised as follows:

common top level folder (facilitates whole-of-library backups and Lr recovery)
- my images or not mine
-- camera or scanner or special purpose folder
--- rating or unsorted
----image files

My filenames are something like this:
camera#yyyy-mm-dd#hhmmss#number
where number is a combination of original camera folder number and image number within that folder.
These files sort nicely into the shooting sequence for each camera, regardless of the software I use.

I used to put other useful text at the end of the filenames but then I found that it prevents Lr from extracting the image number when renaming images and so I removed it. That other info is now only available to me within Lr.

To facilitate unique file names I use consecutive numbering in all of my cameras, never resetting them to zero at each card formatting. That requires that I pay attention when using someone else's cards or cards that have been used in a different camera.

Once in Lr I can apply my own ratings that now represent my initial impression of the image, a colour label that represents its technical quality, and keywords (or phrases) for any number of aspects that I might want to use to separate some images from others (e.g. events, scene details, topics, whatever).

I use collections as required to further group images without having to add new keywords or alter the keyword structure. Smart Collections are used to automatically combine ratings and labels into a single overall scale of goodness or to separate certain folders of images into those that ought to be "private" or "public". Actually putting them into normal collections is done manually because although normal collections can be arranged hierarchically they cannot yet be based on specific smart collections.

With metadata selection I can find images taken with specific gear or exposure settings.
With keywords I can select based on my own terms, and if they are not specific enough then I can add or change keywords and apply them. e.g. I might have a keyword for "aircraft" but I'll need one for "helicopters" only if "aircraft" gets too crowded for me to quickly scan the results to get what I want.

There is something fundamentally important to learn before you begin: It takes a lot of work assigning information to images before you can use that information quickly and easily at a later data. You need to balance what you might need or want to use later against how much work is needed now to facilitate it. So start simple and get more sophisticated as necessary when you discover what is necessary rather than guess what might be necessary. If what you have now allows you to home in on a small group of images that includes the one or two that you want to find then you have probably done enough. If what you want to find is always buried among hundreds or thousands of other images then you obviously need more specific keywords or collections or other metadata to narrow the result down.

I hope you find this info useful. It works well for my almost random collection of image topics but would be less appropriate for specific independent events such as sports, weddings, etc.

- Alan



Sep 07, 2013 at 01:49 PM
Squirrely Eyed
Offline
• •
Upload & Sell: Off
p.1 #7 · p.1 #7 · File management help


GCasey & Alan,

Thanks for your detail on how you have your system setup. It is great food for thought.

I hadn't thought of putting the camera name into the image names, that's an interesting idea that would simplify my existing structure. Having mine split across folders for different cameras ends up being a drag when I use both cameras at the same "event".

Of course for me an "event" might be a trip to the park with the family, or lounging on the couch shooting candids, etc.



Sep 07, 2013 at 02:01 PM
GCasey
Offline
• • •
Upload & Sell: Off
p.1 #8 · p.1 #8 · File management help


Squirrely Eyed wrote:
GCasey & Alan,

Thanks for your detail on how you have your system setup. It is great food for thought.

I hadn't thought of putting the camera name into the image names, that's an interesting idea that would simplify my existing structure. Having mine split across folders for different cameras ends up being a drag when I use both cameras at the same "event".

Of course for me an "event" might be a trip to the park with the family, or lounging on the couch shooting candids, etc.



A directory for "home events" or "family" would work for trips to the park or family candids. I don't identify images by cameras; that doesn't matter to me, though I realize it does to others.

I shoot photos for a group that meets at our church and have a directory, "CAFC," then sub-directories, by date of their events/meetings. One is August 17 Picnic, for example, and right-clicking on it shows 148 files.

The camera labels the images, for example: IMG.1500; when I process that image it may be name the people in the image and the number assigned by the camera in the image, or simply "Image 1500."

Most of my photos are for personal and family use. This system also works for the images that are sold.

If photography were a business, I would want and need a more sophisticated filing system that would include not only the specific ID of the image, but all the pertinent the sales details.



Sep 07, 2013 at 02:50 PM
Mr Joe
Offline
• • • •
Upload & Sell: On
p.1 #9 · p.1 #9 · File management help


+1 on the DAM Book recommendation. There are so many factors to consider here, that a forum post can't cover them all in an organized way.


Sep 07, 2013 at 03:07 PM
Alan321
Offline
• • • • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · File management help


Squirrely Eyed wrote:
GCasey & Alan,

Thanks for your detail on how you have your system setup. It is great food for thought.

I hadn't thought of putting the camera name into the image names, that's an interesting idea that would simplify my existing structure. Having mine split across folders for different cameras ends up being a drag when I use both cameras at the same "event".

Of course for me an "event" might be a trip to the park with the family, or lounging on the couch shooting candids, etc.



By using a keyword for such events (or more accurately, a phrase - they don't have to be single words) you could instantly see all images for such events regardless of cameras and regardless of where the images live on the storage folder structure. So no need to change that structure if there is merit in keeping it - just select the relevant images, click on the appropriate keyword, and you're set to go. Later on you would click on the arrow next to that keyword and there are all of the images. You could narrow them down by date because the dates are already in the image files. So easy to do.

You can have keywords for the family and friends too, so that you can find the images with particular people. You can have separate keywords for "Christmas parties", "Brirthday Parties", etc., and have a parent keyword for all "parties".

More keywords for different locations.

You build up the keyword structure as your needs vary, so that you don't have to think up the ultimate list before you begin using it.


You probably do not even need a file or folder structure at all but having one helps when using less sophisticated software than Lr. You need to understand why you might use such software and don't try catering for situations that will not arise because that leads to a mess. Nearly everything can be done within Lr, including moving files, creating folders, etc.

- Alan



Sep 08, 2013 at 03:35 PM
 

Search in Used Dept. 



MLK-FM
Offline

Upload & Sell: Off
p.1 #11 · p.1 #11 · File management help


Checkout the free tutorials on the Organize Pictures.com website:
http://www.organizepictures.com/organize-photos-tutorials



Sep 08, 2013 at 05:20 PM
15Bit
Online
• • • •
Upload & Sell: Off
p.1 #12 · p.1 #12 · File management help


I use keywords very similarly to Alan, and then use those keywords to generate LR "smart collections" that get uploaded to my web presence. I still maintain a file structure as well though, just in case i migrate away from LR or have some sort of total catastrophe strike my catalogue.

The idea of renaming files to include the camera model is clever too.



Sep 08, 2013 at 05:38 PM
organizepix
Offline

Upload & Sell: Off
p.1 #13 · p.1 #13 · File management help


Essentially I have three "pillars" for a good organization system:'

1) My computer system including my desktop and my external raid hd. I always try to keep my desktop hard drive exactly the same as my backup drive. With storage prices coming down constantly, I figure I will buy larger disks when my media exceeds my hard drive size. Shooting raw makes that possibility closer and closer.

2) Folders. Though they seem outdated, folders are here to stay and they are very important. I break mine down something like this:

-My pictures
---00-Transfer - every time I transfer new pictures they go in here. Very useful.
---01-Family
-----2013
---------2013-09-15-zoo-family trip (date-place-short desc of event)
----02-Export
---------(Various formats....print, web, email, etc)

I also rename my files date-original-filename. Comes in very handy when printing photos. They print the filename on the back of the pictures.

3) Metadata, primarily keywords...including geolocation....when I have time.
I used Xnview, Picasa and now Lightroom for metadata.
The important part is to make sure the software you use writes metadata to to image and not in some proprietary database. The only trouble is with keyword categories...they might be lost since the standard is not well defined.

Hope this helps...focus on your folders first since it seems that you are already doing the backup.

Vlad
Canon T2i with 15-85mm
OrganizePictures.com , TouristBee.com


Edited on Sep 25, 2013 at 01:17 PM · View previous versions



Sep 20, 2013 at 11:20 PM
Bifurcator
Offline
• • • • •
Upload & Sell: On
p.1 #14 · p.1 #14 · File management help


Squirrely Eyed wrote:
Preface:
Please lend me your experience and expertise to assist with my goal of having a simple, efficient file management system that can be created from my existing ad-hoc system. I know there are myriad ways to do this and I have researched it a bit, but much of the advice is for people starting out fresh versus coming from an existing management system.

Much thanks in advance for your time to read my description, consider my goals & dependencies, and provide your expert insight as to what might be good for my situation and why it is a good system
...Show more

I'm doing about the same as you outline in your current setup. The difference is I use OS X smart folders on top of all that. OS X and Windows can both search and sort on tags, labels, keywords, and any other EXIF or IPTC fields there are. For Windows I had to modify some XML definitions IIRC tho. In OS X it's simpler and you can create your own operators to add in if you like - tho it's probably not needed. OS X shows the RAWs at 1:1 without embedded previews (I think) and is WAY faster than LR or other browsers/organizers.







But that's about the only difference between us. I often add the lens name to the file name using ABFR because I shoot m4/3 mostly and almost never use any native lenses that the camera recognizes and records.

For sorting and search then every app can be used or substituted which can read IPTC and EXIF fields - I use the OS mostly tho. For edits however there is no platform-wide standard as you know and every app uses a different standard. I like XMP because it's an ISO standard (16684-1:2012) and was contrived as a conceptual extension to EXIF but even still it seems no vendors can use each other's XMP fields and extent varies greatly as well. The best way I've found to "migrate" edited files is through TIFF or PSD (which is also actually TIFF) as it seems most editors (not organizers) will respect layers and such - and there are a host of utilities for converting PSD and TIFF to whatever to make them more compatible. Some guys use DNG but I've not seen the advantages myself.

Thus with this awareness you're not really "stuck" with LR's management. You can use it along side any other. And, there is no cost in migrating other than preview recreation for whatever app you're using - and DNG or TIFF can usually solve that - if they contain an embedded preview.

The risk of losing something during migration due to file name collisions, etc. is pretty minimal. Most OS's and apps will inform you of collisions and etc. as they occur and offer the choice to rename, overwrite, or skip.

Anyway, try using smart folders (OS X) or virtual folders (Windows) & symbolic links HFS+ & NTFS to expand your structure instead of rearranging the actual locations. Then you can place collections of any type you like in any (every!) structure you can think to create.

http://en.wikipedia.org/wiki/Virtual_folder
http://en.wikipedia.org/wiki/NTFS_symbolic_link




Sep 22, 2013 at 12:03 AM
hugowolf
Offline
• • •
Upload & Sell: Off
p.1 #15 · p.1 #15 · File management help


I really donít see much point in adding camera or lens names into folder or file names. Any half decent DAM, not just Lr (now or in the future) will allow searching for these attributes, which are already stored in the metadata.

Brian A



Sep 22, 2013 at 12:42 AM
Bifurcator
Offline
• • • • •
Upload & Sell: On
p.1 #16 · p.1 #16 · File management help


Not when it's NOT in the metadata at all tho right?

Try placing a any Nikkor, Canon, Minolta, MF Olympus, Or Zeiss lens on any Micro Four Thirds or Sony's Nex body (etc.) and see you get the lens name in the EXIF. So while YOU may not see it, the writing is on the wall as they say - and also in the post for that matter.

Camera names can be useful because camera bodies are often date oriented as well and even selected. If I go to a party I probably bring the Leica but on the job I bring the D800 or whatever. This as you say can indeed be sorted via app or OS for collection and/or smart-folder creation but I understand the initial folder structure creation.




Sep 22, 2013 at 12:52 AM
hugowolf
Offline
• • •
Upload & Sell: Off
p.1 #17 · p.1 #17 · File management help


Bifurcator wrote:
Not when it's NOT in the metadata at all tho right?

Try placing a any Nikkor, Canon, Minolta, MF Olympus, Or Zeiss lens on any Micro Four Thirds or Sony's Nex body (etc.) and see you get the lens name in the EXIF. So while YOU may not see it, the writing is on the wall as they say - and also in the post for that matter.

So ok, not always the lens.

Bifurcator wrote:
Camera names can be useful because camera bodies are often date oriented as well and even selected.


?? ??

Bifurcator wrote:
If I go to a party I probably bring the Leica but on the job I bring the D800 or whatever. This as you say can indeed be sorted via app or OS for collection and/or smart-folder creation but I understand the initial folder structure creation.


Are you saying you understand the need for a camera based folder, or simply understand it?

I would see folders based on camera bodies an impediment rather than a help.

Brian A



Sep 22, 2013 at 04:17 AM
GCasey
Offline
• • •
Upload & Sell: Off
p.1 #18 · p.1 #18 · File management help


The systems mentioned are interesting. And quite varied.

I presented my system earlier, so won't repeat it. Being able to locate the photo images as simply as possible is probably my most important criteria.

If I earned my living at photography (I'm a serious hobbyist yet sell some art-type photos) the process of ID-ing each by date first would appeal to me.



Sep 22, 2013 at 04:29 AM
Bifurcator
Offline
• • • • •
Upload & Sell: On
p.1 #19 · p.1 #19 · File management help


hugowolf wrote:
I would see folders based on camera bodies an impediment rather than a help.

Then you probably shouldn't do that.

GCasey wrote:
Being able to locate the photo images as simply as possible is probably my most important criteria.

Yup, agreed, that's what it's about for me as well.




Sep 22, 2013 at 08:38 AM
mhayes5254
Offline
• • •
Upload & Sell: On
p.1 #20 · p.1 #20 · File management help


If you use LR there are no JPEGS except when you send them somewhere, so that problem goes away. As others have said, I see no reason to include the camera info in the folder/file names, particularly if you use a cross platform processor such as LR/Aperture, etc.

Since LR uses the OS folder structure, that provides at least some measure of portability. If images are organized on disk, you can just import into whatever replaces LR in the future and you have the basic organization structure. Hopefully it will also read XMP files. I also use a LR plugin called "JF folder publisher". This outputs a complete set of my images to JPEG, in a replicate of the source folder structure and it updates with any changes as needed. I use this to give my wife a copy of the family album for her computer.

For the folder structure I have pictures up to 1950 in a folder called "historic" after that it is decade/year/event. I rename on import to include some event name+original camera file number. The rest of the information is key wording (or existing metadata). The only slight disadvantage to this for my wife is the family and photography hobby images are together and is a slight annoyance for my wife. If you are a working pro, it makes sense to separate the work and family folders at a high level.

As for your existing structure, I would just import into LR as is and (perhaps) re-organize later. if you do not do much re-processing, you could consider just importing the JPEGS. A better idea is to import both the JPEGS and RAWs. If the external disk is not connected, LR will still allow you to view, search, keyword, etc and you will only have to put them on-line if you want to edit.



Sep 22, 2013 at 08:12 PM
1
       2       end




FM Forums | Post-processing & Printing | Join Upload & Sell

1
       2       end
    
 

You are not logged in. Login or Register

Username   Password    Retrive password