Archive 2018 · True RAW highlights clipping in a snap

cgarcia
Offline
• •
Upload & Sell: Off

p.1 #1 · p.1 #1 · True RAW highlights clipping in a snap

Aiming to compare the camera zebras with the true RAW zebras (since the highlights metering +2EV doesn't works for me) I have developed a new tool for the PC. It is also useful to exactly know your images exposure, since even RAW converters don't actually show the physically overexposed areas in the images.

The tool generates a highlights clipping representation from the true RAW data and stores it in new sidecar images (appending "_clip" to the original RAW file name). Most camera models should be supported. These images are generated very quickly, in about 2-4 seconds for each RAW photo. This is how they look like:

http://etherpilot.com/photo/expoa7r3/DSC02558_clip_thumb.jpg

Full size available here.

The original picture is developed into B/W to emphasize the overexposed areas. To be more exact, the image is converted to black and bright gray, since pure white is reserved to show clipping in all channels. Anything in color or bright white is overexposed in the RAW data. The color coding is as follows:

Red - only red channel clipped
Green - only green channel clipped
Blue - only blue channel clipped
Yellow - red and green channels clipped
Pink - red and blue channels clipped
Cyan - green and blue channels clipped
White - all channels clipped

These images use half the width and half the height of the full picture, caused by binning (instead of interpolating) the color channels during the bayer demosaicing.

The tool runs in Windows/Linux/Mac and is based on the popular dcraw application. The source code is publicly available on Github.

Windows users can download a standalone binary HERE. Doesn't requires installation: simply, extract the ZIP. Then drag your RAW files into "REVELATOR": the generated image files will be stored along with them. It is possible to drag entire folders (in that case the application will skip the non RAW files).

In addition, a histogram spreadsheet (*.csv file) can be optionally generated (edit REVELATOR.BAT with notepad and change the HISTOGRAM property near the top to 1).

Note that the generated images are big TIFF files, but if you were to store them, they could be converted in batch to jpeg e.g. using ImageMagick (you could even use exiftool to inject on them the original EXIF information, as I did in the example above).

Note: noise in high ISO photos may confuse the application, which may determine a wrong black point. That would cause the image to appear overall much brighter and dull, but has no impact in the highlights clipping. The algorithm to determine the clipping level is far more tricky but fortunately seems to work with lots of photos.

Enjoy!

Feb 24, 2018 at 06:26 PM

rdeloe
Offline
• • • •
Upload & Sell: Off

p.1 #2 · p.1 #2 · True RAW highlights clipping in a snap

Thanks for doing this. I've tried it out and it works fine on my Windows 10 machine. It's in the Sony forum, but it worked just fine on DNG raw files created using IXT from Fuji RAFs.

Feb 24, 2018 at 08:20 PM

cgarcia
Offline
• •
Upload & Sell: Off

p.1 #3 · p.1 #3 · True RAW highlights clipping in a snap

A funny off-topic: I also posted this message in DPreview's Open Talk forum. And it has been later removed by some moderator, who fortunately left me a PM (Dpr do not allow links to hacks). Hummm... was it because the application name? (Hacker's open source toolkit for image sensor characterisation). I wonder if he/she actually readed my post, or the full application title.

In the past I was an active reader on DPr forums (less so a poster) and with the time I moved to FM (by the way, the Sony forum here is super! ... I just bought an A7R3 and are reading here since a couple of months). Rarely I post on Dpr, but this time considering the relevance and that many people don't read several forums, I bit the bullet.

Feb 26, 2018 at 04:58 PM

cgarcia
Offline
• •
Upload & Sell: Off

p.1 #4 · p.1 #4 · True RAW highlights clipping in a snap

I have found that the tool is not so reliable with Sony compressed RAW files, despite I tested them (I personally only use uncompressed files). It is an heuristic job, and I must confess that have no clue about how to safely improve it. In case it picks a too high white point (I think that the correct is 4093) it will underestimate the clipping (it is more rare the case that places it abnormally low). The tool reports the white point after processing each file. Uncompressed Sony RAW files seem to always have a white point of 16383 and until now I have not seen a single error.

It may be difficult to estimate the clipping point because RAW files may have data after it (!?) and gaps (due to compression) on top of that make it a much more difficult job. In Canon, a green channel may clip before the other green (!?). The EXIF information varies across manufacturers and camera models, so it is not useful either. It is possible to manually select the clipping point, but that would be a hard task in the command prompt and even in a graphical interface (I find this feature more useful to occasionally test which areas of a certain image are above certain threshold).

PD: Regarding to my previous comment about DPr, I have to say that they finally reviewed their decision after my reply and un-deleted the thread. I really didn't expected that in such a busy forum, but indeed DPreview listens his forum members and reacts professionally after coming upon an error.

Feb 27, 2018 at 05:21 PM


FM Forums \| Sony Forum \| Join Upload & Sell

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.