Home · Register · Join Upload & Sell

Moderated by: Fred Miranda

New fredmiranda.com Mobile Site

New Feature: SMS Notification alert

New Feature: Buy & Sell Watchlist

FM Forums | Sony Forum | Join Upload & Sell

2 end

Archive 2017 · Shoot compressed or uncompressed?

virtualrain
Offline
• • •
Upload & Sell: Off

p.1 #1 · p.1 #1 · Shoot compressed or uncompressed?

Hi, I'm curious if anyone is shooting uncompressed RAWs for the bulk of their shooting or are most people going with compressed RAWs?

On my A7RII, I shot compressed most of the time, only choosing uncompressed for evening city scape shots on a tripod. Perhaps I'll do the same on my A7RIII but I'm interested to hear what others are planning to do.

Dec 01, 2017 at 02:35 PM

Parariss
Offline
• • •
Upload & Sell: Off

p.1 #2 · p.1 #2 · Shoot compressed or uncompressed?

Compressed, virtually always.

FYI, Jim Kasson did a pretty strenuous technical analysis of the differences in his blog:

http://blog.kasson.com/the-last-word/sony-a7rii-uncompressed-read-noise-and-edr/

Click through to his subsequent pages at the bottom for the continuation of the test.

Dec 01, 2017 at 03:45 PM

pdmphoto
Offline
• • • •
Upload & Sell: Off

p.1 #3 · p.1 #3 · Shoot compressed or uncompressed?

Where is the lossless uncompressed Raw option? Every other digital camera I have owned had this option, except for Sony!

Dec 01, 2017 at 03:53 PM

Isaacheus
Offline
• •
Upload & Sell: Off

p.1 #4 · p.1 #4 · Shoot compressed or uncompressed?

Currently shooting uncompressed on the a7r3, but would be keen on an compressed lossless raw option if it arrived

I've heard that the real difference is in night shots, where artifacts are possible with compressed?

Dec 01, 2017 at 03:57 PM

virtualrain
Offline
• • •
Upload & Sell: Off

p.1 #5 · p.1 #5 · Shoot compressed or uncompressed?

Thanks. While I find Jim’s measurements interesting, I’m often left looking for a conclusion or better summary of the results. It’s almost like... here’s a bunch of charts... draw your own conclusions.

Dec 01, 2017 at 04:01 PM

bwcolor
Offline
• • •
Upload & Sell: Off

p.1 #6 · p.1 #6 · Shoot compressed or uncompressed?

virtualrain wrote:
Thanks. While I find Jim’s measurements interesting, I’m often left looking for a conclusion or better summary of the results. It’s almost like... here’s a bunch of charts... draw your own conclusions.

Dec 01, 2017 at 04:14 PM

virtualrain
Offline
• • •
Upload & Sell: Off

p.1 #7 · p.1 #7 · Shoot compressed or uncompressed?

bwcolor wrote:
You have the camera. Shoot in both modes in whatever environment you are usually shooting and you decide. If you can't decide, then it doesn't matter to you.

Yeah, that's certainly an option, but what's the point of having a forum if we don't share, discuss and debate stuff like this?

Dec 01, 2017 at 04:30 PM

mjm6
Offline
• • •
Upload & Sell: Off

p.1 #8 · p.1 #8 · Shoot compressed or uncompressed?

virtualrain wrote:
After looking at that, I gather there are no significant problems with using compressed RAWs but it really doesn’t enlighten me as to where issues may arise or what situations may create issues.

But there are clearly-seen differences in some high contrast circumstances, even if the basic evaluations that Jim did appear to indicate that the performance is comparable.

It effectively is comparable within a range of normal microcontrast conditions, but when the contrast hits certain levels, artifacts begin to occur. Look here:

https://photographylife.com/reviews/sony-a7-ii/3

Plus, lots of places all over the web for other examples.

For normal shooting, it probably doesn't matter, but for night shooting and high contrast scenes, it might reduce the image quality somewhat if you look carefully.

Further, if you shoot for B&W processing, these artifacts may appear if you start to manipulate the image to lift the shadows. This is really just a potential problem in B&W because there is more latitude for manipulation in B&W by most people's preferences.

Personally, as long as I'm not space limited, I shoot uncompressed for everything. No reason not to unless you are limited by a few things:

1. Memory card space
2. Backup HD space
3. Write speed issues that will slow down the camera (slow cards)
4. If you just don't want to deal with larger files, etc.

I'm utterly baffled that Sony hasn't implemented a lossless compressed RAW file in the a9 or the new a7r3... That, along with the StarEater problem, have identified a clear problem within the company that makes me worried about their ability to maintain the clear technical and market lead they have with mirrorless into the future.

...and I'm saying that as a strong and early adopter of Sony and later Sony full frame mirrorless cameras, not as an anti-Sony troll.

Dec 01, 2017 at 04:48 PM

Matt Grum
Offline
• • •
Upload & Sell: Off

p.1 #9 · p.1 #9 · Shoot compressed or uncompressed?

virtualrain wrote:
Hi, I'm curious if anyone is shooting uncompressed RAWs for the bulk of their shooting or are most people going with compressed RAWs?

Unless I need to shoot in burst mode lot I shoot uncompressed now. I have plenty of memory cards and can compress the data afterwards (losslessly) if necessary. It's a small advantage, but it comes with quite a small cost (unless you are contstrained for resources).

virtualrain wrote:
Thanks. While I find Jim’s measurements interesting, I’m often left looking for a conclusion or better summary of the results. It’s almost like... here’s a bunch of charts... draw your own conclusions.

The bottom line is that there is no effect on dynamic range when shooting compressed... but that shouldn't come as a surprise because dynamic range was never the problem with compressed RAW in the first place. The problems are with posterization.

virtualrain wrote:
How does Sony get rid of half the data in a RAW file without any unwanted effects?

Linear RAW is massively inefficient. Sensors are linear devices (twice the light = twice the signal), whereas human perception is logarithmic (multiplying the light gives a constant increase in perceived brightness). The long and short of it is that a difference of 1 or 2 levels is noticeable whereas a difference of 1 or 2 or 100 or 200 levels is completely undetectable in the highlights.

The first thing the Sony compression algorithm does is apply a tonecurve to the data, allocating more bits (13) to the shadows where bit depth matters and less to the highlights (9) where huge jumps are imperceptible.

The second stage is based on the observation that locally parts of the image only occupy part of the brightness scale, so instead of storing values absolutley they are stored relative to the local maximum and minimum.

The lossy compression algorithm has some interesting properties, it is very fast and very parallelizable, it is highly resistant to byte and bit errors, and perhaps most importantly it yields a constant filesize. Noise is by definition impossible to compress losslessly, so lossless algorithms achieve zero compression in the worst case, so with lossy compression buffers etc. are easier to manage because you can plan for the best case not the worst case.

It also has some disadvantages. Transitions from very bright to very dark will reveal so heavy posterization if you lift the shadows, and bright areas can also yield posterization if you increase contrast.

Isaacheus wrote:
I've heard that the real difference is in night shots, where artifacts are possible with compressed?

Night shots are an example where you are likely to get lots of very bright areas (distant lightsources) adjacent to dark areas you want to lift, so yes some night shots will benefit from uncompressed RAW.

Dec 01, 2017 at 05:14 PM

virtualrain
Offline
• • •
Upload & Sell: Off

p.1 #10 · p.1 #10 · Shoot compressed or uncompressed?

Matt Grum wrote:
Unless I need to shoot in burst mode lot I shoot uncompressed now. I have plenty of memory cards and can compress the data afterwards (losslessly) if necessary. It's a small advantage, but it comes with quite a small cost (unless you are contstrained for resources).

The bottom line is that there is no effect on dynamic range when shooting compressed... but that shouldn't come as a surprise because dynamic range was never the problem with compressed RAW in the first place. The problems are with posterization.

Linear RAW is massively inefficient. Sensors are linear devices (twice the light = twice...Show more →

Awesome explanation... Thanks Matt!

I have three primary types of shooting I do... and here's how I'm likely going to shoot then...
- Travel Daytime Handheld - Uncompressed
- Evening City Tripod - Uncompressed
- Portraits (Family or Candid Street) - Compressed

Dec 01, 2017 at 05:34 PM

bclaff_too
Offline
• •
Upload & Sell: Off

p.1 #11 · p.1 #11 · Shoot compressed or uncompressed?

Matt Grum wrote:
... Transitions from very bright to very dark will reveal so heavy posterization if you lift the shadows, and bright areas can also yield posterization if you increase contrast.
...

Good explanation, one quibble; I would say that the lossy compression can cause artifacts rather than posterization. Posterization is a very specific type of different effect.

Dec 01, 2017 at 05:39 PM

Kronologix
Offline
•
Upload & Sell: Off

p.1 #12 · p.1 #12 · Shoot compressed or uncompressed?

I'll be jumping on the Sony bandwagon after many years of waiting for Canon's move. Since I've been shooting lossless raw forever, I was also wondering about the compressed vs uncompress files in the Sony and their lack of Lossless flavors.

Once I get the new camera I think I would be more inclined to shoot uncompressed, to later compress as lossless DNG. Would that be a viable way to tackle this? or are there any downsides to that approach?

Thx

Dec 01, 2017 at 06:18 PM

mjm6
Offline
• • •
Upload & Sell: Off

p.1 #13 · p.1 #13 · Shoot compressed or uncompressed?

Kronologix wrote:
I'll be jumping on the Sony bandwagon after many years of waiting for Canon's move. Since I've been shooting lossless raw forever, I was also wondering about the compressed vs uncompress files in the Sony and their lack of Lossless flavors.

Once I get the new camera I think I would be more inclined to shoot uncompressed, to later compress as lossless DNG. Would that be a viable way to tackle this? or are there any downsides to that approach?

Thx

This is exactly what I do.

The downside as I understand it (other than the argument about DNG being an Adobe standard, which I believe is fear mongering for the most part), is that some of the metadata that is stored in the original file potentially will not be brought over to the DNG.

It takes longer when importing to LR, but it is mostly happening behind the scenes, so I don't really see that as a workflow issue for my needs.

If I were a shooting pro, processing thousands of images from a weekend wedding, for example, I wouldn't do something like this, but I'd also not be using an a7r3 for the vast majority of the shooting in those circumstances anyway.

Dec 01, 2017 at 06:38 PM

Kronologix
Offline
•
Upload & Sell: Off

p.1 #14 · p.1 #14 · Shoot compressed or uncompressed?

mjm6 wrote:
This is exactly what I do.

The downside as I understand it (other than the argument about DNG being an Adobe standard, which I believe is fear mongering for the most part), is that some of the metadata that is stored in the original file potentially will not be brought over to the DNG.

It takes longer when importing to LR, but it is mostly happening behind the scenes, so I don't really see that as a workflow issue for my needs.

If I were a shooting pro, processing thousands of images from a weekend wedding, for example, I wouldn't do something like...Show more →

Glad to hear. I always converted all my CR2 files into DNG so for me is a non-issue. I much prefer the practicality of DNG than having to track annoying sidecar files.

Dec 01, 2017 at 06:43 PM

JimKasson
Offline
• • •
Upload & Sell: Off

p.1 #15 · p.1 #15 · Shoot compressed or uncompressed?

After looking at that, I gather there are no significant problems with using compressed RAWs but it really doesn’t enlighten me as to where issues may arise or what situations may create issues.

How does Sony get rid of half the data in a RAW file without any unwanted effects?

I just noticed this. Sorry for the delay.

It's easy to find conclusions on the 'net; in my experience, it's hard to find reliable data with sufficient information about the source that you can replicate it yourself if you choose, which is fundamental to the scientific method.

Here are my "See Spot run" conclusions about craw's effect on mages:

1) The reduction in precision from 14 bits to 13 adversely affects the better ADC in the a7RIII at base ISO, but not the a7RII. This effect is IMO noticeable only to testers who are looking for it.

2) The rest of the tone curve has no visual effect; it works like Nikon's similar tone curve.

3) The delta modulation part of craw adversely affects extreme contrast edges, producing bar-code like artifacts. This rarely occurs. When I see a situation with those edges, I switch to uncompressed raw. I do that for well under 1% of my images.

Jim

Feb 10, 2018 at 11:32 AM

SoundHound
Offline
• • • • •
Upload & Sell: On

p.1 #16 · p.1 #16 · Shoot compressed or uncompressed?

My understanding is that, except for the lowest ISOs, the dynamic range of modern cameras (including the newest Sony’s) is 12 bits or less. Also display options are 8 bit for LCDs (a few 10 bits) and, of course, much less for prints.

Feb 10, 2018 at 11:33 PM

Steve Spencer
Offline
• • • • • •
Upload & Sell: On

p.1 #17 · p.1 #17 · Shoot compressed or uncompressed?

Feb 10, 2018 at 11:39 PM

JimKasson
Offline
• • •
Upload & Sell: Off

p.1 #18 · p.1 #18 · Shoot compressed or uncompressed?

SoundHound wrote:
My understanding is that, except for the lowest ISOs, the dynamic range of modern cameras (including the newest Sony’s) is 12 bits or less. Also display options are 8 bit for LCDs (a few 10 bits) and, of course, much less for prints.

When you say "dynamic range", do you mean engineering dynamic range (EDR)? I will assume that you do. Tell me if you meant something else.

You can see the a7RIII EDR here:

http://blog.kasson.com/a7riii/sony-a7riii-edr-vs-iso-setting/

In most shutter modes, the EDR exceeds 12 bits (or 12 stops, take your pick) at ISOs 100 through 320, plus 640, 800, and 1000.

You can't compare the number of bits driving a display with a nonlinearity with that of a linear representation like that in a raw file so casually. You will need to reverse the tone curve.

But more importantly, you can't fairly compare the DR of the original scene with the DR of a print or display without taking into account all the processing that compresses the DR of the raw file to that of the output medium.

Jim

Feb 11, 2018 at 10:38 AM

JimKasson
Offline
• • •
Upload & Sell: Off

p.1 #19 · p.1 #19 · Shoot compressed or uncompressed?

Steve Spencer wrote:
Color is measured in bits. Dynamic range is measured in stops.

There is no reason not to measure DR in bits. Shannon got us comfortable with fractional bits. If F is fullscale, and S is the signal plus noise at the minimum acceptable level of the signal to noise ratio, the DR in bits is log2(F/S). That is also the DR in stops.

Jim

Feb 11, 2018 at 10:42 AM

Mirror
Offline
• •
Upload & Sell: Off

p.1 #20 · p.1 #20 · Shoot compressed or uncompressed?

Tripod/ Night Shots uncompressed, family / portrait compressed

Feb 11, 2018 at 11:27 AM

1 2 end
FM Forums \| Sony Forum \| Join Upload & Sell 1 2 end

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.