fredmiranda.com
Login

Moderated by: Fred Miranda
  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Sony Forum | Join Upload & Sell

1              3              6       7       end
  

Archive 2014 · Lossless RAW files from the Sony A7/A7R?

  
 
Fred Miranda
Offline
Admin
Upload & Sell: On
p.2 #1 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
All, I've written a utility that analyzes A7r raws and generates an overlay PNG that highlights the areas with the maximum lossy compression. This can be used to correlate tonal compression to observed artifacts, to see if they're connected. For more information see my thread on dpreview www.dpreview.com/forums/post/52938557

Fred, can you send me the raw that shows the artifacts you mentioned?


You have the link to one I sent to you previously in a PM. Let me know if you don't find it.
Fred



Jan 19, 2014 at 12:50 PM
snapsy
Offline
• • • • • •
Upload & Sell: On
p.2 #2 · Lossless RAW files from the Sony A7/A7R?


Using Fred's raw image there is an exact correlation between the shadow artifacts he sees and the highly-compressed areas of the image.

100% Crop (Original)
100% Crop +3EV in PP with artifacts circled
100% Crop +3EV in PP with high-compression areas overlayed

Note that the center of the beam is not a high-compressed area; only the left/right edges are.



Jan 19, 2014 at 03:29 PM
philip_pj
Offline
• • • • • •
Upload & Sell: On
p.2 #3 · Lossless RAW files from the Sony A7/A7R?


In the interests of methodological study design please consider expanding this experiment using the other camera employing this sensor from the Nikon range and the best selling Canon equivalent.

You could throw in an a900, which Mr Borg says he is keeping to have all the data retained in the file - to paraphrase him. An odd exercise in vindication at the expense of photographic endeavour, in my humble opinion, as that camera's output is a large quantum behind any modern Sony sensor's output in all respects (see DxO), but this is perhaps it is a matter of belief for some people.
Impartiality is (or should be) key in these areas of investigation, and as said above all design is a compromise.

Disappointees can of course buy another model that may do a better artifact-free recovery job, if it suits their unusual tone lifting needs or pixel inspection activities. The rest can enjoy the amazing clean deep shadows evident in a7/a7r/RX1/a99 output, a creative benefit for many.



Jan 19, 2014 at 04:07 PM
Tariq Gibran
Offline
• • • • • •
Upload & Sell: Off
p.2 #4 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
Using Fred's raw image there is an exact correlation between the shadow artifacts he sees and the highly-compressed areas of the image.

100% Crop (Original)
100% Crop +3EV in PP with artifacts circled
100% Crop +3EV in PP with high-compression areas overlayed

Note that the center of the beam is not a high-compressed area; only the left/right edges are.


Very interesting. Does this software which reveals the areas of most compression work with other Sony models such as RX1 raws?



Jan 19, 2014 at 04:28 PM
snapsy
Offline
• • • • • •
Upload & Sell: On
p.2 #5 · Lossless RAW files from the Sony A7/A7R?


philip_pj wrote:
In the interests of methodological study design please consider expanding this experiment using the other camera employing this sensor from the Nikon range and the best selling Canon equivalent.

I'm hoping to see a D800/A7r raw image pair so we can do differential analysis. But I've never seen the artifacts in Fred's A7r image in any D800 raws I have processed...and I've processed over 10,000 D800 images to date.



Jan 19, 2014 at 07:19 PM
snapsy
Offline
• • • • • •
Upload & Sell: On
p.2 #6 · Lossless RAW files from the Sony A7/A7R?


Tariq Gibran wrote:
Very interesting. Does this software which reveals the areas of most compression work with other Sony models such as RX1 raws?

Yes it does - all recent Sony bodies share the same tonal compression algorithm from what I understand.



Jan 19, 2014 at 07:20 PM
Fred Miranda
Offline
Admin
Upload & Sell: On
p.2 #7 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
I'm hoping to see a D800/A7r raw image pair so we can do differential analysis. But I've never seen the artifacts in Fred's A7r image in any D800 raws I have processed...and I've processed over 10,000 D800 images to date.


I have never seeing these artifacts on Canon 5D II or III files either....just a lot more noise and banding!
Snapsy, don't you have a Nikon D800 as well? Could you find a similar high contrast image with very dark blacks and compare both camera files? It should be easy to replicate.
Fred



Jan 19, 2014 at 07:26 PM
snapsy
Offline
• • • • • •
Upload & Sell: On
p.2 #8 · Lossless RAW files from the Sony A7/A7R?


Fred Miranda wrote:
I have never seeing these artifacts on Canon 5D II or III files either....just a lot more noise and banding!
Snapsy, don't you have a Nikon D800 as well? Could you find a similar high contrast image with very dark blacks and compare both camera files? It should be easy to replicate.
Fred


I do. Now that your artifacts have been correlated to the compression I plan to attempt a similar scene with my D800 and A7r.



Jan 19, 2014 at 07:30 PM
Fred Miranda
Offline
Admin
Upload & Sell: On
p.2 #9 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
I do. Now that your artifacts have been correlated to the compression I plan to attempt a similar scene with my D800 and A7r.


Great



Jan 19, 2014 at 07:52 PM
snapsy
Offline
• • • • • •
Upload & Sell: On
p.2 #10 · Lossless RAW files from the Sony A7/A7R?


First attempt at a D800 vs A7r comparison. I used a third-party flash Yongnuo flash that was flagged to only illuminate the ruler, to simulate the light -> dark transition in Fred's A7r raw, then pushed the exposure +5EV for all. Look at the right edge of the ruler... Not sure this is the same scenario but it's a start

A7r vs D800



Jan 20, 2014 at 01:56 AM
ytwong
Offline
• • • •
Upload & Sell: Off
p.2 #11 · Lossless RAW files from the Sony A7/A7R?


I can see those artifacts with my NEX-6 and NEX-7 files (especially in night cityscape scenes), that's one thing that makes me hesitate to further investing much on Sony, but then I did not know what to blame... 12bit-only raw or raw compression or raw converter..





Jan 20, 2014 at 02:48 AM
lsquare
Offline
• • •
Upload & Sell: Off
p.2 #12 · Lossless RAW files from the Sony A7/A7R?


molson wrote:
This is pretty much just a philosophical debate, because if you don't have an uncompressed file to compare with, you will never know exactly how much you are losing with Sony's "lossless" compression.

I can see some very subtle difference in lossless-compressed vs. uncompressed Nikon D800E files, so I would expect to see the same with the Sony files.


As what another member have said, the difference is minimal and should not be perceptible. I can't quantify that.
Again, why can't there be a choice for users of this camera? For those that want the lossy RAW file, then give that to them. For those that want the lossless RAW file, then give that to them.



Jan 20, 2014 at 05:24 AM
lsquare
Offline
• • •
Upload & Sell: Off
p.2 #13 · Lossless RAW files from the Sony A7/A7R?


Fred Miranda wrote:
It would be great if Sony provided the uncompressed option but it's unlikely we will see compression artifacts in the majority of our images.
I was able to replicate a situation where the compression is visible. When shooting a high contrast subject where the shadows are very close to pure black. When trying to recover the shadows, artifacts become very apparent.
Fred


Fred,

I think someone over at one of the forum said that the odds are about 1 in 10,000 that we'll get a problematic image where the difference will be visible. I don't know how accurate that assertion is, but that guy seems to know what he's talking about.



Jan 20, 2014 at 05:26 AM
lsquare
Offline
• • •
Upload & Sell: Off
p.2 #14 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
Yes it does - all recent Sony bodies share the same tonal compression algorithm from what I understand.


So even the RX100 II and RX10 utilize the same tonal compression algorithm?



Jan 20, 2014 at 05:31 AM
BrianVS
Offline
• •
Upload & Sell: Off
p.2 #15 · Lossless RAW files from the Sony A7/A7R?


The Leica M8 uses a similar algorithm for compressing down to 8-bits. Recently, the uncompressed raw mode was "discovered" in an undocumented diagnostics mode and software was made available to convert to uncompressed DNG. The biggest difference that I've found is in High-ISO images. ISO 2500 M8 images that are much cleaner, the compression algorithm favors higher-intensity portions of the frame.

I would lobby Sony to make a lossless mode available as a firmware update. Post-Processed noise reduction software can most likely benefit, high-ISO files will improve.

I don't have a Sony, but did a comparison with the M8 using the compressed DNG and uncompressed RAW at ISO2500.

http://www.leicaplace.com/showthread.php?t=363&page=3

Leica should have put uncompressed DNG into the M8 years ago, Sony should do it NOW.



Jan 20, 2014 at 08:35 AM
Tariq Gibran
Offline
• • • • • •
Upload & Sell: Off
p.2 #16 · Lossless RAW files from the Sony A7/A7R?


BrianVS wrote:
The Leica M8 uses a similar algorithm for compressing down to 8-bits. Recently, the uncompressed raw mode was "discovered" in an undocumented diagnostics mode and software was made available to convert to uncompressed DNG. The biggest difference that I've found is in High-ISO images. ISO 2500 M8 images that are much cleaner, the compression algorithm favors higher-intensity portions of the frame.

I would lobby Sony to make a lossless mode available as a firmware update. Post-Processed noise reduction software can most likely benefit, high-ISO files will improve.

I don't have a Sony, but did a comparison with the M8 using the compressed
...Show more

The irony here is that Sony did offer a user choice of a compressed versus non-compressed raw format earlier in the Sony a900. Hopefully, now with more attention, it will be offered again as an option for those of us who want to use it.



Jan 20, 2014 at 08:53 AM
AhamB
Offline
• • • • •
Upload & Sell: Off
p.2 #17 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
First attempt at a D800 vs A7r comparison. I used a third-party flash Yongnuo flash that was flagged to only illuminate the ruler, to simulate the light -> dark transition in Fred's A7r raw, then pushed the exposure +5EV for all. Look at the right edge of the ruler... Not sure this is the same scenario but it's a start

A7r vs D800


It's still fairly subtle, but you can see the stair steps at 100%. It would probably be worse if the shadows in your test picture weren't so dark.



Jan 20, 2014 at 09:08 AM
Fred Miranda
Offline
Admin
Upload & Sell: On
p.2 #18 · Lossless RAW files from the Sony A7/A7R?


snapsy wrote:
First attempt at a D800 vs A7r comparison. I used a third-party flash Yongnuo flash that was flagged to only illuminate the ruler, to simulate the light -> dark transition in Fred's A7r raw, then pushed the exposure +5EV for all. Look at the right edge of the ruler... Not sure this is the same scenario but it's a start

A7r vs D800


Could you pm me the link to the RAWs from the A7R and D800? I would like to run the images through different converters.
Thanks for the test.
Fred



Jan 20, 2014 at 11:28 AM
Fred Miranda
Offline
Admin
Upload & Sell: On
p.2 #19 · Lossless RAW files from the Sony A7/A7R?


Snapsy,
I just saw the RAW files. From your lighting, too much Exposure is needed to recover the shadows.
Try shooting an outdoor scene with "bad" lighting in order to get a very high contrast subject. Then only move the LR shadows all the way up (+70). When we increase exposure, the issue is somewhat masked.



Jan 20, 2014 at 02:44 PM
snapsy
Offline
• • • • • •
Upload & Sell: On
p.2 #20 · Lossless RAW files from the Sony A7/A7R?


Yep, I'm going out today with D800+A7r to shoot as many scenes as I can to compare later. Btw I tried that same ruler scene with a smaller EV differential between the ruler and background and still saw the same level of artifacting between the A7r and D800 (amount of artifacting changed with EV but still remained constant between the two cameras).


Jan 20, 2014 at 02:58 PM
1              3              6       7       end




FM Forums | Sony Forum | Join Upload & Sell

1              3              6       7       end
    
 

Welcome back
Log in to your account