p.148 #1 · Official: Sony A7 and A7R Fullframe Mirrorless
akuba wrote:
The encoding algorithm used for the Sony a7/a7R RAWs takes the original 14 bit values and maps them to 11 bit space [...]
The Sony RAWs are instead mapped on a curve and devote more of the 11 bit space to the portions of the exposure range that should benefit from more tonality. I believe this is why Sony considers it "visually lossless".
Is this in any way comparable to what Leica did in the M8? I've read that that camera maps 16 bit (or maybe it was 12?) values to 8 bit, with a curve as well.
p.148 #2 · Official: Sony A7 and A7R Fullframe Mirrorless
douglasf13 wrote:
It may have something to do with the amount of light available. With my NEX cameras, the cameras metered and focused at shooting aperture until you got into low light, where it switched over to focusing wide open.
This happens only f/5.0 and f/5.6, not on any other aperture with FE55. I would understand if it would not go below f/5.6 for example but I see no logical reason to have this only these two apertures.
For clarification exact explanation what happens (and NO this is no issue, just weird):
1. select aperture mode and aperture f/4
2. turn camera so you can see aperture
3. change to f/4.5 and physical aperture changes
4. change to f/5.0 and physical aperture changes
5. change to f/5.6 and there is no change in physical aperture
6. half press shutter, and physical aperture closes from f/5.0 to f/5.6
7. change to f/6.3 and physical aperture changes
(you can continue to f/22 and aperture changes in every 1/3 EV step)
And other way around:
1. select aperture mode and aperture f/6.3
2. turn camera so you can see aperture
3. change to f/5.6 and physical aperture changes
4. change to f/5.0 and there is no change in physical aperture
5. half press shutter, and physical aperture closes from f/5.6 to f/5.0
6. change to f/4.5 and physical aperture changes
(you can continue to f/1.8 and aperture changes in every 1/3 EV step)
I can't figure out any logical reason why this happens. However this will have no effect to practical photography.
p.148 #3 · Official: Sony A7 and A7R Fullframe Mirrorless
Samuli Vahonen wrote:
This happens only f/5.0 and f/5.6, not on any other aperture with FE55. I would understand if it would not go below f/5.6 for example but I see no logical reason to have this only these two apertures.
For clarification exact explanation what happens (and NO this is no issue, just weird):
1. select aperture mode and aperture f/4
2. turn camera so you can see aperture
3. change to f/4.5 and physical aperture changes
4. change to f/5.0 and physical aperture changes
5. change to f/5.6 and there is no change in physical aperture
6. half press shutter, and physical aperture closes from f/5.0 to f/5.6
7. change to f/6.3 and physical aperture changes
(you can continue to f/22 and aperture changes in every 1/3 EV step)
And other way around:
1. select aperture mode and aperture f/6.3
2. turn camera so you can see aperture
3. change to f/5.6 and physical aperture changes
4. change to f/5.0 and there is no change in physical aperture
5. half press shutter, and physical aperture closes from f/5.6 to f/5.0
6. change to f/4.5 and physical aperture changes
(you can continue to f/1.8 and aperture changes in every 1/3 EV step)
I can't figure out any logical reason why this happens. However this will have no effect to practical photography.
I just tried mine again in normal and very low light and didn't see any behavioral difference between the two lighting conditions. However this time I did notice that one f/stop transition does not stop down in LV - for me it's f/5.6 -> f/6.3
p.148 #4 · Official: Sony A7 and A7R Fullframe Mirrorless
Thanks for the details Akuba! Very interesting. It may be that any artifacts of this compression will be nearly invisible, however - perhaps ironically, by not giving users the ability to turn it off, Sony has forever left that uncertainty hanging over it. If you could compare uncompressed and compressed results, then people could see for themselves and then decide if they wanted to use it.
I wonder if they didn't have the processing power to do a more traditional, lossless compression on the RAW files. The space savings going from 14bpp to 8bpp is fairly substantial, but of course with a lossless compression, many images would have only been about 50% of the max possible size anyway.
Another thing about the A7 that I'm noticing - it was probably true of N7 but I didn't use high ISO much on that camera. Anyway - if you shoot at RAW+JPG, when reviewing in-camera it shows you the JPG. For high-ISO images, this can be pretty ugly. Switching to just RAW makes the review slower, but looks a lot better in that case.
p.148 #5 · Official: Sony A7 and A7R Fullframe Mirrorless
akuba wrote:
The statement that 11 bits cannot support a dynamic range of 14 is not strictly correct, as your edit seems to acknowledge. Any bit-depth can express any dynamic range. A greater bit-depth simply allows for more granularity in the levels recorded within a given dynamic range. Whether 11 bits does justice to the dynamic range is a qualitative judgment.
There is nothing unclear about the bit depth of the Sony RAWs. The encoding algorithm used for the Sony a7/a7R RAWs takes the original 14 bit values and maps them to 11 bit space (a bit more on this below) and then the 11 bit values are further compressed into 8 bits per pixel by delta-encoding them in fixed-length 16 pixel blocks as follows: 11-bit minimum value for the block, 11-bit maximum value for the block, 4-bit index of the minimum value in the block, 4-bit index of the maximum value, 14 7-bit deltas from the minimum value for the other 14 pixels. This uses 11+11+4+4+(14x7) = 128 bits per 16 pixels or 8 bits per pixel.
This encoding algorithm is why the a7R always produces files that are just over 36 million bytes (7360 x 4912 single color pixels multiplied by 8 bits per pixel divided by 8 bits per byte plus a small variable overhead for metadata). A good aspect of this algorithm is that it is computationally cheap as it does not need to do any data analysis and simply encodes with one pass over the data. A bad aspect is that it is not adaptive and as a result uses a fixed amount of storage which wastes space for images that don't need it and doesn't give additional space to files that would benefit from it.
Note that this encoding is doubly lossy: There is guaranteed data loss on the first step (the mapping from 14 bits to 11) and potential data loss on the second step (the delta-encoding in 16 bit blocks). The reason the second step may not lose data is that if the difference between min and max is less than 128 (the maximum value that can be expressed in 7 bits) then this step ends up being lossless. Since smooth gradients are most likely to have small deltas in values and since one is most likely to notice dataloss in smooth gradients, the second step is statistically more likely to be lossless when it counts most. It is this delta-encoding step that almost certainly explains the artifacts that some people have noticed on high-contrast transitions but these seem to me to be negligible in the grand scheme of things. The first step is for me much more concerning as it is lossy with regard to the total tonal range expressible in the RAW. Without new firmware, the a7/a7R will not produce a RAW file with more than 2048 values per channel.
Circling back to the "doing justice" comment, the 14 to 11 bit mapping is not quite as bad as it sounds as it is not a simple bit shift. A 3 bit shift would have the effect of binning every set of 8 consecutive 14-bit values into the same single value in 11-bit space. The Sony RAWs are instead mapped on a curve and devote more of the 11 bit space to the portions of the exposure range that should benefit from more tonality. I believe this is why Sony considers it "visually lossless". This mapping approach means that for example the mid tones might consider every set of 2 consecutive values the same instead of every 8. However since there are still only 2048 possible values the piper then gets paid in the shadows where 16 values might get lumped together in the down conversion. At the end of the day no matter how well they slice the 11 bits it still does limit the total tonality of the image and it means the files can't take as much processing as losslessly stored 14 bit files from the same sensor could. ...Show more →
Thanks for the good explanation. We agree then that the mapping is not lossless.
In my opinion, raw files should be as raw as possible.
p.148 #7 · Official: Sony A7 and A7R Fullframe Mirrorless
I could so easily be wrong and it matters little, Taylor, but I always thought all images shown in Sony cameras were jpeg - whether the preview, histo display, review...all I had thought were sourced from in-camera jpegs, either tailor made (sorry) from image format setting choices to shoot jpeg or raw+jpeg, or from the small embedded jpeg created along with raw images. What settings would a camera show for a raw file review? Others will elucidate.
p.148 #8 · Official: Sony A7 and A7R Fullframe Mirrorless
elucidate...
I think you are right Philip. Just imagine, a histogram like today in the viewfinder but based on raw data instead of on the white balanced and calculated JPG and a raw based zebra function! For reviewing I think I prefer a JPG rendition (colours please) as long as it is big enough to check focus and DOF and optical artifacts critically. Well, we can dream on.
I also would like a proper shutter release with a half press trigger point...
p.148 #9 · Official: Sony A7 and A7R Fullframe Mirrorless
Taylor Sherman wrote:
I wonder if they didn't have the processing power to do a more traditional, lossless compression on the RAW files. The space savings going from 14bpp to 8bpp is fairly substantial, but of course with a lossless compression, many images would have only been about 50% of the max possible size anyway.
Perhaps they managed to hit their aggressive price point partly by skimping a little on image processing. I've also noticed the files fall apart faster than I'm used to from my D800 when lifting shadows. Maybe even faster than my X-E1. This is the kind of foible one might expect from a first generation product, though.
I've also noticed the terrible high ISO previews when shooting raw+jpeg.
p.148 #10 · Official: Sony A7 and A7R Fullframe Mirrorless
jcolwell wrote:
+1
+2
Sony, for my birthday, I wish for a true lossless compression, no bit-reduction tricks please. I will gladly deal with file size and potential speed sacrifices.
p.148 #13 · Official: Sony A7 and A7R Fullframe Mirrorless
Any macro shooters here? What's the recommended lenses currently. I have a ton of F mount lens that may be going away but in the interim I did get a metabones NEX-Nikon F/G adapter. I have a humungous Sigma 150/2.8 OS that would be incredibly large on the A7......is there a good, old ai/s manual focus lens by Nikon that's a macro?
p.148 #17 · Official: Sony A7 and A7R Fullframe Mirrorless
philip_pj wrote:
I could so easily be wrong and it matters little, Taylor, but I always thought all images shown in Sony cameras were jpeg - whether the preview, histo display, review...all I had thought were sourced from in-camera jpegs, either tailor made (sorry) from image format setting choices to shoot jpeg or raw+jpeg, or from the small embedded jpeg created along with raw images. What settings would a camera show for a raw file review? Others will elucidate.
I see what you're saying, but I definitely noticed a difference in the previews when I switched between RAW-only and RAW+JPG - same scene, same exposure, ISO6400. The latter showed more blotchiness, essentially what I'd expect from JPG compression of noisy data.
p.148 #19 · Official: Sony A7 and A7R Fullframe Mirrorless
Toothwalker wrote:
Unclear. 11 bits cannot support the purported dynamic range of 14 EV.
Edit: A bit depth of 11 bits does no justice to a dynamic range of 14 EV. Or is it a matter of definition - again? If the dynamic range (the ratio of the largest non-saturating input signal to the smallest detectable input signal) is 14 EV, I would use at least 14 bits to store the information.
All analog to digital converters are imperfect. The ENOB (effective number of bits) of state-of-the-art 14-bit converters sampling at ~180 MHz (as within in the A7R ) is very close to 11 bits, so there is little justification to encode more that 11-bits into the compressed RAW image. The use of 11-bit samples within the signal processing chain in conjunction with the delta-encoding is well proportioned and fully justifiable, and is almost certainly visually-lossless in all but extreme corner cases.
p.148 #20 · Official: Sony A7 and A7R Fullframe Mirrorless
Taylor Sherman wrote:
I wonder if they didn't have the processing power to do a more traditional, lossless compression on the RAW files. The space savings going from 14bpp to 8bpp is fairly substantial, but of course with a lossless compression, many images would have only been about 50% of the max possible size anyway.
Regarding processing power: Venturing into speculation land, I suspect the driving constraint for Sony was either buffer space or data throughput. The algorithm they are using has the properties that it can be pipelined (it doesn't need all of the data to be staged in a buffer but can be applied as the data is streamed) and it produces compressed data with a fixed upper bound on its size (that being the 36M bytes). While many lossless algorithms can be pipelined, none of them can guarantee a fixed-sized upper bound on the output. In fact most can result in the compressed data being larger than the uncompressed data for worst-case inputs. Most implementations fall back to storing the uncompressed data in those cases but that requires either buffering all of the input data or reprocessing the bloated output back into the raw input for final storage. But at this point I'm digressing too far into computer science geekery so I'll leave it at that. I would assume Sony did a bunch of benchmarking with a representative set of images and concluded this algorithm met their design constraints whereas lossless ones did not.
jhenderson0107 wrote:
All analog to digital converters are imperfect. The ENOB (effective number of bits) of state-of-the-art 14-bit converters sampling at ~180 MHz (as within in the A7R ) is very close to 11 bits, so there is little justification to encode more that 11-bits into the compressed RAW image. The use of 11-bit samples within the signal processing chain in conjunction with the delta-encoding is well proportioned and fully justifiable, and is almost certainly visually-lossless in all but extreme corner cases.
The ENOB may be around 11.5 or so, so there may be some signal being lost as well, but I agree that the algorithm should be visually lossless for well-exposed images and that for everything but the deep shadows we're probably losing mostly noise. If I'm not mistaken, I recall mention that Sony may also be cooking deconvolution sharpening into the RAW, so the definition of what is raw may be changing anyway. But for peace of mind and for being able to push exposure in post-processing I personally would still prefer losslessy compressed data. Storage is cheap while paranoia is taxing. I think at this point most of us are fairly convinced that the encoding is good enough, but I'm not sure many of us would choose this encoding if there were a lossless option in the menu.
I think at this point most of us are fairly convinced that the encoding is good enough, but I'm not sure many of us would choose this encoding if there were a lossless option in the menu.