johntruong wrote:
IR has an interview with the head of Sony's mirrorless group.
Great and interesting interview... he again confirmed that uncompressed RAW is in the works (but no timeline) and that the issue could be addresses via firmware.
How they got to the magical 42MP was also interesting.
Here is a snip but you should read the entire article:
"This next question is more of a request maybe, but we've had a lot of questions asking about raw format. And...
KM: Ah, raw. <laughs> 14-bit.
DE: Yeah, well 14-bit is OK, but many people are asking "could we please have uncompressed RAWs?"
KM: Sony RAW is compressed, not uncompressed. But if we're getting a lot of requests for it, we should make such a kind of no-compression raw. Of course we recognize that. But I cannot give you a guarantee when we're going to fix or not fix.
DE: Right. When you're going to address that, yeah.
KM: Sure, sure. And so we recognize the customer's requirement, and actually we are working on it.
DE: So it's something that you're aware of. I'm sure that the image processing pipeline is optimized for the way that it is now, but it seems to me that, while it might involve some trading off some performance, that it could just be a firmware change. Could it? Would you be able to provide uncompressed raw as a firmware update, or would it require new hardware?
KM: Right, yes. So... not hardware.
DE: It is firmware. OK, good! I think people would be willing to accept a slower transfer time or lower frame rate in an uncompressed mode. Some people really, really want that."...Show more →
I'm personally not bothered by the Sony RAW files, but as a journalist, I am bothered by an interview in which the interviewer talks more than the interviewee and seems to put words in his mouth. I'd take this exchange with a big grain of salt.
mogul wrote:
Is there any downside to compressing lossless? Why wouldn't Sony use it?
First it requires more processing. Second the degree of compression that is possible under a lossless scheme is content dependent. Some images will compress better than others. Whereas with lossy compression you can achieve any arbitrary compression ratio by controlling the amount of loss.
johnvanr wrote:
I'm personally not bothered by the Sony RAW files, but as a journalist, I am bothered by an interview in which the interviewer talks more than the interviewee and seems to put words in his mouth. I'd take this exchange with a big grain of salt.
I don't think you will get copious responses from most Japanese so the interviewer setting up the question with a lot of words is OK. This is a far better interview than indicated by your excerpts.
Never had problem with raw files from A7R
Anyway: http://www.sonyalpharumors.com/sony-interview-and-a-good-news-they-are-working-on-uncompressed-raw/
5) Uncompressed RAW: Sony RAW is compressed, not uncompressed. But if we’re getting a lot of requests for it, we should make such a kind of no-compression raw. We recognize the customer’s requirement, and actually we are working on it. And yes Sony could provide that via simple firmware upgrade!
Tariq Gibran wrote:
Well, if you are correct, it's interesting as this is not the typical way compression is talked about generally (not by engineers but such as in Photo schools). The use of the term compression in this setting refers to saved file size by camera vs opened file size, which is where the 2:1 compression comes in for the raw format that Sony once used.
I've never heard "compression" used by engineers, photographers or anyone else to refer to the difference between the RAW file and a demosaiced image - as that demosaiced image can be of a completely arbitrary size, depending on format, bit depth, up/down sampling etc. etc.
Tariq Gibran wrote:
If by "uncomprssed" Sony is talking about offering a raw that is half the size of the opened 8bit file, then I would have no issue with that as it would be the same as they once offered (such as in the A900).
By "uncompressed" they are talking about 73.5 megabyte RAW files instead of 42 megabyte RAWs from the α7ʀ II.
Lossless compressed RAWs would be around 50-67 megabytes.
mogul wrote:
the way I understood raw was uncompressed was about 1/3 more than sensor size but that is a problem that I assumed, probably incorrectly.
Almost, for 12-bit ADCs it's 1.5x, and for a 14-bit ADC it's 1.75x the number of megapixels in megabytes.
jhinkey wrote:
I don't think you will get copious responses from most Japanese so the interviewer setting up the question with a lot of words is OK. This is a far better interview than indicated by your excerpts.
They're not my excerpts. And you generalizing Japanese like that isn't helpful. You obviously don't know anything about Japanese (I'm married to one). If you're saying that you suspect a lack of fluent English led to the short answers, then the interview becomes even more unreliable.
shirozina wrote:
Customers should get what they want and not what they need so let's have uncompressed RAW and compressed lossless RAW as an option.
To be honest I see no use at all for uncompressed RAW if you can have loslessly compressed RAW (as long as the compression isn't somehow hampering interoperability).
Tariq Gibran wrote:
It's a bit concerning that Sony themselves are using this "compressed" vs "uncompressed" terminology when speaking of an issue which is about lossy compression versus lossless compression. The important bit - lossy vs lossless- seems to be missing (which makes me wonder if the Sony folks doing the talking know what they are even talking about).
This is the company that made the Blu-Ray standard require uncompressed audio on every movie disk even though almost every disk also contains losslessly compressed audio using either DTS Master Audio or Dolby's equivalent.
lightskyland wrote:
It's less engineering effort to provide uncompressed RAW than to support two different versions of compression.
Frankly, given the almost-nonexistent problem with the current RAW compression I think an option of uncompressed + the current RAW compression is fair enough.
People can use uncompressed when shooting star trails, and the existing ARW the rest of the time...
I thought this might be the case, but when Nikon released the D750, presumably to cut costs they only offered "compressed" and "lossless compressed" RAW, even though for generations they have offered all three options including uncompressed. Heck, Nikon still offered TIF as a file format up until the D810, and will probably continue to do so in its high-end bodies.
I for one am fine with lossless compressed. It's worked for Canon all these years... ;-)
I want lossless compressed. Not willing to give up processing speed for uncompressed raws. People don't know what they are asking for. Even Phase backs are lossless compressed.
Actually I would not mind having a dummy downed raw as well , not full resolution but half of it. I can get around that with crop mode too. Sometimes I shoot jobs in the thousands that do not need to be 42mpx.
shirozina wrote:
Customers should get what they want and not what they need so let's have uncompressed RAW and compressed lossless RAW as an option.
Lossless compression is somewhat like a shorthand notation. Just to get the concept across lets look at the simplest possible lossless compression scheme. Lets imagine you are recording an image with some white area where you have say 10 pixels in a row which are all 255. The uncompressed format will write this in the file as:
255 255 255 255 255 255 255 255 255 255
The simplest possible lossless compression scheme would be to instead write this as:
255: 10 times
So you record "exactly" the same information just using fewer bytes. Of course I have simplified things a lot but you get the idea. So if you have lossless compression then there is not a huge motivation to also have uncompressed raw which will just take more space to store the same data. I suppose one area where it does help is if there is some data corruption where an uncompressed format will have fewer errors if a few bits gets corrupt.
An uncompressed raw file will be bigger than a losslessly compressed raw, and thus have at greater risk of problems arising from media corruption and media read/write errors. I think the regular compression are pretty fail safe.
If the victim (the person being interviewed) is either uncertain about details, or is under caution not to give out too much information, some brevity of statements often occur...
Compression is mostly good, when done correctly and at the correct places in the processing chain.
Since a lot of the processing has to be optimized for speed (in the in-camera case, also for energy efficiency for battery life) you want the compression type to be sequential and local, i.e not dependent on large surrounding areas and not to complex. I guess that's why Sony chose to do the compression in 8-pixel stripes.
Also, in a camera you want the compression at a late stage, since the data has to pass to the main processor for color interpretation and the jpg-engine. Transferring compressed data from the sensor to the main processor means you have to decompress it before you work on it, and then compress it again - and this also costs energy/time.
Bust just as you have hardware jpg-compression engine pipes going in parallel directly on the processor to the card controller, you can of course have HW raw compression engines that are transparent in speed, and also work at almost no energy cost (low overhead compared to programmed high-level compressors like a software solution). The problem is again the amount of cache needed on the sensor to be able to implement larger support areas (more efficient compression), and the HW implementation. Once you finalize the draft, it's FINAL. you can't change it between models, you can't update it, you're locked in to whatever your first choice was - or you're going to make life even harder for the raw converter software developers...
Actually I don't really care from a pure user PoV if Sony implement the compression or not. An A7Rii wouldn't be my technical camera of choice anyway...
theSuede wrote:
Yes, meant to answer those points earlier, but ran out of battery on the train...
There is no compression on the sensor. Exmor API's normally limit the programming steps to:
-bin type (depends on sensor model, not available on all)
-crop area
-bit depth
-black zero level
Some of the later sensors have a few other options in programmable modes too, but nothing regarding compression.
That's to bad, since gamma-compression in ADC's (non-linear comparator ramps) can increase the read-speeds by up to 30-40% without sacrificing read accuracy or tone resolution. But it makes the later stages of processing harder (to do correctly, from a color-interpretation PoV).
The rest is linear communication protocols that serialize the output data feeds.
And it's pure BS that the Bionz processors are 12-bit precision limited - they've been 16-bit precision since the A700 back in 2007-8 (?year?). It's very hard to make 16-bit and 32-bit processors handle lower-precision data flows, you'd have to incorporate memory-compression stages to work your way around the 32-bit wide memory bus. For a low-energy portable (battery-based) implementation, that's not really an option unless you have the entire ARM division behind you and several million processors per quarter in turnaround. Modern phone and tablet graphics processors do this (memory compression to lower the bandwidth demands on GDDR).
The Bionz-X is unfortunately also 16-bit math precision, I'd have hoped for more since the built in HDR-functions are really good.
Sony do seems to stumble on their own shoelaces on quite a few steps later in the processing chain though, the compression type used in ARW and ARW2-type compression is one of them. Value round-off errors are also quite sloppily handled in some of the implementations....Show more →
I bow down to the Suede, he sounds like he knows what he's talking about.
I was repeating things I've read elsewhere, and don't have firsthand knowledge of.
One thing I'm surprised has not been mentioned in any of the materials, especially with the bragging about the all-new shutter, is shutter lag / shutter blackout. Those, to me, were two of the biggest problems with the first A7 generation. I guess no news probably means bad news in this case.
Not necessarily, since the overall processing latency is down by a very VERY big margin. Unless they messed something else in the EVF chain up thoroughly - which Sony has an unfortunate record of being able to do on a regular basis, unfortunately...
But let's keep the hopes up!
The RX100v4 is a huge improvement in that aspect so far....
Lee Saxon wrote:
One thing I'm surprised has not been mentioned in any of the materials, especially with the bragging about the all-new shutter, is shutter lag / shutter blackout.
Yeah, I have been wondering about that too. Maybe we will have to wait for a stacked Exmor RS sensor for a big breakthrough here though.
theSuede wrote:
just as you have hardware jpg-compression engine pipes going in parallel directly on the processor to the card controller, you can of course have HW raw compression engines that are transparent in speed, and also work at almost no energy cost (low overhead compared to programmed high-level compressors like a software solution).
Do you (or anyone) know where/how the current lossy compression is implemented ie - are there specialized chip(s) in the pipeline and could those chips possibly be re-programmed to implement alternative compression algorithms? Or would the suggested Sony firmware fix be something that had to be done in a generalized processor?
is either uncertain about details, or is under caution not to give out too much information, some brevity of statements often occur... 