http://panasonic.co.jp/corp/news/official.data/data.dir/2013/02/en130204-6/en130204-6.html Since light separated by micro color splitters falls on the detectors in an overlapping manner, a new pixel layout and design algorithm are needed. The layout scheme is combined and optimized using an arithmetic processing technique designed specifically for mixed color signals. The result is highly sensitive and precise color reproduction. For example, if the structure separates light into a certain color and its complementary color, color pixels of white + red, white - red, white + blue, and white - blue are obtained and, using the arithmetic processing technique, are translated into normal color images without any loss of resolution.
Looks like an interesting advancement, more photons hitting the sensor sounds good to me.
p.1 #2 · Panasonic Diffractive Color Filter/Splitter patent
Yay, so we get the sharpness of an extra-heavily-AA-filtered sensor combined with the color separation fidelity of Foveon
In other words, this is a win for low-light / high-ISO sensitivity, but at the expense of resolution and color fidelity (probably similar resolution to typical, stronger-AA'd Bayer sensors).
p.1 #3 · Panasonic Diffractive Color Filter/Splitter patent
mpmendenhall wrote:
Yay, so we get the sharpness of an extra-heavily-AA-filtered sensor combined with the color separation fidelity of Foveon
In other words, this is a win for low-light / high-ISO sensitivity, but at the expense of resolution and color fidelity (probably similar resolution to typical, stronger-AA'd Bayer sensors).
From the Panasonic page I quoted above: The result is highly sensitive and precise color reproduction
So... Less noise and better color, I don't know if there will be a small loss in resolution, it will depend on the math to get RGB from "white + red, white - red, white + blue, and white - blue"
p.1 #4 · Panasonic Diffractive Color Filter/Splitter patent
LightShow wrote:
From the Panasonic page I quoted above:
So... Less noise and better color, I don't know if there will be a small loss in resolution, it will depend on the math to get RGB from "white + red, white - red, white + blue, and white - blue"
I suspect that "highly sensitive and precise color reproduction" is more marketing copy than a quantitative statement of color accuracy. This technology will have similar issues with color reproduction to Foveon sensors, which also strongly mix together colors on the sensor then un-tangle them with math later. Resolving colors from sum/difference values of adjacent pixels necessarily means more noise / lower resolution in color space than individually color-filtered pixels. Where the press release reads "approximately double the color sensitivity," it should really read "double the light sensitivity" (which is what they're actually talking about later), not an increase in color separation accuracy (which will be worse).
p.1 #5 · Panasonic Diffractive Color Filter/Splitter patent
The tech is totally different than Foveon tech though. Foveon has light-absorbing layers, so the colour fidelity is strongly tied to the ability to find materials which absorb very narrow bands of the light, whereas this splits it instead. Maybe it will have similar issues, maybe not, remains to be seen. It looks like there will be 1/2 pixel blur though, which I don't know how compares to current AA filters.
p.1 #6 · Panasonic Diffractive Color Filter/Splitter patent
Foveon actually doesn't have "materials which absorb very narrow bands of the light" (that would be great, if it did). Instead, it relies on the generic fact that longer wavelengths are absorbed more slowly on average (penetrate to a deeper depth) than shorter in silicon. Despite deceptive diagrams, Foveon sensors don't have a blue, red, and green layer. They have a bluish-white, orangeish-yellow, and reddish-orange layer as you go deeper. All colors are fairly strongly mixed together in each layer. This is similar to the results of this Panasonic technology --- colors are scrambled together at the sensor, so you need pixel sum/differences to get back to RGB, which ends up adding together noise contributions from several pixels to the the R,G,B values at each location (decreasing color resolution).
p.1 #7 · Panasonic Diffractive Color Filter/Splitter patent
Yeah, I was being generous I am not a fan of Foveon colours, to put it mildly. I would normally put quotes around the word "colours" when used in reference to the Foveon sensor The sharpness is very high, and the B&W results can look great, but that is where it ends for me.
Anyway, let's see what Panasonic can come up with.
p.1 #8 · Panasonic Diffractive Color Filter/Splitter patent
Here's a simple toy mathematical model to demonstrate what happens with color errors.
Suppose we have a camera with 3 pixels, and light consisting of R+G+B photons is hitting each pixel.
The "usual" scheme is to use an R, G, or B filter on each of the three pixels. This directly gives the color signal one wants, with the usual counting statistics noise proportional to the square root of the number of photons for each channel:
R, dR = sqrt(R)
G, dG = sqrt(G)
B, dB = sqrt(B)
Now, suppose we instead use white, yellow, and cyan filters (not exactly what's happening here, similar). Then, what we directly measure at the three pixels (with its counting statistics noise) is:
W = R+G+B, dW = sqrt(R+G+B)
Y = R+G, dY = sqrt(R+G)
C = G+B, dC = sqrt(G+B)
From this, we can un-tangle the colors to get back to R, G, B:
but now the color errors are about sqrt(5) to sqrt(6) times bigger (assuming roughly equal counts in each channel). We've collected about 7/3 times as much light (so the luminance signal is much better), but the color noise is as bad as if we only had 1/5 the light compared to the R,G,B filter scenario!
p.1 #9 · Panasonic Diffractive Color Filter/Splitter patent
I am not well versed with the math of sensors, but the 1/5th seems rather drastic, are you sure of that number? Doesn't the noise go down dramatically as the camera lowers the ISO to respond to the increased light recorded?
What about for the splitter, which doesn't use any filter?
p.1 #10 · Panasonic Diffractive Color Filter/Splitter patent
My example above isn't exactly what happens with the splitter --- in that case, you get even more light (not throwing away the R,G, or B from any channel, just moving it to another location), but that coincides with inflating the noise on the disentangled color channels even more.
I think I did the "5x worse" estimate right --- yes, it is drastic, but pre-mixing the colors that you want to separate out is pretty harmful! Luminance noise will go down corresponding to the higher light throughout (ability to work at lower ISO) --- the R', G', B' results now have highly correlated errors, so when you add them together the total brightness error is appropriately smaller. But the fractional uncertainty dR'/R', dG'/G', dB'/B' that indicates how well the color is determined is indeed much worse --- by a factor of ~sqrt(5), which is the same increase that you'd get from the un-premixed dR/R, dG/G, dB/B with 1/5 the light.
p.1 #11 · Panasonic Diffractive Color Filter/Splitter patent
I am trying to understand why it is so bad that the light is mixed, for the noise level. Wouldn't the separation of the light rather tend to smooth the differences, rather than adding noise to them? I could understand it better if the 5x factor was an error factor, rather than a noise factor.
p.1 #12 · Panasonic Diffractive Color Filter/Splitter patent
carstenw wrote:
I am trying to understand why it is so bad that the light is mixed, for the noise level.
The increase in noise is a double-whammy effect that you can see in my toy model above:
a) each pixel sees 2-3x more light, which increases the counting statistics by sqrt(2) to sqrt(3). Note, the *relative* error decreases (as it should) for more light; dW/W, dY/Y, dC/C < dR/R, dG/G, dB/B
b) to get back to R,G,B, you need to take sum/difference combinations of W,Y,C. When you do this, the (already larger) errors on each combine together into even bigger errors dR', dG', dB' > dW,dY,dC (bigger again by ~sqrt(2) to sqrt(3)), while the signals (because of the "-" in the combinations) R',G',B' < W,Y,C get smaller.
Combining effects (a) and (b) means the final relative errors dR'/R', etc. are HUGE!
p.1 #13 · Panasonic Diffractive Color Filter/Splitter patent
Hmm, it sort of makes sense to me, although I am still a little fuzzy on the details Thanks for taking the time to explain though. My math is clearly quite rusty, and my sensor knowledge a little lean.
p.1 #14 · Panasonic Diffractive Color Filter/Splitter patent
Remember the brief flirtation with CMY sensors in the late 90's early 00's? Twice as much light at the pixels, low noise! And yet no one uses them now and they didn't pay off then. The reason is outlined by Mendenhall above.
There is a definite risk to the same problem with this approach, but it is still an interesting concept. It could very well make an improvement, but it could also very well come with enough trade-offs to not make it that worthwhile (like CMY, or Foveon).
p.1 #15 · Panasonic Diffractive Color Filter/Splitter patent
Even with fairly severe color accuracy losses, a scheme like this (providing a quite significant luminance gain) could be very attractive for certain market segments. This sensor may not gain plaudits from pixel-peeping image quality purists, but it could be great for typical light-starved point-n-shoots. Since most scenes consist of large regions with fairly constant color, with fine luminance changes defining edges and details, you can get away with a loads of color-smearing noise reduction and still produce a plausible looking image if the luminance detail is there. For folks used to the cooked colors of P&S default JPEGs, the ability to get the same luminance info shooting under half the light would make this sensor really stand out from the competition in image quality.
I think some of the image quality tradeoffs rejected in the late '90s might come back, largely because there is a lot more computer power available in camera now. Sensor artifacts that would be blatantly obvious in crudely-processed '90s images can now be much more effectively "hidden" from casual view by more sophisticated noise reduction tricks.
p.1 #16 · Panasonic Diffractive Color Filter/Splitter patent
mpmendenhall wrote:
I think some of the image quality tradeoffs rejected in the late '90s might come back, largely because there is a lot more computer power available in camera now. Sensor artifacts that would be blatantly obvious in crudely-processed '90s images can now be much more effectively "hidden" from casual view by more sophisticated noise reduction tricks.
Ditching jpg would make me happy, something like JPEG XR.
p.1 #18 · Panasonic Diffractive Color Filter/Splitter patent
Don't assume that the existence of a patent implies production of a product using the concept. Just as in Academia we hear "Publish or Perish" I know from a career in Technology that it is a world of "Patent or Perish." Developers lead a Jeckyl and Hyde life; work on the Project vs work on the Career.
p.1 #19 · Panasonic Diffractive Color Filter/Splitter patent
In this case, there's more than a patent (or, apparently, 21+ of them) --- there's a press release. That indicates management/marketing might want to push this concept a bit further than the graveyard of ideas locked away from competitors. This doesn't necessarily mean a product --- sometimes a press release is purely irrelevant fluff to keep up the perception that a company is innovative --- but there is some chance that this is being circulated to drum up buzz about actual future products.
p.1 #20 · Panasonic Diffractive Color Filter/Splitter patent
mpmendenhall wrote:
Now, suppose we instead use white, yellow, and cyan filters (not exactly what's happening here, similar). Then, what we directly measure at the three pixels (with its counting statistics noise) is:
W = R+G+B, dW = sqrt(R+G+B)
Y = R+G, dY = sqrt(R+G)
C = G+B, dC = sqrt(G+B)
What? no. We are measuring W, Y, and C here, not R, G and B. The way you are adding the noise is as if each pixel was measuring R, G, and B and then using that to come up with it's value of W, Y, or C.
Starting from pixels that actually measure these things directly, we have
W = W, dW = sqrt(W)
Y = Y, dY = sqrt(Y)
C = C, dC = sqrt(C)
I am not a fan of Foveon colours, to put it mildly. I would normally put quotes around the word "colours" when used in reference to the Foveon sensor