Register · Search · Software · Join Upload & Sell · Hosting

Moderated by: Fred Miranda
Username   Password

FM Forum Rules
Canon SLRs, primes, and zooms lenses reviews
FM Forums | Canon-mount SLRs | Join Upload & Sell   
Search Used
1  
2
   3   4   end
  

Archive 2008 · Dynamic Range Measure
  
 
Harvey Moore
Offline
Upload & Sell: On
p.2 #1 · Dynamic Range Measure


Probably the method used. See "Data Processing" on this page from the Clarkvision site. click here

mfurman wrote:

Richard Steer:
retested both cameras today, with two tests each.

350D: 7 1/3 stops
40D: 8 2/3 stops


I am wondering what could explain the discrepancy between these tests and the results found in this article:

http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/index.html

Please see Figure 4, for instance.



Feb 11, 2008 at 12:50 PM
mfurman
Offline
Upload & Sell: Off
p.2 #2 · Dynamic Range Measure


Harvey Moore:
Probably the method used.


Which one is more analytic and "correct" then?

Feb 11, 2008 at 01:02 PM
Harvey Moore
Offline
Upload & Sell: On
p.2 #3 · Dynamic Range Measure


Don't know the answer to this Michael.

The Clarkvision is a lot more detailed and number related, Pondria's method is simpler and ACR related.

From Clarkvision:
The procedure below is based on using ImagesPlus 2.5, hereafter called IP. The image processing system you use must be able to convert the raw data with a linear scale (Photoshop will not do this) and process the data with at least 16-bit math (IP uses 64-bit floating point math).

I think that any judgement I would make, for instance, is prejudiced by my use of the 5D, CS3 w/ACR, and what my eyes see on the screen and in in print.

The relative dr performance should be the same for all valid test methods.

Feb 11, 2008 at 01:27 PM
ejmartin
Offline
Upload & Sell: Off
p.2 #4 · Dynamic Range Measure


mfurman wrote:

Richard Steer:
retested both cameras today, with two tests each.

350D: 7 1/3 stops
40D: 8 2/3 stops


I am wondering what could explain the discrepancy between these tests and the results found in this article:

http://www.clarkvision.com/imagedetail/digital.sensor.performance.summary/index.html

Please see Figure 4, for instance.


You have to carefully read the text of Clark's webpage to see what is being plotted in Fig 4, which has nothing to do directly with any camera's DR. The plot is the number of stops between pixel saturation at the lowest ISO (full well capacity, as it is called), and the lowest read noise at high ISO. In other words, it is the ratio of the maximum possible signal that the camera can record to the smallest possible signal that the camera can record. It is not dynamic range because the former occurs at the lowest ISO and the latter at the highest ISO; but of course when one takes pictures one takes them at fixed ISO -- you don't get to choose one ISO for the part of the frame with highlights and another for shadows!

The figure in that article which has to do with actual DR is figure 5. DR plotted there is the so-called "engineering" DR, which is raw saturation level (max possible raw value of a pixel patch) over read noise (min possible raw value of a pixel patch), both measured at a fixed ISO. This is a rather liberal definition of DR, but has the advantage of being defined in terms of raw sensor data and properties, unfiltered through any raw converter.

Pondria's DR measurement filters the raw data through a particular raw converter, and while of practical use since it tells one how much latitude there is for extracting DR in conversion, it depends strongly on the programming of the converter -- how good the highlight recovery algorithms are, how the converter sets the black and white points, etc. It will be different from converter to converter, and even version to version of a given converter (for instance highlight recovery is much better in ACR 4.x than in earlier versions). The fact that it is a few stops less than the engineering DR could be interpreted as showing that ACR leaves a few stops of DR on the table; though I suspect some of it has to do with the fact that there is a distribution of luminance values being pushed around by the exposure slider, and the criterion being used to decide saturation uses the left edge of the shadow histogram and the right edge of the highlight histogram, so at the vary least it chops out the width of the histogram from inclusion in the DR.

Feb 11, 2008 at 05:41 PM
John Power
Offline
Dedicated FM
Upload & Sell: Off
p.2 #5 · Dynamic Range Measure


Pondria, can't you come up with a testing procedure that involves sex, drugs or guns so I can participate too dammit....

Edited by John Power on Feb 11, 2008 at 03:28 PM GMT


Edited on Feb 11, 2008 at 08:28 PM


Feb 11, 2008 at 07:26 PM
hahr
Offline
Upload & Sell: On
p.2 #6 · Dynamic Range Measure


Pondria wrote:
Any more new data please ?


put me down for 9-2/3 on a 1D III.   i could lie and say '10' but the test proved otherwise.

for archival purposes, i also reached 9-2/3 on my former 1Ds II.

-erik

Feb 11, 2008 at 08:03 PM
Me_XMan
Offline
Upload & Sell: Off
p.2 #7 · Dynamic Range Measure


40D has the same DR as 1DsMK2
Very interesting.

Feb 11, 2008 at 08:11 PM
Tom_W
Offline
Dedicated FM
Upload & Sell: On
p.2 #8 · Dynamic Range Measure


Richard Steer wrote:
I retested both cameras today, with two tests each.

350D: 7 1/3 stops
40D: 8 2/3 stops

Was possibly a bit generous at the dark end yesterday. The fact that I'm in about the right ballpark with the 350D makes me feel a bit happier about these results.

Richard


I'm showing about 8 1/3 on my 40D. I didn't have natural light though, but I'm not sure that matters. I'll retest when I get a chance.

Feb 11, 2008 at 11:29 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #9 · Dynamic Range Measure


ejmartin wrote:

The figure in that article which has to do with actual DR is figure 5. DR plotted there is the so-called "engineering" DR, which is raw saturation level (max possible raw value of a pixel patch) over read noise (min possible raw value of a pixel patch), both measured at a fixed ISO. This is a rather liberal definition of DR, but has the advantage of being defined in terms of raw sensor data and properties, unfiltered through any raw converter.



The Figure 5 DR curves for the 1DmkII and 1DmkIII at low ISO are more in line with what Canon stated in the 1DmkIII white paper. Without quantifying the DR, Canon stated that the DR of the two cameras (1DmkII and 1DmkIII) is about the same at low ISO speeds.

It's very peculiar "to me" that Canon Marketing chose not to say anything positive about "high ISO DR"; they just kind of stopped after the low ISO commentary. You know, it's just not like marketing to stop short of saying anything positive if in fact there is something positive to say in the first place. In fact, even if there isn't anything positive to say marketing will usually figure out some way to create a positive spin without actually lying.

I always figured (maybe "guess" is a better word) in-camera noise reduction was performed at the high ISO settings for the 1DmkIII prior to creation of the RAW data; maybe, just maybe "if" the in-camera noise reduction also wipes out some data, then it would be illegitimate to actually claim an increase in DR at the higher ISO speeds. Do you have any thoughts on this subject? Do you believe the Clarkvision DR numbers at high ISO speeds is legitimate for the 1DmkII and 1DmkIII bodies?

Regards,

Joe Kurkjian

Feb 12, 2008 at 01:10 AM
ejmartin
Offline
Upload & Sell: Off
p.2 #10 · Dynamic Range Measure


jkurkjia wrote:
ejmartin wrote:

The figure in that article which has to do with actual DR is figure 5. DR plotted there is the so-called "engineering" DR, which is raw saturation level (max possible raw value of a pixel patch) over read noise (min possible raw value of a pixel patch), both measured at a fixed ISO. This is a rather liberal definition of DR, but has the advantage of being defined in terms of raw sensor data and properties, unfiltered through any raw converter.



The Figure 5 DR curves for the 1DmkII and 1DmkIII at low ISO are more in line with what Canon stated in the 1DmkIII white paper. Without quantifying the DR, Canon stated that the DR of the two cameras (1DmkII and 1DmkIII) is about the same at low ISO speeds.

It's very peculiar "to me" that Canon Marketing chose not to say anything positive about "high ISO DR"; they just kind of stopped after the low ISO commentary. You know, it's just not like marketing to stop short of saying anything positive if in fact there is something positive to say in the first place. In fact, even if there isn't anything positive to say marketing will usually figure out some way to create a positive spin without actually lying.


Or sometimes they just lie, like that laughable sunset shot showing the superiority of 14-bit files. But as in politics sometimes the Big Lie works well.


I always figured (maybe "guess" is a better word) in-camera noise reduction was performed at the high ISO settings for the 1DmkIII prior to creation of the RAW data; maybe, just maybe "if" the in-camera noise reduction also wipes out some data, then it would be illegitimate to actually claim an increase in DR at the higher ISO speeds. Do you have any thoughts on this subject? Do you believe the Clarkvision DR numbers at high ISO speeds is legitimate for the 1DmkII and 1DmkIII bodies?

Regards,

Joe Kurkjian


I haven't seen any evidence of NR on raws at any ISO in the 1D3.

There is something a bit odd in the data quoted for the 1D3 on Clark's summary page. The gain figures in table 3b look to be off by one stop, ie the gain 4.92 at ISO 50 looks to be about the same as the gain I measured (5.1) at ISO 100. Correspondingly, his read noise figures in table 4b look to be about half of what I see (should be about 4 electrons at high ISO rather than the values of about 2 electrons which are quoted). So I think he made a clerical error early on with the gain and that has propagated through to the rest of the quoted values for the 1D3. Really, read noise in electrons for the 1D3 is not all that different from the 1D2; Canon did quite a good job on that score already in the previous generation; the only thing they improved on was to lower the pattern noise somewhat.

The DR figures Clark quotes at high ISO are OK though, I get about 10.3 stops at ISO 1600 and about 9.3 stops at ISO 3200, about the same as Clark exhibits; I think that means that his clerical error cancels out when taking the ratio of raw saturation to read noise to get DR.

The improvement in DR comes basically from the improvement in gain -- 3.3 electrons/12bit ADU at ISO 400 for the 1D2, 5.1 in the same units for the 1D3 -- 2/3 stop better for the 1D3. Balancing this in the other direction is the change in ISO calibration on the newer Canons which gives back 1/3 stop of this improvement. Finally, it is important to remember that these figures are at the pixel level; if we look on a per area basis, the 1D3 is getting this DR and S/N ratio with pixels that are 36% smaller, and so gets nearly a half stop additional advantage if we compare equal size patches of the frame. Putting it all together, ignoring the change in ISO calibration the 1D3 outperforms the 1D2/1D2N by a bit more than a full stop on a per area basis, perhaps 2/3 stop or a bit more if one accounts for the change in ISO calibration.

These results are tabulated in tables 8 and 9 of
http://theory.uchicago.edu/~ejm/pix/20d/posts/tests/D300_40D_tests/

Note from these tables that the 1D2/1D2N is really a rather poor performer, masking its dismal quantum efficiency by having huge pixels, and thereby doing OK on measures that are pixel-based rather than image percentage based (as they should be IMO).



Feb 12, 2008 at 03:04 AM
 



Pondria
Offline
Dedicated FM
Upload & Sell: Off
p.2 #11 · Dynamic Range Measure


John Power wrote:
Pondria, can't you come up with a testing procedure that involves sex, drugs or guns so I can participate too dammit....



The procedure should be easy enough for even a lawyer to follow. You won't be held liable for the numbers that you post here. Please participate


Feb 12, 2008 at 03:07 AM
jkurkjia
Offline
Upload & Sell: Off
p.2 #12 · Dynamic Range Measure


ejmartin wrote:

I haven't seen any evidence of NR on raws at any ISO in the 1D3.



Just out of curiosity, how can you deduce the "absence" of NR at high ISO settings?

Regards,

Joe Kurkjian


Feb 12, 2008 at 09:34 AM
mfurman
Offline
Upload & Sell: Off
p.2 #13 · Dynamic Range Measure


to ejmartin

Thank you for such an interesting post and the link to the article.

Feb 12, 2008 at 11:46 AM
ejmartin
Offline
Upload & Sell: Off
p.2 #14 · Dynamic Range Measure


jkurkjia wrote:
ejmartin wrote:

I haven't seen any evidence of NR on raws at any ISO in the 1D3.



Just out of curiosity, how can you deduce the "absence" of NR at high ISO settings?

Regards,

Joe Kurkjian


Any averaging of neighboring pixels would reduce the photon shot noise; the "gain" derived from the shot noise at high ISO would not be consistent with the gain derived at low ISO. For instance, the "gain" at ISO 1600 should be 1/16 of the gain at ISO 100. I really dislike that "gain" terminology -- really it's the conversion factor from raw levels to electrons, so it goes down in inverse proportion to the ISO, which is a true gain (electronic amplification). Anyway, if the shot noise were less at ISO 1600 due to NR, the inferred gain would be higher than it should be. High ISO samples I have seen show no such behavior.

Feb 12, 2008 at 01:56 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #15 · Dynamic Range Measure


ejmartin wrote:
jkurkjia wrote:
ejmartin wrote:

I haven't seen any evidence of NR on raws at any ISO in the 1D3.



Just out of curiosity, how can you deduce the "absence" of NR at high ISO settings?

Regards,

Joe Kurkjian


Any averaging of neighboring pixels would reduce the photon shot noise; the "gain" derived from the shot noise at high ISO would not be consistent with the gain derived at low ISO. For instance, the "gain" at ISO 1600 should be 1/16 of the gain at ISO 100. I really dislike that "gain" terminology -- really it's the conversion factor from raw levels to electrons, so it goes down in inverse proportion to the ISO, which is a true gain (electronic amplification). Anyway, if the shot noise were less at ISO 1600 due to NR, the inferred gain would be higher than it should be. High ISO samples I have seen show no such behavior.


To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian




Feb 12, 2008 at 04:56 PM
ejmartin
Offline
Upload & Sell: Off
p.2 #16 · Dynamic Range Measure


jkurkjia wrote:

To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian


If you put the lens cap on, there is no photon signal and what is measured is entirely read noise and no photon noise. It does not go down inversely with ISO, though it does go down by nearly that much between low, main ISO's (say between 100 and 200, and between 200 and 400) because of the optimized, on-sensor amplification associated to those ISO's. Read noise is constant across intermediate ISO's drawn from the same main ISO (eg 250 and 320 are the same as 200) and plateaus at high ISO's (800 and above). The D300 is much the same (I've only looked at the main ISO's); I haven't seen quality data from the D3.

But read noise need not bear any relation to ISO. What I was mentioning in my previous post was that one can use photon shot noise measurements (photon shot noise is independent and unrelated to sensor read noise) to deduce the conversion between raw levels and electrons, and that conversion factor halves when the ISO doubles. If one monkeys with the raw data in a way that decreases photon shot noise, the inferred conversion factor is thrown off in a way inconsistent with the way it should scale with ISO. And I haven't seen that. Some people have comlained about long exposure NR on the Nikons; I wouldn't know about that, all my tests were done with relatively short exposures.

Edited on Feb 12, 2008 at 05:44 PM


Feb 12, 2008 at 05:43 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #17 · Dynamic Range Measure


ejmartin wrote:
jkurkjia wrote:

To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian


If you put the lens cap on, there is no photon signal and what is measured is entirely read noise and no photon noise. It does not go down inversely with ISO, though it does go down by nearly that much between low, main ISO's (say between 100 and 200, and between 200 and 400) because of the optimized, on-sensor amplification associated to those ISO's. Read noise is constant across intermediate ISO's drawn from the same main ISO (eg 250 and 320 are the same as 200) and plateaus at high ISO's (800 and above). The D300 is much the same (I've only looked at the main ISO's); I haven't seen quality data from the D3.

But read noise need not bear any relation to ISO. What I was mentioning in my previous post was that one can use photon shot noise measurements (photon shot noise is independent and unrelated to sensor read noise) to deduce the conversion between raw levels and electrons, and that conversion factor halves when the ISO doubles. If one monkeys with the raw data in a way that decreases photon shot noise, the inferred conversion factor is thrown off in a way inconsistent with the way it should scale with ISO. And I haven't seen that. Some people have comlained about long exposure NR on the Nikons; I wouldn't know about that, all my tests were done with relatively short exposures.


Assuming the design has carefully minimized Johnson noise as a factor the root cause of noise for the "lens cap on" case should be determined by the equivalent noise input of the amplifiers. To me it would seem incomprehensible that Canon would use more than one amplifier in the signal processing chain prior to A/D conversion (i.e. they use one amplification stage and change gains either by reducing the signal or the amplifier's gain at low ISO settings and/or increasing the amplifier's gain at high ISO settings). It just seems to me that read noise should be inversely proportional to ISO settings "unless" there are external factor in play (e.g. NR).

I hear what you are saying about screwing with the RAW data but if "only" the equivalent of the highest spatial frequencies are "monkeyed" with it's possible to miss (maybe even easily miss) the fact that some form of NR is applied. Anyway, I greatly appreciate that you are willing to share your thoughts on the subject and just wanted to toss out a few things in the back of my mind.

Regards,

Joe Kurkjian




Edited on Feb 12, 2008 at 07:45 PM


Feb 12, 2008 at 07:44 PM
GeneO
Offline
Dedicated FM
Upload & Sell: Off
p.2 #18 · Dynamic Range Measure


I do believe they have different amplifiers optimized for different ISO.
Gene

jkurkjia wrote:
ejmartin wrote:
jkurkjia wrote:

To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian


If you put the lens cap on, there is no photon signal and what is measured is entirely read noise and no photon noise. It does not go down inversely with ISO, though it does go down by nearly that much between low, main ISO's (say between 100 and 200, and between 200 and 400) because of the optimized, on-sensor amplification associated to those ISO's. Read noise is constant across intermediate ISO's drawn from the same main ISO (eg 250 and 320 are the same as 200) and plateaus at high ISO's (800 and above). The D300 is much the same (I've only looked at the main ISO's); I haven't seen quality data from the D3.

But read noise need not bear any relation to ISO. What I was mentioning in my previous post was that one can use photon shot noise measurements (photon shot noise is independent and unrelated to sensor read noise) to deduce the conversion between raw levels and electrons, and that conversion factor halves when the ISO doubles. If one monkeys with the raw data in a way that decreases photon shot noise, the inferred conversion factor is thrown off in a way inconsistent with the way it should scale with ISO. And I haven't seen that. Some people have comlained about long exposure NR on the Nikons; I wouldn't know about that, all my tests were done with relatively short exposures.


Assuming the design has carefully minimized Johnson noise as a factor the root cause of noise for the "lens cap on" case should be determined by the equivalent noise input of the amplifiers. To me it would seem incomprehensible that Canon would use more than one amplifier in the signal processing chain prior to A/D conversion (i.e. they use one amplification stage and change gains either by reducing the signal or the amplifier's gain at low ISO settings and/or increasing the amplifier's gain at high ISO settings). It just seems to me that read noise should be inversely proportional to ISO settings "unless" there are external factor in play (e.g. NR).

I hear what you are saying about screwing with the RAW data but if "only" the equivalent of the highest spatial frequencies are "monkeyed" with it's possible to miss (maybe even easily miss) the fact that some form of NR is applied. Anyway, I greatly appreciate that you are willing to share your thoughts on the subject and just wanted to toss out a few things in the back of my mind.

Regards,

Joe Kurkjian





Feb 12, 2008 at 10:50 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #19 · Dynamic Range Measure


GeneO wrote:

I do believe they have different amplifiers optimized for different ISO.
Gene



Gene, wow, where did you come up with that information? Has Canon published anything about this subject?

It's certainly possible to switch between multiple amplifiers as long as power and space don't kill you; I guess this approach can be made to work if the switches and amplifiers are integrated into one chip (otherwise the wiring and pads for interconnects would really eat into available space). Oh well, enough speculation on my part, I'll wait to hear back from you regarding the source of the information, thanks.

Joe


Feb 13, 2008 at 01:19 AM
mttran
Offline
Dedicated FM
Upload & Sell: Off
p.2 #20 · Dynamic Range Measure


Following the instructions I got 9 2/3 for the 1ds2, 8 2/3 for the 1d2 and 7 1/3 for the 300D

Edited on Feb 13, 2008 at 08:09 AM


Feb 13, 2008 at 07:01 AM
1  
2
   3   4   end




FM Forums | Canon-mount SLRs | Join Upload & Sell

1  
2
   3   4   end
    
 

You are not logged in. Login or Register

  Username   Password  
Lost your password?