Register · Search · Software · Join Upload & Sell · Hosting

Moderated by: Fred Miranda
Username   Password

FM Forum Rules
Canon SLRs, primes, and zooms lenses reviews
FM Forums | Canon-mount SLRs | Join Upload & Sell   
Search Used
1
2
3 end
  

Archive 2008 · Dynamic Range Measure
  
 
hahr
Offline
Upload & Sell: On
p.2 #1 · Dynamic Range Measure


Pondria wrote:
Any more new data please ?


put me down for 9-2/3 on a 1D III.   i could lie and say '10' but the test proved otherwise.

for archival purposes, i also reached 9-2/3 on my former 1Ds II.

-erik

Feb 11, 2008 at 08:03 PM
Me_XMan
Offline
Upload & Sell: Off
p.2 #2 · Dynamic Range Measure


40D has the same DR as 1DsMK2
Very interesting.

Feb 11, 2008 at 08:11 PM
Tom_W
Offline
Dedicated FM
Upload & Sell: Off
p.2 #3 · Dynamic Range Measure


Richard Steer wrote:
I retested both cameras today, with two tests each.

350D: 7 1/3 stops
40D: 8 2/3 stops

Was possibly a bit generous at the dark end yesterday. The fact that I'm in about the right ballpark with the 350D makes me feel a bit happier about these results.

Richard


I'm showing about 8 1/3 on my 40D. I didn't have natural light though, but I'm not sure that matters. I'll retest when I get a chance.

Feb 11, 2008 at 11:29 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #4 · Dynamic Range Measure


ejmartin wrote:

The figure in that article which has to do with actual DR is figure 5. DR plotted there is the so-called "engineering" DR, which is raw saturation level (max possible raw value of a pixel patch) over read noise (min possible raw value of a pixel patch), both measured at a fixed ISO. This is a rather liberal definition of DR, but has the advantage of being defined in terms of raw sensor data and properties, unfiltered through any raw converter.



The Figure 5 DR curves for the 1DmkII and 1DmkIII at low ISO are more in line with what Canon stated in the 1DmkIII white paper. Without quantifying the DR, Canon stated that the DR of the two cameras (1DmkII and 1DmkIII) is about the same at low ISO speeds.

It's very peculiar "to me" that Canon Marketing chose not to say anything positive about "high ISO DR"; they just kind of stopped after the low ISO commentary. You know, it's just not like marketing to stop short of saying anything positive if in fact there is something positive to say in the first place. In fact, even if there isn't anything positive to say marketing will usually figure out some way to create a positive spin without actually lying.

I always figured (maybe "guess" is a better word) in-camera noise reduction was performed at the high ISO settings for the 1DmkIII prior to creation of the RAW data; maybe, just maybe "if" the in-camera noise reduction also wipes out some data, then it would be illegitimate to actually claim an increase in DR at the higher ISO speeds. Do you have any thoughts on this subject? Do you believe the Clarkvision DR numbers at high ISO speeds is legitimate for the 1DmkII and 1DmkIII bodies?

Regards,

Joe Kurkjian

Feb 12, 2008 at 01:10 AM
ejmartin
Offline
Upload & Sell: Off
p.2 #5 · Dynamic Range Measure


jkurkjia wrote:
ejmartin wrote:

The figure in that article which has to do with actual DR is figure 5. DR plotted there is the so-called "engineering" DR, which is raw saturation level (max possible raw value of a pixel patch) over read noise (min possible raw value of a pixel patch), both measured at a fixed ISO. This is a rather liberal definition of DR, but has the advantage of being defined in terms of raw sensor data and properties, unfiltered through any raw converter.



The Figure 5 DR curves for the 1DmkII and 1DmkIII at low ISO are more in line with what Canon stated in the 1DmkIII white paper. Without quantifying the DR, Canon stated that the DR of the two cameras (1DmkII and 1DmkIII) is about the same at low ISO speeds.

It's very peculiar "to me" that Canon Marketing chose not to say anything positive about "high ISO DR"; they just kind of stopped after the low ISO commentary. You know, it's just not like marketing to stop short of saying anything positive if in fact there is something positive to say in the first place. In fact, even if there isn't anything positive to say marketing will usually figure out some way to create a positive spin without actually lying.


Or sometimes they just lie, like that laughable sunset shot showing the superiority of 14-bit files. But as in politics sometimes the Big Lie works well.


I always figured (maybe "guess" is a better word) in-camera noise reduction was performed at the high ISO settings for the 1DmkIII prior to creation of the RAW data; maybe, just maybe "if" the in-camera noise reduction also wipes out some data, then it would be illegitimate to actually claim an increase in DR at the higher ISO speeds. Do you have any thoughts on this subject? Do you believe the Clarkvision DR numbers at high ISO speeds is legitimate for the 1DmkII and 1DmkIII bodies?

Regards,

Joe Kurkjian


I haven't seen any evidence of NR on raws at any ISO in the 1D3.

There is something a bit odd in the data quoted for the 1D3 on Clark's summary page. The gain figures in table 3b look to be off by one stop, ie the gain 4.92 at ISO 50 looks to be about the same as the gain I measured (5.1) at ISO 100. Correspondingly, his read noise figures in table 4b look to be about half of what I see (should be about 4 electrons at high ISO rather than the values of about 2 electrons which are quoted). So I think he made a clerical error early on with the gain and that has propagated through to the rest of the quoted values for the 1D3. Really, read noise in electrons for the 1D3 is not all that different from the 1D2; Canon did quite a good job on that score already in the previous generation; the only thing they improved on was to lower the pattern noise somewhat.

The DR figures Clark quotes at high ISO are OK though, I get about 10.3 stops at ISO 1600 and about 9.3 stops at ISO 3200, about the same as Clark exhibits; I think that means that his clerical error cancels out when taking the ratio of raw saturation to read noise to get DR.

The improvement in DR comes basically from the improvement in gain -- 3.3 electrons/12bit ADU at ISO 400 for the 1D2, 5.1 in the same units for the 1D3 -- 2/3 stop better for the 1D3. Balancing this in the other direction is the change in ISO calibration on the newer Canons which gives back 1/3 stop of this improvement. Finally, it is important to remember that these figures are at the pixel level; if we look on a per area basis, the 1D3 is getting this DR and S/N ratio with pixels that are 36% smaller, and so gets nearly a half stop additional advantage if we compare equal size patches of the frame. Putting it all together, ignoring the change in ISO calibration the 1D3 outperforms the 1D2/1D2N by a bit more than a full stop on a per area basis, perhaps 2/3 stop or a bit more if one accounts for the change in ISO calibration.

These results are tabulated in tables 8 and 9 of
http://theory.uchicago.edu/~ejm/pix/20d/posts/tests/D300_40D_tests/

Note from these tables that the 1D2/1D2N is really a rather poor performer, masking its dismal quantum efficiency by having huge pixels, and thereby doing OK on measures that are pixel-based rather than image percentage based (as they should be IMO).



Feb 12, 2008 at 03:04 AM
Pondria
Offline
Dedicated FM
Upload & Sell: Off
p.2 #6 · Dynamic Range Measure


John Power wrote:
Pondria, can't you come up with a testing procedure that involves sex, drugs or guns so I can participate too dammit....



The procedure should be easy enough for even a lawyer to follow. You won't be held liable for the numbers that you post here. Please participate


Feb 12, 2008 at 03:07 AM
jkurkjia
Offline
Upload & Sell: Off
p.2 #7 · Dynamic Range Measure


ejmartin wrote:

I haven't seen any evidence of NR on raws at any ISO in the 1D3.



Just out of curiosity, how can you deduce the "absence" of NR at high ISO settings?

Regards,

Joe Kurkjian


Feb 12, 2008 at 09:34 AM
mfurman
Offline
Upload & Sell: Off
p.2 #8 · Dynamic Range Measure


to ejmartin

Thank you for such an interesting post and the link to the article.

Feb 12, 2008 at 11:46 AM
ejmartin
Offline
Upload & Sell: Off
p.2 #9 · Dynamic Range Measure


jkurkjia wrote:
ejmartin wrote:

I haven't seen any evidence of NR on raws at any ISO in the 1D3.



Just out of curiosity, how can you deduce the "absence" of NR at high ISO settings?

Regards,

Joe Kurkjian


Any averaging of neighboring pixels would reduce the photon shot noise; the "gain" derived from the shot noise at high ISO would not be consistent with the gain derived at low ISO. For instance, the "gain" at ISO 1600 should be 1/16 of the gain at ISO 100. I really dislike that "gain" terminology -- really it's the conversion factor from raw levels to electrons, so it goes down in inverse proportion to the ISO, which is a true gain (electronic amplification). Anyway, if the shot noise were less at ISO 1600 due to NR, the inferred gain would be higher than it should be. High ISO samples I have seen show no such behavior.

Feb 12, 2008 at 01:56 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #10 · Dynamic Range Measure


ejmartin wrote:
jkurkjia wrote:
ejmartin wrote:

I haven't seen any evidence of NR on raws at any ISO in the 1D3.



Just out of curiosity, how can you deduce the "absence" of NR at high ISO settings?

Regards,

Joe Kurkjian


Any averaging of neighboring pixels would reduce the photon shot noise; the "gain" derived from the shot noise at high ISO would not be consistent with the gain derived at low ISO. For instance, the "gain" at ISO 1600 should be 1/16 of the gain at ISO 100. I really dislike that "gain" terminology -- really it's the conversion factor from raw levels to electrons, so it goes down in inverse proportion to the ISO, which is a true gain (electronic amplification). Anyway, if the shot noise were less at ISO 1600 due to NR, the inferred gain would be higher than it should be. High ISO samples I have seen show no such behavior.


To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian




Feb 12, 2008 at 04:56 PM
ejmartin
Offline
Upload & Sell: Off
p.2 #11 · Dynamic Range Measure


jkurkjia wrote:

To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian


If you put the lens cap on, there is no photon signal and what is measured is entirely read noise and no photon noise. It does not go down inversely with ISO, though it does go down by nearly that much between low, main ISO's (say between 100 and 200, and between 200 and 400) because of the optimized, on-sensor amplification associated to those ISO's. Read noise is constant across intermediate ISO's drawn from the same main ISO (eg 250 and 320 are the same as 200) and plateaus at high ISO's (800 and above). The D300 is much the same (I've only looked at the main ISO's); I haven't seen quality data from the D3.

But read noise need not bear any relation to ISO. What I was mentioning in my previous post was that one can use photon shot noise measurements (photon shot noise is independent and unrelated to sensor read noise) to deduce the conversion between raw levels and electrons, and that conversion factor halves when the ISO doubles. If one monkeys with the raw data in a way that decreases photon shot noise, the inferred conversion factor is thrown off in a way inconsistent with the way it should scale with ISO. And I haven't seen that. Some people have comlained about long exposure NR on the Nikons; I wouldn't know about that, all my tests were done with relatively short exposures.

Edited on Feb 12, 2008 at 05:44 PM


Feb 12, 2008 at 05:43 PM
jkurkjia
Offline
Upload & Sell: Off
p.2 #12 · Dynamic Range Measure


ejmartin wrote:
jkurkjia wrote:

To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian


If you put the lens cap on, there is no photon signal and what is measured is entirely read noise and no photon noise. It does not go down inversely with ISO, though it does go down by nearly that much between low, main ISO's (say between 100 and 200, and between 200 and 400) because of the optimized, on-sensor amplification associated to those ISO's. Read noise is constant across intermediate ISO's drawn from the same main ISO (eg 250 and 320 are the same as 200) and plateaus at high ISO's (800 and above). The D300 is much the same (I've only looked at the main ISO's); I haven't seen quality data from the D3.

But read noise need not bear any relation to ISO. What I was mentioning in my previous post was that one can use photon shot noise measurements (photon shot noise is independent and unrelated to sensor read noise) to deduce the conversion between raw levels and electrons, and that conversion factor halves when the ISO doubles. If one monkeys with the raw data in a way that decreases photon shot noise, the inferred conversion factor is thrown off in a way inconsistent with the way it should scale with ISO. And I haven't seen that. Some people have comlained about long exposure NR on the Nikons; I wouldn't know about that, all my tests were done with relatively short exposures.


Assuming the design has carefully minimized Johnson noise as a factor the root cause of noise for the "lens cap on" case should be determined by the equivalent noise input of the amplifiers. To me it would seem incomprehensible that Canon would use more than one amplifier in the signal processing chain prior to A/D conversion (i.e. they use one amplification stage and change gains either by reducing the signal or the amplifier's gain at low ISO settings and/or increasing the amplifier's gain at high ISO settings). It just seems to me that read noise should be inversely proportional to ISO settings "unless" there are external factor in play (e.g. NR).

I hear what you are saying about screwing with the RAW data but if "only" the equivalent of the highest spatial frequencies are "monkeyed" with it's possible to miss (maybe even easily miss) the fact that some form of NR is applied. Anyway, I greatly appreciate that you are willing to share your thoughts on the subject and just wanted to toss out a few things in the back of my mind.

Regards,

Joe Kurkjian




Edited on Feb 12, 2008 at 07:45 PM


Feb 12, 2008 at 07:44 PM
GeneO
Offline
Dedicated FM
Upload & Sell: Off
p.2 #13 · Dynamic Range Measure


I do believe they have different amplifiers optimized for different ISO.
Gene

jkurkjia wrote:
ejmartin wrote:
jkurkjia wrote:

To ensure I'm not reading anything into your answer, let me ask a very specific question; hopefully you have the data in hand.

Regarding the 1DmkIII, if you put a lens cap on and measure noise at ISO settings of 100, 200, ..., 1600, are the noise levels at the higher ISO speeds related to ISO-100 by factors of two? Just in case you have the data at your disposal, let me ask the same question for the D3 and D300 at ISO 200, 400, ..., 3200, and high ISO speeds referenced to ISO-200.

Regards,

Joe Kurkjian


If you put the lens cap on, there is no photon signal and what is measured is entirely read noise and no photon noise. It does not go down inversely with ISO, though it does go down by nearly that much between low, main ISO's (say between 100 and 200, and between 200 and 400) because of the optimized, on-sensor amplification associated to those ISO's. Read noise is constant across intermediate ISO's drawn from the same main ISO (eg 250 and 320 are the same as 200) and plateaus at high ISO's (800 and above). The D300 is much the same (I've only looked at the main ISO's); I haven't seen quality data from the D3.

But read noise need not bear any relation to ISO. What I was mentioning in my previous post was that one can use photon shot noise measurements (photon shot noise is independent and unrelated to sensor read noise) to deduce the conversion between raw levels and electrons, and that conversion factor halves when the ISO doubles. If one monkeys with the raw data in a way that decreases photon shot noise, the inferred conversion factor is thrown off in a way inconsistent with the way it should scale with ISO. And I haven't seen that. Some people have comlained about long exposure NR on the Nikons; I wouldn't know about that, all my tests were done with relatively short exposures.


Assuming the design has carefully minimized Johnson noise as a factor the root cause of noise for the "lens cap on" case should be determined by the equivalent noise input of the amplifiers. To me it would seem incomprehensible that Canon would use more than one amplifier in the signal processing chain prior to A/D conversion (i.e. they use one amplification stage and change gains either by reducing the signal or the amplifier's gain at low ISO settings and/or increasing the amplifier's gain at high ISO settings). It just seems to me that read noise should be inversely proportional to ISO settings "unless" there are external factor in play (e.g. NR).

I hear what you are saying about screwing with the RAW data but if "only" the equivalent of the highest spatial frequencies are "monkeyed" with it's possible to miss (maybe even easily miss) the fact that some form of NR is applied. Anyway, I greatly appreciate that you are willing to share your thoughts on the subject and just wanted to toss out a few things in the back of my mind.

Regards,

Joe Kurkjian





Feb 12, 2008 at 10:50 PM
 



jkurkjia
Offline
Upload & Sell: Off
p.2 #14 · Dynamic Range Measure


GeneO wrote:

I do believe they have different amplifiers optimized for different ISO.
Gene



Gene, wow, where did you come up with that information? Has Canon published anything about this subject?

It's certainly possible to switch between multiple amplifiers as long as power and space don't kill you; I guess this approach can be made to work if the switches and amplifiers are integrated into one chip (otherwise the wiring and pads for interconnects would really eat into available space). Oh well, enough speculation on my part, I'll wait to hear back from you regarding the source of the information, thanks.

Joe


Feb 13, 2008 at 01:19 AM
mttran
Offline
Dedicated FM
Upload & Sell: Off
p.2 #15 · Dynamic Range Measure


Following the instructions I got 9 2/3 for the 1ds2, 8 2/3 for the 1d2 and 7 1/3 for the 300D

Edited on Feb 13, 2008 at 08:09 AM


Feb 13, 2008 at 07:01 AM
Pondria
Offline
Dedicated FM
Upload & Sell: Off
p.2 #16 · Dynamic Range Measure


Data updated upto mttran's. Thank you everyone for the contribution.

I'm very glad that the data are pretty consistent although measured by different individuals.

One thing to notice.
I have been suspecting that 5D and 1D-2 are sharing the sensors with the same technology. So are 1Ds-2 and 1D-3. What I mean by "Same technology" is that the two sensors use the identical design and process except that one just have larger die ( more cells ) than the other. The DR data supports the theory.

Another thing:
Canon has been managing to reduce the pixel pitch ( to add more pixels ) while improving the DR ( or noise ) UNTIL 1Ds-3. Looks like they finally ran out of magic. It seems that they have reached the maximum entitlement of 35mm sensors. We will get more pixels only at the expense of the quality of each pixel.


Feb 13, 2008 at 04:38 PM
ejmartin
Offline
Upload & Sell: Off
p.2 #17 · Dynamic Range Measure


Pondria wrote:

One thing to notice.
I have been suspecting that 5D and 1D-2 are sharing the sensors with the same technology. So are 1Ds-2 and 1D-3. What I mean by "Same technology" is that the two sensors use the identical design and process except that one just have larger die ( more cells ) than the other. The DR data supports the theory.


While the pixel pitches are similar, the 5D pixels have substantially larger efficiency than the 1D2 (4.1 vs 3.3 electrons/raw level at ISO 400); similarly the 1D3 has substantially larger efficiency than the 1Ds2 (5.1 vs 3.3 in the same units). I don't think they are the same technology.

Feb 13, 2008 at 09:03 PM
stanj
Offline
Dedicated FM
Upload & Sell: On
p.2 #18 · Dynamic Range Measure


Where do these electron numbers come from?

Feb 13, 2008 at 09:06 PM
John Black
Offline
Dedicated FM
Upload & Sell: On
p.2 #19 · Dynamic Range Measure


Pondria - I think you're using the numbers the best way possible - not as absolutes, but as relative comparisons and as some type of general trend analysis. It is disappointing to see the 1Ds3 at 9 stops. Hopefully others can add to the dataset to either validate or augment Stan's results.

In regards to the 40D's 9 2/3 stops, you'd probably want to know if that was with or with out highlight tone priority.

Feb 13, 2008 at 10:07 PM
ejmartin
Offline
Upload & Sell: Off
p.2 #20 · Dynamic Range Measure


stanj wrote:
Where do these electron numbers come from?



http://theory.uchicago.edu/~ejm/pix/20d/posts/tests/D300_40D_tests/

Table 7. Most of the data here is taken directly from Roger Clark's website; 1D3 measurement is my own. There seems to be a clerical error in Clark's 1D3 reported measurement, it is off by one stop (ISO 100 gain is actually the ISO 200 gain, 200 gain is actually the 400 gain, etc).

Edited on Feb 13, 2008 at 10:15 PM


Feb 13, 2008 at 10:11 PM
thedigitalbean
Offline
Dedicated FM
Upload & Sell: On
p.2 #21 · Dynamic Range Measure


ejmartin wrote:
http://theory.uchicago.edu/~ejm/pix/20d/posts/tests/D300_40D_tests/

Table 7. Most of the data here is taken directly from Roger Clark's website; 1D3 measurement is my own. There seems to be a clerical error in Clark's 1D3 reported measurement, it is off by one stop (ISO 100 gain is actually the ISO 200 gain, 200 gain is actually the 400 gain, etc).


A fascinating and informative read, I enjoyed the report.

Feb 13, 2008 at 10:23 PM
stanj
Offline
Dedicated FM
Upload & Sell: On
p.2 #22 · Dynamic Range Measure


Me_XMan wrote:
40D has the same DR as 1DsMK2
Very interesting.


I measured / tested 3x, same result. If my method was flawed then it was flawed for both cameras in the same way...

Feb 13, 2008 at 10:51 PM
stanj
Offline
Dedicated FM
Upload & Sell: On
p.2 #23 · Dynamic Range Measure


John Black wrote:
Pondria - I think you're using the numbers the best way possible - not as absolutes, but as relative comparisons and as some type of general trend analysis. It is disappointing to see the 1Ds3 at 9 stops. Hopefully others can add to the dataset to either validate or augment Stan's results.

In regards to the 40D's 9 2/3 stops, you'd probably want to know if that was with or with out highlight tone priority.


Without. That mode means absolutely nothing to me, so I didn't use it. Same with the 1Ds3.

I'd also like to see more values for the 1Ds3, as I find the number by itself disappointing; then again I am not disappointed with the actual image quality, and I feel that it's easier to recover highlights with the 1Ds3 than with the 1Ds2. Naturally the devil may lie in the shadows.

While my absolute value of 9 for 1Ds3 may be off, given that I measured three cameras (40D, 1Ds2, 1Ds3) at the same time using the same method, and repeated the test a total of 3 times, always with same results (I even sent the raws to Pondria for verification), I think it's safe to say that _my_ 40D has more DR as measured by Pondria than _my_ 1Ds3. I still prefer the 1Ds3 over my 40D, though

Feb 13, 2008 at 10:55 PM
Alan321
Offline
Dedicated FM
Upload & Sell: On
p.2 #24 · Dynamic Range Measure


ejmartin, please forgive me if I'm being a bit dense (it happens more frequently these days ) but how is measuring DR on a per image area basis any different from a per pixel basis ? Wouldn't you be scaling both noise and signal by the same amount and hence getting the same DR result ? Or am I talking about the wrong quantity, being a signal to noise ratio instead of an absolute electron count ?

- Alan

Feb 13, 2008 at 11:20 PM
Alan321
Offline
Dedicated FM
Upload & Sell: On
p.2 #25 · Dynamic Range Measure


I'm also a bit surprised and disappointed to see the 1Ds3 result lower than the 1Ds2, 1D3 and 40D results. Can anyone (Stan ?) confirm what these numbers mean anything in practice - i.e. does the 1Ds3 seem to have less DR than the 1D3 or 1Ds2 ? If it did then I'd expect it to handle high iso less cleanly. If it seems to be cleaner or at least as clean as the 1D3 then it would seem there is a discrepency in the test results (despite being re-tested several times). Maybe it is mathematically noisier afterall but the noise is visibly finer and less intrusive because there are more pixels per image ?

- Alan

Feb 13, 2008 at 11:30 PM




FM Forums | Canon-mount SLRs | Join Upload & Sell
1
2
3 end
    
 

You are not logged in. Login or Register

  Username   Password  
Lost your password?