Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Canon Forum | Join Upload & Sell

1       2       3      
4
       5       6       7       end
  

Archive 2013 · Why Do Canon Lenses Score so Low?

  
 
splathrop
Offline
• • •
Upload & Sell: Off
p.4 #1 · p.4 #1 · Why Do Canon Lenses Score so Low?


Most published tests of lenses usually leave out too much, especially variations in how different lenses render color and contrast. Sharpness is a subjective impression, not equivalent to resolution. Resolution is objectively measurable at least in principle. But resolution measured with a black and white test target can't show the ability of a lens to distinguish subtle shadings among close-together colors. Nor does it show very well whether the lens suffers color blindness in parts of the spectrum, nor what color casts it delivers. Good performance in those areas adds much to the subjective impression of sharpness, but tends to go unmeasured in published tests.

For that reason, published test results tend to be more useful for weeding out clunkers than for distinguishing among better-serviceable lenses. Among the latter, adequate resolution is the rule, but unmeasured differences in color and contrast nevertheless manifest as notable differences in the subjective impression of sharpness.

To take an example, the Canon 50mm f/1.4 delivers high resolution when used at f/5.6. The excellent and generally conservative Photozone test site rates it very well. At that aperture, the site gives the Canon a slight resolution edge over the Zeiss 50 f/2.0 MP. But they are close, and Photozone's charts compare resolution without measuring color rendering (except for aberrations, measured separately from resolution).

In side-by-side image comparisons that I make myself, I rate the Zeiss as subjectively sharper, and the difference does not seem particularly close. Better Zeiss color and contrast account for the difference. I doubt many image viewers who enjoy normal color vision would reach any different conclusion. Advantageously rendered subtle colors make the Zeiss images at least seem to show more detail. I suspect the difference is not merely subjective, but real, and would be readily measurable by methods different than those used to measure resolution.

Edited on Dec 22, 2013 at 04:11 AM · View previous versions



Dec 21, 2013 at 11:04 PM
gdanmitchell
Offline
• • • • • • •
Upload & Sell: Off
p.4 #2 · p.4 #2 · Why Do Canon Lenses Score so Low?


And most tests lead readers to over-value trivial or inconsequential differences or to look for a "winner" when multiple compared lenses are actually very, very good. They also encourage people to buy lenses because they are "best" rather than because the provide the functionality that the photographer needs.

There is nothing wrong with reading tests and lens comparisons - I read them, too. However, it is clearly the case that folks trying to find a way, any way, to make a decision that seems smart often gravitate to simple rating systems or declarations of this or that lens as being The Best Lens.

Dan



Dec 22, 2013 at 12:49 AM
John_T
Offline
• • • •
Upload & Sell: Off
p.4 #3 · p.4 #3 · Why Do Canon Lenses Score so Low?


...speaking of leading, I find loaded and misleading pulp press-style thread titles to be irresponsible, misplaced here and doesn't speak very well of the poster.


Dec 22, 2013 at 02:57 AM
verbiage
Offline
• • •
Upload & Sell: Off
p.4 #4 · p.4 #4 · Why Do Canon Lenses Score so Low?


Eric Gottesman wrote:
Even on the same camera I find it suspect that the 50 1.8II scores better than the Carl Zeiss Planar T 50mm f/1.4 ZE. No first hand comparison, but really?

http://www.dxomark.com/Reviews/Best-lenses-for-your-Canon-EOS-6D/Moderate-telephoto-and-standard-models

Also the Tamron 70-200mm 2.8 tests better than the Canon 70-20mm 2.8 II.

I'm really surprised at their lens results when compared on the same camera body.


+10

I cannot find the link now, but vaguely recall looking at their comparison of one of the 300 F/2.8 to a consumer zoom that went to 300, and supposedly left the 300 prime in the dust by a huge margin. I never looked at their lens ratings again after that.



Dec 22, 2013 at 04:34 AM
chez
Offline
• • • • • • •
Upload & Sell: Off
p.4 #5 · p.4 #5 · Why Do Canon Lenses Score so Low?


gdanmitchell wrote:
And most tests lead readers to over-value trivial or inconsequential differences or to look for a "winner" when multiple compared lenses are actually very, very good. They also encourage people to buy lenses because they are "best" rather than because the provide the functionality that the photographer needs.

There is nothing wrong with reading tests and lens comparisons - I read them, too. However, it is clearly the case that folks trying to find a way, any way, to make a decision that seems smart often gravitate to simple rating systems or declarations of this or that lens as being The
...Show more

Dan, why do you think so lowly about other people, not being able to use their own minds in looking at scores and thinking they blindly accept the ratings as gospel. Don't you think others might just use these ratings as 1 criteria in many when making their purchase decisions? Do you think most other photographers just look at the list and purchase the highest rated lens with disregard to need. I personally think more highly of fellow photographers and feel they are able to make intelligent decisions and having many different sites thst review gear is good as you can do your own homework reading these sites and getting a overall view. DXO s just one of many out there and I have no problems reading their tests on lens and cameras and taking their results in when making purchasing decisions.



Dec 22, 2013 at 08:20 AM
molson
Offline
• • • • • • •
Upload & Sell: On
p.4 #6 · p.4 #6 · Why Do Canon Lenses Score so Low?


chez wrote:
Dan, why do you think so lowly about other people, not being able to use their own minds in looking at scores and thinking they blindly accept the ratings as gospel. Don't you think others might just use these ratings as 1 criteria in many when making their purchase decisions? Do you think most other photographers just look at the list and purchase the highest rated lens with disregard to need. I personally think more highly of fellow photographers and feel they are able to make intelligent decisions and having many different sites thst review gear is good as you
...Show more

I think it's just a case of Dan never having met an argument he didn't like...



Dec 22, 2013 at 09:42 AM
Mark K
Offline
• • •
Upload & Sell: Off
p.4 #7 · p.4 #7 · Why Do Canon Lenses Score so Low?


ratsnest74 wrote:
http://www.dxomark.com/Lenses/Ratings/Optical-Metric-Scores

First page (top 28 lenses):

10 Nikon Lenses. 1 Canon. Even Tamron and Sigma each have 2 in the top 28...

The rating is a camera lens rating. So if Canon has a 36MP sensor, the result will be different



Dec 22, 2013 at 09:51 AM
Sneakyracer
Offline
• • • •
Upload & Sell: Off
p.4 #8 · p.4 #8 · Why Do Canon Lenses Score so Low?


ratsnest74 wrote:
http://www.dxomark.com/Lenses/Ratings/Optical-Metric-Scores

First page (top 28 lenses):

10 Nikon Lenses. 1 Canon. Even Tamron and Sigma each have 2 in the top 28...


They test the Camera / Lens combination not the lens by itself. Since both the Nikon D800/E and the D600/610 have a higher resolution the the highest res. Canon DSLR (the 5D3) the Nikon lenses on the Nikon bodies have higher scores.

So, the DxO rating or tests are good for comparing lenses in a single camera platform and give you an idea of the capability between platforms but are pretty useless for determining the ultimate resolution capability of the lens.



Dec 22, 2013 at 11:42 AM
Mikael Risedal
Offline
• •
Upload & Sell: Off
p.4 #9 · p.4 #9 · Why Do Canon Lenses Score so Low?


jpeter wrote:
I have never been to the dxo site before today. Not a very useful comparison system for me.

I notice a few Canon lenses that seem out of place. The 35f2 rates higher than 24-70vii, 400f2.8, 600f4 and 500f4 ?

It's hard to beat the digital_picture lens reviews and test charts for something more concrete.

JP


a real MTF test of the lens conducted by Hasselblads MTF lab in Gothenburg Sweden, old Photodo with the world largest MTF tests were performed by Hasselblad in Sweden , today the magazine Photo (Foto) here in Sweden runs a number of real tests every year

Edited on Dec 22, 2013 at 12:07 PM · View previous versions



Dec 22, 2013 at 11:59 AM
unclechuck
Offline
• • •
Upload & Sell: On
p.4 #10 · p.4 #10 · Why Do Canon Lenses Score so Low?


Sneakyracer wrote:
So, the DxO rating or tests are good for comparing lenses in a single camera platform and give you an idea of the capability between platforms but are pretty useless for determining the ultimate resolution capability of the lens.


Great proof of this observation in progress, live @ the Sony A7R Images thread in Alt Gear

50+ pages with a wide variety of glass, old and new ... just keep flippin' pages till you find your flavor



Dec 22, 2013 at 11:59 AM
Mikael Risedal
Offline
• •
Upload & Sell: Off
p.4 #11 · p.4 #11 · Why Do Canon Lenses Score so Low?


skibum5 wrote:
I just typed some comments on the sigma 24-105, parts of which have some relevance to your question, a quick cut and paste:

Count me super dubious of DxO when it comes to lenses.

First, why do they rank lenses based solely upon best aperture and focal length, what sense is there in that?
Second, since they seem to rate so many zooms best at wide open, like, just for starters here, they claim the 24-70 f/4 IS does best at 24mm and f/4? What are they measuring like the 4 center most pixels in the frame? Because I know that none of
...Show more

Heavily generalization in this post

If you look at DXO findings and compare them with for example Lenstip,Photozone, IR the results are often similar.
Tested lenses also have large variations



Dec 22, 2013 at 12:11 PM
gdanmitchell
Offline
• • • • • • •
Upload & Sell: Off
p.4 #12 · p.4 #12 · Why Do Canon Lenses Score so Low?


I think that quite a few of the observations made recently in this thread are roughly in line with my earlier points: Tests often produce interesting raw results, but the simple rating values don't reveal the context or assumptions behind the tests. As I wrote earlier, I read and try to understand test results and their context. It sounds like at least some participating in this thread do, too. The main concern comes when the final rating values (lens X scores value A) are assumed to tell us things that they don't really tell us - e.g. that lens A that scores X is not as "good" as lens B that scores X+1.

DXO (along with others) no doubt provides a useful service in doing the testing, and a lot of interesting data results from their tests. The problem is that most of the context is missing when the final lens "numbers" are brought up in the way that they were at the start of this thread. The numbers don't mean much unless you understand the context of what format, camera, and brand was tested and the way that the various factors that go in to the ratings are weighted. The problem isn't that someone is testing things - it is that too many people make presumptions based on the final ratings values that are unwarranted.

I stand by my observation - which also seems to be shared by many in this thread - that, given what we know about various lenses from actually using them and what we discover when we look more closely at the methodology of testing and how the final values are calculated, the actual meaning of the final lens ratings as a way of determining real world lens performance is of often very little value and all too often over-rated.

If you want to use and understand lens ratings - and I often do - I think that the best way to go about getting useful information from them includes:

- checking the results of one rating system against several others. (I like the interactive blur charts from http://www.slrgear.com/ for example, recognizing their presumptions and limits, too. There are others, including several mentioned in this thread.) Look for overall trends and areas of general agreement among these tests.
- paying attention also to subjective ratings and evaluations, especially those by people whose opinions have some validity and to reports that pool the feedback from larger groups of users. (Also learn avoid those photo writers and bloggers who offer up opinions that are inconsistent and unreliable.)
- trying to understand the assumptions behind and context of the factors that go into the ratings - just what do those numbers actually mean?
- balancing the technical testing data against other important factors including functionality and the reputation of the lens for design integrity, along with how these things relate to your photography.
- being aware that tests often speak to differences among things that are all very good, in which cases small differences in test performance that are measurable on the test bench may be insignificant and other factors will make a bigger difference.

Take care,

Dan

Edited on Dec 22, 2013 at 10:25 PM · View previous versions



Dec 22, 2013 at 12:50 PM
Scott Stoness
Offline
• • • • • •
Upload & Sell: On
p.4 #13 · p.4 #13 · Why Do Canon Lenses Score so Low?


Larate wrote:
Once again, you've made a definitive judgement*. Even if I don't agree with it**, I really prefer your opinion given in your last sentence.

* Although I understand it's your opinion, it's only to tell you that some "not knowledgeable" readers might take it as a set in stone fact. That was the sole purpose of my messages.

** Incidentally, I tend to prefer my 1Dx instead of my newly 5D3 for landscape photos. Photos shot with my 1Dx have less noise, require less sharpening than the ones shot with my 5D3 (talking about Raws). And the difference in resolution doesn't yield
...Show more

Larate: I agree - I try to make my opinions to look like opinions rather than facts but sometimes I stray. The 1dx is a good camera and to most, the difference between the 5diii and the 1dx would not be perceptible. However - my opinion - is that if you want the best landscape camera in the canon system - its the the 5diii.



Dec 22, 2013 at 03:20 PM
Sneakyracer
Offline
• • • •
Upload & Sell: Off
p.4 #14 · p.4 #14 · Why Do Canon Lenses Score so Low?


One crucial variable that most lens tests do not mention is the focusing distance used for the testing. I know and have seen first hand lenses that work great at close focusing distance but the performance at infinity drops of noticeably.

A test done at 10 feet of a lens that is mostly going to be used at or near infinity might not be useful to you if your are a landscape photographer. The test might give you a general idea of lens performance but do not completely rely on it.

Also slight field curvature that might make a lens test rather bad in the lab against a flat target might not be a huge issue in landscape situations for example since most scenes are quite deep (to infinity) and a slight bend in the focus plane might not be noticeable.



Dec 22, 2013 at 07:35 PM
Rajan Parrikar
Offline
• • • • •
Upload & Sell: On
p.4 #15 · p.4 #15 · Why Do Canon Lenses Score so Low?


Sneakyracer wrote:
One crucial variable that most lens tests do not mention is the focusing distance used for the testing. I know and have seen first hand lenses that work great at close focusing distance but the performance at infinity drops of noticeably.

A test done at 10 feet of a lens that is mostly going to be used at or near infinity might not be useful to you if your are a landscape photographer. The test might give you a general idea of lens performance but do not completely rely on it.

Also slight field curvature that might make a lens
...Show more


Diglloyd is one tester who does take into account these factors in his reports.



Dec 22, 2013 at 07:55 PM
Paul Mo
Offline
• • • • •
Upload & Sell: Off
p.4 #16 · p.4 #16 · Why Do Canon Lenses Score so Low?


The most reliable lens tests for me have always been whether or not I am envious of another photog's work.


I often find their test charts on the walls at World Press Photo, VII, or Magnum exhibitions, or inexplicably hidden inside things called 'books' in so-called libraries.



Dec 22, 2013 at 08:03 PM
gdanmitchell
Offline
• • • • • • •
Upload & Sell: Off
p.4 #17 · p.4 #17 · Why Do Canon Lenses Score so Low?


Paul Mo wrote:
... I often find their test charts on the walls at World Press Photo, VII, or Magnum exhibitions, or inexplicably hidden inside things called 'books' in so-called libraries.


Perhaps your best post ever. :-)

Dan



Dec 22, 2013 at 08:27 PM
scalesusa
Offline
• • • •
Upload & Sell: Off
p.4 #18 · p.4 #18 · Why Do Canon Lenses Score so Low?


DXO does not test lenses, the test lens / camera combinations. You get different scores for each lens based on the camera.

You CANNOT compare across brands. Most reputable web sites will tell you that the scores are only for a particular lens / camera combination.

For example, the Nikon 200-400mm F/4G which I own, rates a 12 on my D300s. If I change the camera to a D800, it suddenly rates a 25!\

How can a lens rate both a 12 and a 25? Its the same lens, only the camera body is different.

That's why you cannot compare across brands, or even across cameras.

Their lens score is meaningless, it will improve again if they come up with a 50mp camera.



Dec 22, 2013 at 08:42 PM
Pixel Perfect
Offline
• • • • • • •
Upload & Sell: On
p.4 #19 · p.4 #19 · Why Do Canon Lenses Score so Low?


Canon's lenses score so low because clearly they cannot reproduce that 3D look and microcontrast you can only get from spending a squillion dollars on Leica and a bit less on Zeiss glass.





Dec 22, 2013 at 08:56 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.4 #20 · p.4 #20 · Why Do Canon Lenses Score so Low?


I don't put much stock in the overall DXO scores for lenses. They don't seem to care much about the corner performance or some other important parameters. I recently received the Sigma 24-105/4 and OMG what a poor lens it is compared to the ratings! The Mpix rating is a joke, since the "best" results are only useful at one aperture and focal length. Look elsewhere for more detailed and better lens test reports.

EBH



Dec 22, 2013 at 09:05 PM
1       2       3      
4
       5       6       7       end




FM Forums | Canon Forum | Join Upload & Sell

1       2       3      
4
       5       6       7       end
    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.