Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | Canon Forum | Join Upload & Sell

1
       2       end
  

Archive 2014 · So what does DXO=29 really mean?

  
 
OntheRez
Offline
• • • • •
Upload & Sell: On
p.1 #1 · p.1 #1 · So what does DXO=29 really mean?


I'm looking carefully at the latest version of the Sigma 120-300mm f/2.8 OS, Sport (etc.) lens in the EF mount. I've never paid much attention to DXO, but decided to check them out this time. DXO assigns it a global score of 29 (on a 5DIII) and gives the Canon 300mm f/2.8L ver II a 32 on the same camera.

Obviously there are a number of other factors in play when comparing these lenses starting with the fact that the 120-300 is incredibly flexible and not forgetting that the Canon 300mm is 2.5X the price (give or take a few hundred, but who's counting?

I've been shooting HS night football with the 70-200mm f/2.8L IS ver II - a truly excellent lens. (DXO incidentally rates the 120-300 higher than this lens. ) I cover 8-man ball (played on a standard sized field). There are a lot of long plays and empty spaces thus I'm FL limited. The 200mm just doesn't reach that far. This can also be a problem in sports like baseball.

So setting aside price for a moment just what does 32 vs 29 tell me? I haven't read deeply at the DXO site, but looking at the three lenses in question the composite number seems a bit arbitrary. In fact DXO assigns a higher mark to the Sigma than it does to the 70-200mm f/2.8L a "fact" that amazes me.

So how useful are these numbers? Yes the 300 prime is "better" than the much cheaper zoom. If this were not so then I suspect all of you that own the 300mm would be jumping off tall buildings about now

Is the DXO mark linear, i.e. the 300mm is ~10% better than the 120-300mm? Or what?

Beyond that does anyone have real world experience with the "new" Sigma? I read varying things about its AF acquisition not being that good and others saying it's great. Huh?

Two different questions in one post but thought on either or both appreciated.

Robert



May 15, 2014 at 10:34 AM
mogul
Offline
• • • •
Upload & Sell: On
p.1 #2 · p.1 #2 · So what does DXO=29 really mean?


DXO is like the alchemists of old, turning lead into gold. No one understands their arbitrary scores.


May 15, 2014 at 10:49 AM
Ian.Dobinson
Offline
• • • • • •
Upload & Sell: Off
p.1 #3 · p.1 #3 · So what does DXO=29 really mean?


Robert , I have no idea what the difference between 29 and 32 is (other than 3 obviously ) but all I can say is .

id rather have a 300/2.8 IS (one of my 2 dream whites) but as I cant I have to say my non S 'BigSig' can really rock

just over 120mm f4 on a 7DIMG_0599.jpg by dobbie99, on Flickr

and then moments later 300mm
IMG_0618.jpg by dobbie99, on Flickr

id still love a white 300 (and 500) though . maybe one day



May 15, 2014 at 12:04 PM
RCicala
Offline
• • • •
Upload & Sell: Off
p.1 #4 · p.1 #4 · So what does DXO=29 really mean?


I'm not certain, but I think the DxO score is reached using this equipment. You add the three highest scores, then subtract the others from that to reach the overall total.







Courtesy Wikipedia Commons




May 15, 2014 at 12:18 PM
curious80
Offline
• • • •
Upload & Sell: Off
p.1 #5 · p.1 #5 · So what does DXO=29 really mean?


The trick with DxoMark is to always ignore their "scores" and head to the graphs and plots. The have a ton of useful data (specially about sensors), but in an attempt to simplify the findings they try to summarize all of that into these scores which often don't tell much. For lenses the scoring is particularly bad because they measure performance at various different apertures and focal lengths and then try to summarize all of that with one number.

Of course even when you look at their plots and graphs you need to know any hidden assumptions - for example the focus distance at which measurements were done - lenses often perform differently at close distances versus far.



May 15, 2014 at 12:43 PM
IndyFab
Offline
• • • • •
Upload & Sell: On
p.1 #6 · p.1 #6 · So what does DXO=29 really mean?


Ian, any experience using the Canon TC 1.4 & 2X III w/120/300 2.8

I read one individual say he had both III versions, and the Sigma TC's gave better performance with that lens.



May 15, 2014 at 01:28 PM
super35
Offline
• •
Upload & Sell: Off
p.1 #7 · p.1 #7 · So what does DXO=29 really mean?


The DXO lens scores are not arbitrary at all. But it takes some time reading the individual measurements if you want that score to have any real meaning.

Sigma 120-300mm F2.8 DG OS HSM S Canon vs Canon EF 300mm f/2.8L IS II USM

1. The Sigma has better transmision. 2.8 TStop vs 3.2 TStop

2. The Canon has lower distortion 0% vs .2%

3. The Canon has less vignetting -1EV vs -1.1EV

4. The Canon has less chromatic aberration 4µm vs 7µm

Would you agree based on these measurements that the Canon is the overall better lens? If so, then you must also agree that the Canon 32 vs Sigma 29 score has some useful meaning.

Edited on May 15, 2014 at 02:34 PM · View previous versions



May 15, 2014 at 02:04 PM
Scott Stoness
Offline
• • • • • •
Upload & Sell: On
p.1 #8 · p.1 #8 · So what does DXO=29 really mean?


The score you are refering to is based on their weighting of effective mpx, dynamic range, and ISO performance - I think.

I agree that it is not overly useful because we each woudl have different weighting for these and we would have other things we woudl weight as well like weather sealing, reliability, glass available, fps.....

Their subscores for lens body combinations are more useful if you understand them. Eg a 20 effective mpx yields more than a 18 by 2 mpx. Unfortunately this is not well defined either but it is directionally useful.

Ditto on dynamic range and ISO. Its good to know that how sensor/lens line up on a standard test relatively but it needs to have a grain of salt because its a one dimensional test and you would do it differently.

However, more data is better data, and I check other sources to cross reference their conclusions where avaialbe.

My favorite tools are photozone.de, thedigitalpicture picture comparison, and dxomark. Usually between the 3 you can get a pretty good appreciation of a lens/body combination.

But the d800 and a7r, which are rated very highly are not for me the best mulitpurpose bodies. 5diii or 1dx, if you had to go to something that counts are much better.

Most people would not choose a house on weighting but its useful to have a scale to test your choice. Because my wife would reject any house that has concrete counter tops.... I woudl reject all that are far from work. .... a weighting could not capture vetoes.



May 15, 2014 at 02:10 PM
Ian.Dobinson
Offline
• • • • • •
Upload & Sell: Off
p.1 #9 · p.1 #9 · So what does DXO=29 really mean?


IndyFab wrote:
Ian, any experience using the Canon TC 1.4 & 2X III w/120/300 2.8

I read one individual say he had both III versions, and the Sigma TC's gave better performance with that lens.


No , Ive been on the hunt for a good 2x for it for a while now . (I posted a thread about that a while back)

I cant find any decent priced sigma 2x around (Im not paying new price for something I wont use that often) .
I did try a mk1 Canon which seemed to be pretty good . I was tempted but wanted to find a sigma first to see if it got even better .

If I find the test shots of the Canon Mk1 I'll post them .

as for 1.4 TC's . Ive only used a Kenko 1.4 . results were decent enough . If I can pick up a pair of siggies I probably will .
from what I've seen (there's a TC'd Sigma 120-300 group on flickr thats well worth a look) the sigma or canon ones produce great results .



May 15, 2014 at 05:09 PM
AJSJones
Offline
• • • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · So what does DXO=29 really mean?


For good lenses, the score is determined mostly by the camera the lens is tested on. So it's hard to extract a number relating just to the lens's actual capability, only on the cameras they can test it on. They can't test the Canon on the D800E: (For most lenses, the T stop, aberration, vignetting were reproducibly measured from camera to camera (phew) Here's some points of reference on the score "scale"


Sigma 120-300mm F2.8 DG OS HSM S

Camera------"Score"------"Perceived MP"
D800 E----------32--------------29
D800------------30---------------21
5D3--------------29---------------20
5D----------------21---------------12

Zeiss Otus 55mm 1.4

Camera------"Score"------"Perceived MP"
D800 E----------50---------------33
D800------------45---------------29
5D3--------------38---------------21
5D----------------29---------------13

Sigma 50 mm EX DG HSM 1.4

Camera------"Score"------"Perceived MP"
D800 E----------36--------------23
D800------------32---------------16
5D3--------------29---------------15
5D----------------22---------------10




May 15, 2014 at 05:10 PM
skibum5
Offline
• • • • • •
Upload & Sell: Off
p.1 #11 · p.1 #11 · So what does DXO=29 really mean?


nothing

And I say this as someone who is NOT a DxO sensor plot naysayer. But I don't really get their lens test at all. First of all even their lens data plots have had some VERRRRRY weird results in the past so I don't know that I even trust the raw data. Second, they make it harder to get to the good data for lenses and their overall scores are just....

Apparently they try to put a single number on a lens, nevermind f/1.4 vs f/10 and 16mm vs 300mm performance on it might be very different (never mind there are no 16-300mm f/1.4 lenses, but you get my point) and it's weird since they rate them by effective perceptual megapixels (which is it's own can of worms) at what they consider to be the ideal best point for the lens. The latter sounds reasonable enough until you find that that means they decide to rate the 50mm 1.4 at 1.4, because, umm, yeah, everyone knows that is where THAT lens performs the best . But even that might be one thing, compare them all wide open then. But.... some lenses are f/4 and some f/1.0 wide open so how fair is it to compare f/1.0 to f/4? Even for the same type of lens, say 50mm prime, they have done stuff like compare one at f/1.2 to another at f/2?!! And then add in zooms and they may compare a 24-105 at f/5 50mm to a 24-70 f/4 IS at 24mm f/4 or whatnot, it doesn't make sense. So they may be comparing a 50mm 1.2 at 1.2 to a 24-70 II at 24mm f/2.8 to a 24-105 at 50mm f/5, etc. (don't recall exact numbers, just using hypotheticals there).

Apparently they love near wide open performance and actual light gathering and they seem to find that focal length and aperture lets the most light through while maintain some minimum score relative to the lens overall or something or other. All nonsense IMO. Sometimes the scores might make sense for most purposes, many of times not as much.





May 15, 2014 at 05:51 PM
ggreene
Offline
• • • • •
Upload & Sell: Off
p.1 #12 · p.1 #12 · So what does DXO=29 really mean?


Scott Stoness wrote:
But the d800 and a7r, which are rated very highly are not for me the best mulitpurpose bodies. 5diii or 1dx, if you had to go to something that counts are much better.


The low light sports category is really puzzling. DXO's actual high ISO measurements show the 1DX equal to or better then the D800 (much better at high ISO DR) yet the summary score has the D800 pulling ahead. The only place the D800 is ahead in the measurements is low ISO DR which has what to do with high ISO sports?



May 15, 2014 at 05:59 PM
goosemang
Offline
• • •
Upload & Sell: Off
p.1 #13 · p.1 #13 · So what does DXO=29 really mean?


It means you should turn off the computer and go take some photographs.


May 15, 2014 at 06:11 PM
super35
Offline
• •
Upload & Sell: Off
p.1 #14 · p.1 #14 · So what does DXO=29 really mean?


I don't particularly like DXO, but I don't get why people take the best and most complete scientifically measured lens data on the planet and throw it away because they don't like the way the final scores are presented? WTH? This makes zero sense, especially considering the raw measurements are also provided for you to arrive at your own conclusions.

Just consider the DXO optical metric P-Mpix score as simple way to describe the resolution of a lens on a particular camera, as a system. Do you agree that the 70-200 2.8 II outresolves the 24-105L? Well, so do they, and on every Canon camera they have tested them on. I've never found one of their lens measurements that was out of line with my own real world experience with multiple lens/body combinations.

As for the DXOmark lens score, just think of it as a gross oversimplification of the image quality of the lens, aside from resolution. If you want to know more detail, the data is in the graphs they provide.



May 15, 2014 at 06:42 PM
OntheRez
Offline
• • • • •
Upload & Sell: On
p.1 #15 · p.1 #15 · So what does DXO=29 really mean?


Lots of interesting ideas here. Appreciate the responses.

super35, as you note the Sigma has a better T-stop by a fair margin. I'd say 0.2% difference in distortion is probably not visible to the eye. The .1 EV in vignetting might be noticeable but easy to correct. I don’t know the scale on which CA is measured but I do know 3 micro-meters is a very small distance. Whether this all adds up to a DXO mark difference of 3 points doesn't seem to follow. Don't get me wrong. I'm sure the Canon 300mm f/2.8L IS v. II is a superior lens at its only focal length. Does 3 points on the DXO scale make it ~$4500 better?

AJSJones, I immediately noticed that altering the camera made a lens "better" or "worse." Fascinating. Gives me more questions than answers and clearly suggests that the rating of a lens is not "clean" that is, it is not constant across multiple bodies.

Ian, thanks for the excellent pix. I don't know what you bribed the cat with but the lens obviously can be sharp. My intended use would be in sports (in lousy light) so AF acquisition is something I'm very interested in. Have you done any low light work with the lens? (Like say ISO 6400, 1/800 sec on a 1Dx)

Roger, thank you for a thoroughly scientific explanation. I'd been told it was the sum of the purple, yellow, and red minus the the sum of the green and orange divided by the blue value. Some people just insist on making things more complex than they need to be.

Scott, I have looked at the sub-scores as they do purport to actually measure something and tend to agree with your suggestion to consider their direction more than say their magnitude because as you note things don't seem to be well defined. The veto factor is also ignored. I doubt anyone would disagree with the statement that "the Canon 300mm f/2.8L IS ver II is better than the Sigma 120-300mm f/2.8 OS Sport at 300mm." Of course the Sigma just destroys the Canon at 120mm

Skibum I really did blow beer thru my noise when I stumbled onto effective perceptual megapixels. Say what? Anyway this was my first in-depth encounter with the DXO mark and I've come away rather bemused by the whole thing. All I really want to know is how well does the Sigma work on fast moving things in very bad light? I don't have $3.5K to pick it up as an "experiment."

Robert



May 15, 2014 at 07:15 PM
skibum5
Offline
• • • • • •
Upload & Sell: Off
p.1 #16 · p.1 #16 · So what does DXO=29 really mean?


super35 wrote:
I don't particularly like DXO, but I don't get why people take the best and most complete scientifically measured lens data on the planet and throw it away because they don't like the way the final scores are presented? WTH? This makes zero sense, especially considering the raw measurements are also provided for you to arrive at your own conclusions.

Just consider the DXO optical metric P-Mpix score as simple way to describe the resolution of a lens on a particular camera, as a system. Do you agree that the 70-200 2.8 II outresolves the 24-105L? Well, so do they, and
...Show more

Because I don't think their raw lens data is the best measured lens data on the planet. I see lots of weird results. Results that nobody else has ever gotten.

Their raw sensor data seems to be good though (if the overall scores are somewhat whatever).



May 15, 2014 at 07:19 PM
skibum5
Offline
• • • • • •
Upload & Sell: Off
p.1 #17 · p.1 #17 · So what does DXO=29 really mean?


ggreene wrote:
The low light sports category is really puzzling. DXO's actual high ISO measurements show the 1DX equal to or better then the D800 (much better at high ISO DR) yet the summary score has the D800 pulling ahead. The only place the D800 is ahead in the measurements is low ISO DR which has what to do with high ISO sports?


probably the difference is chroma noise and color sensitivity at high iso under natural, but dim, lighting



May 15, 2014 at 07:20 PM
Paul Mo
Offline
• • • • •
Upload & Sell: Off
p.1 #18 · p.1 #18 · So what does DXO=29 really mean?


IMHO, careful scrutiny of images on photo sharing sites tells us more about a lens than DXO ever could - what real people are doing with the lens(es) in question. That being said, nothing beats actual use as the Internet can be a devious place with regards to (ahem) 'reviews'.


May 15, 2014 at 07:36 PM
ggreene
Offline
• • • • •
Upload & Sell: Off
p.1 #19 · p.1 #19 · So what does DXO=29 really mean?


skibum5 wrote:
probably the difference is chroma noise and color sensitivity at high iso under natural, but dim, lighting


The color sensitivity chart is virtually a tie at higher ISO with the D800 ahead by a tiny margin. In the meantime the 1DX has a full stop DR advantage past 1000 ISO. That's a hell of a bias towards color sensitivity.



May 15, 2014 at 07:50 PM
super35
Offline
• •
Upload & Sell: Off
p.1 #20 · p.1 #20 · So what does DXO=29 really mean?


Can you give an example of a weird result in the current DXO lens test data? I remember one about 4 years ago with the 70-200mm f/2.8L IS II, but that lens has since been retested. This is the same data the used in DXO Optics Pro for lens corrections.

Who else publishes body/lens combo results of almost every EF lens on almost every EOS body sold in the past 10 years? If you use it wisely this data is worth thousands of dollars and they provide it free of charge. No one else even comes close. All other lens testing is done on a single body, many times 2 to 4 generations old.


skibum5 wrote:
Because I don't think their raw lens data is the best measured lens data on the planet. I see lots of weird results. Results that nobody else has ever gotten.

Their raw sensor data seems to be good though (if the overall scores are somewhat whatever).




May 15, 2014 at 09:16 PM
1
       2       end




FM Forums | Canon Forum | Join Upload & Sell

1
       2       end
    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.