Upload & Sell: Off
| p.1 #11 · Do you think DxOMark is accurate? |
lens testing is tricky,
also some sites focus once in the center while others refocus for edge and corner, the former means they also need to be utterly parallel which is very tricky to do, insanely tricky and field curvature changes by distance, in some real world photography the tests leaving FC in play might be a better truth teller but in other cases not at all
some sites appear to use test chart only a few feet away, others try for 20-25' away
copy to copy variation can exist too
Agreed that lens testing is extremely tough! As a scientist by training, I can see that there are far too many independent variables to precisely control, that may make one lens measure better than others. Usually no attempt is made to optimize or measure these myriad of variables, or their interaction. The distance dependent field curvature mentioned above, is one such example. Lenstip and TDP use charts at close to minimum focal distance for some of their testing, and many complain that this creates results which do not reflect actual use. Its very difficult to compare different, incompatible brands like Canon and Nikon, since different camera bodies have different sensors, and one cannot separate the lens from the capture and processing of the data. So arbitrary bodies and test conditions are chosen to test different lenses just like arbitrary lenes are used to test different bodies. Its actually a very small design space of probable users situations.
At best, the testers choose one condition (or range) for each variable and can compare various lenses on that same body. The tests are limited, so the interpretation of the data has to also be very limited in scope. There are many ways of measuring performance, so it is meaningless to compare the absolute results of one testing lab to another. There are simply too many combinations of variables to be comprehensive and exacting. While this does not mean lens testing useless, it does mean that one should take any data set with a grain of salt, the scope of the testing is very narrow.
On the copy to copy variation, read this
Statistics and error bars are rarely presented in the realm of lens testing.