Like with most DxO Mark tests, it seems the testing portion is relatively solid. It's how we are interpreting them that can be the problem. I'd like to re-remind those that didn't see theSuede's opinion on this subject from last page:
So if I did interprete the results correctly, the 21 must be very soft at f8 everywhere but in the centre. Of course they don't mention at what distance, but my guess is at extremely close. I never used this lens but I have no problem in believing the results since the lens has been designed for landscapes not macro reproduction work. However, since they do not mention this clearly as far as I know, a misinformed reader would tend to believe the 21 is very bad in the corners. I do find this a bit dishonest.
Makes sense to me, Edward. Focusing distance seems to be the big question mark from the various testing sources, and, as we all know, that can make things vary wildly.
douglasf13 wrote:
Makes sense to me, Edward. Focusing distance seems to be the big question mark from the various testing sources, and, as we all know, that can make things vary wildly.
The second big question after focusing distance, is curvature of field. DxOmark test for a flat target, and lenses with curvature of field will yield bad results in the corners. Photozone refocuses for corners and have the inverse problem of not quantifying the curvature of field. Still, their results correspond better with my real world experience than DxOmark.
douglasf13 wrote:
Like with most DxO Mark tests, it seems the testing portion is relatively solid. It's how we are interpreting them that can be the problem. I'd like to re-remind those that didn't see theSuede's opinion on this subject from last page:
For the most part I agree with this, but like I mentioned initially, the MTF plots for nearly all lenses at nearly all apertures show a modulation transfer of 0.91 +/- 0.01 at 10 lp/mm. Are really all lenses so close in micro contrast, or is the DxOmark method extremely bad for measuring it?
Alundeb: yes, when measured in a low-contrast scene - as black-on white prints really are, they give a 150:1 contrast, or 7Ev of easy-to-handle contrast - the 10lp/mm measurement at "a certain" F-stop is VERY similar for almost any lens imaginable. Certainly within +/- 0.02 units as the MTF scales that DxO (and ALL other camera-based measurements, PZ, SLRgear and so on!) are all based on the ~1 lp/mm being the "100% reference" - making a 10 lp/mm relative contrast of 91% pretty universal. And not very relevant, truth be told.
The REAL wide open optical center measurement MTF at 10 lp/mm can range between 75% (short F1.4 lens, or a crappy zoom) and 95% (Very good F2.8 macro). That's QUITE a noticeable difference, as anyone that's ever been mindful of picture quality would know.
As I said; mind what you're watching, and know the ramifications. Otherwise those "camera-based" measurement sites are pretty useless.
Close range focus field curvature. The chart looks very plausible to me. It's also totally irrelevant to a landscape shooter.
If I could wish for just one thing in a lens test - and you still have to use a camera to do the test - it would be a backlit pinhole / slit measurement at somewhere between 100-200x focal length. Focused to "best focus", and with an indication of total field curvature. This could be:
50mm lens tested. Performance at:
center, 100xFL=5.00m distance initial focus > [test result]
mid field, best focal distance at 4.5m (90%) > [test result]
corner, best focal distance 3.8m (75%) > [test result]
Doing this would be very work intensive, as you'd have to have a linear rail dolly to refocus mid-field and corner tests. And a really long corridor for 600mm lenses... :-)
theSuede wrote:
The REAL wide open optical center measurement MTF at 10 lp/mm can range between 75% (short F1.4 lens, or a crappy zoom) and 95% (Very good F2.8 macro). That's QUITE a noticeable difference, as anyone that's ever been mindful of picture quality would know.
.
Yes, the "contamination" of the blacks is somewhere in the range of (1-MTF), so the difference between 75% and 95% translates to (100-75) : (100-95) = 5:1 or 2.3 Ev darker blacks locally.
theSuede wrote:
Close range focus field curvature. The chart looks very plausible to me. It's also totally irrelevant to a landscape shooter.
If I could wish for just one thing in a lens test - and you still have to use a camera to do the test - it would be a backlit pinhole / slit measurement at somewhere between 100-200x focal length. Focused to "best focus", and with an indication of total field curvature. This could be:
50mm lens tested. Performance at:
center, 100xFL=5.00m distance initial focus > [test result]
mid field, best focal distance at 4.5m (90%) > [test result]
corner, best focal distance 3.8m (75%) > [test result]
Doing this would be very work intensive, as you'd have to have a linear rail dolly to refocus mid-field and corner tests. And a really long corridor for 600mm lenses... :-)...Show more →
+1
Do we really not know what distance the DXO tests are performed at? I mean, come on, if these tests are being conducted at very close range with wide angle lenses, they become completely meaningless and irrelevant for 99% of users and often have no correlation with how a lens will perform at the sort of distances the lens will actually be used at. If this is indeed the case, the tests then become extremely misleading. The question then becomes, are these review sites really that dense?
I don't think it is overblown. For many lenses I care about shooting at or near infinity focus. These tests are nowhere near that, so I have no idea how such a lens would perform at the distance that I would typically use to shoot with it. Lenses do perform differently at minimum focus, mid or portrait type range, and infinity focus. It seems to me that any measurement of sharpness/resolution that doesn't take into account or at least describe the focussing distance is giving at best limited and at worst misleading information.
What I mean with overblown, is that the distance effect in reality is significantly smaller than the other error sources (field curvature and inability to differentiate micro contrast on low to medium frequencies) on the DxOmark lens tests specifically.
Steve Spencer wrote:
I don't think it is overblown. For many lenses I care about shooting at or near infinity focus. These tests are nowhere near that, so I have no idea how such a lens would perform at the distance that I would typically use to shoot with it. Lenses do perform differently at minimum focus, mid or portrait type range, and infinity focus. It seems to me that any measurement of sharpness/resolution that doesn't take into account or at least describe the focussing distance is giving at best limited and at worst misleading information.
Hmmm, the target looks to me like a standard 80x120 cm sheet.
IMO the close focus range point is overblown and the focus method is more questionable.
+1000
At that distance even at f8, I don't think field curvature can explain their poor results for the 21.
Like I have said, I have close up stream shots with the 21 which exhibit the usual incredible sharpness across the frame at f8.Some thing else is contaminating their results.
