freedolin wrote:
Allow me to comment, especially on our methodology
Scott Stoness wrote:
I really like optical limits however:
1) OL tests uwa by bringing the image in close to the lens until the target fills the page. Thus centre is proportionately way closer than edges. This is a broken method.
No, this is how all MTF tests are done, and there is simply no alternative to it. Interestingly, it's never an issue for lenses with good results, but always a reason to question our testing approach for lenses with fairly low resolution numbers.
Lenses have a flat image field (theoretically). So testing a flat target is the natural approach. For wide angle lenses, yes that brings up the issue that we're not testing at infinity (you can't, really), but if you happen to have a lens with a distance scale (the RF 16 doesn't qualify here), you can still see that with our typical test distance we're still closer to infinity on the lens' scale than to its MFD.
Scott Stoness wrote:
2) Similarly I am only interested in performance at infinity. Which is not tested.
As I wrote already: this is simply not possible. Unless you happen to own a landscape you can fill with a giant MTF test target.
Scott Stoness wrote:
3) They test corrected. The lens is really 14mm at uncorrected. For landscape. The vignetting can be cropped out at 16mm. I will shoot uncorrected and the IQ will be much higher. Most times I crop anyway to 16x9. Unless you are shooting architecture this conclusion is misleading. Uncorrected the iq at corners is much higher (albeit distorted).
We test corrected in this case, and it's clearly stated in the text that results without distortion correction naturally show higher resolution in the image corners. Testing without distortion corection simply would not have been possible due to the massive vignetting. The corners are pitch black, there is nothing to measure there for Imatest. Testing with cropping would have led to results that are simply not comparable to all other RF reviews, since the resolution of the test image would be different.
On top, as also clearly stated in the review: this lens is very obviously designed to be used with software correction enabled. Still, we're interested in the native performance of a lens, since software correction never comes for free. Many manufacturers heavily rely on software correction nowadays, true, but no other manufacturer currently offers lenses that are underdesigned to such a degree as Canon does. I think it's fair to point that out and demonstrate it by numbers.
It's a very personal decision if you can live with the drawbacks of such a lens. And as we also pointed out, it's definitely a price/performance king. On the other hand it's also fair to ask, if the lens could have been significantly better (optically) for a slightly higher price. Personally, I think a $400 oder $450 lens, that under the hood fills the sensor to the corners and is not a hidden fisheye would have sold similarly well.
Markus (OL team)
Let me reiterate - I really appreciate OL. They provide great reference material. I go there often to check on lens. I just understand that the UWA tests need to be taken with a grain of salt outside of centre given the methodology.
re 1. Lensrentals does test at infinity - it can be done and really needs to be done to test UWA lens. If you compare their tests in their blog vs your tests of 11-22 Canon you will see dramatic differences, for example on 5dsr.
re 2. Testing at perhaps 10% in from corner uncorrected would at least give a better appreciation of how good the lens is. No one is going to use the lens with vignetting in at 14mm - they will all crop but some may not correct distortion. But someone (many including me) will use it uncorrected for landscape where distortion is not a significant factor.
I really appreciate you participating here. Testers should be accountable for their product.
[and note you fixed your method after the 24-104 test where the lens got less resolution on 45mpx than 30mpx which is nonsensical I suspect a mislabling somewhere.]
Thanks for the great efforts.
Feb 22, 2022 at 12:16 PM
Previous versions of Scott Stoness's message #15871586 « Optical Limits reviewed Canon RF 16mm f/2.8 STM »
