Re: Official Nikon Z 180-600mm f5.6-6.3 VR Image Thread
Everyone comparing things on Digital Camera World seem to have utterly forgotten what we’ve known for decades:
1. Imatest MTF50 numbers are NOT COMPARABLE ACROSS SYSTEMS. This is the absolute basics of lens reviews, and if you are honestly still making this mistake in 2023 then please do yourself a favor and stop even looking at these numerical results.
2. They are also not comparable across different bodies or processing chains even in the same system.
3. Any review site that does not specifically tell you what body they tested on is not fit to present this data in the first place (DCW seems to fit into this category, I can’t find this specified anywhere but perhaps it is just well hidden).
4. Any review site that randomly selects bodies to test a lens on, rather than sticking to a single body across reviews within a system, are not fit to present this data either (E-Photozine is notorious for this, but at least they document they did it).
5. Properly running Imatest on a super telephoto lens is notoriously difficult to get right, even the quality review sites can have issues.
In this specific case the charts are screaming at us that the comparison data is largely useless. Note the F/22 center results. Every modern lens has identical performance by F/22 in the center - performance is well into being entirely diffraction limited here. Note how every zoom lens has all its results for various focal lengths converge by F/22 illustrating this principle.
Now note how for each DCW test of a different lens the F/22 results are not the same as other lenses - even in the same system. This is a huge red flag telling us comparing lenses by MTF50 results from Imatest on DCW is a fool’s errand. Don’t waste your time doing it.
Now - the behavior of the curve within a single test may still be meaningful. For example, determining a lens’s “sharpest” aperture does not involve comparisons between reviews. Here we see the 180-600 gets sharper at F/8 than at F/6.3 which matches some reviewer’s field results. But that could of course still just be due to a focus error when DCW did their test. Since I couldn’t find any documentation on their test methodology we don’t really know if they depend on AF or do MF for their Imatest shots.
CameraLabs provides crops from their test charts instead for visual comparison. This of course is not without its own potential issues, but as linked earlier clearly their lens and test setup is getting a different result (F/6.3 is the sharpest aperture at 600mm) than DCW.
Re: Official Nikon Z 180-600mm f5.6-6.3 VR Image Thread
Everyone comparing things on Digital Camera World seem to have utterly forgotten what we’ve known for decades:
1. Imatest MTF50 numbers are NOT COMPARABLE ACROSS SYSTEMS. This is the absolute basics of lens reviews, and if you are honestly still making this mistake in 2023 then please do yourself a favor and stop even looking at these numerical results.
2. They are also not comparable across different bodies or processing chains even in the same system.
3. Any review site that does not specifically tell you what body they tested on is not fit to present this data in the first place (DCW seems to fit into this category, I can’t find this specified anywhere but perhaps it is just well hidden).
4. Any review site that randomly selects bodies to test a lens on, rather than sticking to a single body across reviews within a system, are not fit to present this data either (E-Photozine is notorious for this, but at least they document they did it).
5. Properly running Imatest on a super telephoto lens is notoriously difficult to get right, even the quality review sites can have issues.
In this specific case the charts are screaming at us that the comparison data is largely useless. Note the F/22 center results. Every modern lens has identical performance by F/22 in the center - performance is well into being entirely diffraction limited here. Note how every zoom lens has all its results for various focal lengths converge by F/22 illustrating this principle.
Now note how for each DCW test of a different lens the F/22 results are not the same as other lenses - even in the same system. This is a huge red flag telling us comparing lenses by MTF50 results from Imatest on DCW is a fool’s errand. Don’t waste your time doing it.
Now - the behavior of the curve within a single test may still be meaningful. For example, determining a lens’s “sharpest” aperture does not involve comparisons between reviews. Here we see the 180-600 gets sharper at F/8 than at F/6.3 which matches some reviewer’s field results. But that could of course still just be due to a focus error when DCW did their test. Since I couldn’t find any documentation on their test methodology we don’t really know if they depend on AF or do MF for their Imatest shots.
CameraLabs provides crops from their test charts instead for visual comparison. This of course is not without its own potential issues, but as linked earlier clearly their lens and test setup is getting a different result (F/6.3 is the sharpest aperture at 600mm) than DCW.
Re: Official Nikon Z 180-600mm f5.6-6.3 VR Image Thread
Everyone comparing things on Digital Camera World seem to have utterly forgotten what we’ve known for decades:
1. Imatest MTF50 numbers are NOT COMPARABLE ACROSS SYSTEMS. This is the absolute basics of lens reviews, and if you are honestly still making this mistake in 2023 then please do yourself a favor and stop even looking at these numerical results.
2. They are also not comparable across different bodies or processing chains even in the same system.
3. Any review site that does not specifically tell you what body they tested on is not fit to present this data in the first place (DCW seems to fit into this category, I can’t find this specified anywhere but perhaps it is just well hidden).
4. Any review site that randomly selects bodies to test a lens on, rather than sticking to a single body across reviews within a system, are not fit to present this data either (E-Photozine is notorious for this, but at least they document they did it).
5. Properly running Imatest on a super telephoto lens is notoriously difficult to get right, even the quality review sites can have issues.
In this specific case the charts are screaming at us that the comparison data is largely useless. Note the F/22 center results. Every modern lens has identical performance by F/22 in the center - performance is well into being entirely diffraction limited here. Note how every zoom lens has all its results for various focal lengths converge by F/22 illustrating this principle.
Now note how for each DCW test of a different lens the F/22 results are not the same as other lenses - even in the same system. This is a huge red flag telling us comparing lenses by MTF50 results from Imatest on DCW is a fool’s errand. Don’t waste your time doing it.
Now - the behavior of the curve within a single test may still be meaningful. For example, determining a lens’s “sharpest” aperture does not involve comparisons between reviews. Here we see the 180-600 gets sharper at F/8 than at F/6.3 which matches some reviewer’s field results. But that could of course still just be due to a focus error when DCW did their test. Since I couldn’t find any documentation on their test methodology we don’t really know if they depend on AF or do MF for their Imatest shots.
CameraLabs provides crops from their test charts instead for visual comparison. This of course is not without its own potential issues, but as linked earlier clearly their lens and test setup is getting a different result (F/6.3 is the sharpest aperture at 600mm) than DCW.
Sep 13, 2023 at 12:47 PM
Previous versions of kwalsh's message #16341525 « Official Nikon Z 180-600mm f5.6-6.3 VR Image Thread »
