Upload & Sell: On
Thanks Roger. How confident are you in the accuracy and repeatability of results for each individual copy? You're probably aware of slrgear's old article about how tiny focus differences (even manually focused) translate into large variations of measured resolution, which lead them to create a dolly contraption to find ideal focus (Article Link)
I work with SLRGear.com quite a bit, we've visited and exchanged methodology. When we first started testing I spent a day with Dave and Rob, learning to align properly (Atlanta is close to Memphis).
I don't go into this in the posts trying to keep them brief, but for each lens we do the following:
1) confirm alignment via adjusting a 6 way mount analyzing images to be within 0.5 Degress of horizontal, vertical, and rotational tilt, and within 0.5cm of center of the chart.
2) manually focus using 10X Live view.
3) focus bracket 5 shots, 2 on either side of live view focus, analyze all 5 and choose the best.
4) repeat the test and take the better result of the two tests.
5) plot the data point.
6) go back and take the lowest 10% fo the test run, retest all of those lenses to be certain there's no error.
So I'm pretty confident in the data point (and sorry for boring the crap out of everyone who didn't want to know this).
From past experience, if we redo the entire test run on a different camera, the range, pattern, average and SD will be the same (assuming a reasonably large series). However the individual data points will shift around. Like many variation measurements the repeat will tend to have a 'travel toward the center' appearance. That is, the very best and worst lenses on this camera will be more toward the middle on another camera, while the middle ones on this camera will spread out higher or lower. That's just simple camera to camera variation.
As to the earlier question, for zooms, this is about what most f/2.8 lenses look like. Primes have about half as much variation as a very general rule. Consumer zooms seem to vary less actually, but part of thatis that the absolute numbers are lower (a 10% variation from 500 seems smaller than a 10% variation from 800) and because the apertures are smaller. If I tested these lenses at f/4, the variation would be reduced by nearly half.
</ boring lecture>