Sample variation yes but when it's stacked against one of the best zooms ever released to have it come in some cases slightly worse off is no "disaster".
If compared to the old one that would be a different story but anything that's at least as good as the 70-200II is a killer lens..

splathrop wrote:
Roger, thanks for that information.

One question it suggests for further investigation is how much of the variation observed is attributable to each lens individually (as a permanent feature), and how much to the spread of results you might get from a single lens if it were re-zoomed and re-focussed between each of multiple tests.

If on multiple tries any particular lens always grouped tightly, we would know that the question of getting a good copy is more important than it would be if each lens produced a spread of results similar to the one on your graph. Maybe your experience allows you to just answer the general question from your present knowledge?

Just one other question. The subjective quality measurement you use is based on an 8x10 print, and of course some of us print much larger. Would smaller subjective quality variation numbers become more evident in say, a 30x20 print? I'm basically wondering if getting a great copy is more important to a landscape photographer who prints big than to a wedding photographer, for instance.

Thanks again for the terrific work.

I did that sort of thing a bit in this post: http://www.lensrentals.com/blog/2011/10/notes-on-lens-and-camera-variation
It's an older post and our technique with testing is a little better now, but it shows the principle (third graph down):

1) If I take 6 straight shots, the points are nearly touching but there's a bit of variation (5 points or so either way) which is simply variation in vibration or lighting.
2) If I manually focus 6 separate shots with live view, there's a bit more variation, maybe 20 points. That's the limit of Roger's ability to focus and why we focus bracket and take the best result.
3) If I autofocus (phase), though, there's quite a bit more variation shot-to-shot. AF variation on one lens can be around 30% of the variation seen for best shot on all lenses.
4) If I test with a different camera (next graph down) variation is again, about 30-40% for each lens.

Lets also add that I'm testing 70mm at 18 feet. Since the focus elements will change position at infinity or minimum focusing distance, there would be some variation between lenses if I was able to test at infinity.

Items 3, 4 and 5 are why I'm always saying (trying to explain it to a well known reviewer right now) you can't say "I want your best 24-70". I don't know which is best on your camera, at different focal lengths, or at different distances. There's a reason you don't shoot shotguns at a bullseye target.

For the other question, yes, if you're printing a huge print, than a difference in SQF of 5 would be noticeable, at least in a side-by-side comparison. But for all the reasons mentioned above I don't know if it's possible to predict more closely than that. I can tell you that even on lenses we're great at optically adjusting, that's about the best we can do, and that's about what the factory service center can do too.

After living with and shooting the new 24/70II here is what I know.

I can now confidently shoot this lens at full aperture yes F2.8 when I want and get very good results. I could not do this with the previous version. For me this justifies the expense.

Very happy with the new lens.

I think the suggestion about real world images someone made is a good one, so I've appended the article with 100% crops of an ISO chart shot with one of the highest and one of the lowest resolving lenses.

I think it helps demonstrate that the number differences, while real, seem a lot greater than what you'd notice in the real world.

I shot the old one at f/2.8 without worry and ill continue to do so.
The new lens is fantastic and ill buy one eventually.

Man...folks have too much time on their hands

Hello, for a newbie who plans to buy two or possibly three of these top notch lenses (16-37, 24-70ii and 70-200 ii) I am concerned how do I check my lenses with professional eyes and how good is Canon repair.

RCicala wrote:
I think the suggestion about real world images someone made is a good one, so I've appended the article with 100% crops of an ISO chart shot with one of the highest and one of the lowest resolving lenses.

I think it helps demonstrate that the number differences, while real, seem a lot greater than what you'd notice in the real world.

Here is another visual demonstration the best 70mm f/2.8 and the worst I saw over a few (many less than you tried of course) copies. I actually was prompted to do the careful test because I noticed, in real world shots, that one didn't seem to be quite popping as much as the others (although this one did seem to have the most even corner performance of them all, oddly enough, so you picks your poison).

These are 200% to help rid jpg artifacts from having any affect on things. Difference is easiest to spot if you save both and flip between them in an image viewer. One should have crisper darks next to brighter whites, better micro-contrast for more image pop.

Thanks Roger for the test. I'm with Robert; I will own one eventually. However, not sure how many months in my eventually.