Upload & Sell: On
Lee, I would not be too fussed on the multiple sample front, if your aim is not purely a 'sharpness' derby. Though admittedly the thread is bending off in that general direction. I am much more interested in the overall perceivable photographic quality of images of the sort that regular photographers take, which looks to place me in the small niche of photographers down at the end of the distribution.
I'm definitely with you on real world testing and visual examples. There will be a mixture of normal field shooting and deliberate, Lloyd-Chambers-esque A-B comparisons. I'm just going to do an Imatest "sharpness derby" also
The multiple samples aren't really about maximizing the "sharpness derby" results, though. Roger Cicala's blogs make it clear 2-3 samples isn't nearly enough for that. It's more about screening for actual bad samples. For example, the Zeiss Jena 180/2.8 is supposedly the same optical design as the very first 180mm from 1936. My expectations for that lens aren't very high; I could get a bad sample and not realize it.