The Sonnar ZM and Biogon 25, I purchased pairs in black and chrome, tested them extensively on film Leicas as well as on the Nex-5N, could not detect any sample variations worth to mention. With the FD 50/1.4s however (4 samples) sample variations were more obvious at full aperture, especially CA.
tomb57 wrote:
Thank you all for your replies. Much appreciate.
Being new, I am not sure I can test the lens since I have no reference point, as well as I would have the tendency to think it's my mistake, not the lens.
The only safe way I can do is to try two or more lens and get the best one. It would be harder for me to do this way by buying from out-of-town shop.
Cheers...
Well, if you have the money to do that then go for it. Unless you're saying the local guys will test the lenses for you or allow you to conduct tests of their stocked items. Otherwise hey splurge! Buy 8 or 10 and do the tests. Tho such things (even at two) might classify in the "more money than sense" category. Local is better if there's very little price difference - otherwise frack'em! And with all this worrying who's to say that all of the copies at the local dealership aren't inferior to the one that's out of town. It might be that special one!
Testing a lens is pretty simple really. You don't need to doubt yourself unless this will be like, your first photograph ever. Photographing a yard-stick (ruler) at a shallow angle from several distances will tell you if the AF is front or back focusing. And test charts that go up to 2000LPI are readily available for free on the internet. Just DL and print one out, set up on a tripod, mirror-up, manually LV focus, check the results. If you're even in the ballpark it's not a bad copy. Remember what the site proved above all other things: "A bad copy isn't slightly softer, it's way softer!".
RCicala wrote:
I don't have a large data sets for that - we test the lenses in rotation so in general a lens tested for the second and third time has been rented several times between tests, which adds the variable of wear-and-tear, either from use or shipping. Our primary goal is not really to look at the small copy-to-copy variability that exists between most lenses, it's to find the much larger difference in those few lenses that are bad, either out-of-the-box or from damage in use and get them out of stock. (The difference in these is dramatic and far removed from the pattern of minor copy-to-copy variation).
For what it's worth we have found (about 1,500 lenses with repeated tests, although I emphasize weeks or months and several rentals passed between tests) that as long as the same test camera is used a variation of more than + / - 2.5% (best shot of a 5 shot focusing bracket) is fairly normal. With very few exceptions change greater than that is MUCH greater than that and always negative, indicating some damage has happened to the lens.
If we change test cameras, though, the variation is at least double that - even if we change test cameras without making any other changes to the setup, simply changing the camera and immediately running the same lenses again. Similarly, if we lock one lens in place and simply test a whole series of cameras, the variation is about 5% (with the exception of a few outliers - bad cameras happen at least as frequently as bad lenses). ...Show more →
Good information. I assume then that your lens sample graph on this page is based on tests with a single camera. Concerning the + / - 2.5% for the lens alone, is that to be interpreted as a deviation in the range from -2.5% to +2.5%? Min/max spread or standard deviation? Or do you mean with + / - 2.5% that the spread is about 2.5%? In the former case, a spread of 5% is a significant part of the total spread in the graph: 40 on a total of 800, whatever the unit may be along your y-axis.
It would be interesting to see how these numbers develop as your statistics improve over time.
Toothwalker wrote:
Good information. I assume then that your lens sample graph on this page is based on tests with a single camera. Concerning the + / - 2.5% for the lens alone, is that to be interpreted as a deviation in the range from -2.5% to +2.5%? Min/max spread or standard deviation? Or do you mean with + / - 2.5% that the spread is about 2.5%? In the former case, a spread of 5% is a significant part of the total spread in the graph: 40 on a total of 800, whatever the unit may be along your y-axis.
It would be interesting to see how these numbers develop as your statistics improve over time.
Thank you, Toothwalker - I wasn't very clear. Testing using best focus of a 3 bracket, same camera but different day's setup otherwise, I expect a given lens result to be within a range of + or - 2.5% of it's original result (range of 5%). I don't have enough repeated measurements of each copy to try calculating an S.D. or S.E for repeated samples. We basically have a few measurements of many different lenses, but not many measurements of any one lens.
As you point out, about a 5% range is equivalent to a 40 point variation on the scale of 800 lp/image height we're using. Generally the range of all copies of a given lens is about 100 lp/IH. If we change test cameras (same model, different copy), not surprisingly, the range of the overall pattern is very similar although the individual lenses within the pattern shift positions more than the pattern itself changes. (I'm probably not saying this well either. The range of the pattern is basically unchanged, although the center of the range may change slightly. More noticeably, though, each lens copy within the pattern shows more variation than the pattern does overall when the same group of lenses is tested on different bodies.
As you have ably pointed out pointed out, testing using an on-camera method has it's uses but introduces an additional set of variables that make things less clear. Hopefully when we have off-camera bench testing available it will really help us clarify where the variability is coming from. Personally I suspect lens and camera mounts along with sensor alignment are a large part of it. Hopefully comparing on and off camera results for a group of lenses will help eliminate some of the variables and clarify things.
Local is better if there's very little price difference - otherwise frack'em! And with all this worrying who's to say that all of the copies at the local dealership aren't inferior to the one that's out of town. It might be that special one! 