Monito
Offline
• • • • • •
Upload & Sell: Off
|
 p.38 #20 · ['NEW Fix' UPDATE!] - MkIII AF still broken | |
Hrow wrote:
A number of people have made some appreciated suggestions and here are the results of way too much testing. [...] Major culprit is a 20% reduction in shutter speed though that doesn't help with the color too much.
Testing is notoriously difficult. If you go looking for problems, you will find them. Something people don't realize is that though reducing the number of variables is good testing technique, it also puts greater stress on the remaining variables. I'm not saying that there might or might not be problems with this lens, but this test is not convincing and definitely would be shredded in a court of law, as one poster asks.
Did you take a photo of the test setup with another of your cameras to show us? If you only have one camera, did you take the camera off the lens, leaving the lens in place and photograph the test setup? (Assuming you mounted the lens on the tripod, not the camera)
Over what length of time did the shoot take place? Did you interleave the manual focus pictures with the autofocus pictures?
You may think that outdoor sunlight filtered indirectly in through windows is going to be constant, but it is not. Did you make a picture showing the light source? Were there any thin high cloud conditions? Since you said the day was windy outside, was the window partially obscured by moving tree shadows? Reflected light from the ground onto the ceiling and into the picture can also vary if the shadows on the ground move. Vegetation moving in a windy day can reflect varying amounts of light and green into a room depending on the direction and strength of gusts and of calms.
Were all room lights turned off? Were there any sources of specular reflections inside or outide? If you block reflections off one shiny surface inside, a slightly different position of the camera operator can change the amount blocked or not blocked. Was there any street traffic moving? Parked car windows or other buildings reflect varying amounts of light as the sun moves, even a single degree.
What shutter speed? What aperture? What ISO? What metering technique? The pictures look underexposed. Why vary the exposure if you are testing focus? Why weren't they taken all at the same exposure? Have you tested your shutter to see about variability of the shutter duration?
Mirror lockup? How many seconds did you leave it to stabilize? Was the tripod mounted on a wooden floor or a concrete slab? Were there children playing upstairs? Pets running around? Is the site a detached house in the country far from truck traffic, subways, jet aircraft overflights, construction activity? Were other devices like washing machines and refrigerators turned off? Did you stand absolutely frozen for the entire time between touching the lens/camera and clicking the shot?
What Image Stabilization settings? I think it was said that IS was left on. Why? If you are testing focus, best to leave it off. Or make a full set of tests with IS in each of the three settings (0, 1, 2) times manual versus auto for a total of six sets, all interleaved in a random order determined by a random number generator or table of random numbers.
Why did you use a printed picture to introduce all kinds of moire issues? If you wanted flat, you could have mounted a chopping board or other flat wood surface with grain pattern. Moire issues interacting with the sensor can affect color balance. The color balance of lenses can be slightly different at different apertures.
Did you apply trigonometry to ensure parallel surface to the sensor plane?
Did you not have a gray card and color swatch to include? Did you include some pure white paper in the scene? Without neutral gray in the scene to set a color pick point White Balance, AWB will vary extremely. Depending on AWB for comparison between shots is bogus. AWB is meant as guidance for simple programmed automatic picture taking, not as a crutch to lean on. Further, leaving AWB on is to vary over the map under these variable lighting and exposure conditions will lead to variable results.
Without a scene picture, preferably with the photographer in it, it is hard to judge other variables. What color shirt and clothing were you wearing? Where were you standing in relation to the test subject? Did you always stand in exactly the same position with exactly the same pose? Probably not. People are frequently surprised by how much colored surfaces can reflect into a scene. If you are on the opposite side from the light source you will be reflecting a tremendous amount of color into the scene. If you are on the same side as the main light source, your different positions will change the amount of light.
Individual variations of factors show up most strongly in tests such as these. They do not make any practical difference in almost all of the photography we are likely to do in real world conditions. If we are photographing a hawk flying overhead with a 300 mm lens, it does not really matter if we are wearing a red shirt or if we are angled to reflect more or less of the red into the hawk's underbelly. Likewise, if we are making a series of portrait exposures with one lighting setup, we don't leave the camera on AWB from shot to shot, or if we do since we are shooting Raw mode, we figure out the best color balance in post-processing and apply that to all the pictures in the set.
The claim is made that there are differences between the manual exposure series, but I expect that the tests were not interleaved, and so that conditions of lighting and other factors will have changed from one set to the other, since so many factors were uncontrolled.
This test, though well intentioned, and though conducted with some amount of thought and care, is basically worthless for many reasons. At most it is only suggestive, indicating much more rigorous testing is required to isolate or eliminate problems with testing methodology and see if actual equipment problems remain. We buy cameras that are extremely complex instruments designed to be very flexible to handle a huge range of variable situations, but we don't think like scientists when it comes to testing them. Each individual variable will not have much effect, but they are interactive and compounding, such that for example where automatic exposure is being used, when the exposure is near the middle between two shutter settings, it can edge it one way or the other.
Nice try. No prize. No meaningful result. Sorry.
|
