gdanmitchell
Online
• • • • •
Upload & Sell: Off
|
Turns out I actually do know something about assessment and testing methods, though I don't often offer up the full story in photography forum posts. I'm boring enough as is! :-)
The test I propose is most certainly not "bogus," much less "purposefully bogus," though that does not mean that there are not other ways to test such things.
What I'm after is not so irrelevant to actual photography as suggesting that so-called objective equipment tests that use some arbitrary baseline method of conversion and apply it to all cameras equally tell us something useful about the results that photographers will obtain in real use.
For example, if camera A turns out to be "more red" (to make up a difference) than camera B - which could equally be described at "camera B is less red than camera A" - this tells us little of value if a) color balance is almost always adjusted in post and b) the results after typical post adjustments are essentially the same.
In addition, having some background in testing and the analysis of test results, perhaps you have heard of an alternative to your AB test, which I have heard described as an ABX test. In this type of test, three samples are presented to the subject. One is case A, one is case B, and the other ("X") could be either A or B, but the subject is not told which.
Several things are possible in this sort of test, and it addresses the psychological issues you raise, notably the tendency of test subjects to find a way to a) see differences whether or not they are there when they are asked to compare - e.g. look for differences - among two test cases, and b) determine whether or not these purported subjective ratings are based on something they can really see or not.
I first heard of this a few decades ago when an audio magazine used it to test the reported ability to distinguish between the audio outputs of some (at that time) very, very expensive compact disk players and the output of some rather inexpensive players. CD playback was synchronized between the A and B machines (one of which was the "high priced spread" and the other was the relative cheapie) and audio levels and so forth were equalized. The test subjects were, as I recall, given a three position switch with A, B, and X positions, and were invited to switch among the three inputs as they listened to music. The subjects included audiophiles, musicians, regular folks, and so forth.
Almost all, as I recall, expressed a preference between the A and B options. However, there was a second request, namely to identify whether X matched A or B. It turned out that those claiming prefer A or B were unable to determine in any statistically meaningful way whether X was their favored A/B option or not.
So, yes, I'm somewhat familiar with a range of issues related to testing.
Meanwhile, back in the world of photography, I think it is quite important to step away from the supposedly objective world of measurable but trivial differences between things and ask some basic questions about this focus. If we are so sure that A is substantially better than B, why is it that really fine photographers seem to be able to produce outstanding work using A and using B? If we are really concerned about the "quality" of our work, technically or aesthetically (he timidly proposes in the "technical" forum...) why do we focus so much on trivialities that make so little difference to our photography and spend (some of us, at least) so little time and energy focusing on the things that demonstrably do have a very real effect on the photographs we produce?
So many of the "differences" that we debate here - though not quite all of them - are differences that don't matter or differences that cannot be seen in the product of our work, either because they are vanishingly tiny anywhere other than the test bench and/or because they are utterly swamped by other factors.
I can "see" a difference between ISO 100 and 200 if I stare long enough and careful enough at things like pure sky or similar and jack up contrast or create a very steep curve and look at 100% magnification and compare two samples side by side. But I cannot see a difference between ISO 100 and ISO 200 in a large photographic print from my 5D2 - and, as a consequence, while I tend to shoot at 100 out of pure habit, I will switch to 200 when I want an extra stop without worrying about it in the least. I can more readily "see" a difference between 100 and 400 if I compare the same way (100%, side-by-side, smooth gradient or solid area, staring very closely) but I can produce a print from a ISO 400 image that shows none of this - I simply use a slightly different balance of NR and masking and so forth. On my screen as I write this is a natural light photograph of a violinist shot at ISO 3200, displayed at 100% and containing tones from pure black to pure white. I apply a slight amount of NR (20-25 in Lightroom) along with a bit of masking (about 40-50) of Amount:25 sharpening and it looks great - will make a fine 18" x 24" print.
About: "Being an artist is about control of the materials and techniques."
No. And the nature of "artist" and what constitutes "art" happens to be something that I know a lot about by professional training and decades of professional work. Much of it is mystery, but if by "control of materials and techniques" you intend to draw a parallel to something as mundane as reducing noise by some tiny increment in a photograph, you are very, very wrong. Control of the methods and materials of one's art is not the same as things like minimizing noise. To that point, the "control" of noise as an aspect of the creation of art often means decisions about the role of noise in the work and it can and does involve decisions to increase or control the quality of noise as much as to diminish it. If not, how to explain artists whose style is at least partially defined by using a gritty, grainy style.
Mastery of the materials of an art is often a characteristic of an "artist," but that simply means understanding those materials so well, to the level of intuition, that he/she can manipulate them it whatever way is best for the work at hand.
In any case, we do have choices regarding noise that makes a difference in our work. We can apply all sort of effective NR strategies in those few situations where this is necessary or we can add it in post if we think that is useful. Or we could even choose to shoot at ISO values that create a lot of noise and then leave it - sort of like how folks used to (and a few still do) shoot ISO 400 films and even shoot/develop to accentuate the grain.
I'll stop the philosophical and theoretical business about art here - though it seems to me to be far more interesting and important than the difference between noise at ISO 160 and ISO 200. To anyone who thinks that fretting about whether to use full or fractional ISO values will make a sensible difference in their work... I urge you to look elsewhere for things that will make your work better in ways that people will actually notice.
Take care,
Dan
Monito wrote:
Ordinarily I agree with you, Dan, but here I must differ slightly.
gdanmitchell wrote: B. Even the relatively small differences between whole ISO values (e.g. - 100 and 200) are barely visible in most cases. I challenge anyone to look at a 16" x 24" print and tell me whether it was shot at 100 or 200. In fact, it would be interesting to expand that to include a 400 ISO print.
The test you describe is bogus. It is obvious you do not know how to do psychophysiological experiments; or if you do, you are being purposefully bogus.
Your suggested "test" is as ridiculous as showing a print of a landscape and asking someone if it was made with a SoNikon or a Canon or a Pentax.
The proper way to do this kind of test is to show a number of people pairs of prints, one at 100 and one at 200 and ask them to decide which is which. Then you see if the performance of the group rises above chance. My experience comes from having been a research assistant, but the real scientists are even more sophisticated.
The visual difference between ISO 100 and ISO 200 will usually be slim to none. It is more about the 320 / 640 / 1250 set where it counts more, on modern Canons (after 5D classic), compared to the 400 / 800 / 1600 set. Also, much depends on how much shadow area is in the images and how crucial they are to the quality of the image.
In my tests of the 5D classic, I can see the difference in noise between ISO 400 and 500. Also between 800 & 1000 and 1000 & 1250. It's in the shadows.
gdanmitchell wrote: C. The amounts of noise that folks are fretting over are not only entirely insignificant, but a bit of noise in an image often creates a more pleasing photograph than one in which noise has been overly reduced. In some cases we actually choose to add a bit of noise for this purpose.
Being an artist is about control of the materials and techniques.
It is much better for artists to have the choice to add the noise or not, rather than having the noise there without the choice.
|
