I have a novel idea: How about we compare images using a metric fitted to our respective, general output?
If you print at a certain size, compare at that size... printed. (And then tell us about your results.)
If you post online at a certain size, compare at that certain size... on the web.
For the most part, I would imagine these discussions would start to disappear. Because the vast majority of people here-- and even a higher percentage of photographers not on FM-- would find the discussion useless. For most of us, the differences once we get to final output are minimal. Let those who suffer at the extremes figure out their various solutions.
justruss wrote:
I have a novel idea: How about we compare images using a metric fitted to our respective, general output?
If you print at a certain size, compare at that size... printed. (And then tell us about your results.)
I am not aware of a method of uploading a print. This forum allows people to share knowledge and image data. Reading members' impressions of their own prints is useful, but perhaps not as useful as seeing actual results. For the sharing to be worthwhile, the methods of comparison between cameras need to be fair. The method that I suggest, endlessly, is as close as possible to examining equal sized prints. Methods which compare crops at different magnification are like comparing portions of DIFFERENT sized prints, per square inch, without allowing for the descrepancy, and inevitably lead to incorrect conclusions and the tower of Babel that is online discussion of camera performance.
Once again, this is a thread of observations between two cameras with 5 years of technology development between them. I've never said the 5D was better, had more detail, etc. It appears to me, that the 5D has better noise performance with noise reduction and sharpening turned off. With that said, the files from the 1D4 are clearly better, and they will certainly withstand substantial adjustments that would likely cause the 5D files to fall apart.
Am I surprised by what I've seen and posted here? Yes! Is it catastrophic? Not even close!
It would be interesting to see how the two cameras compare in very low light, but equal shutter speeds and apertures should be used. The camera which shows less gain should have the file brightened in conversion to match the brighter file. A subject containing a wide range of EV's would help in showing the performance at different exposure levels. The structure of noise in the shadows is of particular interest, because that often affects images at higher isos and gives an insight into how well a camera maintains tones in blacker areas.
brainiac wrote:
Here are the results with a typical dose of chroma NR applied. Interestingly, in the crop showing the native 1D4 against the uprezzed 5D, the 1D4 holds its own better than in the other crops. That's because in that case the 1D4 is not being penalised by incorrect enlargement...
Are you suggesting that uprezzing the file of a smaller camera will result in a penalty for that camera? Isn't this the basis for all your tests? Help me understand.
brainiac wrote:
It would be interesting to see how the two cameras compare in very low light, but equal shutter speeds and apertures should be used. The camera which shows less gain should have the file brightened in conversion to match the brighter file. A subject containing a wide range of EV's would help in showing the performance at different exposure levels. The structure of noise in the shadows is of particular interest, because that often affects images at higher isos and gives an insight into how well a camera maintains tones in blacker areas.
That would indeed be a good test. My 1D4 exposes differently than the 5D, with the 5D being about 1/3 - 1/2 stop brighter. This too, may have worked to the 5Ds advantage in the earlier tests. In the latter test, the exposures were much closer.
32067dlm wrote: brainiac wroteHere are the results with a typical dose of chroma NR applied. Interestingly, in the crop showing the native 1D4 against the uprezzed 5D, the 1D4 holds its own better than in the other crops. That's because in that case the 1D4 is not being penalised by incorrect enlargement...Are you suggesting that uprezzing the file of a smaller camera will result in a penalty for that camera? Isn't this the basis for all your tests? Help me understand.
No, I'm suggesting the converse. Uprezzing has practically no detrimental effect on files, and therefore it provides a good basis for the comparison of output from cameras with different numbers of pixels. In other words, I think that disparity of magnification is the worst error in comparisons, but downrezzing (like DxO) also somewhat penalises higher rez files in cases where image detail or fitness for noise reduction ought to be taken into account. Uprezzing one or both files to a common number of pixels is the simplest way of ensuring that an A/B comparison quantifies real image differences correctly.
brainiac wrote:
Uprezzing has practically no detrimental effect on files, and therefore it provides a good basis for the comparison of output from cameras with different numbers of pixels.
Personally, I don't have a major disagreement with what you have said. However, unfortunately there is always an effect (post interpolation) which, depending on the individual may be acceptable to some but unacceptable to others.
AFAIC the best way to compare two (and only two cameras) is to normalize them to the computed mean size; the arithmetic mean is just fine unless there is a significant difference in file size, then I'd suggest using the geometric mean. By interpolating both outputs you have now compromised both cameras and put them on a level playing field (rather than compromising just one camera if the smaller file is interpolated up to the larger file size), YMMV.
brainiac wrote:
Uprezzing is precisely what your printer does before it puts all those ink drops on the paper. It is drivel to suggest that uprezzing makes comparison between 18 Mpixel and 13 Mpixel cameras unfair. Uprezzing using bicubic (not a variant) produces an image which will be indistinguishable in print from the original, UNLESS the print is so large that the boundaries between pixels are resolved.
Printing images is very different from using a digital program. Most photo printers use inkjet designs. When the ink hits the paper, it spreads out and co-mingles with the other ink drops that hit the paper. This is non-discrete. Digital upresing, as seen in Photoshop, is a discrete method of resizing. Even laserjet printers don't have the discrete fineness of detail as you see on your monitor.
brainiac wrote:
The reason why uprezzing is a fairer method of comparison than downrezzing is that downrezzing has the potential to destroy detail, and therefore can hide an advantage that the higher rez file may have over the uprezzed file. For comparison of noise alone, downrezzing the higher rez file is perfectly adequate EXCEPT that noise reduction algorithms are less destructive when performed on the full rez file than on the downrezzed file. In tests like the ones shewn on this thread, the iso is such that noise reduction would very likely be used in real world output, so any comparison of noise reduced output should have the noise reduction applied BEFORE downrezzing....Show more →
This is a fallacious argument. The only non-destructive method of resizing an image is nearest neighbor. This, of course, produces the worst results. As such, bicubic and bilinear resize algorithms are used frequently by those upresizing. Both of these algorithsm interpolate the image in way or another. Thus, the resize filter you use also determines the quality of the image and resize. Moreover, as I've already stated, by using the upresing method, you give an unfair advantage to cameras that have higher resolution as you magnify the noise when you upres.
Please! Everyone try to understand what brainiac is saying otherwise it is vary hard to do these comparisons. Up rezzing doesn't destroy detail and you can't "throw away" data recorded by the bigger sensors.
But thanks for the discussion of this topic because: yes it does appear that there's some consensus that the Mk IV is inferior to the D3 and, especially, D3s in hi ISO and not very far in front of (image area adjusted-brainiac) even the 5DmK II and 1Ds Mk III maybe 1/2 a stop.
There's something else here which is not, explicitly, detailed and which could be the topic of a new metric. At some point (by varying the ISO) the noise of the file obscures the data, which could have been, recorded by more pixels on the sensor.
This is indicated in these posts, somewhere, by the observation that the Mk IV's extra resolution (12MP vs 16) is not an advantage at around ISO 3200 because the noise obscures the extra resolution of its sensor.
So don't buy a 1Ds Mk III, 5DMk III or Nikon D3x and expect 21/25Mp of resolution at ISO 3200 and above. That is to say the real "King of Hi ISO" the 12 Mp Nikon D3s has as good or better resolution at ISO 3200+ as the hi Pixel crowd.
thw2 wrote:
Have you guys seen this which shows RAW files comparison at the PIXEL level between 1D4, 1Ds3, 7D and D3s?
Outcome:
1D4 is 1 stop better than 1Ds3 (which is equivalent to 5D2).
1D4 is 1.5 stops better than 7D.
1Ds3 is merely 0.5 stop better than 7D.
D3s is 1 stop better than 1D4.
This means D3s is TWO whole stops better than 1Ds3.
OK... better duck for cover... I can see all the guns pointing at me already..
umm #1 problem i see is again you are comparing per PIXEL! useless!
SoundHound wrote:
Please! Everyone try to understand what brainiac is saying otherwise it is vary hard to do these comparisons. Up rezzing doesn't destroy detail and you can't "throw away" data recorded by the bigger sensors.
I understand what he's saying. I think his theory is flawed.
Moreover, upresing an image DOES destroy detail unless you're using nearest neighbor. The problem is that any resize algorithm, save for nearest neighbor, uses some form of interpolation. When you resize an image, you have to interpolate in some fashion. This interpolation cannot be undone. If you take an image, upres it and then resize it to the original size, the image will be different
It's easy to test if you use some discrete shapes. If make a black square on a white background, upres to a size that's not a multiple of the original resolution (using bicubic) and then downres the image to the original resolution, there are significant artifacts on the image.
Moreover, digital upresing of an image is NOT typical. As I've previously mentioned, printing an upresed image is not the same as viewing an upresed image on your screen. The print is analog and the co-mingled ink droplets make the resize artifacts difficult to discern. This is compared to viewing an image at 100%+ on your monitor where each, separate pixel is easy to see.
SoundHound wrote:
This is indicated in these posts, somewhere, by the observation that the Mk IV's extra resolution (12MP vs 16) is not an advantage at around ISO 3200 because the noise obscures the extra resolution of its sensor.
So don't buy a 1Ds Mk III, 5DMk III or Nikon D3x and expect 21/25Mp of resolution at ISO 3200 and above. That is to say the real "King of Hi ISO" the 12 Mp Nikon D3s has as good or better resolution at ISO 3200+ as the hi Pixel crowd.
not likely so at all!
a 7D per area downsized looks better than 5D2 at any ISO (i.e. when reach limited)
anyway maybe the 5D2 will lose some of the 21MP detail but once you downscale to D3s size you average away all sorts of demosaic errors and you always get better detail by the oversampling before downsampling, I doubt the D3s high iso is nearly enough better to pull it ahead of 5D2 detail at any iso.
kakomu wrote: SoundHound wrotePlease! Everyone try to understand what brainiac is saying otherwise it is vary hard to do these comparisons. Up rezzing doesn't destroy detail and you can't "throw away" data recorded by the bigger sensors.I understand what he's saying. I think his theory is flawed.
Moreover, upresing an image DOES destroy detail unless you're using nearest neighbor. The problem is that any resize algorithm, save for nearest neighbor, uses some form of interpolation. When you resize an image, you have to interpolate in some fashion. This interpolation cannot be undone. If you take an image, upres it and then resize it to the original size, the image will be different
It's easy to test if you use some discrete shapes. If make a black square on a white background, upres to a size that's not a multiple of the original resolution (using bicubic) and then downres the image to the original resolution, there are significant artifacts on the image.
Moreover, digital upresing of an image is NOT typical. As I've previously mentioned, printing an upresed image is not the same as viewing an upresed image on your screen. The print is analog and the co-mingled ink droplets make the resize artifacts difficult to discern. This is compared to viewing an image at 100%+ on your monitor where each, separate pixel is easy to see....Show more →
And you would be wrong.
You better read some texts on interpolation of data sets before you make such a blatantly incorrect assertion and embarrass yourself even further.
I was prepared to write a long treatise on the matter but realized that it would be futile. You are completely misunderstanding the nature of interpolation and the fact that there are other things going on with respect to the storage of the data that are not intrinsic to the interpolation algorithm.
You better read some texts on interpolation of data sets before you make such a blatantly incorrect assertion and embarrass yourself even further.
I was prepared to write a long treatise on the matter but realized that it would be futile. You are completely misunderstanding the nature of interpolation and the fact that there are other things going on with respect to the storage of the data that are not intrinsic to the interpolation algorithm.
The first image is the source image (black rectangle on white square). The second image is what happens to the (simple) image when you do a bicubic resize to a larger image (in this case from 400x400 to 1600x1600) and then a 2nd bicubic resize to the original resize. The edges are software. Edge fidelity is lost. Both images are saved as PNG, which is lossless, so any changes in the images are not caused by image compression.
You can easily reproduce the test in photoshop. The effect on more complex images is certainly going to be the same.
My point is that the quality of an image will change when you upres. The notion that upresizing doesn't "destroy" the image is clearly FALSE as interpolation will ultimately change the image. The only method of resizing an image to a larger size and back to a smaller size is to use nearest neighbor and only if you resize an image to a multiple of the original dimensions. Then again, who uses nearest neighbor for photo work?
Also, I'm not wrong as far as ink jets are concerned. THe ink does spread out once it hits the paper. This spreading and co-mingling of ink ultimately hides a lot of the digital resize artifacts that are present when you upres an image.
Interpolation is best done with unsharpened images. If sharpening is applied after resizing, there is no longer a problem with soft edges.
Images out of a RAW converter with sharpening disabled never have edges like kakomu's example.
BTW, most printer drivers use Nearest Neighbour interpolation before the data is sent to the printer where the RIP calculates all those ink dots. The resolution to which this interpolation takes place, does even change if you print borderless.
alundeb wrote:
Images out of a RAW converter with sharpening disabled never have edges like kakomu's example.
This is only correct with really non sharp images, as in color blotches with wide color gradation. Any image that has sharp contrasts of any sort will lose edge fidelity with a resize.
The best way to see the differences in images when you resize is the following process:
- Open your image of choice (doesn't matter what has been done to it) in Photoshop.
- Select the entire image and copy it.
- Select Image->Resize (or hit alt+ctrl+I) and choose whatever larger resolution you want
- Repeat the above step but resize the image back to its original resolution
- Create a new layer and paste the original, untouched image into that layer.
- Set that layer's Blending style to "Difference"
- Add an "invert" layer mask at the top to more clearly see the differences.
After all the above, you'll find an image. If both layers were the same, you'd arrive at a white image. However, because the layers are different, you'll find an outline showing the differences between both images. The results are imperfect for showing the extent to which the images are changed, but shows that a change does in fact exist.
Each interpolation will yield different artifacts. Bilinear resize tends cause "ringing" around the edges of sharp contrast.
As far as noise performance is concerned, most of the tests I see are flawed and not at all stringent enough to base any assumption. Considering noise levels change depending on color and available light, I cannot see how any conclusion can be made in any of the uncontrolled experiments I've seen thus far in this thread. They do a good job of showing "real world" application, but no conclusion of any scientific merit can be garnered from any of it.
The best way to test noise is to compare a photograph of an object of known color with a known amount of light (of a known color) against the color and test the SnR between the photograph and the color.
e.g., given a white wall with very high color uniformity with a light source of x luminance with y color temperature, compare the SnR between the color white and the photograph taken. This will give you a tangible number to use as far as noise is concerned. This number can then be used as a ratio in comparison to the resolution which will allow you to compare against other sensors of different resolution.