if i were there i would gladly hook up at least for a free ale
you are being disingenuous tho if you are claiming "neutrality" in your thread, that really is the crux of it for me. I always like to have fun with tests, but you have asserted things as truth in that thread like 'nikon Always uses NR on their high iso shots, it canot be turned "off"' or things to that effect. But it is also a known fact that one of the compelling reasons canon pursued CMOS in the first place was so they could use DIGIC processing for on site at the pixel level "massaging" of the data to reduce noise, so where is the nod that recognizes that fact?
We have bowed and kissed the proverbial ring for many years over our choices of cameras , if you had been told on so many occasions that "your" gear trumped ours in one regard or another you would realize how tiring that really is. The tide has turned , and on the last frontier of dominance by canon it has been challenged and really we are ALL winners in the end.
If you had not had that threat things would be different , but you did and would you insist on downsizing your files or uprezzing the D3 files to satiate your curiosity? is it your way and none other in the testing?
MY last comment , i hope you find someone for the test, i do wish it were me , tho i do not regularly shoot above 1600 i have made some amazing prints at 6400 and am truly loving the new SB900 flash controlling my more difficult to configure pair of SB800s
jmcfadden wrote:
if i were there i would gladly hook up at least for a free ale
You'ld be welcome to it.
> you are being disingenuous tho if you are claiming "neutrality" in your thread, that really is the crux of it for me.
It's not about my neutrality, it's about the neutrality of the test, i.e. doing the test in a scientific way. IMO comparing 100% crops is thoroughly misleading when the files have radically different numbers of pixels. My intention, in suggesting comparing prints, is to remove the confusing bit so that we don't have to argue about the misleading error of comparing files at different magnifications.
> I always like to have fun with tests, but you have asserted things as truth in that thread like 'nikon Always uses NR on their high iso shots, it canot be turned "off"' or things to that effect.
I stand corrected, although every sample I have ever seen posted online appears to have had some kind of chroma-blending, so I hope you can understand the source of my confusion on that.
> But it is also a known fact that one of the compelling reasons canon pursued CMOS in the first place was so they could use DIGIC processing for on site at the pixel level "massaging" of the data to reduce noise, so where is the nod that recognizes that fact?
As I have said before, both files should be processed as they would be in real life use, i.e. optimally. I would suggest DPP and NX, as the least controversial options, since they are each supplied by the manufacturers.
> We have bowed and kissed the proverbial ring for many years over our choices of cameras , if you had been told on so many occasions that "your" gear trumped ours in one regard or another you would realize how tiring that really is. The tide has turned , and on the last frontier of dominance by canon it has been challenged and really we are winners in the end.
I am sorry, but that's your personal issue. Let's leave angst and envy out of this, and understand that brands and products do not define us.
> If you had not had that threat things would be different , but you did and would you insist on downsizing your files or uprezzing the D3 files to satiate your curiosity? or is it your way and none other in the testing?
I expect that I and my co-tester could each keep the files we generate and do with them what we like. If I post stuff online to share (kindly!) then I will make clear exactly how I have done the comparison, including the process I used for noise reduction in each file. I think it would be a good idea to compare both by resizing the D3 file to 21 megapixel, and also resizing the 1Ds3 file to 12 megapixel. That way we can fairly examine noise crops in both ways, even though the second method may handicap the 1Ds3 file by throwing away any detail advantage.
But one thing is for sure: I will not compare the files at 100% without resizing one or the other so that there is parity in magnification. To compare at 100% without resizing is subliminally misleading, and non-neutral, because it favours the less magnified image, i.e. the one with fewer pixels.
Avi B wrote:
Please explain your testing methodologies. Just curious.
Avi, since that can of worms has been opened, I've had to go into more detail as you can see in the above comment. Your question now deserves a more thorough answer:
- When it comes to making on-screen comparisons, I won't compare 100% crops at different magnifications, because I think it is a very misleading arithmetical error.
- At very high ISOs, I shoot raw, to be able to use the full 14 bits when pushing (the 1Ds3 files will need to be pushed since it has no 6400 and 12800 settings).
- I typically apply noise reduction (Photoshop dust & scratches) to high ISO files before handing them over to clients, and I think it's sensible in a real world test to allow each file to have noise-reduction applied, according to the file's needs, in NX/DPP and/or other programme, e.g. Photoshop.
- The same lens and aperture should be used on both cameras, with the help of a Nikon->EOS adaptor, so that light levels are identical.
- The same shutter speed should be used, to ensure we are seeing identical exposures.
- Photoshop may be used on the 1Ds3 file to fine tune colour balance and contrast so that it matches as closely as possible the D3 file (a concession to the Nikon, since such adjustments are lossy).
I should point out that I am well aware that the D3's high ISO settings and in-camera noise reduction are a huge convenience for any photographer in a hurry, especially photojournalists and sports shooters, and that makes the 1Ds3 a much less appealing camera for those photographers.
I simply want to compare best possible high iso output from the two cameras.
Sounds like an interesting test. It would also be interesting to do a follow-up test with anything new that gets released – in the near future - by either Canon or Nikon. Some of use who use either system to shoot with will be paying close attention to the results. Good luck.
Anyway, why don't you keep the Canon colours as-is? Or change the Nikon colours instead?
Let me suggest a ISO3200 shot as well so that the 1Ds3 has at least one output at its native ISO range.
Also you should print several different large sizes. I'd honestly be surprised if you could tell any real difference in them. Just my hypothesis
I was also going to suggest a DX shot out of the D3 just for kicks. But then that changes the FL so maybe not so much? Perhaps a centre crop at the DX crop size (for both D3/D700 and 1Ds3) printed large would be interesting
brainiac wrote:
I should point out that I am well aware that the D3's high ISO settings and in-camera noise reduction are a huge convenience for any photographer in a hurry, especially photojournalists and sports shooters, and that makes the 1Ds3 a much less appealing camera for those photographers.
I simply want to compare best possible high iso output from the two cameras.
just to let you know i use NO in camera noise reduction in my images. all is off on both the D3 and D700. i HATE in camera NR.
Part of the problem for me with previous comparisons (and why I haven't really bought into them fully as very valid tests), is due to resizing either the DsIII or the D3 files so they match the same resolution.
Take both files and print them to a standard size, say 16x20 and 24x36 with no post processing or resizing of the files other than allowing the RIP software to handle it as best for the printer. As we all know, viewing on screen doesn't always translate the same as final print. And after all, isn't the concern final print output?
jmcfadden wrote:
you are being disingenuous tho if you are claiming "neutrality" in your thread, that really is the crux of it for me.
J
You have highlihgted the key point here - "Brainiac's" level of neutrality, IMO, rivals his "scientific" approach. I doubt there will be much of value produced here - other than his entertainment!
You can say what you like about my scientific approach, but it's unlikely ever to be as unscientific as comparing crops for per pixel noise at different magnifications. That is scientifically wrong, and guaranteed to lead to the wrong conclusion. Luminous Landscape does it, DPR does it, and I seem to be the only person who has noticed that a great many people are being misled by the unscientific method of these reviewers.
I have done the most unscientific test of all. I bought both cameras (D3,1Ds Mk III and the MK III too) and exchange them while shooting 500/1000 images per evening in dim stage lighting. Then I spend a minimum of 2 hours per image in PS and often print (Z3100 HP) to 24x36" glossy paper.
There are many, MANY variables which bear on each camera's utility. Presently I am setting my D3 for ISO 6400 and the 1Ds Mk III to ISO 3200. I use the Nikkor F2.8 24-70 zoom and a F2.0 or faster prime on my Mk IIIs (I sold my 24-70 Canon which suffers a bit verses the Nikkor). I process the images side by side.
I have to deal with underexposed black costumes and so I have not found it useful to advance the ISO past 6400/3200. I really don't know what will happen when both cameras are pushed another stop. I don't use a lot of NR but I do make a whole lost of processing adjustments to get what I want.
The 1Ds Mk III images can have more detail "enlarge/crop-ability." The NEFs yield a semi poetic filmlike texture which allows, seemingly, unlimited enlargement but, obviously no more detail. Were I to have the 4' carriage Z3100 I'm sure, with genuine fractals, that I would be successful in making 4x5' prints which could be viewed from way too close up.
Don't misunderstand me the 1Ds Mk III doesn't suffer in IQ, indeed at it's best, there's a certain sophisticated quality and grace which I hesitate to describe with numbers but more difficult to obtain. That's because the FPS is much slower, the buffer is smaller, the AF boxes are too far inside the frame,
If you want the extra resolution the 1Ds offers over its brother Mk III you will want an L prime which precisely places the (thin) plane of focus. Otherwise you can wind up with a double size file that has no more resolution than 10/12 Mp files from the D3 or Mk III. Your mileage will certainly vary.
brainiac wrote:
You can say what you like about my scientific approach, but it's unlikely ever to be as unscientific as comparing crops for per pixel noise at different magnifications. That is scientifically wrong, and guaranteed to lead to the wrong conclusion. Luminous Landscape does it, DPR does it, and I seem to be the only person who has noticed that a great many people are being misled by the unscientific method of these reviewers.
I have no dog in this fight. And I like my Nikons so much more than the Canons I've used, for so many reasons, that I have no reason to care about the results of the test either. But, I love the science and the art of photography.
So I'm very curious: why do you believe it's wrong to compare 100% pixel-by-pixel images from two cameras with different resolutions?
I agree that it makes sense to compare the noise for images at the same magnification (pixel dimensions), as this replicates what is happening when the image is printed. You should be able to get a good idea of the difference without actually printing. Just resample the 1Ds3 image to D3 resolution and compare. Interested in seeing the results, I suspect they will be very similar.
I would like to see the difference between the following two tests:
D3 up-ressed to 21Mp vs native 1Ds3 image
D3 native image vs 1Ds3 downsampled to 12 Mp
In other words, does the 21Mp resolution actually get you anything or is the benefit offset by the higher per-pixel noise.
I would also see comparisons of the two cameras shot at f22 and f16 where diffraction noise becomes an issue for cameras with a small pixel pitch such as 1Ds3. Can the diffraction noise issue be overcome by downsampling to lower resolution or is it an inherent problem with capture on a high-res sensor?
Andrew, with so much magnification and reduction of images going on, in the end the software is becoming almost as important as the camera, no? How can one claim to really be testing the camera, when image sizes are changed so dramatically that almost none of the original pixels remain?
andrewd01 wrote:
I agree that it makes sense to compare the noise for images at the same magnification (pixel dimensions), as this replicates what is happening when the image is printed.
Yes - exactly, and hence not resizing is like cutting a square inch out of a 24x36" print from one camera, and a square inch out of a 24x16" print from the other camera, looking at the two square inches and declaring that one has less noise than the other. It is a total failure to compare visible noise in an image. There is no point comparing noise from different cameras in this way. It is wholly misleading.
> I would like to see the difference between the following two tests:
D3 up-ressed to 21Mp vs native 1Ds3 image
D3 native image vs 1Ds3 downsampled to 12 Mp
In other words, does the 21Mp resolution actually get you anything or is the benefit offset by the higher per-pixel noise.
Yes - both comparisons should be done to satisfy the skeptics, although IMO the upsizing one is the more instructive since you are not throwing away any extra detail that may exist in the higher rez file. Extra detail is very rare though, due to resolution limits at wide aperture, camera shake, and the detail destroying effects of high iso. However, it does make sense to apply any NR before downrezzing, regardless of how much detail there is, since NR is not the same as just averaging, so NR at full rez may be more successful than NR at diminished rez.
> I would also see comparisons of the two cameras shot at f22 and f16 where diffraction noise becomes an issue for cameras with a small pixel pitch such as 1Ds3. Can the diffraction noise issue be overcome by downsampling to lower resolution or is it an inherent problem with capture on a high-res sensor?
The short answer is that diffraction is a lens effect. It sets a ceiling on resolution. It can not under any circumstances afflict a higher rez file more than a lower rez file. It simply levels the playing field so that the higher rez camera loses its advantage.
Rodolfo Paiz wrote:
Andrew, with so much magnification and reduction of images going on, in the end the software is becoming almost as important as the camera, no? How can one claim to really be testing the camera, when image sizes are changed so dramatically that almost none of the original pixels remain?
Interpolation is a statistically reliable process. Downrezzing does not affect the quality of the information, only the quantity. It is like standing further away.
Rodolfo Paiz wrote:
So I'm very curious: why do you believe it's wrong to compare 100% pixel-by-pixel images from two cameras with different resolutions?
Some people can see why this is true, and others can't. Imagine you print a picture at 16x24 and at 24x36. You then cut out a square inch of print from each, and hand the two crops to a judge. She will say that the cropped square inch from the more enlarged print has worse (more visually intrusive) noise than the crop from the smaller enlargement. The noise will be bigger. The less enlarged crop will clearly be the noise winner. But it's the same file. Get me now? This is precisely how Luminous Landscape and DPReview compare noise samples. Go and look at their sites if you don't believe me. The Queen's head on the postage stamp is more enlarged whenever the number of megapixels is higher. As a result, their noise tests are skewed in favour of the lower rez cameras.
Meanwhile, I am criticised for being unscientific, or biased. I really have no loyalty to any brand, and when I do my test I will do it fairly, not least because I wish to know the facts of this matter as much or more than anyone else.
I think DPReview are right on this one. They're testing at a given DPI, so two cameras with a different number of pixels will produce different print sizes at a given DPI. What you want to test is noise on a per pixel basis so you need to have a 100% crop.
Otherwise resizing an image becomes totally reliant on whatever interpolation software you choose to use, which has nothing to do with the camera.
At the end of the day I see this as being a totally pointless and worthless test. The 1Ds3 is not competing with the D3 and nor is the D3 competing with it.
The short answer is that diffraction is a lens effect. It sets a ceiling on resolution. It can not under any circumstances afflict a higher rez file more than a lower rez file. It simply levels the playing field so that the higher rez camera loses its advantage.
I agree it's a lens effect, but I am interested to know what the ceiling on resolution is. I have heard that the ceiling at f16 to f22 is around 12MP but I have not seen this proven in any scientific test. For the discussion let's assume we are dealing with "best possible" lenses here, eg Canon L, Zeiss ZF Nikon Pro lenses etc. Hope you can address this issue in your tests.
