I already had the f4 70-200L and the f4 300 L IS lenses when I tested the 70-300 DO lens, which I ended up purchasing. My own test was conducted in my own worst case, real world situation, where I probably wouldn't have brought along nor considered using my full-size telephotos. The "test" was shot in an available, mixed light environment (in a Palo Alto, California camera store), using a 1D Mark II, which the store let me use.
I was very favorably impressed. If I can provide a link to my results, they're at: http://www.rlk.com/do_70_300.
After purchasing the 70-300 DO, my initial thought was to put my 70-200 and 300 up on eBay. But, it's now occured to me that each lens serves a useful purpose. When I'm travelling light, I almost never could fit the 70-200 and/or the 300 in my little shoulder bag -- so unless I was certain of the need, they got left behind. The 70-300 DO is so compact, there's always room for it in the small shoulder bag, just in case. I was also able to prove to myself, that in spite of its relatively slow f-stop, it's a great little available light people lens. I was also able to prove to myself that, even without a lens shade, the results were crisp wide open, and the IS was a big help at the 1/40th - 1/70th shuuter speeds I was using.
FredericB wrote:
Thank you very much for the test - it finished convincing me that the 70-300DO will not meet my needs (even though I would love a lighter lens than my 100-400 since I do most of my shooting while hiking in the rockies).
I do not know if you've had a chance to do similar tests on the 400 DO vs 500 F:4 but I'd love to see it if you have for I am in the hunt for a long telephoto (not for the hikes this time!)
I don't want to bash Amir for his tests but all of his comparison shots look bad to me. Are you shooting test charts or landscapes? I have had the DO for a month now and haven't seen anything even remotely that bad looking come out of my 1D.
This review mirrors the differences I'm seeing between the DO and the 70-200 f2.8 for the kind of pictures I'm taking:
1. Yes, I am shooing a flat chart against the wall because I don’t know of any other way to assure proper focus center to edge. I am also shooting at closer focusing distances than what the photo mag is doing. The distance was close to 10 feet from what I recall. Perf may be different at other distances. I also used a 1Ds which is more revealing wrt to resolution than your 1D and takes in the entire image, not a portion of it as with 1D.
2. I made no attempt to make my pictures pretty. All of them are horrible looking when judged in this area due to poor lighting and white balance among other reasons. This is actually an asset because pretty looking things make you think something is better than it is. So yes, all the images look “bad” but this doesn’t impede the measurements that I performed.
3. I see no methodology details in the magazine report. How did they assure proper focus which I found to be super important? Did they use AF? If so, with a changing, 3-D scene outside, how did they make sure the same focus was achieved? Did they have a computer with a large screen to test focus or did they rely on the tiny viewfinder with no focusing aid?
4. I have no idea how one does a proper A/Btest outdoors. The light level changes as the test goes on, eliminating proper contrast evaluations. The results are not repeatable and incremental testing can not be done like I did to achieve good focus.
5. I see a ton of Canon logos everywhere. People are wearing Canon hats, shirts, shooting sailboats with Canon sails, etc. Not saying there is any bias here but I did not get any of these before I did my test or my results may have been different .
6. It is easy to come up with tests that show no difference between two systems. Just pick an easy target and everything looks the same unless something is badly broken. As an analogy, imagine timing how long it takes for two cars to stop from 5 miles/hour. They would both stop in less time than you can measure by hand. Increase the speed to 60 mph and you separate the real players from the others. I had to work hard to find targets that were revealing. Indeed, even the typical resolution charts that I used here did not show the proper difference. It was the little bars in the PDI image that was the winner.
The cropped areas that they show do not have high frequency information. So any resolution differences are not going to show up there.
Here is the bottom line. I have no idea if their tests are better than mine. Testing lenses with a camera is half science, half luck. You are testing a system here, not just the lens. There are ton of variables. I tried to eliminate most of them and feel that they didn't go as far. But heck, their results could more representative, I don't know. I also don't want to make anyone feel bad about their purchases as I am in that pool myself .
What I do know is that Canon's theoretical MTF charts clearly show the DO to be worse than the L lenses. Just look at the charts below from Canon. The F4 wide open has a center score no less than 0.77. The DO is way lower at 0.64. At F8 (the blue lines), the L lens achieves ruler flat perf with a near 1.0 maximum score (which is traditionally reserved for Canon’s best super telephotos). The DO tops out at 0.92. Yes, .92 is very respectable but the F4 peformance is out of this world especially for a zoom.
So the magazine's results go not only against mine, but also against much more objective (but theoretical) testing done by Canon. There is a reason Canon doesn’t give the L designation to these lenses. They want to distinguish these as a different class of product. Lightweight, smaller but with some compromise in image quality. As long as one understands this, you can make the right purchase decision.
This was measured on an image that was blurred in photoshop up to 11 times and the MTF values read off each time - each BxU represnts the effect of one application of "blur more" in Photoshop.
I appreciate your testing Amir and am not accusing you of doing a bad test any more than you should accuse that article of being biased. I simply said your shots didn't look very good to me, don't reflect what I'm seeing from my 1D with this lens and that my experiences shooting outdoors (pretty much the only place I'm using this lens) reflect the results as shown in that Euro link I put up. Your report doesn't reflect the DO in a good light so I was simply showing another side to those on this thread who are interested. You're welcome to post all the MTF charts, pictures of resolution charts you want but it won't change my opinion of the lens. I've found it to be exactly what I thought it would be, a compact, well built, inconspicious walk around lens with great image quality. I owned the 70-200 f4 and currently own the f2.8, they are both great lens - no argument there.
BTW I see you own the 400 DO, are you happy with it? I want to get a 400 tele next and really like that lens.
I am sorry rd4tile. I was not taking a shot at you and didn't mean to get so defensive . In my day job I deal a lot with magazine reviewers (in Audio/Video domain) and am always frustrated by how little the understand the importance of doing tests right and how difficult it is in general, to properly benchmark things.
Yes, I have the 400DO. But rarely use it. I was not happy with the quality of the images when I got it. So I rented the 400f2.8, 500f4 and 600f4. I was surprised how much better they were. So I went ahead and bought the 500f4 and that's all I use now. Having said this, my tests then were not as rigorous as the ones here. So I will repeat them again when I get a chance. But from what I recall, it was the lack of contrast that bothered me last.
Of course, the 500 has a longer reach which I tend to need anyway so the quality is not the only reason I use that lens.
You probably know the weight difference is quite a bit here. The savings in size and weight is quite remarkable when compared to the 500. So even with lower image quality, you may still want to consider buying it. There are days which I regret lugging the whole setup with me. But when I go on a wildlife shoot, I am serious about it, no matter how tough it is on my back .
Andy, thanks for the remarks and interesting experiment. I actually ran into the mechanical equiv of the same thing running this test. I noticed that both contrast and resolution would improve as focus became more accurate so the two do track. The effect though, was not following the same curve with each lens though as I noted with the 70-200f4.
It is interesting that the 10lpm happens to be linear bu thte 30lpm is not. I can understand the latter but it must be a coincident that the former is linear. I wonder if it becomes non-linear with higher "blur more" factors?
I agree it is odd that DXO spits out this factor rather than any other specific measure. Maybe they are just measuring contrast and expressing them in these units. I would think measuring resolution is harder for them.
The thing that would help all of these benchmarks btw, is computer controlled AF. Canon should let their remote capture application control the AF servo. Using that, one could achieve near perfect focus which would be wonderful for still life work in the studio besides lens testing.
Amir, thanks a lot for the test! It's interesting to see that the 70-200Ls with TC are still better than the 70-300DO. But I also think that this test doesn't show that the 70-300DO is a bad performer - it's just not as good as the two 70-200Ls.
And the 10lpm on the graph: it's not linear. If it was linear, we would get a negative MTF value after approximately 25 "Blurs more" (just extrapolating the line) ... not very likely, huh?
Have to be careful when using geometric MTF calculations as it assumes a perfect optical system, and the performance is based as a function of defocusing.
Rainer - if the extrapolated line went below "0" it would indicate a phase shift in the image (blacks become whites, white blacks). This happens and is known as spurious resolution.
Measuring MTF is complex. And one really needs a controlled environment and proper equipment to achieve realistic results.
Interesting observation in this thread and in many Internet digital camera forums is the lack of mentioning how important contrast is when considering lens performance. It is equal to - if not more important than - resolution depending on the application (in this case "the shot"). And this is where I find Canon's L lenses really excel, in producing very good contrast. Even the less sharp ones (24mm F1.4 for example) produce wonderful imagery due to the high contrast.
Being in the imaging business, I was quite intrigued about Canon's DO lenses. We use diffractive (or binary) lenses and many other types in our infrared systems. But it is challenging to make DO elements work in the shorter wavelengths, and make them cost effectively. Kudos to Canon.
I picked up the DO last week but, to be fair, haven't had much time to take it out. Have a couple of 'quick and dirty' backyard shots posted here:
(Also note I'm not a "pro" so don't expect examples of award winning compositions! Just getting back into the hobby over the past few months.)
Initial experience: yes, it's lower in resolution than other L lenses, but has very good contrast and color reproduction. The DO's ability to null out any lateral color issues (CA) is also a very nice quality as seen in the fluorite equipped lenses.
EDIT: Sorry - should have added my compliments to Amirm for taking time to do the tests!
RK
Edited by Kubicide on Jun 06, 2004 at 09:31 PM GMT
.
.