Ok, guys, your objections are similar so I'll answer you all at once. Yes, I like aliasing and I've said so before (look at two posts back). Aliasing can be problematic when you have periodic patterns where the incomplete signals sampled at a too low frequency can introduce new objectionable patterns. Usually however this is not the case. All digital medium format backs work under this assumption as do the Leica M8 & M9. The price is that you can get moire where you have periodic patterns. The benefit is that you get a much crisper picture.
You can see the moire pattern in the blinds as a consequence of no AA filter on the M9. But guess what, there's just as much aliasing in every part of the picture. All that fine detail that you see consists for the most part of imperfectly reconstructed signals. It's what makes it so crisp. And you want to nuke that with a low pass filter just because it isn't a 100% accurate reconstruction? No thanks. Lowpass filtering won't make it any more accurate but it will make it much softer.
Now, how do I want my resized images to look? Crisp with moire or soft without? I'll take the former any day and deal with the rare cases of problematic moire by hand.
Does that sufficiently explain my reasoning? I'm definitely not alone in this - the designers of the AA-less cameras share my priorities.
kwalsh wrote:
There are pitfalls in drawing parallels between communications sampling theory and what "looks good" in image processing - so I've been watching this discussion with interest. But the concept of "preserving information" through the introduction of ambiguity due to improperly bandlimited downsampling raised the DSP hairs on the back of my neck to the point I felt like posting
Thanks to both of you for the continuing good read!
Ken
Thank you for an eloquent contribution. I agree that an image is successful when it "looks good", regardless of whether strict attention was paid to the theory of digital sampling during its materialization. Like you I doubt that massive aliasing positively contributes to the appreciation of a real-life photograph.
For some time I have been wondering about the multi-action resize scripts that are so vividly described in this forum. Downsampling in steps, sharpening before downsampling, avoiding certain resampling factors ... I think there is no theoretical base for any of these ingredients if a proper downsize method is used. However, the widely used bicubic resize in popular photo editing programs is apparently not so good, and people have indeed demonstrated that a multi-step approach can produce images that look better than a single bicubic resize. Still the question remains whether downsampling in steps can achieve something that cannot be achieved with a single action of a proper algorithm. If it can, that something probably falls in the category of "looking good" rather than being a faithful representation of the original scene.
Technically any sharpening adds false detail. A bicubic resize (including lanczos3) won't eliminate aliasing completely - it will just lower the micro contrast significantly. When you add sharpening you again boost the micro contrast and thereby the aliasing.
Good examples, and I get where you are coming from with cameras without AA filters.
Now I think I see where some of us are disconnecting here (or at least I am, can't really speak for anyone else). It is in the analogy between resizing (re-sampling) within the digital domain and the original spatial sampling of the continuous image (with or without an optical AA filter).
As you point out many camera systems forgo the use of an optical AA filter. There are some very good reasons to do this:
- The PSF of an AA filter is a horrible beast to release upon top notch optics!
- The roll off of an AA filter is rather shallow and thus can have a negative impact on "micro-contrast", that is it is significantly attenuating spatial frequencies well below Nyquist that really should be preserved.
- Even the best optics often have spatial bandwidth (resolving power) not very much in excess of the imager so extreme aliasing isn't very likely even without the AA filter. In other words there actually isn't very much energy above Nyquist for the AA to remove.
None of these conditions hold in digital resampling, however:
- The PSF of the resampling kernel can be arbitrarily good without any of the bizarre characteristics of an optical AA filter.
- The roll off of a resampling kernel can be nearly brick wall in nature which allows perfect preservation of micro contrast while completely blocking aliasing components.
- For large resampling steps there is enormous amounts of signal energy above the output Nyquist frequency in the original image ready to alias in the resized image unless suppressed by the kernel.
In short I think the two of us are drawing different conclusions from cameras without AA filters. You seem to be saying the aliased image data let into the digitally sampled RAW file produces a visually desirable result. I've drawn a different conclusion, namely that the aliased image data really isn't desirable at all but it is less objectionable than what the poor roll off and PSF of the AA filter does to unaliased spatially frequencies. In my interpretation the lack of the AA filter isn't a benefit because it allows aliasing, rather removing the AA filter is a benefit because it is such a bad filter that it does more damage to in band spatial frequencies than it does good by eliminating aliasing frequencies. I think we agree there are good reasons to get rid of an AA in the camera, I think we attribute it to different effects.
Because of these different interpretations of AA-less cameras from my perspective I can't see why one would choose a resampling kernel that allows aliasing. We aren't constrained to awful filters like birefringent OLPFs in the digital domain - we can make nearly perfect resampling kernels. We don't have to make the trade between "tolerate some aliasing" and "lose some microcontrast" - we can get the best of both worlds with nearly optimal kernels that don't exist in optical systems. Bicubic resampling is a poor kernel that is a hold over from ancient 16-bit processors that needed fast algorithms. We can make larger kernels now with excellent single pass resampling results as Paul has demonstrated.
So anyway, that's where I think we are approaching the problem differently. I don't think the AA-less camera analogy is a good one. That analogy constrains one to comparing the truly awful kernel (PSF) of an OLPF (way worse than the worst bicubic kernel) with a still so-so kernel (PSF) of the lens in a system where often the Nyquist frequency is already in a region of high attenuation. This doesn't correspond well at all to resizing where near ideal kernels are available that are superior to bicubic by a large measure and there is a whole bunch of energy above Nyquist.
Ken
P.S. On an unrelated note, I really liked your MTF overlay illustration earlier in the thread.
denoir wrote:
... how do I want my resized images to look? Crisp with moire or soft without? I'll take the former any day and deal with the rare cases of problematic moire by hand.
Does that sufficiently explain my reasoning? I'm definitely not alone in this - the designers of the AA-less cameras share my priorities.
+1
The DMR's sensor also has no AA filter and it's a problem for me only rarely. I'd rather deal with the occasional problem by hand, and only in the areas where it's a problem, rather than have 100% of my photos softened 100% of the time.
I haven't read through this entire thread but frankly I'm astonished at the degree of manipulation that is thought to be nessesary to get good sharp prints. I usually adjust white balance, re-size for output, and sharpen once. That's all. For web it's Photoshop's Filter->Sharpen->Sharpen, for prints it's light USM.
Toothwalker wrote:
For some time I have been wondering about the multi-action resize scripts that are so vividly described in this forum. Downsampling in steps, sharpening before downsampling, avoiding certain resampling factors ... I think there is no theoretical base for any of these ingredients if a proper downsize method is used.
If the sharpening steps are equivalent to the linear convolution of a kernel then they of course could simply be convolved with the next resampling kernel and be perfectly equivalent (basic USM for example). If, however, the sharpening steps use any sort of non-linear functions (thresholding, shadow or highlight haloing protection) then that of course wouldn't be true.
Since the bicubic kernels do allow some aliasing then the multistep resampling is adding multiple different aliasing frequencies along the way. I don't know why that would necessarily be desirable, but it would certainly be different from a single kernel and down sample.
However, the widely used bicubic resize in popular photo editing programs is apparently not so good, and people have indeed demonstrated that a multi-step approach can produce images that look better than a single bicubic resize. Still the question remains whether downsampling in steps can achieve something that cannot be achieved with a single action of a proper algorithm. If it can, that something probably falls in the category of "looking good" rather than being a faithful representation of the original scene.
I find it unlikely that bastardizing existing poor kernels and sharpening algorithms by applying them in multiple steps is likely to produce a better result than a single well thought out linear kernel followed by a single non-linear sharpening algorithm. However, when what matters is aesthetics then we enter the realm of trial and error and to do this practitioners are going to use the tools at hand. So I don't find it surprising at all that good results are coming from seemingly Rube Goldberg approaches since the tools at hand seem to be forcing the people doing the most experimentation into that approach while the folks more likely to be using more direct algorithms by nature don't seem to be approaching the problem as much through aesthetic experimentation. Different horses, different courses - and a lot to choose from in the end!
Doug, if you want to, send me a full-res version of one of your shots, and I will down-res it, then we can compare it to what you do. Only that way will you know if there is a difference or not. I think it would help the shots (in web size), but keep in mind that printing is very different than resizing for the web.
Ken, very good points and an interesting take on the subject.
I think however that you may be overestimating how good the kernels are at removing aliasing. I'll show some examples below but from what I've seen to truly eliminate aliasing you need to be more aggressive in the low pass filtering than the regular bicubic algorithms are - including lanczos3. And to top it off, to get a crisp picture you need to add sharpening which to a varying extent undoes what the resampling kernel did in the way of removing aliasing. For instance a deconvolution sharpening can recover amazing (false) detail.
I'll aim to show:
1) That bicubic/lanczos3 does not remove aliasing
2) That sharpening undoes any anti-aliasing effect the down sampling algorithm had
3) That an unsharpened image looks less visually appealing than a sharpened one and that the sharpened one closer resembles qualitatively the higher resolution original.
4) That to truly remove all aliasing you need to low-pass filter the image beyond what most people would consider acceptable
1) Bicubic/lanczos3 & aliasing.
We'll use the 16x16 -> 8x8 example here that contains two basic spatial frequencies plus a diagonal line.
The last image has been low pass filtered before resize to remove all spatial frequencies that cannot be represented in the resized image without aliasing.
Which do you think looks best and in which one is the texture qualitatively closest to the 100% crop? My answer is 2) without hesitation.
If you agree, what conclusions can be drawn? Well, you made a good point that it's not aliasing I like but rather that I dislike the brute nature of the anti aliasing process - even with a sophisticated kernel such as the lanczos3. In practice it makes no difference though when it comes to the processing. If you skip/counteract the anti aliasing, you end up with aliasing and if it looks better, well, that's it then.
Edit: Point 3 was incorrectly expressed. I meant of course that the sharpened was closer to the original crop.
kwalsh wrote:
However, when what matters is aesthetics then we enter the realm of trial and error and to do this practitioners are going to use the tools at hand. So I don't find it surprising at all that good results are coming from seemingly Rube Goldberg approaches since the tools at hand seem to be forcing the people doing the most experimentation into that approach while the folks more likely to be using more direct algorithms by nature don't seem to be approaching the problem as much through aesthetic experimentation. Different horses, different courses - and a lot to choose from in the end!...Show more →
Precisely! This discussion started with the question if we could make something better on theoretical basis rather than the improvised (as you aptly named it) Rube Goldberg approaches. We soon however encountered the problem that we realized that we had very different ideas of what would constitute an 'ideal' resized image. So that's what we've been discussing now.
If it is possible to describe a theoretical best solution then one could perhaps work backwards and develop a good algorithm to get there. There has to however be an agreement of what 'best' is.
denoir wrote:
Ok, guys, your objections are similar so I'll answer you all at once. Yes, I like aliasing and I've said so before (look at two posts back).
You can see the moire pattern in the blinds as a consequence of no AA filter on the M9. But guess what, there's just as much aliasing in every part of the picture. All that fine detail that you see consists for the most part of imperfectly reconstructed signals. It's what makes it so crisp.
I have to disagree. You don't prove that the crispness (detail) is due to aliasing. The detail is due to perfectly rendered signal components up to the Nyquist frequency.
The fall-off of the anti-aliasing filter, that starts well below the Nyquist frequency, prevents this in other cameras. You don't like aliasing. You like the benefit of having no filter for the rendering of detail and accept the drawback of aliasing. (Visible in the blinds as well as via color artifacts in several other places.)
Since the pixel density is high the transfer function of the lens already acts as an anti-alias filter and the artifacts are not too overwhelming. I don't think that you would like a sensor with a low pixel density and without AA filter.
(There is also an effect of sensor cells having a finite size rather than that they point-sample the image cast by the lens.)
Finally, as Ken wrote you can approach the best of both worlds once you are in the digital domain, so this example is not a good one to illustrate what you want to say about downsizing techniques.
I prefer a sensor with AA filter by the way, certainly for the purpose of the relatively small images typically shown on the net. Once you downsize the high-frequency detail in the original capture is not very important and aliasing in the original capture cannot be removed by downsizing.
Toothwalker wrote:
I have to disagree. You don't prove that the crispness (detail) is due to aliasing. The detail is due to perfectly rendered signal components up to the Nyquist frequency.
The fall-off of the anti-aliasing filter, that starts well below the Nyquist frequency, prevents this in other cameras. You don't like aliasing. You like the benefit of having no filter for the rendering of detail and accept the drawback of aliasing. (Visible in the blinds as well as via color artifacts in several other places.)
As I said, it doesn't really matter which way it is - if I object to anti aliasing or if I like aliasing. It ultimately comes down to the same thing. AA as digital signal processing isn't much better than the optical variety. If you truly want to eliminate aliasing you'll have to ruin the image first by excessive low pass filtering. And if you don't sharpening will restore it (there's a question of degree of course, but it doesn't detract from the principal argument). I've shown as much in my previous post.
However, I'm not willing to say that I don't want aliasing. I still maintain that more information is preserved even when a signal is subsampled at a frequency below the Nyquist limit.
An aliased (sampling frequency below the Nyquist limit) reconstruction will still reflect the increase in amplitude rather than the flat 0 an anti-aliased solution would present. You'd preserve both the information of periodicity (not the period, but that it is periodic) and the increase in amplitude. That's real information.
Now in a real world image things are more complex where you have a blend of multiple spatial frequencies. And in effect you'd be blending the poorly reconstructed signal with a properly reconstructed one. So which is worse, sabotaging the good one or throwing away information from the poor one? I don't know and I have so far not heard any convincing arguments either way.
Since the pixel density is high the transfer function of the lens already acts as an anti-alias filter and the artifacts are not too overwhelming. I don't think that you would like a sensor with a low pixel density and without AA filter.
Ken brought up this point as well, but I think you underestimate the lenses in question. For instance I used a Zeiss 25/2.8 ZM in a previous example which is diffraction limited at f/4 and resolves up to 400 lp/mm center frame. It is way beyond the resolving power of any 35mm sensor. The crop above was shot with a lens that is around 90% MTF at 40 lp/mm, so you can rest assured that it outresolves the sensor by a large margin. So I very much doubt there is any AA effect from the lens.
This however gave me an idea of a rather simple experiment to perform. One could point a high resolving lens on an AA-less camera at a subject where you know moire patterns will be evident. Then by very slight steps you move focus and for each move take a shot. At some point the moire pattern will disappear as the misfocused lens will act as an AA filter. Unlike the camera based AA filters this will be a very high quality AA filter. At that point where the moire disappears one should check how much detail has been lost (if any) in other regions where there was no apparent aliasing. If tuned precisely the lens should at one point cut off at the nyquist limit and in theory there should be no loss of detail according to the "aliasing is bad" theory. Correct?
James Burden wrote:
You guys are too damn smart for photography. Shouldn't you be out designing spacecraft or discovering alternate fuels or something?
That's my day job
Seriously: I'm writing software that will help developers build zero-net-energy homes.
James Burden wrote:
You guys are too damn smart for photography. Shouldn't you be out designing spacecraft or discovering alternate fuels or something?
Hehe, actually since you mention it.. My company develops artificial neural network software that is used by both ESA and NASA (JPL actually). As for alternative fuels..err..we're currently helping an oil company decide how they should treat their shale oil wells before extraction so that they can maximize total extraction over time.. Does that count?
Thanks for the resizing examples. I see what you are driving at there. The limitation is that the spatial domain support of the kernels is still very small so the transition band in the frequency domain is still going to be very broad. So you are still going to get a lot of attenuation in the "microcontrast" frequencies if you try to severely attenuate the aliasing frequencies just as your examples show when restricted to such a small kernel.
My last image processing work was nearly a decade ago and was more about detection than aesthetics, as a result I could use very large kernels with very nice frequency domain responses. I can see how as a practical matter in these cases a large kernel would cause trouble at image borders (there are ways to deal with this of course, but it is a complication) and potentially other aesthetically objectionable effects in the spatial domain like ringing near contrast edges. Similarly, in communications DSP we tend to use very long filters as no one cares what the time domain looks like except in a very narrow way.
I haven't tried it, and don't have the time at the moment, but I can imagine aesthetic constraints and not just processing constraints require a kernel size too small for a sharp resampling filter roll off. That would result in the aliasing vs. "microcontrast" trade off we've been discussing. As you mention if that is truly a constrained trade off due to aesthetics it doesn't really matter what specifically is the root cause as you would be forced to pick your trade off point based on aesthetics and leave it at that.
I can certainly imagine with said insight creating a far more sensible kernel design and resampling approach without the multiple resizing and sharpening steps or excluded resampling ratios. That said, processing power is cheap and it isn't a real time task so if someone has an empirically designed approach that produces a desirable result with a multi-step action or script I can't see a strong motivation to invent a new tool to reproduce it.
As a side note, for your experiment proposal I'd think maybe an easier an more controlled approach would be to just keep stopping down the aperture to increase the size of the Airy disc. Much easier to model and back out what exactly happened I'd expect.
That tickles a memory of some tests and discussion I saw on a Luminous Landscape forum awhile back. There were some pretty impressive results of adaptive deconvolution sharpening recovering detail from small aperture diffraction limited shots compared to wider aperture shots. I wonder how a single pass downsample with a standard kernel followed by such an algorithm would compare to the more complicated multistep procedures. Or maybe someone has already tried that - I haven't followed all of this closely enough.
Again, thought provoking discussion. Thanks for the nice examples.
James Burden wrote:
You guys are too damn smart for photography. Shouldn't you be out designing spacecraft or discovering alternate fuels or something?
Well, recently I've been more worried about the communications ground stations for spacecraft rather than the spacecraft themselves. It hasn't helped my photography one bit though!
denoir wrote:
Hehe, actually since you mention it.. My company develops artificial neural network software that is used by both ESA and NASA (JPL actually). As for alternative fuels..err..we're currently helping an oil company decide how they should treat their shale oil wells before extraction so that they can maximize total extraction over time.. Does that count?
Good enough for me JPL's using some of my software too.
As already suggested, there is a potential disconnect between proper signal processing and what looks pretty. I am regularly plagued by moire because the 1Ds has weak AA filtering, and my lenses are too sharp. This can turn large expanses of my subject (textile) into a chromatic disaster. At least with distance shots, one can stop down until diffraction lends a hand.
I also thought edge sharpening was an abomination at one time, and tried to unify the effect with Lanczos-windowed sinc in my interpolator. Eventually, I realized that sharpening is a distinct issue relating to visual acuity and the viewing geometry. Specifically, you will sharpen edges in the viewed image (photograph, CRT) across a certain angle subtended to the eye. Pixel-level signal-processing issues - like sinc "ringing" or aliasing - are operating at a different level, and don't relate to the "eye candy" issue of edge sharpening. Note that edge sharpening is relevant in analog photography, too: think unsharp masking, and the film/developer feature called "acutance".
