AhamB wrote:
The test image. In PS both are indeed neutral so it looks like Firefox is rendering your image test_denoir with a green cast here for some reason...

Monitor color profile... Firefox is not properly color managed.


Regarding the relevance of the test image: "line pairs" are not often seen in nature, so I wonder how useful it will be to be looking at the appearance of set of line pairs processed/resized in different ways. Are you hoping to find some "golden mean" conversion procedure that creates least artifacts while retaining most detail?

Heh, resolution is resolution regardless of what you show. Line pairs are a simple representation of spatial frequency and that's why they're commonly used for tests. MTF charts are another example. That does not however mean that it's only applicable to test charts.

Here's an example from an old post - MTF, theory vs practice:

This is the 35/1.4 Lux ASPH @ f/5.6:

denoir wrote:
I generated an image in Matlab with four sets of line pairs - 10 lp/mm, 20 lp/mm, 40 lp/mm and (sensor max) lp/mm. I used the M9 sensor as a model - i.e 24x36mm that produces 5212x3468 pixel images. Sensor max in that case is 72 lp/mm. For each set the lines go from 50% gray to black.

I notice that the gray lines are two pixels wider than the white lines, except at 72 lp/mm.
Not that it matters much for this test.


Here's the full image - run your script on it.


I don't have a single script, as I am in a phase of testing and evaluation. The following examples are all the result of a single downsize operation.

Bicubic resize in PSP 7.
Horrendous aliasing.

Bicubic resize in PSP X3.
Much better suppression of aliasing, but with deviating grayscale values. The mean pixel value is much higher than that of all other results. (This is the same algorithm that lost the green of the Berlin green fringing.)

Bicubic resize in Matlab.
The best of the three. Apparently, there is no uniform definition/implementation of bicubic resizing. All four bicubic results (including your photoshop output) are different.


FFT resize in Matlab.
Here I made a script that approaches the ideal brick-wall filter, i.e. a sinc filter in the spatial domain or a rectangular window in the frequency domain. It has higher resolution than any other method and uncompromising anti-aliasing filtering, but unfortunately the ringing in the spatial domain (which is clearly noticed around the text) renders it less suitable for actual applications.

This one looks useful:
Lanczos3 kernel in Matlab.
It avoids the ringing of the sinc filter at the expense of mild aliasing in the 20 lp/mm field.

Toothwalker wrote:
I notice that the gray lines are two pixels wider than the white lines, except at 72 lp/mm.
Not that it matters much for this test.


Oops, my bad. Error in the generating script (Matlab indexes start with 1 not 0).

Fixed:
Full image
Resize bicubic (PS)
Resize denoir


I don't have a single script, as I am in a phase of testing and evaluation. The following examples are all the result of a single downsize operation.


Interesting, but you can see in all the bicubic cases (except the photoshop & PSP) the 72 lp/mm lines are continuous gradients - i.e you don't see them as line pairs any more. There's also a drop in overall contrast.

What would one want ideally from a resize algorithm?

1) The preservation of detail. On your resized image you'd still want to see the 40 lp/mm stuff even though the picture size only allows for 20 lp/mm. This is possible by boosting the micro contrast enough (i.e. sharpening) so that any weighted resize (like bi-cubic) becomes more like direct sub-sampling (reducing the impact of the averaging).

2) The preservation of contrast - both absolute and relative. In the example above, you'd want the gradients to remain continuous gradients and to have the same start and end points.

3) The reduction of aliasing.


In the case above, the image has been resized by a factor of about 4.3. It can correctly show up to about 17 lp/mm. So the ideal resize would look something like this:

denoir wrote:
Toothwalker wrote:
I notice that the gray lines are two pixels wider than the white lines, except at 72 lp/mm.
Not that it matters much for this test.


Oops, my bad. Error in the generating script (Matlab indexes start with 1 not 0).

Fixed:
Full image
Resize bicubic (PS)
Resize denoir


I don't have a single script, as I am in a phase of testing and evaluation. The following examples are all the result of a single downsize operation.


Interesting, but you can see in all the bicubic cases (except the photoshop & PSP) the 72 lp/mm lines are continuous gradients - i.e you don't see them as line pairs any more.


That is good, because we don't want to see line pairs that should not be there.


There's also a drop in overall contrast.

What would one want ideally from a resize algorithm?

1) The preservation of detail. On your resized image you'd still want to see the 40 lp/mm stuff even though the picture size only allows for 20 lp/mm. This is possible by boosting the micro contrast enough (i.e. sharpening) so that any weighted resize (like bi-cubic) becomes more like direct sub-sampling (reducing the impact of the averaging).

2) The preservation of contrast - both absolute and relative. In the example above, you'd want the gradients to remain continuous gradients and to have the same start and end points.

3) The reduction of aliasing.

In the case above, the image has been resized by a factor of about 4.3. It can correctly show up to about 17 lp/mm. So the ideal resize would look something like this:

Denoir: Is there a reason why you convert to lab mode before doing the sharpening steps?

shoenberg3 wrote:
Denoir: Is there a reason why you convert to lab mode before doing the sharpening steps?

He doesn't only convert to Lab, but selects the Lightness channel, so that the color saturation is completely untouched by sharpening. When doing sharpening on the color channels as well, saturation is affected.
Demonstration of the differences between Lab and RGB mode sharpening: http://www.fredmiranda.com/forum/topic/860134/113#8733724

An alternative method is to sharpen a separate (duplicate) layer and set blend mode to luminosity.

denoir wrote:
AhamB wrote:
The test image. In PS both are indeed neutral so it looks like Firefox is rendering your image test_denoir with a green cast here for some reason...

Monitor color profile... Firefox is not properly color managed.


I know that Firefox's color management implementation isn't perfect, but that doesn't explain why the bicubic resized test image was perfectly neutral in the browser (while the one resized with your routine had a green cast).

Is there an advantage of doing it in the lab mode, as opposed to your alternative method (duplicate layer)? It seems the latter method would be much quicker to do.

Have a look at those images in the thread I linked to. If you like the boost in saturation, you can sharpen in RGB mode, but if you want to keep the colors as they are, Lab (or separate layer in luminosity blend mode) is needed. In some cases the difference is almost imperceptible, but in others RGB sharpening causes a bit of a color cast in parts of the image.

I think you misunderstood my second question. I was wondering if there would be any differences in doing it lab or separate layer in luminosity blend mode, as the latter method seems simpler.

Toothwalker wrote:
I disagree. Since the image can support up to about 17 lp/mm, it cannot support 20 lp/mm, 40 lp/mm, and 72 lp/mm. Thus I don't want to see the 40 lp/mm stuff and not even the 20 lp/mm stuff. You are talking about spatial frequencies that cannot be represented by the smaller image. Hence any detail in these fields after downsizing has to be an artifact. The ideal resize would show line pairs in the 10 lp/mm field, and smooth continuous gradients with the proper contrast in all other fields - without any high-frequency pattern superimposed.


Not at all. The only place where it is problematic is when you have a strictly periodic patterns. Otherwise you're just doing a subsampling . Say for instance that you photo of a gravel road. The individual particles of the gravel may very will be below the 17 lp/mm that you can show directly. However by a proper resize & sharpen you can boost them to be visible at the 17 lp/mm.

Here's an example that I've posted in a previous discussion. First a 100% crop (M9 so 72 lp/mm maximum resolving power):

I'd like to add three things about what has been written the last few pages (I didn't read to diligently, I just skimmed it...).

1) Whitebalance in raw converters are NEVER, EVER done in Lab. That would be very performance degrading, and actually a very inaccurate way to correct colour.
You have to do WB in the first raw stage, before you even put the raw values through any interpolation. Done at this stage, you get as close to zero conversion artefacting as the value accuracy will allow.

2) Most of the resampling colour faults you see in pictures with high spatial frequencies stem from the fact that you are trying to apply linear theory math to RGB values that are both S-curve- and gamma-treated. You can get around some of those issues by actually doing the resampling in Lab, but you can never get around the fact that you're trying to interpolate between values that are already messed up (have the wrong scene-related contrast).

Working on curve- and gamma-treated values will always have a large impact on the relative detail contrast. The contrast that remains after a rescale depends on the average value of the surrounds - midrange contrasts get a boost, and bright areas are often undersharpened, dark areas oversharpened.

Unfortunately, there's no (practical) way to do things "in the right order" until we have raw-converters that actually give you that option. Until then, we have to make do with what's "visually pleasing", and this is somewhat of an arcane art - it isn't quantifiable.

3) As in most other things, the large software manufacturers are terrified of things that "can go wrong". So almost every editing action that's available is tuned for safety (non-aliasing, non-artefacting) and for performance - performance as in "time spent" or "processing cycles spent", not in "accuracy or pictorial performance". The same goes for resampling algorithms. BiCubic is chosen because of the almost total resistance to aliasing in natural scenes (not testcharts), even though there are other easy ways to do it with much higher accuracy and not much more processing power involved.

Lanczos in the 3- and 5-lobe versions are very good, almost as good as a complete Sinc reconstruction (which would be the same as the FFT toothwalker showed).

denoir wrote:
Not at all. The only place where it is problematic is when you have a strictly periodic patterns.


A periodic pattern just reflects one of the constituent frequencies of a normal photographic image. In terms of audio, you want to hear evidence of a 72-Hz tone in an audio file sampled at 33 Hz. You won't hear any, and if you do you are listening to artifacts such as a tone at a lower frequency due to aliasing.


Otherwise you're just doing a subsampling . Say for instance that you photo of a gravel road. The individual particles of the gravel may very will be below the 17 lp/mm that you can show directly. However by a proper resize & sharpen you can boost them to be visible at the 17 lp/mm.


An individual pebble is represented by a broad distribution of spatial frequencies, and after the resize only those <17 lp/mm can remain. All frequency contents >17 lp/mm is necessarily lost.


Here's an example that I've posted in a previous discussion. First a 100% crop (M9 so 72 lp/mm maximum resolving power):

Toothwalker wrote:
You can't see it, because it is not there.


No, don't think you've understood what I mean. My fault I'm sure. I'm not suggesting that you can cheat mr Nyquist.

However, consider two things - first, what happens when you sample with an insufficient frequency? Aliasing. You don't end up with nothing - you end up with something with a lower frequency. You still record a signal - it just isn't an accurate representation of the full high frequency signal.

Second, consider how bicubic (and most other) downsampling works. Each target pixel is weighed with neighboring pixels in the source image. So it's not a plain selection of every N:th pixel.

Let's take a look at an pixel level example. First up is photoshop bicubic resize. We start with a 16x16 image and reduce it to 8x8 and finally to 4x4. Note that 8x8 is the last level where we can theoretically capture the high frequency pattern - the lines have a width of just one pixel.

Toothwalker wrote:
It appears you did, because a single lanczos3 resize followed a single sharpening step looks similar:

denoir wrote:
To demonstrate this with a better example, consider a mixed 16x16 image that contains both N lp/mm and 2*N lp/mm where 2*N is the maximum supported as they're only one pixel wide.

Bicubic first:

Toothwalker wrote:
It appears you did, because a single lanczos3 resize followed a single sharpening step looks similar:







That does look good. That's matlab's kernel implementation to be used with imresize, right?

Ok, I see I'm still not getting my point through. Let's go back to basics. What does Mr Nyquist tell us?

If we have a signal we must sample it at the double maximum frequency to be able to perfectly reconstruct it:

denoir wrote:
What I'm saying is that this is a much better reconstruction (although of course imperfect) than a flat line at 0 would be - which you are saying is the optimal answer.

I'm not sure you'll find universal agreement with that. In fact, for repeating patterns that aren't a perfect integer multiples of the sample rate the result is called Moire which is rarely considered desirable.

I think you are driving at the fact that perhaps aliased information isn't always a bad thing aesthetically, which seems reasonable in some cases. I think what Paul is driving at is that plenty of detail is recoverable for small isolated impulse functions in the image (e.g. pebbles) within the Nyquist bandwidth if you use the proper kernel and so there is more risk than benefit in allowing aliasing to occur. Apologies to both of you if I'm failing to parse your intent and just muddying the waters by injecting myself here!

It definitely preserves more information than just not bothering with any reconstruction just because you can't represent as high a frequency as the original signal had.

Hmmm.... I have a hard time with this interpretation. What is "information"? If I alias and cause visually objectionable Moire across part of the image have I "preserved" information or have I in fact irreversibly corrupted the lower frequency scene information with aliased artifacts that have no representation in the real scene? Isn't that more akin to destroying information?

Certainly in some cases the aliasing "false information" may be visibly pleasing or may increase "apparent detail" but is this really "preserving" information or just serendipitous image enhancement that could equivalently and more safely be performed with a proper downsampling kernel and sharpening?

I may be influenced here by my "day job" in which your aliasing proposal is considered an unmitigated disaster clearly corrupting information. The only case in which I know aliasing is used to any advantage is radar signal processing, but in those cases multiple sample rates are used along with things akin to the Chinese remainder theorem to resolve the ambiguity - these techniques depend on the signal source being quasi-stationary which isn't at all applicable to image data. (And I exclude undersampling of bandpass signals here which is sometimes somewhat erroneously considered aliasing despite the fact there is no ambiguity in the sampled data).

There are pitfalls in drawing parallels between communications sampling theory and what "looks good" in image processing - so I've been watching this discussion with interest. But the concept of "preserving information" through the introduction of ambiguity due to improperly bandlimited downsampling raised the DSP hairs on the back of my neck to the point I felt like posting

Thanks to both of you for the continuing good read!

Ken

denoir wrote:
What I'm saying is that this is a much better reconstruction (although of course imperfect) than a flat line at 0 would be - which you are saying is the optimal answer.

I believe that what Toothwalker is saying is that 50% gray, not 0, is a better representation of existing lines than "new" lines with a different width. This given that the lines are too narrow. With your "sampling" the result would be the latter - if interpreted as e.g. a horizontal lineapproximation in a jpg image. Your sinus curve would build up the left part of his example image, the one with too narrow lines.

Sorry if i missed your point and thanks for this enlightening discussion

denoir wrote:
Ok, I see I'm still not getting my point through. Let's go back to basics. What does Mr Nyquist tell us?

If we have a signal we must sample it at the double maximum frequency to be able to perfectly reconstruct it:

shoenberg3 wrote:
I think you misunderstood my second question. I was wondering if there would be any differences in doing it lab or separate layer in luminosity blend mode, as the latter method seems simpler.

If you want your sharpening to be destructive (you're sure that you're not going to tweak it at some later stage), you may as well forego creating extra layers and work in Lab.