Home · Register · Join Upload & Sell

Moderated by: Fred Miranda

New fredmiranda.com Mobile Site

New Feature: SMS Notification alert

New Feature: Buy & Sell Watchlist

FM Forums | Sony Forum | Join Upload & Sell

2 3 … 15 16 end

Archive 2017 · Pixel shift and lens testing thoughts...

mjm6
Offline
• • •
Upload & Sell: Off

p.1 #1 · p.1 #1 · Pixel shift and lens testing thoughts...

A thought occurred to me that the pixel shift capability of the new a7RIII will potentially lend itself to lens testing as long as the shooter is able to use a suitable subject. It does this for a few reasons potentially:

1. The higher resolving potential of the file will push the lenses more, taking an effective 10.5MP file to 42MP.
2. This also eliminates the Bayer filter effects. I'm not sure how much this impacts some or any of the optical characteristics like chromatic aberrations, but I suspect at least part of those issues has to do with the way the lenses render off axis through a Bayer filter, not just through the cover glass.
3. If this multi-shot approach also includes some noise reduction that is associated with the use of the four shots, it may be possible to also help highlight the differences in the lens performance be reducing the sensor noise that begins to show up at the pixel level in a single shot image.

Actually, #3 could be done right now with frame averaging or smooth reflections, but I don't think anyone is doing that for lens testing to my knowledge.

I don't expect to get an RIII anytime soon, but I'd be really curious to see some lens testing done single-shot compared to pixel shifted to see if the performance differences between lenses becomes more apparent with the potentially higher resolving capability of the camera. The differences should be greater than the difference seen by comparing an original a7 to an a7RII, and probably about equal to that from an original a7S to the A7RII, given the actual megapixel values that are present in these cameras.

Remember how 'easy' the a7S is on some lenses (in particular the adapted lenses)? I suspect the pixel shifted a7RIII will be absolutely punishing on most lenses, even the best that Sony is making... Of course, this will only occur if Sony builds a camera that is actually consistently making the shift mechanism work correctly at the scale needed for the pixel shift to work well.

---Michael

Oct 25, 2017 at 07:46 PM

Fred Miranda
Offline
Admin
Upload & Sell: On

p.1 #2 · p.1 #2 · Pixel shift and lens testing thoughts...

mjm6 wrote:
A thought occurred to me that the pixel shift capability of the new a7RIII will potentially lend itself to lens testing as long as the shooter is able to use a suitable subject. It does this for a few reasons potentially:

1. The higher resolving potential of the file will push the lenses more, taking an effective 10.5MP file to 42MP.
2. This also eliminates the Bayer filter effects. I'm not sure how much this impacts some or any of the optical characteristics like chromatic aberrations, but I suspect at least part of those issues has to do with the way the...Show more →

Michael,
I curious about #3 as I wrote here:
https://www.fredmiranda.com/forum/topic/1514359/16#14229848

The biggest gains in SNR can be seen with only 4 shots and then more modest gains after 8 shots with 16 shots (2-stop) being the best compromise in terms of number of shots vs reduction of random noise.

I do not see extra resolution from my lenses when averaging shots but I see a huge benefit in shadow noise where I can push my files many stops without seeing a hint of color noise.

Oct 25, 2017 at 08:16 PM

k-h.a.w
Offline
• • • • •
Upload & Sell: Off

p.1 #3 · p.1 #3 · Pixel shift and lens testing thoughts...

How do you average, say in C1Pro?

K-H.

Oct 25, 2017 at 08:45 PM

Fred Miranda
Offline
Admin
Upload & Sell: On

p.1 #4 · p.1 #4 · Pixel shift and lens testing thoughts...

k-h.a.w wrote:
How do you average, say in C1Pro?

K-H.

I don't think it can be done in CaptureOne or Lightroom. Prior to using the Smooth Reflections app, I would bring my images to Photoshop as layers, convert them to a smart object and then averaging the smart object using a Mean or Medium stack.

Oct 25, 2017 at 09:40 PM

k-h.a.w
Offline
• • • • •
Upload & Sell: Off

p.1 #5 · p.1 #5 · Pixel shift and lens testing thoughts...

Fred Miranda wrote:
I don't think it can be done is CaptureOne or Lightroom. Before using the Smooth Reflections app, I would bring my images to Photoshop as layers, convert them to a smart object and then averaging the smart object using a Mean or Medium stack.

Thanks Fred. I have CS6.
Unfortunately it seems the Bridge app got hosed when I installed macOS High Sierra.
Installing CS6 from scratch doesn’t work because of the new file system.

K-H.

Oct 25, 2017 at 09:46 PM

engardeknave
Offline
• •
Upload & Sell: Off

p.1 #6 · p.1 #6 · Pixel shift and lens testing thoughts...

Fred Miranda wrote:
I don't think it can be done in CaptureOne or Lightroom. Prior to using the Smooth Reflections app, I would bring my images to Photoshop as layers, convert them to a smart object and then averaging the smart object using a Mean or Medium stack.

How do the gains in shadow detail compare with a simple HDR merge in Lightroom? (Say three frames, three stops apart.)

Oct 25, 2017 at 09:50 PM

Fred Miranda
Offline
Admin
Upload & Sell: On

p.1 #7 · p.1 #7 · Pixel shift and lens testing thoughts...

engardeknave wrote:
How do the gains in shadow detail compare with a simple HDR merge in Lightroom? (Say three frames, three stops apart.)

Based on the graph above (and I can confirm with my own results), we would have this:

2 shots = 70% (0.5 stop)
4 shots = 50% (1 stop Noise Random reduction)
8 shots = 35% (1.5 stop)
16 shots = 25% (2 stop Noise Random reduction)
32 shots = 18% (2.5 stop)
64 shots = 12.5% (3 stop Noise Random reduction)
128 shots = 9% (3.5 stop)
256 shots = 6.25% (4 stop Noise Random reduction)

So theoretically we would have the following SNR improvement based on "Noise Random reduction stops":

1 stop (4 shots)
2 stop (16 shots)
3 stop (64 shots)
4 stop (256 shots)

--------

Regarding which is better "bracketing" or "averaging"...It depends on your bracket sequence but by looking at the numbers above, you can do the math. However, image averaging does more than significantly improve SNR. (shadow noise)...It also acts as a ND filter and that's why I love it because it basically replaced my ND filters. (Sony's Smooth Reflection was great at this)

From my calculations here are the number of images needed to match the "equiv. ND filters":

1 stop (2 shots)
2 stops (4 shots)
3 stops (8 shots)
4 stops (16 shots)
5 stops (32 shots)
6 stops (64 shots)
7 stops (128 shots)
8 stops (256 shots)

Oct 25, 2017 at 09:57 PM

Fred Miranda
Offline
Admin
Upload & Sell: On

p.1 #8 · p.1 #8 · Pixel shift and lens testing thoughts...

In practical terms, I loved using the SR app because it did image averaging in-camera. So, I could expose my landscape scene to the highlights and take a number of images from 4 to 64 shots, depending on the scene DR needs.
Then in post, I would recover the shadows and mid-tones up to 3-stops and they would still be clean! As a bonus I would have a ND effect to the sky or water without adding filters to my lenses. I will definitely miss this app on the A7RIII.

Oct 25, 2017 at 10:04 PM

engardeknave
Offline
• •
Upload & Sell: Off

p.1 #9 · p.1 #9 · Pixel shift and lens testing thoughts...

Fred Miranda wrote:
Based on the graph above (and I can confirm with my own results), we would have this:

2 shots = 70% (0.5 stop)
4 shots = 50% (1 stop Noise Random reduction)
8 shots = 35% (1.5 stop)
16 shots = 25% (2 stop Noise Random reduction)
32 shots = 18% (2.5 stop)
64 shots = 12.5% (3 stop Noise Random reduction)
128 shots = 9% (3.5 stop)
256 shots = 6.25% (4 stop Noise Random reduction)

So theoretically we would have the following SNR improvement based on "Noise Random reduction stops":

1 stop (4 shots)
2 stop (16 shots)
3 stop (64 shots)
4 stop (256 shots)

--------

Regarding which is better "bracketing"...Show more →

Thanks for explaining that. Yes, I've tried averaging once or twice. It was a really long process that didn't yield worthwhile results for my application (interiors). Exposure fusion seems better for what I'm doing. You know, I think I did it with raw files, which is probably a bad idea. Might have to try with jpg and maybe electronic shutter.

Oct 25, 2017 at 10:08 PM

kwalsh
Offline
• • • •
Upload & Sell: Off

p.1 #10 · p.1 #10 · Pixel shift and lens testing thoughts...

I wrote a small C program that takes multiple RAW exposures and averages them so you can do the equivalent of Smooth Reflections outside the camera. In general if you are primarily interested in reducing shadow noise a HDR stack is better and a more efficient workflow assuming you don't have Smooth Reflections in your camera.

As Fred says if you want an ND effect then stacking/averaging is what you want. If you want true ND use mean (moving people in the scene become blurred ghosts). If you want moving objects to dissappear (remove people from a tourist spot) take lots of exposures and use median.

In *theory* averaging will reduce noise in the highlights too while HDR only helps the shadows. In practice I've found HDR to be equivalent as it is really shadows and midtones that need noise help and HDR does that just fine.

Lastly I can't understand why all camera manufacturers don't have a "synthetic" super low ISO option that averages multiple exposures using rolling shutter. It is trivial to do and is just the kind of "gee-whiz" feature that would be easy to market. Instead we have Sony removing the app that does this!

On the lens testing based on the Olympus implementation yes you really see lens warts with image shifting - especially in the corners of course. Be aware though that it does nothing to improve slant edge MTF testing. Slant edge MTF tests already use a super resolution technique which is why they report values above Nyquist. Pixel shift adds no new information to a slant edge test. But for visual comparisons it really does make previously acceptable lenses look kind of sad.

Oct 25, 2017 at 10:27 PM

Fred Miranda
Offline
Admin
Upload & Sell: On

p.1 #11 · p.1 #11 · Pixel shift and lens testing thoughts...

kwalsh wrote:
I wrote a small C program that takes multiple RAW exposures and averages them so you can do the equivalent of Smooth Reflections outside the camera. In general if you are primarily interested in reducing shadow noise a HDR stack is better and a more efficient workflow assuming you don't have Smooth Reflections in your camera.

As Fred says if you want an ND effect then stacking/averaging is what you want. If you want true ND use mean (moving people in the scene become blurred ghosts). If you want moving objects to dissappear (remove people from a tourist spot) take...Show more →

Great post! Yes, "mean" averaging for the ND effect and "median" for making objects in movement disappear.

I would love to test that app. Is it Mac compatible? I really need to find a way to replace smooth reflections and going through Photoshop takes just too much time.

It's not very known by some Canon bodies offer a 'multiple exposure' mode (up to 9 shots). One of the options was doing 'mean averaging' in camera. I'm not sure if their current bodies still offer this option though.

Oct 25, 2017 at 10:52 PM

engardeknave
Offline
• •
Upload & Sell: Off

p.1 #12 · p.1 #12 · Pixel shift and lens testing thoughts...

kwalsh wrote:
I wrote a small C program that takes multiple RAW exposures and averages them so you can do the equivalent of Smooth Reflections outside the camera. In general if you are primarily interested in reducing shadow noise a HDR stack is better and a more efficient workflow assuming you don't have Smooth Reflections in your camera.

As Fred says if you want an ND effect then stacking/averaging is what you want. If you want true ND use mean (moving people in the scene become blurred ghosts). If you want moving objects to dissappear (remove people from a tourist spot) take...Show more →

My understanding is that averaging would also yield the equivalent of an extremely low iso image. I'm always looking for any schemes that can make my work (https://imgur.com/a/Tl3oP) pop a little more. Interiors are particularly amenable to many-frame compositions blended together through various alchemic processes. I have an idea to throw in a pixel-shift frame. Probably not worth it though.

Re: lens testing, I'm never going to say I don't want more data about lenses I'm considering buying. But it's hard to excited about yet more sharpness data. There's so little difference between modern high-end lenses, it's splitting hairs.

Oct 25, 2017 at 10:56 PM

mjm6
Offline
• • •
Upload & Sell: Off

p.1 #13 · p.1 #13 · Pixel shift and lens testing thoughts...

Fred and everyone,

Yes, that old discussion about SR was what got me to add the #3 thought in regarding the sharpness differences. There is obviously a big difference between a non-shifted image stack and a shifted image stack, but it would be possible to actually run some NR on the final image by using the Bayered frames, and then go in and pull the raw pixel info to produce another image with that are then combined to produce this ultra high resolution and noise reduced file. That's exactly what I was thinking that Sony may do in the external program.

If this pixel shift technology really works well, it's going to lay bare the performance of the various lenses in a way that is going to make so many things really, really critical (optical quality, choosing the best aperture, focus point, shooting technique, etc.) to achieve the optimal performance out of the system. Forget about IBIS, and use a massive tripod. Use lots of dampening and the electronic shutter, etc...

I remember that there was discussion about 5-10 years ago that the majority of 35mm lenses will be out resolved by a camera at about 50MP. That may have been at a 50MP image with a real 50MP resolving capability, not the much lower resolving capability of a Bayer filtered 50MP image. If that is the case, we will see it happen with this pixel shifting probably, or at least come close to it at 42MP.

The easiest way to see if that starts to happen is to look for moire in the images, and see how close to the center you can get moire to occur. If you can't get it into the center, you are out-resolving the lens, even at it's best. In the corners, I'm sure most lenses wont even come close.

Go out and find some taffeta or other fabric with fine stitching and use that to drive the camera mad with moire. At 42MP, it will take some pretty small patterns. Kick in the pixel shift and see what happens...

I don't believe for a moment that the newest lenses are substantially sharper than lenses from 10 years ago, so I think it is probably reasonable to consider that as a reasonable upper limit still.

---Michael

Oct 26, 2017 at 01:31 AM

kwalsh
Offline
• • • •
Upload & Sell: Off

p.1 #14 · p.1 #14 · Pixel shift and lens testing thoughts...

Fred Miranda wrote:
I would love to test that app. Is it Mac compatible? I really need to find a way to replace smooth reflections and going through Photoshop takes just too much time.

I just put the source code and a Mac binary on Dropbox:

https://www.dropbox.com/sh/b83mrs729rm53z7/AACmVdrOcz3FCE608miERe4Aa?dl=0

It is a command line utility so you should be able to just run dngavg from the terminal. With no options it will give a quick help line. The key point to this code and why it is so simple is that it works only on *uncompressed* DNG files. So steps would be:

- Export the files from LR as *uncompressed* DNG
- Run dngavg with the name for the merged file first followed by the names of all the files you want to merge
- Import the merged DNG back into LR

In theory for someone with the time and competency it would be a small matter to have a plug-in that did the above steps so that you didn't need to leave LR and do all the manual crap. I looked at it briefly but there wasn't a trivial solution from existing plugin templates to select multiple files and then re-import a single output file. This doesn't seem like rocket science since it is identical to an HDR merge plugin workflow but I just couldn't find a good starting point in my very brief search.

There is a little shell script in the same dropbox folder called dnguncomp which calls the DNG converter with the appropriate parameters to generate uncompressed DNG in case LR export isn't letting you do that or someone wants to make a more automated process.

The source code is in there as well (dngavg.cpp) and it has *zero* dependencies on any libraries - it does all the TIFF/DNG parsing on its own. So it would be trivial for folks to build on their system. If you've got gcc or the equivalent installed you are good to compile.

As a side note I completely avoided the DNG SDK from Adobe because it looks like and based on other's experience is the kind of software development abomination I have no desire to get near. Thus I went the other software abomination route that dcraw takes and made a single C source file that reinvents the wheel where it needs to and eschews any libraries!

Anyway - give it a try if you'd like. Let me know if you have any issues. Maybe if someone like you finds it useful and people that are hooked on Smooth Reflections get an A7RIII we'd get enough notice and interest for one of the LR plugin authors like Friedl to bang out a plugin to do this. I expect someone like that could bang out a plugin that did export/call dngavg/reimport in their sleep.

It's not very known by some Canon bodies offer a 'multiple exposure' mode (up to 9 shots). One of the options was doing 'mean averaging' in camera. I'm not sure if their current bodies still offer this option though.

Olympus allows you to combine RAW files in camera post exposure, but it only allows you to do two or three at a time. But you can then combine those as multiple exposures as well. So you can in theory do an arbitrary number. This is unfortunately pretty fussy to do on the camera interface and the ORF format is 12-bit so pretty quickly you'll be limited by the ORF format making the exercise pointless as far as noise goes - but still get a nice ND effect.

Oct 26, 2017 at 08:16 AM

znikon5
Offline
• •
Upload & Sell: On

p.1 #15 · p.1 #15 · Pixel shift and lens testing thoughts...

With the Micro 4/3 models, the pixel shifting high-resolution modes definitely result in a higher resolution image. I am wondering how this is achieved? Will removing the Bayer organization to one where you have RGB represented separately at each pixel result in higher resolution as well for the Sony A7Riii? What is the effective megapixel output from a pixel-shifted Sony image? Are there any publicly available images of a pixel-shifted image from Sony? Will this rival medium format for landscape work with still objects?

Here is a video URL showing the Olympus pixel shift in action and how you get increased resolution: https://youtu.be/toYpuK7WaTY

Oct 26, 2017 at 09:46 AM

jhinkey
Offline
• • • • • •
Upload & Sell: On

p.1 #16 · p.1 #16 · Pixel shift and lens testing thoughts...

Now that full electronic shutter is 10 fps and 14 bit, I can potentially fire off 4 frames over 0.4 seconds and average them in post if my subject is not moving much or at all.

What would be cool is if there is something moving in the frame during the 4 exposures that software could average the parts of the scene that don't move and intelligently use the parts of the image that are moving - kind of stitching + averaging.

Oct 26, 2017 at 11:31 AM

mjm6
Offline
• • •
Upload & Sell: Off

p.1 #17 · p.1 #17 · Pixel shift and lens testing thoughts...

znikon5 wrote:
With the Micro 4/3 models, the pixel shifting high-resolution modes definitely result in a higher resolution image. I am wondering how this is achieved? Will removing the Bayer organization to one where you have RGB represented separately at each pixel result in higher resolution as well for the Sony A7Riii? What is the effective megapixel output from a pixel-shifted Sony image? Are there any publicly available images of a pixel-shifted image from Sony? Will this rival medium format for landscape work with still objects?

Here is a video URL showing the Olympus pixel shift in action and how you get increased...Show more →

In response to this list of questions, I'll try to answer them:

1. This is done exactly the same way as the Olympus method, but it does not do the half-pixel option (I think Olympus uses 8 shots for that). It shifts the sensor over by the effective distance of a single pixel and takes another capture. Then it shifts down and takes another, then back and another, for a total of four captures.

Then, the individual raw photosite locations that are the uninterpolated (actual readings) from the RAW files are combined to produce the RGB values at each location essentially without any interpolation of the file. It's a bit of data wizardry that works, and is creating "real" improved resolution.

2. Yes, higher resolution will happen as long as Sony is able to move the sensor accurately enough for each of the shifts to do this, and assuming the optical path (through the cover glass and the on-chip fresnel lenses) isn't limiting the performance in some manner.

3. A pre-shifted a7RII is 42MP interpolated, but the R and B channels each only have about 10.5MP of real data, and the G channel has 21MP of real data. So the answer is somewhere between those values, depending on the color channel you are looking at and the colors represented in the final image.

4. The effective output of a shifted pixel image will be 42MP of real data in each channel. Assuming they are able to execute this to match the theoretical capabilities.

5. I haven't seen many images yet, but look at this video from Sony for a few examples:

https://www.youtube.com/watch?time_continue=121&v=6-811lVi0Qc

6. I think it will likely not rival MF because the lens optics will be out resolved (the limitation on the image will be the lenses, not the sensor away from the center).

Also, the larger photosites in a MF sensor will offer other benefits, like lower noise. Lastly, sone MF sensors are 16 bit, and will probably have a bit better characteristics in other ways as well.

Oct 26, 2017 at 11:41 AM

Jonas B
Offline
• • • •
Upload & Sell: Off

p.1 #18 · p.1 #18 · Pixel shift and lens testing thoughts...

I owned the Olympus OM-D EM-1 Mk II for a while.
The pixel-shift tech worked great. One, or two, benefits over the new Sony solution is that the camera processes the images and delivers one big ORF file, and that the Olympus is takes the images fast when compared.

I played with that a little. If you are interested in getting an idea about what the differences between this Hi-Res images and the standard images look like you can go here.

The images were taken with the Olympus 17mm lens which is a medium performer. I took other images with the Zuiko D. 25/1.2 as well. The 25mm is a better lens than the 17mm and the results improved further.

Oct 26, 2017 at 11:43 AM

mjm6
Offline
• • •
Upload & Sell: Off

p.1 #19 · p.1 #19 · Pixel shift and lens testing thoughts...

Here is a link to a sample video from Sony.

https://www.youtube.com/watch?time_continue=121&v=6-811lVi0Qc

What I said about the moire a few posts above is apparent on one of the sample photos they show in the video.

If moire disappears, the sensor is out resolving the optical system, and that appears to be the case in that sample.

Oct 26, 2017 at 11:44 AM

Jonas B
Offline
• • • •
Upload & Sell: Off

p.1 #20 · p.1 #20 · Pixel shift and lens testing thoughts...

mjm6 wrote:
(...)
6. I think it will likely not rival MF because the lens optics will be out resolved (the limitation on the image will be the lenses, not the sensor away from the center).

Also, the larger photosites in a MF sensor will offer other benefits, like lower noise. Lastly, sone MF sensors are 16 bit, and will probably have a bit better characteristics in other ways as well.

It's too early to tell of course but you might be in for a surprise. See how the Olympus High-Res images often are cleaner and sharper than the A7R MkII images. (That I say as an A7R MkII owner and a former Olympus owner.

Oct 26, 2017 at 11:46 AM

1 2 3 … 15 16 end
FM Forums \| Sony Forum \| Join Upload & Sell 1 2 3 … 15 16 end

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.