Can someone explain what the DxOMark scores expressed as Megapixels mean? When I was looking at the current FE lenses reviewed on the website I noticed that very few were right up near the 42 MP sensor resolution of the A7r II (the highest MP Sony camera I could choose). The FE 90 2.8 macro could resolve the full 42 MP.
When translating these numbers to the A7r iv, what would happen? I assume it would be relative to the sensor, but does a sharpness of 36MP on a 42MP sensor translate to (36/42) x 61, an absolute value of 36 MP (I assume not), or something in between for the A7r iv?
I realize that we all have the lenses we like to use and will continue to use them with the newer cameras that are available, but I wonder if there’s a point where the lens is the limiting factor and not the sensor.
The DxOMark Lens Score is a linear scale related to the largest print size that provides excellent image quality. Doubling the size of the print requires doubling the DxOMark score. A score difference of less than 10% can be considered irrelevant.
The DxOMark Lens Score is an open scale, limited by the lens and camera resolution and by sensor noise. As we can expect these to improve over time, the maximum DxOMark score is bound to increase with technological development.
You can read more about why we base our testing on the RAW image format here. But now let’s have a closer look at the setups and methodologies for the individual test criteria and how the sub-scores that feed into the final score are computed.
MTF and sharpness
The Modulation Transfer Function (MTF) of a camera (body and lens) is measured in line with the ISO 12233 standard SFR method (see MTF measurement definition). The target is a pattern of white and black squares tilted at a 5° angle and filling the field of the camera. DxO Labs designed the target and produced it using a high-resolution printer to achieve sharp transitions between black and white areas without aliasing. The target is attached to a frame made with aluminum profiles to provide the necessary rigidity to the target assembly.
The target is evenly illuminated with halogen lights that are filtered to provide a daylight color temperature of 5500K.
To guarantee absolute stability and prevent any motion blur, the camera is mounted on a geared tripod head that is fixed to a heavy-duty studio stand. A graduated rail on ball bearings permits very accurate adjustment of the distance between camera and target. To minimize vibrations, we use the reflex mirror lockup function when available and we release the shutter with a remote control or the self-timer. Before shooting, we ensure that the camera sensor and target planes are parallel by using a mirror placed flush against the target. Perfect alignment is achieved when the reflected image of the lens appears at the center of the camera viewfinder.
The reflected image of the lens in the center of the camera viewfinder indicates perfect alignment.
We select the lowest actual ISO speed of the camera to acquire images with a minimum level of noise. We set the exposure so that the white squares of the target are just below sensor saturation in RAW format, to ensure that the entire dynamic of the sensor is used. Of course, we deactivate all sharpening options and stabilizing systems of the camera or lens. For each focal length and aperture of the lens, we take pictures at 60 different focusing positions around the focusing point set by the camera’s autofocus system. We then use the sharpest image to measure the camera’s MTF.
We use these results to generate the DxOMark resolution score. The score represents the sharpness performance of a lens-camera combination averaged over its entire focal length and aperture ranges and is computed as follows:
For each focal length and each f-number sharpness is computed and weighted across the image field, with the corners being less critical than the image center. This results in a number for each focal length / aperture combination. We then choose the maximum sharpness value from the aperture range for each focal length. These values are then averaged across all focal lengths to obtain the DxOMark resolution score that is reported in P-MPix (Perceptual Megapixels).
It’s worth noting that for lenses with a wide zoom range the differences between sharpness at different focal lengths can be quite significant. For most lenses, sharpness in P-Mpix is typically between 50% and 100% of the sensor pixel count, and differences smaller than 1 P-MPix are usually not noticeable. The best resolutions are usually achieved by prime lenses at apertures between f/2.8 and f/8.
----------------------------
Not sure either, this is from their website. I did use them since they did have relative sharpness tests (based on the same sensors). Practically speaking, I think for lenses, I would just pixel peep the finest detail you actually need in your work and see if it's resolving it. Lenses being close enough to eachother sharpness wise, other qualities may be more important.
It's come up before, my take away was that they don't really mean much in terms of mpix and their measurements may be sometimes wrong to boot. Single copy and all to boot afaik.
I think they have an A7II and A7RII in their DB. You could compare the sharpness field maps between 24mpx and 42mpx, if you see substantial improvement throughout the frame (not just center, best ignore the mpix number and go to details/measurements), then the lens is probably also up to the A7rIV.
Also when looking at other sites, looking at MTFs (or mtf/50 center/mid/edge) on a 42mpx sensor, if the lens gets nearly identical numbers stopped down 1-2 stops almost across the frame it usually has potential - if the edges never quite catch up, it's probably not going to cut it on higher res sensor.
Problems off the top of my head with DXO are that afair it's center weighted (so eg the FE 28/2 scores better than I'd personally rate it) and calculated at the best aperture, and afair shooting a target at less than infinity.
Also, I remember a thread finding _huge_ discrepancies between the optically same ZE/ZF and Milvus lenses on the same camera, many of them the same optical formula with maybe better coatings and new shell. I suspect they didn't properly manually focus for a while or maybe didn't refocus at every aperture at one point. Just makes me wonder...
Personally, I find infinity shots and crops here on FM and close target/infinity shots and comparison with other lenses much more useful even though you don't get "one number"/score. Guess there's also lensrentals MTFs.
I find the DxOMark reviews and scores pretty much worthless.
Fred Miranda, Lensrentals, Philipp Reeve, Christopher Frost, Dustin Abbott - these are the dudes to look for when you want to know something useful about lenses.
In short: I don't think the score is a helpful tool to learn about a lens' sharpness. Quite the opposite actually: It can lead to confusion instead of clarifying things.
As you see the 2.8/90 is a pretty sharp lens but at f/2.8 it isn't as sharp as at f/4 and the corners are softer than the center. And that is at 24MP. It obviously doesn't make full use of the sensor at any aperture. If I was awarding that lens a sharpness number (which I don't do for good reason) it wouldn't be 100% of the theoretical maximum. But since DxO doesn't really tell us how they cook up the number we don't know how they get to the number.
The there are other factors which have an effect on sharpness:
They are an average of 10 copies and they also tell us that the lens has very strong copy-to-copy variation.
They give information on different "defintions of sharpness" (I am simplifying a lot here). These different definitions can all be very relevant, depending on application. The more you read up on the topic the more complicated it gets.
DxO is very simple: they imply that they can measure the resolution of a lens like that of a sensor where you just need to count pixels. They can't. All they do is
to measure different things (we don't know what they actually measure) and do some calculations with their results (again: we don't know what kind of calculations) to come up with a number they call perceptive Megapixels. They do that because people have bought cameras depending on megapixel numbers for decades.
Buying a camera based on its megapixel count has never been a good idea. Buying a lens based on DxO's MP score is an even worse idea. There are much better tools than the DxO Score.
I agree with everything that Phillip says here, with the caveat that DXO data, if you ignore the headlines, can be useful if you know what you are looking for.
The provide sharpness data across the field, and at different apertures. This is hard to find but can be useful just so long as you:
(1) Only compare data with other lenses tested on the same camera
(2) Realize that this data is from a chart distance, likely 4 metres or so but AFAIK they don't tell you.
Sometimes this is useful just because performance does vary at different distances. (The Sony 2.8/90 for example, while fine at longer distances, is superb in the macro range it was designed for - not that DXO can tell you that)
The biggest problem with DXO data (as opposed to the useless headline numbers) is that they are from one copy, but that sadly is true of just about all reviews including my own - the only exception not really being a review, but the lens rental infinity MTF data which is the average of ten copies.
Phillip Reeve wrote:
In short: I don't think the score is a helpful tool to learn about a lens' sharpness. Quite the opposite actually: It can lead to confusion instead of clarifying things.
As you see the 2.8/90 is a pretty sharp lens but at f/2.8 it isn't as sharp as at f/4 and the corners are softer than the center. And that is at 24MP. It obviously doesn't make full use of the sensor at any aperture. If I was awarding that lens a sharpness number (which I don't do for good reason) it wouldn't be 100% of the theoretical maximum. But since DxO doesn't really tell us how they cook up the number we don't know how they get to the number.
The there are other factors which have an effect on sharpness:
They are an average of 10 copies and they also tell us that the lens has very strong copy-to-copy variation.
They give information on different "defintions of sharpness" (I am simplifying a lot here). These different definitions can all be very relevant, depending on application. The more you read up on the topic the more complicated it gets.
DxO is very simple: they imply that they can measure the resolution of a lens like that of a sensor where you just need to count pixels. They can't. All they do is
to measure different things (we don't know what they actually measure) and do some calculations with their results (again: we don't know what kind of calculations) to come up with a number they call perceptive Megapixels. They do that because people have bought cameras depending on megapixel numbers for decades.
Buying a camera based on its megapixel count has never been a good idea. Buying a lens based on DxO's MP score is an even worse idea. There are much better tools than DxO.
