Register · Software · Search · Image Upload · Buy & Sell · Reviews · Hosting

	Moderated by: guardian
	Username   Password Visit the FM Store · Image Upload · Buy & Sell FM Forum Rules Nikon SLRs, primes, and zooms lenses reviews FM Forums | Nikon-mount SLRs | Join Image Upload end

Roel Online Buy and Sell: On p.1 #1 · Compressed NEFs - D2x Hi there I have been doing some (basic) testing on compressed NEFs on my D2x... there is "supposed" to be some loss of quality compared to uncompressed NEFs, however, after converting in NC, my eye cannot see a difference between the two. Has anyone else found this to be true? Compressed NEFs are only 50% the filesize of uncompressed ones... therefore, the space savings are huge. Looking forward to hearing from you! Edited on Sep 30, 2005 at 06:57 PM Sep 29, 2005 at 09:10 PM

rhyder Offline Image Upload: Off p.1 #2 · Compressed NEFs - D2x I've never heard of a loss of quality. Where did you hear this? Jacko will know!! Edited on Sep 30, 2005 at 06:57 PM Sep 29, 2005 at 09:39 PM

Jack OBrien Offline Image Upload: Off p.1 #3 · Compressed NEFs - D2x I think it might be more accurate to say that there's "not supposed" to be any loss. Nikon's terminolgy from the D series camera manuals is "virtually lossless". Some have asserted that there is some loss of highlights using compressed nefs, but no one has shown me any substantiating data. I have done comparison shots, and everything seems equal when converted from raw, and all the rgb values are identical. There is no evidence of any kind of compression artifacts, loss of color, pixelation, or anything. If someone has further knowledge on this, I'd like to hear it, but I can't find anything. FWIW, I shoot compressed raw 100% of the time, and there's just not any difference. Jack Edited on Sep 30, 2005 at 06:57 PM Sep 29, 2005 at 10:47 PM

John DeMott Offline Image Upload: Off p.1 #4 · Compressed NEFs - D2x The compressed NEF file does lose a little data--the amount of loss is greater in the brighter parts of the image. The result is that highlights in a compressed NEF file will have less data (more gaps between adjacent values) than would be present in a true 12 bit file. It is unlikely, but still possible, that the difference would be visually apparent. Most likely the difference would be apparent if you made extreme adjustments in Curves or Levels in the highlights--you might then see some posterization. I have seen examples demonstrating the difference between 12 bit and 8 bit files (where 8 bit files exhibit posterization) but I haven't seen anything similar for compressed NEFs and I haven't done any experiments myself. I would expect posterization to be significantly less pronounced with compressed NEFs than with 8 bit files. Edited on Sep 30, 2005 at 06:57 PM Sep 29, 2005 at 11:25 PM

Jack OBrien Offline Image Upload: Off p.1 #5 · Compressed NEFs - D2x John DeMott wrote: The compressed NEF file does lose a little data--the amount of loss is greater in the brighter parts of the image. The result is that highlights in a compressed NEF file will have less data (more gaps between adjacent values) than would be present in a true 12 bit file. How is it not a 'true' 8 bit file, just because there's compression? Where is it documented that data loss is in the brighter parts of the image? Why not the darker parts? John, I'm not trying to be a smart-aleck, but I can't find any technical documentation to substantiate these types of claims. I too have seen the comparisons between the different bit-depth files, but that comparison can't hold true for compressed/uncompressed. Maybe when I get the time, I can shoot some more tests, but the last round I did, I just can't find any differences. Anyway, not to belabor the point, I really would just like to know. Kindest Regards, Jack Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 01:23 AM

John DeMott Offline Image Upload: Off p.1 #6 · Compressed NEFs - D2x My source on the compression methodology is Thom Hogan's book, Complete Guide to the Nikon D2X, which has a lengthy discussion of the NEF compression method, that I merely attempted to summarize (I believe accurately). BTW, I didn't say that the file is not a 12 bit file, I said it has less data than a true 12 bit file, i.e., one that had not been compressed. Thom Hogan's book doesn't say, but I assume the reason the compression takes place in the highlights, rather than the shadows, is because RAW data are linear, unlike tiff or jpeg data, meaning that there are many more RAW data points in the highlights than in the shadows. Hence the advice to "expose to the right" with RAW files because (unless the highlights are blown) it is much better to correct a RAW file by bringing down the highlights than trying to bring up the shadows. Given that there are more data points in the highlights, if one were to chooose to discard any data points through compression, they would first be in the highlights not the shadows. As I said, I haven't done any tests either and I don't claim to have seen any differences. In fact, based on the little I know, I think it is unlikely to see any differences except under extreme circumstances. I was just trying to answer the original question. John Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 03:26 AM

jmcfadden Online Image Upload: On p.1 #7 · Compressed NEFs - D2x The last stop of capture contains more data than all the other stops COMBINED. Think of it in musical terms, one octave is a doubling of the hertz in the octave below it. IOW go from 20hz which is the lower limit of human hearing to 40hz and you have gone One octave. and you go one octave from 10,000hz (10Khz) to 20Khz ! , the first octave contains only 20hertz, and in the last octave contains 10,000hertz! This is the same principle as far as light and our cameras as well. Get as much captured to the right as possible , this is the fundamental rule of digital to maintain the highest signal to noise ratio. Since the darker regions have Less data per stop any lifting of the EV in post is acting on a part of the exposure which has the lowest signal and the highest noise J Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 03:44 AM

turnert Online Image Upload: Off p.1 #8 · Compressed NEFs - D2x jmcfadden wrote: and you go one octave from 10,000hz (10Khz) to 20Khz ! , the first octave contains only 20hertz, and in the last octave contains 10,000hertz! This is the same principle as far as light and our cameras as well. Get as much captured to the right as possible , this is the fundamental rule of digital to maintain the highest signal to noise ratio. J That's the coolest factoid I've learned all week. Thanks John. Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 04:49 AM

paulbaarn Offline Image Upload: Off p.1 #9 · Compressed NEFs - D2x That's all very well, theoretically, but when you have a scene with a contrast that exceeds the camera's dynamic range, the choice to expose to the right is not always there. At least if blown out highlights go against your religion. I accept blown out highlights if it means my subject is correctly lit. I find that viewers who are not photographers, don't even realize that there is something like blown out highlights, also because they are bombarded with them in magazines. When a scene can be captured within the dynamic range of the camera, do you deliberately overexpose it (withouth blowing highlights), and bring it down again in the RAW conversion? And do you do one conversion or combine several conversions of the same file to expand the dynamic range? At the moment I expose for how I want the image to look without EV, but I would be interested to see if I can improve my results and how. Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 05:01 AM

jmcfadden Online Image Upload: On p.1 #10 · Compressed NEFs - D2x paulbaarn wrote: That's all very well, theoretically, but when you have a scene with a contrast that exceeds the camera's dynamic range, the choice to expose to the right is not always there. At least if blown out highlights go against your religion. I accept blown out highlights if it means my subject is correctly lit. I find that viewers who are not photographers, don't even realize that there is something like blown out highlights, also because they are bombarded with them in magazines. When a scene can be captured within the dynamic range of the camera, do you deliberately overexpose it (withouth blowing highlights), and bring it down again in the RAW conversion? And do you do one conversion or combine several conversions of the same file to expand the dynamic range? At the moment I expose for how I want the image to look without EV, but I would be interested to see if I can improve my results and how. IT is more about "seeing" the end from the beginning and understanding the format (digital in this case) and how to leverage the strength of the format to achieve a visual end. So to answer you , there is no right answer or magic bullet for this stuff, sometimes I decide to let them go (highlights) and other times I preserve them at all costs and process for the shadows later. The ball is in my court from the beginning Sometimes I do overexpose intentionally and then just leave it that way , I see this stuff in the magazines like you Everyday. Make art, make someone dream, move someone , move yourself , that is my motto J Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 05:08 AM

jmcfadden Online Image Upload: On p.1 #11 · Compressed NEFs - D2x turnert wrote: jmcfadden wrote: and you go one octave from 10,000hz (10Khz) to 20Khz ! , the first octave contains only 20hertz, and in the last octave contains 10,000hertz! This is the same principle as far as light and our cameras as well. Get as much captured to the right as possible , this is the fundamental rule of digital to maintain the highest signal to noise ratio. J That's the coolest factoid I've learned all week. Thanks John. da nada J Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 05:28 AM

Fundy Offline Image Upload: Off p.1 #12 · Compressed NEFs - D2x Nikon says there is no visible loss in quality. To me that means that there is a loss but the human eye cannot see it. Good enough, I'll take the space savings and shoot compressed. The D70 only shoots compressed NEFs and the quality is great in my opinion. Andrew Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 05:44 AM

rhyder Offline Image Upload: Off p.1 #13 · Compressed NEFs - D2x John DeMott wrote: My source on the compression methodology is Thom Hogan's book, Complete Guide to the Nikon D2X, which has a lengthy discussion of the NEF compression method, that I merely attempted to summarize (I believe accurately). BTW, I didn't say that the file is not a 12 bit file, I said it has less data than a true 12 bit file, i.e., one that had not been compressed. Thom Hogan's book doesn't say, but I assume the reason the compression takes place in the highlights, rather than the shadows, is because RAW data are linear, unlike tiff or jpeg data, meaning that there are many more RAW data points in the highlights than in the shadows. Hence the advice to "expose to the right" with RAW files because (unless the highlights are blown) it is much better to correct a RAW file by bringing down the highlights than trying to bring up the shadows. Given that there are more data points in the highlights, if one were to chooose to discard any data points through compression, they would first be in the highlights not the shadows. As I said, I haven't done any tests either and I don't claim to have seen any differences. In fact, based on the little I know, I think it is unlikely to see any differences except under extreme circumstances. I was just trying to answer the original question. John Help me out here, I believe I read somewhere that raw data is linear as compared to film, not tiff or jpeg. Tiff and jpeg are file formats and have no inherent gamma. Raw data is just that raw data. It is not a file format. NEF is a file format that contains raw data. Tiff and jpeg are file formats that contain modified data. As far as exposing to the "right", you should expose for the scene as you normaly would, any pushing to the right should be done very minimally. I can fix a dark shadow, there is very little you can do for a blown highlight. Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 05:48 AM

witwald Offline Image Upload: Off p.1 #14 · Compressed NEFs - D2x Roel wrote: I have been doing some (basic) testing on compressed NEFs on my D2x... there is "supposed" to be some loss of quality compared to uncompressed NEFs, however, after converting in NC, my eye cannot see a difference between the two. Has anyone else found this to be true? Nikon say that their compressed NEFs (cNEFs) are "visually" lossless. This ties in with your own experience where you are unable to see a difference between NEFs and cNEFs. If that were not the case, I doubt that Nikon would have used cNEFs. Thom Hogan has this to say about cNEFs in his Nikon D70 review: The Compressed NEF format is the only one that comes close to retaining the full data set the D70 is capable of acquiring. I say "close to retaining" because the compression scheme Nikon uses is not lossless. Basically, the camera takes the highlight data and places them into groups (essentially a rounding of many of the data points), producing the equivalent to somewhere between 9 and 10 bits of data. When this is returned to 12-bit form, there's a bit of posterization in the highlight data. The reason this works as a visually lossless scheme is that our eyes really can't resolve more than about an 8-bit value can produce (and our eyes aren't linear in response to light, either). For the most part it isn't a big thing that the compression loses data, though there may be some post-processing manipulations that will render slightly differently because of the data rounding. I have worked with Dave Coffin's dcraw.c program, which is able to read and process compressed and uncompressed NEF files. In cNEFs, there is a lookup table of 683 elements that is used to index the amplitude values obtained from the sensor. This compares to the 4096 values that normally cover the range of a 12-bit analog-to-digital converter. Hence, it seems that we are getting an effective resolution of around 9.5 bits. Interestingly, the first 216 levels in the lookup table correspond linearly to the sensor levels on a one-to-one basis (e.g. curve<0>=0, curve<128>=128, curve<215>=215). Beyond that point, there is a parabolic-like relationship (example points are: curve<256>=263, curve<512>=1951, curve<682>=4095). This means that data associated with darker sections of the image is stored at full resolution, while brighter sections of the image are stored with progressively larger steps between levels. This approach helps to maintain as high a tonal resolution as possible in the shadows of the image. Further information on the exact nature of the lossy compression scheme used in cNEFs is available here. In particular, take a look at the transfer curve used by a cNEF. There you can clearly see the linear part of the curve for the shadow data, and the parabola-like curve for the brighter image data. Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 07:06 AM

Jack OBrien Offline Image Upload: Off p.1 #15 · Compressed NEFs - D2x John DeMott wrote: My source on the compression methodology is Thom Hogan's book, Complete Guide to the Nikon D2X, which has a lengthy discussion of the NEF compression method, that I merely attempted to summarize (I believe accurately). BTW, I didn't say that the file is not a 12 bit file, I said it has less data than a true 12 bit file, i.e., one that had not been compressed. Thom Hogan's book doesn't say, but I assume the reason the compression takes place in the highlights, rather than the shadows, is because RAW data are linear, unlike tiff or jpeg data, meaning that there are many more RAW data points in the highlights than in the shadows. Hence the advice to "expose to the right" with RAW files because (unless the highlights are blown) it is much better to correct a RAW file by bringing down the highlights than trying to bring up the shadows. Given that there are more data points in the highlights, if one were to chooose to discard any data points through compression, they would first be in the highlights not the shadows. As I said, I haven't done any tests either and I don't claim to have seen any differences. In fact, based on the little I know, I think it is unlikely to see any differences except under extreme circumstances. I was just trying to answer the original question. John Thanks John, appreciate your response. As to answering the original question, you did well and I wasn't trying to take away from that, just clarity for my numb brain. Jack Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 11:58 AM

John DeMott Offline Image Upload: Off p.1 #16 · Compressed NEFs - D2x An article about the linear nature of RAW data and why that suggests we should "expose to the right" can be found here. Of course, you don't want to over-expose to the point of blowing out the highlights, but if you have latitude on exposure, you will give yourself more flexibility in post processing if you expose to the right. Whether that is worth doing for you is a personal choice. When a RAW file is converted, it is typically output as a jpeg or tiff file with the brightness values for each channel normalized so that each change of one stop in brightness more nearly corresponds to a change in an equal number of levels as expressed in the file data. These are the sort of jpeg and tiff files I was referring to in a shorthand way. Rhyder is correct that jpeg and tiff are file formats, not gamma curves or color spaces, so I suppose one could have a tiff file with linear data, e.g., the output from a RAW file that had been de-mosaiced but not normalized. I haven't seen such a file and I was simply referring to the usual sorts of files we all work with. Edited on Sep 30, 2005 at 06:57 PM Sep 30, 2005 at 03:34 PM

burner Offline Image Upload: Off p.1 #17 · Compressed NEFs - D2x Another question regarding compressed NEF's. Does anyone who has experience with both compressed and uncompressed know if it takes any longer for Capture to open up a compressed file? I'm just wondering at what point does the uncompressing take place, and how much time does it require? Sep 30, 2005 at 06:57 PM

	FM Forums | Nikon-mount SLRs | Join Image Upload end

You are not logged in. Login or Register

Username   Password   Lost password?