A Qualitative Analysis of NEF Compression

Nikon's advanced digital cameras can save images in three file formats: jpeg, straight Raw, or compressed Raw. Whether compressed or not, the Raw files are named with the “.NEF” (Nikon Electronic File) extension. With the 10.2 megapixel D200, an uncompressed NEF is about 16 megabytes; the compressed NEF is about half that size. The reduction in size is appealing, but does it come at a reduction in quality?

In an excellent writeup, Fazel Majid looks at the mathematical ramifications of NEF compression from a theoretical point of view. The summary is that while compression does not change the dynamic range, it “compresses 12 bits worth of data into 9.4 bits worth of resolution,” and due to the way images are encoded, the lost resolution is in the highlights. That's fine in theory, but what does this mean for an actual image? Can a human ever tell the difference?

UPDATE: Nikon's newer cameras (D3, D300, D700, ...) also have a compressed mode that is lossless, making three types of NEF files that they can produced (uncompressed, lossy compressed, and lossless compressed). This post is not about the new lossless NEF, which should be, well, lossless.

The Test

To see exactly what information is lost during compression, I set up a high-contrast scene in my living room. I took a single picture, from which I had the camera create both a compressed NEF and an uncompressed NEF. Thus, I had two files whose sensor data was pixel-for-pixel identical except for whatever differences were caused by the compression.

(To be clear, I did not take snap two similar pictures; both the compressed NEF and uncompressed NEF files were the result of the same, single shutter release.)

To illustrate the full range of detail captured in each NEF, I “developed” them three ways, producing three jpegs each:

  • A “dark” version, to bring out highlight detail
  • A “mid” version, balancing highlights and shadows
  • A “bright” version, to bring out shadow detail

These images are shown in the mini application further down on this page. Mouseover the “dark,” “mid,” and “bright” buttons to see that version, in either the “full” (no compression) column, or the “compressed” column.

(I should note here that because my livingroom is mostly white, I sprinkled a bunch of my three-year-old's toys all around to add some color. The house is not normally this cluttered!)

I didn't expect to be able to see any differences at the 8-bit jpeg level, and indeed there are no visual difference between the pairs: the dark version produced from the compressed NEF looks identical to the dark version produced from the uncompressed NEF, and the same goes for the mid and bright pairs as well.

But since there clearly are differences (the compression is not lossless, so by definition, there must be differences), I moved into the realm of raw sensor data to investigate further...

Raw Sensor Data

I compared the two NEFs in 16-bit mode, producing a “mask” image that's black where the two NEFs are identical, and green where there are sensor-level differences of any magnitude. Mouseover the “Mask” button to see the mask. Switch back and forth between the “Mask” and one of the “Mid” versions and notice that dark areas of the scene have no differences. Fazel's analysis of the compression (that it's lossy in the highlights) is spot on.

Note that even though I've shown three versions from each NEF (dark, mid, and bright), there's only one mask image. Remember that the dark/mid/bright versions are derivations of the NEF that I happened to choose to illustrate the various kinds of detail in the image. The mask compares the raw sensor data of each NEF directly.

The mask image shows where there are differences, but not the magnitude of the differences, which is the real question. So, I then produced a 16-bit “difference” image. For each color channel of each pixel, I subtracted the data from one NEF from the other. The result is pure black where there are no differences (where the compression is lossless), and various colors of increasing brightness for increasing differences/loss. This is the “true difference” image.

You can mouseover the “True” button below to see the true-difference image, but I'll save you the trouble by telling you that it's black: the differences are all so subtle that they become black when represented in an 8-bit jpeg.

But there are differences, and I want to see them, so I created an image with the differences exaggerated. Wildly exaggerated. Spectacularly exaggerated. Not only did I stretch the differences so that they filled the whole 8-bit color space of the jpeg used here, but in reducing the size (from the 3,900 × 2,616 original), each pixel of the result received the combined differences from 36 pixels. This means that the differences are further enhanced by an extra factor of 36.

Mouseover the “Exaggerated” button so see that version.


Under-Exposed exposure: from uncompressed NEF

Under-Exposed exposure: from compressed NEF

Under-Exposed exposure: mask showing location of compression-related differences

Under-Exposed exposure: compression-related differences wildly exaggerated

Under-Exposed exposure: compression-related differences

Under-Exposed exposure: from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: from compressed NEF (darker, to show highlight detail)

from uncompressed NEF

from compressed NEF

mask showing location of compression-related differences

compression-related differences wildly exaggerated

compression-related differences

from uncompressed NEF (brighter, to show shadow detail)

from compressed NEF (brighter, to show shadow detail)

from uncompressed NEF (darker, to show highlight detail)

from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: from uncompressed NEF

Over-Exposed exposure: from compressed NEF

Over-Exposed exposure: mask showing location of compression-related differences

Over-Exposed exposure: compression-related differences wildly exaggerated

Over-Exposed exposure: compression-related differences

Over-Exposed exposure: from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: from compressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from uncompressed NEF

Under-Exposed exposure: 100% crop from compressed NEF

Under-Exposed exposure: 100% crop of mask showing location of compression-related differences

Under-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Under-Exposed exposure: 100% crop of compression-related differences

Under-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

100% crop from uncompressed NEF

100% crop from compressed NEF

100% crop of mask showing location of compression-related differences

100% crop: compression-related differences wildly exaggerated

100% crop of compression-related differences

100% crop from uncompressed NEF (brighter, to show shadow detail)

100% crop from compressed NEF (brighter, to show shadow detail)

100% crop from uncompressed NEF (darker, to show highlight detail)

100% crop from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from uncompressed NEF

Over-Exposed exposure: 100% crop from compressed NEF

Over-Exposed exposure: 100% crop of mask showing location of compression-related differences

Over-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Over-Exposed exposure: 100% crop of compression-related differences

Over-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from uncompressed NEF

Under-Exposed exposure: 100% crop from compressed NEF

Under-Exposed exposure: 100% crop of mask showing location of compression-related differences

Under-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Under-Exposed exposure: 100% crop of compression-related differences

Under-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

100% crop from uncompressed NEF

100% crop from compressed NEF

100% crop of mask showing location of compression-related differences

100% crop: compression-related differences wildly exaggerated

100% crop of compression-related differences

100% crop from uncompressed NEF (brighter, to show shadow detail)

100% crop from compressed NEF (brighter, to show shadow detail)

100% crop from uncompressed NEF (darker, to show highlight detail)

100% crop from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from uncompressed NEF

Over-Exposed exposure: 100% crop from compressed NEF

Over-Exposed exposure: 100% crop of mask showing location of compression-related differences

Over-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Over-Exposed exposure: 100% crop of compression-related differences

Over-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from uncompressed NEF

Under-Exposed exposure: 100% crop from compressed NEF

Under-Exposed exposure: 100% crop of mask showing location of compression-related differences

Under-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Under-Exposed exposure: 100% crop of compression-related differences

Under-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

100% crop from uncompressed NEF

100% crop from compressed NEF

100% crop of mask showing location of compression-related differences

100% crop: compression-related differences wildly exaggerated

100% crop of compression-related differences

100% crop from uncompressed NEF (brighter, to show shadow detail)

100% crop from compressed NEF (brighter, to show shadow detail)

100% crop from uncompressed NEF (darker, to show highlight detail)

100% crop from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from uncompressed NEF

Over-Exposed exposure: 100% crop from compressed NEF

Over-Exposed exposure: 100% crop of mask showing location of compression-related differences

Over-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Over-Exposed exposure: 100% crop of compression-related differences

Over-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)
View  (mouseover to select)
FullCompressed
Dark Dark
Mid Mid
Bright Bright
(click for help)
Difference
Mask
True
Exaggerated
(click for help) (click for help) (click for help)

Let me stress again that the “exaggerated” version is wildly exaggerated, both because the differences have been stretched to fill the breadth of the 8-bit color space, and also because the differences are further exaggerated (by a factor of 36!) by the way the size reduction was done.

It's instructive to see all these versions, masks, and differences at 100% zoom. Click this button to enable several 100% crops in the mini application above:

Click Here To Enable Crop Views

While the reduced-size “Exaggerated” view might seem to show a lot of detail has been lost, looking at the 100% crops shows that the actual differences are much less, even though the exaggerated view is still wildly exaggerated.

These Tests with Additional Pictures

Playing around with the mini application above, you can see many different views of the same picture (9 views for each of the four “crop” buttons). All are derived from one shutter-release event.

In looking at these results, it's clear that the detail lost via compression is extremely subtle, but perhaps this is only because my test image doesn't have enough of the highlights that are susceptible to the compression's lossiness. Perhaps I need to test with an exposure that has more photons hitting the sensor?

With that in mind, I took another picture of the same scene, but this time I purposefully overexposed it by two stops. To keep a balance to my tests, I took one underexposed by two stops, as well.

It makes no sense to do pixel-by-pixel comparisons among these three shots, as they're indeed different shots and subtle changes in lighting and composition are likely much larger than any compression-related differences. Still, each of the new pictures can be looked at, individually, with the same attention to detail as the first. Click on the this button to enable additional photos in the mini application above:

Click Here To Enable Other Photos

(Clicking on it has your browser download an additional 72 images — 9 views for each of the four crops, per each of the two additional pictures)

As additional confirmation that the loss of detail in NEF compression is in the highlights, select the Mask view and cycle among the three photos: the over-exposed photo has changes across the entire scene, while the under-exposed photo has changes only in the brightest areas.

Conclusions

Consider the exaggerated view of the over-exposed photo (click to select, then scroll up to view). Looking at this may lead one to believe that a lot of detail has been lost. Clearly, much more has been lost in this over-exposed shot than with the others, yet, the differences still don't make it into any of the developed images, even at 100% crop. “More” is clearly relative.

Nevertheless, all the detail in the reduced-size exaggerated view is detail that has been removed. Even though such detail may seem insignificant when spread out over the entire 10.2 megapixel image, it is detail none the less.

In the end, the only reason to use compression in the first place is for convenience (smaller files, faster write times, and more images on a card), and that convenience is paid for by irrevocable loss of image detail, albeit a very small amount of image detail.

It's up to each photographer to determine whether the lost detail is at all relevant. It is not relevant to me, so I enjoy the convenience of the smaller, compressed files. However, if I were likely to have a lot of over-exposed shots from which I absolutely had to extract the most information possible, I might reconsider. (I'd probably continue to use compression, but I'd have to at least reconsider.)


Technical Details

Some notes about how I did the tests...

  • The original compressed and uncompressed NEFs were converted to 16-bit “pseudo PPM” files of raw, linear sensor data with no gamma and no whitebalance, using David Coffin's most excellent dcraw.

  • I wrote a program to compare the two PPMs and generate a new 16-bit PPM representing only the per-pixel sensor-level differences, either as a mask or as a “true differences” image.

  • The dark/mid/light versions were produced from each NEF (using dcraw) with different -b (brightness) arguments.

  • The various jpeg versions of these files were created from PPM files using ImageMagick's convert utility.

  • The exaggerated versions were created using the convert utility's -normalize argument.

  • I spent considerably more time building the javascript application for showing the results than I spent generating the results.


All 35 comments so far, oldest first...

Absolutely excellent analysis – I suspect that the compression exploits the linear nature of the coding in the highlights (a 1-bit step in the shadows is much more noticeable than a 1-bit step in the highlights). Thus we can get away with much higher granularity (larger step sizes) in the linear values in the highlights without compromising the tonal transitions perceived by the eye.

This is something I’ve long suspected – but I never got round to testing the science.

Many thanks… Martin

— comment by Martin on October 10th, 2006 at 1:40am JST (17 years, 5 months ago) comment permalink

This revelation although possibly insignificant to most of us, indicates the emerging improvements that should be coming before long. These will give those high priced pro lenses the sensors and firmware they deserve.

— comment by Paul Linder on October 10th, 2006 at 8:31am JST (17 years, 5 months ago) comment permalink

I am switching from Canon to Nikon and I was wondering about this raw compression. I want to thank you for a very detailed analysis. Now I’m curious what Canon is using. I had previously thought it was lossless compression, now I’m not sure. And what is the difference in compression rate between Nikon’s lossful compression and the best lossless compression? Is Nikon saving a lot by going to lossful? I suppose if they’re cutting the file size by half they are.

— comment by Michael Mattox on November 20th, 2006 at 2:36pm JST (17 years, 4 months ago) comment permalink

beautiful

— comment by kaishaku on December 17th, 2006 at 3:25am JST (17 years, 3 months ago) comment permalink

Most of my photos have extreme contrast and I am struggling to keep detail in the highlights. Recently I decided to change to compressed raw, now, having read your excellent article, I think that I will go back to uncompressed raw. I need all the detail that I can get in the highlights. Space is not a problem, I have a number of CF cards, time is not a problem I don’t do this for a living, so why should I compromise? Thank you for your time, effort and expertise which makes a complex subject understandable even to me, (well almost).

— comment by Eric on December 17th, 2006 at 7:53am JST (17 years, 3 months ago) comment permalink

Your analysis is the most objective one I have come across on the web. Thank you so much for your work! I have just started down the RAW-path and now my computer is struggling to juggle my D200 NEF’s. Now I just have to test which factor would slow me down the most – the size or the de-compression of the NEF files? Since I can now breathe easily over supposedly lost highlight detail, I think I’m leaning towards using compressed files.

— comment by Stefan on December 19th, 2006 at 9:04pm JST (17 years, 3 months ago) comment permalink

This is pure (but slightly informed) speculation on my part, but I would think that the win due to smaller data transfers to/from disk far, far, far exceed the small extra costs in CPU needed for decompression. Even if it were a much more expensive decryption, rather than the trivial decompression in question, the smaller data transfers would cause a win.

— comment by Jeffrey Friedl on December 20th, 2006 at 9:23am JST (17 years, 3 months ago) comment permalink

Great analysis. Thanks for doing this. The only thing I have a question about is how did you get the D200 to produce both a compressed NEF and an uncompressed NEF from the same shutter click? Or did I not understand what you did?

I can get a NEF and a JPG with one click, but I can’t find in the menus how to get two NEFs.

— comment by Leroy on December 29th, 2006 at 3:48pm JST (17 years, 3 months ago) comment permalink

> The only thing I have a question about is how did
> you get the D200 to produce both a compressed NEF
> and an uncompressed NEF from the same shutter click?

Step 1: capture an uncompressed-NEF image of your choice

Step 2: while compression still turned off, perform an in-camera image overlay with the step-1 picture and another image (a blank frame being best).

Step 3: change camera settings, turning NEF compression on

Step 4: repeat step 2

You now have two copies of the same image — the results of steps 2 and 4 — that are pixel-for-pixel identical except for NEF compression.

Note that the initial image (result of step 1) is not used in the end; it’s used only as the source image for the two image-overlay steps.

The correctness of this procedure does not depend on the other source image being blank, although the more blank it is, the more the orignal scene is preserved. For what it’s worth, the blank image I used was a NEF in which I had manually zeroed out all sensor data, so it was a true, pure blank image. Again, it doesn’t matter at all for the correctness, but I thought it was more tidy to do it that way.

Jeffrey

— comment by Jeffrey Friedl on December 30th, 2006 at 7:10pm JST (17 years, 3 months ago) comment permalink

Thanks Jeffrey. It worked like a charm. Just take a quick snap with the cap on for the blank and follow your instructions. This will help me determine if compressed will affect the kind of shots I do. Thanks again.

— comment by Leroy on January 1st, 2007 at 3:01pm JST (17 years, 3 months ago) comment permalink

Hello,
this test is bad, because have two different photo.
Why ?, light, temperature of sensor, cloudy and more and more factor, battery have different voltage in time
This is error and you can correct from statistic error repair: take 100 picture and analyze.

When we have program to compress uncompressed NEF then we must test from single foto!

Is possible code hight bite to less bite with lossless !!!
Theory of compression, Shannon definition …. RAR, ZIP, Gzip etc and mathematical can calculate maximum teorticaly compression, but this compression is slow, real program have optional algorithms for speed and size

Sorry to my horrible english, i am from Czech Republic

The tests are from a single photo from which I produced both compressed and uncompressed versions. The test is valid, at least in this sense. Your English is much better than my Czech, so I appreciate your using it 🙂 —jfriedl

— comment by Izak on February 27th, 2007 at 6:32am JST (17 years ago) comment permalink

This analysis was very helpful in finally confirming earlier discussions on highlight loss in compressed NEFs. I will continue to use Compressed NEFs because I shoot so many frames and don’t want to spend twice as much time backing them up.
My only question is whether Compressed NEFs take longer to open and process with a program like Adobe Camera Raw and therefore hinder the performance of the computer.

Kevin, although I haven’t tested it, the engineer in me is certain that compressed NEFs are faster to open, because the minor extra CPU time needed to uncompress the data is certainly far overshadowed by the savings due to the smaller file reads from disk. —Jeffrey

— comment by Kevin Trowbridge on March 5th, 2007 at 4:09pm JST (17 years ago) comment permalink

Great analysis. And some of the best written explanation I have seen on the Web or anywhere else (and I used to be a technical writer and editor). The question of NEF compression has been nagging at me for some time. I tried your trick of overlaying two versions of one photo. Worked great. I was going crazy trying to take two identical photos, one compressed and one uncompressed. Here’s my take on the quality changes in compressed NEF.
I took a simple still life with lots of color and lots of highlights (with a few blown out). I used your trick to get identical NEF images, one compressed, one uncompressed. I opened both images in PhotoShop (identical settings) and then tried a few somewhat subjective tests.

1- I overlaid the images and looked at them in ‘difference’ mode. As expected, there were no visible differences.

2- I looked at the histograms of the individual images. By toggling back and forth between the images, I could see very minor changes in brightness levels at several points along the histogram. The changes were almost imperceptible. Individually, you cannot tell the difference. I then made a very minor white balance change in one of the images (you would have to look at the images side by side to see the difference). But the change in the histogram WAS obvious.

3- Then I just looked at the images, side by side. Again, it’s impossible to tell them apart, even at 100% magnification. I zoomed in and looked around. At 1600% I could start to see some artifacts in the form of differences in random pixels. Not very many, probably (wild guess) one in several hundred or more. At 3200%, the differences were fairly easy to spot (or I maybe my eye was getting tuned in). I checked the color difference of a few (quite a few) pixels and none of the comparisons had a difference of more than one unit in one of the color channels. That is, a pixel in one image with an RGB value of 221:190:121, changed to 220:190:121 in the other.

4-Before using the overlay trick. I tried to take duplicate images one at a time. When I compared compressed and uncompressed, all comparisons had distinct but subtle differences (some more than others). It is now clear these duplicate images taken under controlled conditions had differences as a result of normal mechanical and electrical variations. And further, these differences were greater, much greater, than the artifacts caused by NEF compression.

Conclusion: I was skeptical about how good the compression was going to be. There is a lot of misinformation and inflated opinions floating around the Web. Your technical analysis and my subjective analysis have convinced me that Nikon’s NEF compression is, well, pretty damn good. The compressed image is virtually identical to the uncompressed. For purists, that is not good enough. There are changes, but they are not perceptible when viewed at normal magnification; and certainly not perceptible in a print. If you manipulate an image with an editor, ANY modification will, by far, exceed any compression artifacts (probably by orders of magnitude). There are least two unknowns. One, it’s not clear if editing might exaggerate artifacts. It’s possible, but still probably not perceptible. Two, I only looked at one image. There may be types of images that will show more compression artifacts. I’m going to start using compress NEF.

Sorry for being so wordy – Bill

— comment by Bill Souza on March 11th, 2007 at 6:34am JST (17 years ago) comment permalink

I am a hobbyist (I hate that term), not a professional and I didn’t understand half of what Jeffrey did or those who commented on his work but I care alot about how my pictures look (notice I did not say images). So I did my own experiment. First I took two pictures of one of my wife’s daffodils, then two pictures of the Colosseum in Rome and finally two pictures of my grandson. Both compressed and uncompressed pictures of the daffodils looked pretty good but then everyone says it is easy to take great pictures of flowers. The two pictures of the Colosseum frankly look identical. As to the two pictures of my grandson, Nathaniel, 4 years old, he looked perfect prescious in both altho I think his little smile in the compressed one was a bit cuter. But then you can’t take a bad picture of him.

— comment by Charles M. Luther on March 14th, 2007 at 3:38am JST (17 years ago) comment permalink

Hello Jeffrey! Very interesting report, thanks for sharing. In my opinion, however, there is a point you are missing: what happens BEFORE the compression in Nikon cameras. Looking around I found out that Nikons perform a median filter even when saving NEF files. This is contrary to the philosophy of RAW files. Actually, this is the main reason for Canon are preferred to Nikon in astrophotography. I think Nikon is cheating here.

— comment by Marko Germani on April 9th, 2007 at 7:27pm JST (16 years, 11 months ago) comment permalink

You have shown the absolute proof that in order to have detail in an image be affected by compression, you need to pretty much completely ignore your camera’s light meter, create a heinously badly exposed picture, and then torture the image until it’s not a photograph.

— comment by Ed on April 30th, 2007 at 6:50am JST (16 years, 11 months ago) comment permalink

“completely ignore your camera’s light meter, create a heinously badly exposed picture, and then torture the image until it’s not a photograph”

I’m right with you up until those last five words. Who are you to say what is and isn’t a photograph? Some of us have technical needs for images that involve “torturing” the image to wring every last subtle variation in tone out of a particular set of pixels. Others might want to do so just for fun. Either way, it’s still photography, whether it’s what floats your boat or not.

— comment by Dave on May 18th, 2007 at 3:50am JST (16 years, 10 months ago) comment permalink

Interesting comparison! It is clear that the variation between two shots of the same scene taken with the same camera settings will vary more than uncompressed NEF and compressed NEF images of the same exposure. If one is really concerned about the highlights, then take several images of the same scene with different exposures and combine them, e.g., using PhotoShop’s HDR technique. This will give much better results in terms of highlight details than uncompressed NEF. Of course, this will not work if there is any movement in the scene.

I can see absolutely no practical advantage to using uncompressed NEF other than a psychological feeling of purity.

— comment by Bernard Waxman on July 2nd, 2007 at 11:27pm JST (16 years, 9 months ago) comment permalink

This is an excellent analysis of where the photo software industry is today in terms of NEF processing. Things being what they are with technology we will undoubtly see substintial changes down the road in this area. I would bet that RAW processing in the future will give greater reason to shoot in an uncompressed format. If you compare the digital imaging industry from its inseption verses today then tack on 10 more years you can only imagine what results will be achievable. Have you ever looked at an old photograph and wished you could make some changes? Think about what you will be able to do with your current uncompressed NEF files in the future.

— comment by Howard P on July 7th, 2007 at 12:24am JST (16 years, 8 months ago) comment permalink

Nice test and text. Very useful for those, like me, struggling to choose between the D200 and the D80.

The D80 only has the compressed NEF format, and it seems, at least to me, the only real image quality difference between them. Exposure metering, another one, is really irrelevant for those coming from the pre-AE era like me, I still use my Leica M3 and, most of the time, guess the (correct) exposure, or use the wonderful Lunasix to help.

Since DSLRs are not good investments, at least for now, it is better to go with the cheaper thing (the nice metal body of the D200 will be very soon obsolete, some say later this year).

— comment by R Pedrosa on July 25th, 2007 at 4:09am JST (16 years, 8 months ago) comment permalink

Oh, I forgot, any comment on the comment above which suggets that Canon’n RAW is rawer than Nikon’s?

— comment by R Pedrosa on July 25th, 2007 at 4:12am JST (16 years, 8 months ago) comment permalink

Hi there

A Good write up & from the point of the end user not from a pure mathematical perspective.

But if in doubt and you really need to stretch dynamic range in your landscapes then you really want as much data as possible in case the highlights are borderline & so I won’t use compression.

As for the new ” Loseless Compression ” this will need looking at.

thanks Roy

— comment by Roy on September 28th, 2007 at 10:27pm JST (16 years, 6 months ago) comment permalink

I am wondering is are all the experiments really testing the compressed NEF V/S the NEF, or are they testing the software capability of discerning the difference. I don’t mean to be picky, but there is an important difference in interpreting the results.

— comment by Zan on October 28th, 2007 at 12:06am JST (16 years, 5 months ago) comment permalink

Nice analysis, thanks for taking the time to cover this topic. I just got a d200 and Ken Rockwell recommends shooting in JPG Basic mode. Which I did for a week or so but I think its going to be compressed NEF from here on out.

— comment by michael on November 21st, 2007 at 3:11pm JST (16 years, 4 months ago) comment permalink

The ‘lossy’ version of NEF compression is a clever use of the properties of light in order to thin the levels used in raw recording in a way that loses zero information (image content). Each pixel counts photons, but any such counting process has statistical fluctuations (so-called Poisson or shot noise) whose magnitude is the square root of the average count. For instance, if the average photon count of a group of pixels is 10000, there will be variations in the count from pixel to pixel of order sqrt[10000]=100. Typical CCD Nikon sensors collect about 8 photons per raw level at ISO 100, so a fluctuation of 100 photons is about twelve raw levels. So in highlights where one is collecting thousands of photons, there are random fluctuations of several raw levels. The spacing of raw levels is too fine to efficiently encode the highlight data, since the last 2-3 bits of the 12-bit raw number are purely random and encoding no image data. On the other hand, in shadows, one is collecting relatively few photons; 100 photons comes with fluctuations of order sqrt[100]=10 photons, which translates to only a little over one raw level — no thinning of levels is possible in shadows.

This then is the idea behind ‘lossy’ NEF compression — thin the levels in highlights where they are too finely distributed relative to the inherent noisiness of light, but keep most or all of the levels in shadows where the magnitude of the fluctuations is smaller. This is why the NEF compression table is linear at the low end, and quadratic further up. The quadratic part is a reflection of the quadratic relationship between noise and signal of light, the higher the signal the more levels can be thinned out. The linear part of the compression table kicks in when the signal is low enough that the noise fluctuations are less than or of order one raw level — then no compression is possible and the output value must equal the input value.

The ‘lossy’ NEF compression is really a clever use of information theory to save space by eliminating redundant raw levels. The noise which is unavoidably present in light effectively dithers tonal transitions so that the compression is *perceptually lossless*.

— comment by emil on April 24th, 2008 at 5:56am JST (15 years, 11 months ago) comment permalink

Excellent analysis, thank you very much for posting.

— comment by Robby on October 4th, 2008 at 11:12pm JST (15 years, 5 months ago) comment permalink

Hi Jeffrey –

First of all, thanks for a very informative post, and thanks for your diligence and objectivity in gathering and interpreting your data. You answer a lot of questions.

However, there’s one circumstance that your tests don’t really cover, and in my own tests (much less meticulously done than yours, I’ll happily admit) I have found that NEF compression does create visible artifacts (though still not very offensive ones).

Specifically, using a D200 set to ISO 800, I took photos of a sunny Arizona scene (my back yard) with lots of blue sky included (we have no shortage of that here). I then visually compared, at 100 % magnification, the photos made with NEF compression turned on and off, after converting them to tiffs via ACR using identical settings (note that I used different shots for this, but they were made within a minute or two of each other, so I don’t expect that to be a factor).

What I found is that in the blue sky areas, the noise was pretty clearly more evident in the compressed NEFs than in the uncompressed ones. The individual noise “grains” (viewed at 100%) were a bit harsher in the compressed NEF files.

My guess is that the noise in the sky constitutes a particularly difficult challenge for compressed NEFs because the camera’s blue channel, specifically, is recording values in the highlight range. Under this circumstance, the compression algorithm can magnify small differences (caused by noise) between neighboring blue photosites, since the effective spacing between the discrete levels that can be recorded in the highlight range is increased in compressed NEFs (i.e., the quantization error is higher).

The practical consequence of this observation is, no doubt, up for debate. When viewed at more reasonable, print-size magnifications (25-33%), it was hard (but still possible) for me to detect the differences in the noise artifacts in the sky areas of the photos, but I suspect most viewers wouldn’t be bothered by them. However, just to be on the safe side, I always shoot uncompressed NEFs.

You don’t mention what ISO setting you used in your tests, but I assume it was lower than 800. If you’re interested in reconstituting your test setup, it would be interesting to see if your test methods (much more rigorous than mine) could detect significant differences between compressed and uncompressed NEFs at higher ISO settings.

Anyway, thanks again for your great work!

Best regards, -Jim

— comment by Jim Peterson on July 30th, 2009 at 7:23am JST (14 years, 8 months ago) comment permalink

Jeffrey: thanks for the useful analysis and clearly written explanation. I also think the comment from emil dated April 24th is especially relevant. Peter, Kingston, Canada

— comment by peter on September 9th, 2009 at 10:31am JST (14 years, 6 months ago) comment permalink

Thank you very much. Only after reading this page did I become convinced that “compressed” is the best option for me. I may still use 14-bit when I need to maximize post-processing capability, but it will always be compressed. I belive that this information is more important today than ever before with full frame cameras D600 and D800/D800E and their HUGE 24MP and 36MP file sizes.

— comment by Reez on January 10th, 2013 at 12:45am JST (11 years, 2 months ago) comment permalink

Thanks for your test – its helping us saving a lot of storage in nearby future….

— comment by Anne und Björn on August 18th, 2014 at 6:05am JST (9 years, 7 months ago) comment permalink

Excellent analysis… It was bugging me about the compressed RAW in my D3200. But I guess, I am convinced now that I am not facing any huge image detail when I decided to go for this camera. I, along with so many others I’m sure, appreciate the pain you took in this detailed analysis and especially the awesome javascript! Thanks a ton!

— comment by Samik on September 15th, 2014 at 2:26pm JST (9 years, 6 months ago) comment permalink

This is a bit late but related:

https://www.degruyter.com/downloadpdf/j/teme.2016.83.issue-1/teme-2015-0093/teme-2015-0093.pdf

Proves that this is effectively lossless as per emil’s earlier post.

— comment by Robert Smith on July 1st, 2017 at 8:43am JST (6 years, 9 months ago) comment permalink

What about NEF to DNG? Are we loosing something or is it just re-write 0/1 from one format to another?

I don’t know much about it, but in its most basic form I believe it’s bringing over the raw sensor data, and bundling the appropriate sensor-specific profile with it. The converter can also do compression (lossless and lossy, the later of course which loses some data to get smaller space). The question I’d have is about metadata… does it bring over everything you’d ever need? I don’t know. You can, however, embed the original NEF into the DNG, at the cost of doubling the size, but then you know you’re not losing anything. —Jeffrey

— comment by Martin on January 13th, 2020 at 7:18pm JST (4 years, 2 months ago) comment permalink

I have wondered a lot about NEfs and you give an exhaustive and informative answer to all my questions!

Today with cameras with over 100MP compression seems even more sensible!

— comment by Tord S Eriksson on June 11th, 2021 at 7:46pm JST (2 years, 9 months ago) comment permalink

Thanks for your in depth Explanations. Now I understand whats it all about. I´m thinking about changing to Nikon and this was one of the questions to answer. Thanks a lot!
Best
Matthias

— comment by Matthias on July 26th, 2022 at 4:47am JST (1 year, 8 months ago) comment permalink
Leave a comment...


All comments are invisible to others until Jeffrey approves them.

Please mention what part of the world you're writing from, if you don't mind. It's always interesting to see where people are visiting from.

IMPORTANT:I'm mostly retired, so I don't check comments often anymore, sorry.


You can use basic HTML; be sure to close tags properly.

Subscribe without commenting