In an excellent writeup, Fazel Majid looks at the mathematical ramifications of NEF compression from a theoretical point of view. The summary is that while compression does not change the dynamic range, it “compresses 12 bits worth of data into 9.4 bits worth of resolution,” and due to the way images are encoded, the lost resolution is in the highlights. That's fine in theory, but what does this mean for an actual image? Can a human ever tell the difference?

UPDATE: According to the advanced literature, Nikon's D3 and D300 also have a compressed mode that is lossless, making three types of NEF files that they can produced (uncompressed, lossy compressed, and lossless compressed). This post is not about the new lossless NEF, which should be, well, lossless.

The Test

To see exactly what information is lost during compression, I set up a high-contrast scene in my living room. I took a single picture, from which I had the camera create both a compressed NEF and an uncompressed NEF. Thus, I had two files whose sensor data was pixel-for-pixel identical except for whatever differences were caused by the compression.

(To be clear, I did not take snap two similar pictures; both the compressed NEF and uncompressed NEF files were the result of the same, single shutter release.)

To illustrate the full range of detail captured in each NEF, I “developed” them three ways, producing three jpegs each:

• A “dark” version, to bring out highlight detail
• A “mid” version, balancing highlights and shadows
• A “bright” version, to bring out shadow detail

These images are shown in the mini application further down on this page. Mouseover the “dark,” “mid,” and “bright” buttons to see that version, in either the “full” (no compression) column, or the “compressed” column.

(I should note here that because my livingroom is mostly white, I sprinkled a bunch of my three-year-old's toys all around to add some color. The house is not normally this cluttered!)

I didn't expect to be able to see any differences at the 8-bit jpeg level, and indeed there are no visual difference between the pairs: the dark version produced from the compressed NEF looks identical to the dark version produced from the uncompressed NEF, and the same goes for the mid and bright pairs as well.

But since there clearly are differences (the compression is not lossless, so by definition, there must be differences), I moved into the realm of raw sensor data to investigate further...

Raw Sensor Data

I compared the two NEFs in 16-bit mode, producing a “mask” image that's black where the two NEFs are identical, and green where there are sensor-level differences of any magnitude. Mouseover the “Mask” button to see the mask. Switch back and forth between the “Mask” and one of the “Mid” versions and notice that dark areas of the scene have no differences. Fazel's analysis of the compression (that it's lossy in the highlights) is spot on.

Note that even though I've shown three versions from each NEF (dark, mid, and bright), there's only one mask image. Remember that the dark/mid/bright versions are derivations of the NEF that I happened to choose to illustrate the various kinds of detail in the image. The mask compares the raw sensor data of each NEF directly.

The mask image shows where there are differences, but not the magnitude of the differences, which is the real question. So, I then produced a 16-bit “difference” image. For each color channel of each pixel, I subtracted the data from one NEF from the other. The result is pure black where there are no differences (where the compression is lossless), and various colors of increasing brightness for increasing differences/loss. This is the “true difference” image.

You can mouseover the “True” button below to see the true-difference image, but I'll save you the trouble by telling you that it's black: the differences are all so subtle that they become black when represented in an 8-bit jpeg.

But there are differences, and I want to see them, so I created an image with the differences exaggerated. Wildly exaggerated. Spectacularly exaggerated. Not only did I stretch the differences so that they filled the whole 8-bit color space of the jpeg used here, but in reducing the size (from the 3,900 × 2,616 original), each pixel of the result received the combined differences from 36 pixels. This means that the differences are further enhanced by an extra factor of 36.

Mouseover the “Exaggerated” button so see that version.

Under-Exposed exposure: from uncompressed NEF

Under-Exposed exposure: from compressed NEF

Under-Exposed exposure: mask showing location of compression-related differences

Under-Exposed exposure: compression-related differences wildly exaggerated

Under-Exposed exposure: compression-related differences

Under-Exposed exposure: from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: from compressed NEF (darker, to show highlight detail)

from uncompressed NEF

from compressed NEF

mask showing location of compression-related differences

compression-related differences wildly exaggerated

compression-related differences

from uncompressed NEF (brighter, to show shadow detail)

from compressed NEF (brighter, to show shadow detail)

from uncompressed NEF (darker, to show highlight detail)

from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: from uncompressed NEF

Over-Exposed exposure: from compressed NEF

Over-Exposed exposure: mask showing location of compression-related differences

Over-Exposed exposure: compression-related differences wildly exaggerated

Over-Exposed exposure: compression-related differences

Over-Exposed exposure: from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: from compressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from uncompressed NEF

Under-Exposed exposure: 100% crop from compressed NEF

Under-Exposed exposure: 100% crop of mask showing location of compression-related differences

Under-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Under-Exposed exposure: 100% crop of compression-related differences

Under-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

100% crop from uncompressed NEF

100% crop from compressed NEF

100% crop of mask showing location of compression-related differences

100% crop: compression-related differences wildly exaggerated

100% crop of compression-related differences

100% crop from uncompressed NEF (brighter, to show shadow detail)

100% crop from compressed NEF (brighter, to show shadow detail)

100% crop from uncompressed NEF (darker, to show highlight detail)

100% crop from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from uncompressed NEF

Over-Exposed exposure: 100% crop from compressed NEF

Over-Exposed exposure: 100% crop of mask showing location of compression-related differences

Over-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Over-Exposed exposure: 100% crop of compression-related differences

Over-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from uncompressed NEF

Under-Exposed exposure: 100% crop from compressed NEF

Under-Exposed exposure: 100% crop of mask showing location of compression-related differences

Under-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Under-Exposed exposure: 100% crop of compression-related differences

Under-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

100% crop from uncompressed NEF

100% crop from compressed NEF

100% crop of mask showing location of compression-related differences

100% crop: compression-related differences wildly exaggerated

100% crop of compression-related differences

100% crop from uncompressed NEF (brighter, to show shadow detail)

100% crop from compressed NEF (brighter, to show shadow detail)

100% crop from uncompressed NEF (darker, to show highlight detail)

100% crop from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from uncompressed NEF

Over-Exposed exposure: 100% crop from compressed NEF

Over-Exposed exposure: 100% crop of mask showing location of compression-related differences

Over-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Over-Exposed exposure: 100% crop of compression-related differences

Over-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from uncompressed NEF

Under-Exposed exposure: 100% crop from compressed NEF

Under-Exposed exposure: 100% crop of mask showing location of compression-related differences

Under-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Under-Exposed exposure: 100% crop of compression-related differences

Under-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Under-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Under-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

100% crop from uncompressed NEF

100% crop from compressed NEF

100% crop of mask showing location of compression-related differences

100% crop: compression-related differences wildly exaggerated

100% crop of compression-related differences

100% crop from uncompressed NEF (brighter, to show shadow detail)

100% crop from compressed NEF (brighter, to show shadow detail)

100% crop from uncompressed NEF (darker, to show highlight detail)

100% crop from compressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from uncompressed NEF

Over-Exposed exposure: 100% crop from compressed NEF

Over-Exposed exposure: 100% crop of mask showing location of compression-related differences

Over-Exposed exposure: 100% crop: compression-related differences wildly exaggerated

Over-Exposed exposure: 100% crop of compression-related differences

Over-Exposed exposure: 100% crop from uncompressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from compressed NEF (brighter, to show shadow detail)

Over-Exposed exposure: 100% crop from uncompressed NEF (darker, to show highlight detail)

Over-Exposed exposure: 100% crop from compressed NEF (darker, to show highlight detail)

These buttons show representative jpeg views from the two NEFs. The three dark/mid/bright buttons on the left are for the uncompressed NEF, while the three on the right are for the compressed NEF.

There are three versions for each NEF to illustrate the detail in different areas of the high-contrast image.

Differences due to compression should theoretically show up when comparing the pairs (e.g. the right-side Dark with the left-side Dark), but no differences are apparent because the sensor-level differences are too subtle to show up in an 8-bit jpeg.

(click window to close)

The “Difference Mask” button shows which pixels have any sensor-level difference between the compressed and uncompressed raw files.

To be clear, the mask is related the two original raw files – compressed and uncompressed – derived from a single shutter-release event. The mask is not directly related to the six dark/mid/bright views, which are merely derived from the raw files.

The mask differences are checked in 16-bit, raw sensor-data mode.

(click window to close)

The “True Differences” button shows the difference between two raw files – compressed and uncompressed – derived from the same shutter release. The difference is computed with raw 16-bit sensor data, prior to any gamma or color correction.

The 16-bit-mode result was then converted to an 8-bit sRGB jpeg. Because the differences are far too minor to show up in an 8-bit color space, the image shown here is black.

(click window to close)

The “Exaggerated Diff” button shows the raw 16-bit differences between the two raw files – compressed and uncompressed – expanded to fill the entire 8-bit RGB range of an sRGB jpeg. This is to vividly illustrate the difference between the two raw files. Any detail that can be seen in this version represents detail lost during NEF compression.

Remember that the true lost detail is exceedingly more subtle than shown in this exaggerated version; see the text for details.

(click window to close)

The “Crop” buttons select what part of the original photo is viewed with the nine “View” buttons on the right.

The “Full Scene” button selects a small-sized version of the original 10.2 megapixel (3,900 × 2,616) image. The other buttons select 100% crops of selected areas of interest in the full scene.

(click window to close)

The three “Photo” buttons select which photo all other buttons/views work with.

Each photo is a separate shutter-release event. The two non-“primary” shots were over/under exposed by two stops, resulting in a lot of highlight/shadow clipping. NEF compression is lossy in the highlights, so the over-exposed photo shows the most differences.

(click window to close)

Let me stress again that the “exaggerated” version is wildly exaggerated, both because the differences have been stretched to fill the breadth of the 8-bit color space, and also because the differences are further exaggerated (by a factor of 36!) by the way the size reduction was done.

It's instructive to see all these versions, masks, and differences at 100% zoom. Click this button to enable several 100% crops in the mini application above:

While the reduced-size “Exaggerated” view might seem to show a lot of detail has been lost, looking at the 100% crops shows that the actual differences are much less, even though the exaggerated view is still wildly exaggerated.

These Tests with Additional Pictures

Playing around with the mini application above, you can see many different views of the same picture (9 views for each of the four “crop” buttons). All are derived from one shutter-release event.

In looking at these results, it's clear that the detail lost via compression is extremely subtle, but perhaps this is only because my test image doesn't have enough of the highlights that are susceptible to the compression's lossiness. Perhaps I need to test with an exposure that has more photons hitting the sensor?

With that in mind, I took another picture of the same scene, but this time I purposefully overexposed it by two stops. To keep a balance to my tests, I took one underexposed by two stops, as well.

It makes no sense to do pixel-by-pixel comparisons among these three shots, as they're indeed different shots and subtle changes in lighting and composition are likely much larger than any compression-related differences. Still, each of the new pictures can be looked at, individually, with the same attention to detail as the first. Click on the this button to enable additional photos in the mini application above:

(Clicking on it has your browser download an additional 72 images — 9 views for each of the four crops, per each of the two additional pictures)

As additional confirmation that the loss of detail in NEF compression is in the highlights, select the Mask view and cycle among the three photos: the over-exposed photo has changes across the entire scene, while the under-exposed photo has changes only in the brightest areas.

Conclusions

Consider the exaggerated view of the over-exposed photo (click to select, then scroll up to view). Looking at this may lead one to believe that a lot of detail has been lost. Clearly, much more has been lost in this over-exposed shot than with the others, yet, the differences still don't make it into any of the developed images, even at 100% crop. “More” is clearly relative.

Nevertheless, all the detail in the reduced-size exaggerated view is detail that has been removed. Even though such detail may seem insignificant when spread out over the entire 10.2 megapixel image, it is detail none the less.

In the end, the only reason to use compression in the first place is for convenience (smaller files, faster write times, and more images on a card), and that convenience is paid for by irrevocable loss of image detail, albeit a very small amount of image detail.

It's up to each photographer to determine whether the lost detail is at all relevant. It is not relevant to me, so I enjoy the convenience of the smaller, compressed files. However, if I were likely to have a lot of over-exposed shots from which I absolutely had to extract the most information possible, I might reconsider. (I'd probably continue to use compression, but I'd have to at least reconsider.)

Technical Details

Some notes about how I did the tests...

• The original compressed and uncompressed NEFs were converted to 16-bit “pseudo PPM” files of raw, linear sensor data with no gamma and no whitebalance, using David Coffin's most excellent dcraw.

• I wrote a program to compare the two PPMs and generate a new 16-bit PPM representing only the per-pixel sensor-level differences, either as a mask or as a “true differences” image.

• The dark/mid/light versions were produced from each NEF (using dcraw) with different -b (brightness) arguments.

• The various jpeg versions of these files were created from PPM files using ImageMagick's convert utility.

• The exaggerated versions were created using the convert utility's -normalize argument.

• I spent considerably more time building the javascript application for showing the results than I spent generating the results.