Lightroom Catalog Maintenance: Clearing Out Edit History For Potential Performance Gains
desktop background image of a garden at the Toji Temple (東寺) in Kyoto Japan during spring -- Serene Japanese Garden 'cause nuts-n-bolts digital maintenance like this post is about can give a headache Toji Temple (東寺), Kyoto Japan, April 2013 -- Toji Temple (東寺) -- Copyright 2013 Jeffrey Friedl, http://regex.info/blog/ -- This photo is licensed to the public under the Creative Commons Attribution-NonCommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/ (non-commercial use is freely allowed if proper attribution is given, including a link back to this page on http://regex.info/ when used online)
Nikon D4 + Voigtländer 125mm f/2.5 — 1/800 sec, f/2.5, ISO 140 — map & image datanearby photos
Serene Japanese Garden
'cause nuts-n-bolts digital maintenance like this post is about can give a headache
Toji Temple (東寺), Kyoto Japan, April 2013
Desktop-Background Versions
1280×800  ·  1680×1050  ·  1920×1200  ·  2560×1600  ·  2880×1800

This post is about trimming down the file size of an Adobe Lightroom catalog. In keeping with the off-season them of my previous post (which was filled with fall-foliage photos), this post is sprinkled with a few photos from Kyoto springs past, chosen more for their mentally-calming effect than for photographic quality, which they generally lack. (Sorry)

In a nutshell, if you have a large Lightroom catalog, and if you don't need the detailed editing history for your photos, you might be able to significantly reduce the size of your Lightroom catalog by flushing that edit-history data, which might generally improve overall responsiveness when working in Lightroom.

As one data point, my personal Lightroom catalog holding data for 134,026 photos and videos went from 3.9GB to 1.7GB, a savings of 56%.

desktop background image of cherry blossoms in Japan -- Cherry-Blossom Whiteout -- Ryouanji Temple (龍安寺) -- Kyoto, Japan -- Copyright 2013 Jeffrey Friedl, http://regex.info/blog/ -- This photo is licensed to the public under the Creative Commons Attribution-NonCommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/ (non-commercial use is freely allowed if proper attribution is given, including a link back to this page on http://regex.info/ when used online)
Nikon D4 + Voigtländer 125mm f/2.5 — 1/800 sec, f/2.5, ISO 100 — map & image datanearby photos
Cherry-Blossom Whiteout
Desktop-Background Versions
1280×800  ·  1680×1050  ·  1920×1200  ·  2560×1600  ·  2880×1800

Before you can do this, you have to decide whether the edit history is data you don't want to keep. It's a log of every change made in the develop module, as illustrated here:

Lightroom allows you to return the image to any given step by clicking on the step in the list, which can be quite convenient while you're working on an image. But if you're like me, it's of practical use only while actively working on an image. For editing states that I want to save over time, I use either snapshots (such as automatically made via my Snapshot on Export plugin), or virtual copies.

So I personally don't need the step-by-step editing history to be saved for years and years in the catalog, so I hope I'll get some benefit by clearing it out.

desktop background image of the rock garden at the Ryouanji Temple (龍安寺), Kyoto Japan, during cherry blossom season -- Rock Garden Ryouanji Temple (龍安寺), Kyoto Japan, April 2013 -- Ryouanji Temple (龍安寺) -- Copyright 2013 Jeffrey Friedl, http://regex.info/blog/ -- This photo is licensed to the public under the Creative Commons Attribution-NonCommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/ (non-commercial use is freely allowed if proper attribution is given, including a link back to this page on http://regex.info/ when used online)
Nikon D4 + Nikkor 14-24mm f/2.8 @ 14mm — 1/100 sec, f/11, ISO 100 — map & image datanearby photos
Rock Garden
Ryouanji Temple (龍安寺), Kyoto Japan, April 2013
Desktop-Background Versions
1280×800  ·  1680×1050  ·  1920×1200  ·  2560×1600  ·  2880×1800

Just how much benefit the clearing-out provides remains an open question. It's obviously better to have a smaller catalog file, though I haven't run any quantitative tests to put a number to it as I have in some earlier write-ups (such as I did in An Analysis of Lightroom JPEG Export Quality Settings and A Qualitative Analysis of NEF Compression).

A reduction in catalog size clearly makes backups smaller and faster, and you are backing up your Lightroom catalog, right? It contains a lot of data that would be painful if not impossible to recreate if you lost it. But one assumes a smaller file makes any of its database access quicker, and so day-to-day interaction with those aspects of Lightroom should, one reasons, be faster

The procedure is easy, but before doing it, it's perhaps interesting to know the size of your catalog file before flushing edit-history data, as a point of comparison with after. To check the file size, first optimize the catalog (in Lightroom, invoke File > Optimize Catalog..., exit Lightroom, then check the size of the *.lrcat catalog file using Finder or Explorer.

Then make a backup of the catalog file, just in case. You can do so by zipping up a copy in Finder or Explorer, or you can use Lightroom's backup mechanism, though the latter can be excruciatingly slow for large catalogs.

Searching for Improved Performance Toji Temple (東寺), Kyoto Japan, April 2013 -- Toji Temple (東寺) -- Copyright 2013 Jeffrey Friedl, http://regex.info/blog/ -- This photo is licensed to the public under the Creative Commons Attribution-NonCommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/ (non-commercial use is freely allowed if proper attribution is given, including a link back to this page on http://regex.info/ when used online)
Nikon D4 + Voigtländer 125mm f/2.5 — 1/800 sec, f/2.5, ISO 140 — map & image datanearby photos
Searching for Improved Performance
Toji Temple (東寺), Kyoto Japan, April 2013

Now to flush the edit-history data. Back in Lightroom, select the images that you want to do this to. I selected everything, but you can select a subset. You might select all but recent photos, for example.

Then enter the Develop module, and invoke Develop > Clear History... menu item, and if you've selected more than one image, you'll be presented with a dialog:

Choose Selected Photos.

You then have to wait a while for the process to complete. Lightroom gives no indication that anything is going on under the hood, but if you have a CPU-activity monitor running, you'll see it working for a while. It's unfortunate that Lightroom doesn't expose this background task's progress so you can know for sure when it's done. If you don't have a CPU-activity monitor, I'll blindly suggest waiting at least a minute for every 25,000 items selected.

You can use Lightroom while this is going on, or just let it sit.

Once you're ready to see the results, again invoke File > Optimize Catalog..., exit Lightroom, and inspect the new size.

Poof your catalog file might be much smaller now -- Copyright 2013 Jeffrey Friedl, http://regex.info/blog/ -- This photo is licensed to the public under the Creative Commons Attribution-NonCommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/ (non-commercial use is freely allowed if proper attribution is given, including a link back to this page on http://regex.info/ when used online)
Nikon D4 + Sigma “Bigma” 50-500mm OS @ 500mm — 1/2000 sec, f/6.3, ISO 1000 — image data
Poof
your catalog file might be much smaller now

Mine reduced to less than half of what it had been, but your results will depend on your own editing history, of course. Lots of local-correction brush strokes, for example, makes for much larger history data, so when clearing it out, for much more savings.

If you chose to do this, how much savings for how large a catalog did you get?

desktop background image of cherry blossoms in Japan -- Cherry Blossoms a nice way to end any post -- Nagaoka Tenmangu Shrine (長岡天満宮) -- Nagaokakyo, Kyoto, Japan -- Copyright 2014 Jeffrey Friedl, http://regex.info/blog/ -- This photo is licensed to the public under the Creative Commons Attribution-NonCommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/ (non-commercial use is freely allowed if proper attribution is given, including a link back to this page on http://regex.info/ when used online)
Nikon D4 + Voigtländer 125mm f/2.5 — 1/800 sec, f/2.5, ISO 160 — map & image datanearby photos
Cherry Blossoms
a nice way to end any post
Desktop-Background Versions
1280×800  ·  1680×1050  ·  1920×1200  ·  2560×1600  ·  2880×1800

All 13 comments so far, oldest first...

While my LR catalogue is still small enough that I won’t be clearing out editing history, thanks for pointing out your “Snapshot on Export” plugin.

— comment by Hajime on June 27th, 2014 at 9:02am JST (2 years, 9 months ago) comment permalink

Thank you for this great tip.

In my case I’ve started with a catalog of 3.89 GB and deleted the History for just the photos unedited in the last 2 weeks (55k photos). Final result: 1.84 GB

Hopefully this will also translate in better Lightroom speed… 🙂

— comment by Gabriel on June 27th, 2014 at 5:50pm JST (2 years, 9 months ago) comment permalink

Jeff,

Thanks for this post. Can’t believe that the History log is occupying so much space.

Just want to share my stats for everyone’s reference:

My LR catalog has 130,743 photos, of which only 22,262 (17%) have Develop edits (me lazy boy!)
Catalog file size (optimized): Before = 1,396MB ; After = 1,080MB (dropped by 23%)
It took approx 38secs to delete the History log (timed by a stop watch and looking at the CPU activity meter). I run my LR catalog on an old SSD.

And, your plugins are really really useful!

— comment by Bosco on June 28th, 2014 at 12:07am JST (2 years, 9 months ago) comment permalink

Jeffrey,
Thanks for this useful tips. In my case my LR Cat has 50.687 photos and was 980kB in size. After deleting history, it’s now 780kB.
And thanks for your wonderful plugin. The only one I’m still expecting is a plugin allow us to see the focus point selected when the picture was taken. (some guy started to develop it but was never released http://luc.lakephoto.org/journal/viewfinder-plugin/ ).
Best Regards
Alain

I probably won’t do such a plugin because I just don’t see the value in knowing what focus points were active when focus was locked, because that may have nothing to do with the scene in the final image (because it was made not when the focus points were locked, but when the shutter was actually released; the camera and/or the subject may well have moved in between). I think you’re fooling yourself to think there’s much data there that you can rely on. —Jeffrey

— comment by AlainD on June 29th, 2014 at 3:57pm JST (2 years, 9 months ago) comment permalink

Hi Jeffrey,

great tipp, but I like my history. Therefor I use a plug-in from Rob Cole called “Dev History Editor”. This plug-in deletes repetitive development steps (e.g. multiple brush strokes, on some images, I have hundreds of them!). You can also delete history steps manually. Here the results for my catalog (50000 images, 15000 with ajustments):
original: 3.125 GB
Dev hist: 1.933 GB
Del. hist: 1.227 GB

— comment by Thorsten on July 13th, 2014 at 4:56am JST (2 years, 9 months ago) comment permalink

Jeffrey,

Just discovered your blog as I was researching nuances of the Lightroom catalog. I’ll spend much more time here. Thus far I’ve not seen any mention of a point that really puzzles me. I have LR5 loaded on my SSD boot drive. I keep my photos on a traditional HDD and I’ve directed LR to put the catalog backups on that same HDD. I recently noticed that my backup.lrcat is MUCH bigger than the “active” catalog the program automatically stores on my SSD (C:\Users\GWH\Pictures\Lightroom\Lightroom 5 Catalog.lrcat). Have you observed this and/or do you have any thoughts why there is so much difference. Both files have the .lrcat extension, but the backup is 100 times larger.

Thanks for any thoughts on this.

Regards from Washington, DC

They should be the same size… that’s really strange. I’d contact Adobe with specific details. —Jeffrey

— comment by GWH on August 18th, 2014 at 2:25am JST (2 years, 7 months ago) comment permalink

Just want to drop by with my results. I have Lightroom CC, been using lightroom since 2006. Have 600,000+ images in my catalog. After cleaning history from ALL of my photos, since i don’t ever use history. my catalog went from 12.2GB to 6.5GB. Almost Half! nice work!

— comment by Nick Collura on May 1st, 2015 at 12:07am JST (1 year, 11 months ago) comment permalink

Thanks, Jeffrey! I had noticed my catalog swell in size and optimization time recently, when I did heavy touch-ups on one album of scans of dirty negatives. With clearing history of just those 174 images (not the other 30,700 images with only light editing), the catalog went from 4.5 GB to 550 MB! Ah, that’s better.

— comment by Wallace White on September 20th, 2015 at 7:10am JST (1 year, 6 months ago) comment permalink

Hi Jeffrey,

I came across your catalog blog while doing some lower level Lightroom research and even though I am using LRCC/6 I decided to try the history deletion. My Catalog was 32 GB for 166,000 photos. After I deleted the history it dropped to 4 GB. Amazing.

I did take your suggestion and turned on the task manager performance tab to watch the purge. I noticed one thing you did not mention or I missed it. Lightroom appears to stuff all the photos into memeory while deleting the history. I recently built a new computer (you may remember my asking some questions about maximum memory and performance) with 64 GB of memory. During the purge Lightroom continued to stuff data into memory until 37.9 GB was showing as used. I waited 15 minutes and until memory did not increase any longer and then I optimized the catalog. Even after the catalog showed that it dropped from 32 GB to 4 GB Lightroom did not release the memory until I exited. After reopening Lightroom memory looked usage normal.

This looks like a bug to me. Have you seen anything like this?

Thank you for the great blogs,

Jim from Germany

Yeah, it does seem like a bug, though I’m not familiar enough with Windows memory management to know for sure. You might report it to Adobe, but it seems to be a small thing since restarting Lightroom after the optimization (to check the catalog size) seems natural, so the memory leak won’t really bother much. —Jeffrey

— comment by James Palik on June 13th, 2016 at 2:08am JST (9 months, 18 days ago) comment permalink

Hi Jeffrey,

My catalog file was 4.41GB in size. After cleaning up the history it reduced to 1GB in size. Great improvement. With only 33566 photo’s it took only a couple of minutes to perform the history cleanup.

Like Jim from Germany mentioned, Lightroom CC release 2015.6 kept using a lot of CPU even after the cleanup. I had to restart Lightroom to go back to normal operation.

A tip I like to mention is related to backing up the Lightroom catalog upon exit. During the backup process, my catalog of 1GB is compressed by Lightroom in a ZIP of 198MB. This my seems great but with KeKa my 1GB catalog can be compressed in 7-Zip format with maximum compression down to 98MB. On my MacBook Pro (mid 2010, 2.66GHz i7, 8GB RAM) it took 5-7 minutes. When performing a regular compression using OS X Finder Compress, it will be compressed to a ZIP of 157MB.

Using KeKa compression in 7-Zip format seems the way to go to minimize disk space on catalog backups. Unfortunately, there is no option in Lightroom to disable compression during the backup process when exit Lightroom. I have to unpack the Zip created by Lightroom and recompress it with KeKa (or compress the catalog file directly 😉

With greetings from The Netherlands,

Dennis

Perhaps suggest a re-compress feature to Lr/Backup? —Jeffrey

— comment by Dennis on June 15th, 2016 at 5:43am JST (9 months, 16 days ago) comment permalink

Oh my,
I was close to give up. Following your suggestion brought the catalogue down from 8,6 GB to 1,2 GB for 22500 pictures.
Thanks, Lars

— comment by Lars on July 25th, 2016 at 6:38am JST (8 months, 6 days ago) comment permalink

Wow, great suggestion! By clearing history of all photos not touched for over two months my main lrcat went from 895MB down to 440MB, less than half as big!
Thanks!

— comment by John Johnson on September 4th, 2016 at 8:58am JST (6 months, 27 days ago) comment permalink

The size of my Lightroom catalog went from 6.71 GB to 3,33 GB after purging the history.
That sounds like a success !

There is no immediate speed difference to be noted though 😉 (which is what I was trying to achieve … )

Cheers,
Christiaan

— comment by Christiaan Hofland on December 6th, 2016 at 5:34pm JST (3 months, 24 days ago) comment permalink
Leave a comment...


All comments are invisible to others until Jeffrey approves them.

Please mention what part of the world you're writing from, if you don't mind. It's always interesting to see where people are visiting from.


You can use basic HTML; be sure to close tags properly.

Subscribe without commenting