icon next to them have been artificially shrunk to better fit your screen; click the icon to restore them, in place, to their regular size.
Nikon D700 + Voigtländer 125mm f/2.5 — 1/400 sec, f/4, ISO 800 — map & image data — nearby photos
Be My Guest...
Koumyou-in Temple (光明院), Kyoto Japan
So, Adobe has released the first official update for Lightroom 4, Lightroom 4.1. The release announcement on Adobe's Lightroom Journal blog lists the major changes, bug fixes, and new camera support (including recent heavy hitters like the Nikon's D4 and D800, and Canon's 1D X and 5D Mark III). However, the release notes don't even mention two changes that will be welcome and important for many users, so I want to tell you a bit about them here.
(The photos on this post, from my first visit earlier today to the small but elegant Koumyou-in Temple (光明院) in eastern Kyoto, have nothing to do with the prose of the post. They're just pretty pictures.)
Nikon D700 + Nikkor 24mm f/1.4 — 1/400 sec, f/10, ISO 2000 — map & image data — nearby photos
Japanese Aesthetic
The Problem
Upload Progress in Lr4.0 and earlier
Historically in Lightroom, when uploading one's photos to an online service such as Facebook, Flickr, etc., one sees a little progress bar in the upper left that chunks along with each picture, such as the one shown at right. With my plugins, the progress bar makes two steps for each image, one after Lightroom has rendered the copy to be uploaded, and another after the copy has actually been uploaded. However, there's no movement to the progress bar while the image is actually being rendered, and while the image is actually being uploaded. That's just how Lightroom has always worked.
So, during these potentially time-consuming processes, there's no indication that anything is happening. This is all the more regrettable when working with large videos, as they can take quite a long time to render and upload.
But even worse, the way Lightroom worked internally, large uploads were extremely inefficient. This was perhaps fine for most folks' images, but became a real problem with videos with sizes stretching into the hundreds of megabytes. On my machine, Lightroom would simply crash if I tried to upload a video weighing in at more than 30 or 40 megabytes.
Nikon D700 + Voigtländer 125mm f/2.5 — 1/125 sec, f/2.5, ISO 6400 — map & image data — nearby photos
Built-in Navigation
this guy's socks reminded me of this photo on my recent post about my wife's ballet shoes
(Update some years later: the guy I met that day with the socks turned out to become a good friend)
The Solution
I made a suggestion to Adobe about how to allow plugin uploads to be more efficient, and they added this support in Lightroom 4.1. The result is extremely efficient, and plugins that make use of this support can now upload without file-size limit, at least as far as Lightroom is concerned. If your network and upload destination can handle a terabyte video, so can Lightroom.
While testing this stuff with a sneak-peek version of Lightroom that Adobe kindly provided to me last week, I realized that one of the by-products of using this new internal support was that I could have the plugin update the progress bar in real time while the upload progressed.
Upload Progress in Lr4.1
In the progress bar at right, notice the tiny little “7%” that's been added? That's the only difference from before, but it's huge from a user-interaction point of view. Now while the upload is happening, you can watch that percent-complete value march up, and you can see the progress bar creep toward completion. It's such a small thing that makes a surprisingly big difference: you no longer have to wonder whether the thing is stuck or not, waiting for the progress bar to jump once the upload has completed.
Nikon D700 + Nikkor 50mm f/1.4 — 1/400 sec, f/1.4, ISO 200 — map & image data — nearby photos
Contemplation
Koumyou-in Temple (光明院), Kyoto Japan
The Joy
I spent the majority of the weekend working on this. It required writing a bunch of new code to use the new internal upload-handling stuff, and then retooling the plugins to work it in. It wasn't always smooth: I filed four separate bugs with Adobe on things related to this.
Sometimes I make murmurs about what I don't like about plugin development, most recently in “Ongoing Pitfalls of My Lightroom Plugin Development”, but also over the years in posts like “What To Do When a Hobby Becomes Work” and the most-unfortunate “Breakdown: When Good Intentions Don’t Scale”. But the days I spent wrestling with this were a joy because I was back getting my hands dirty with complex code that ended up actually working (I was able to work around most of the bugs), and the first time I tested a large upload and actually watched the progress bar slowly progress from 1% to 100% was positively giddy.
Positively giddy.
I don't expect many folks to understand that, but it was gratifying both as a user of Lightroom (the whole “no longer need to wonder” thing), and as a developer because I overcame obstacles to solve the real problem of handling large uploads. Before, some users would have trouble at 10-megabyte uploads, but now, 10 gigabytes would be a breeze.
Nikon D700 + Voigtländer 125mm f/2.5 — 1/400 sec, f/2.5, ISO 1800 — map & image data — nearby photos
Dainty Propeller
So far I've pushed this change only in my uploader for PicasaWeb, which is also used a lot by folks for G+.
(UPDATE: I've now pushed this update for my Flickr, SmugMug, Zenfolio, Facebook, and Photobucket plugins as well.)
Google currently allows videos up to 100 megabytes, and I tested right up to the limit without problems. (Note: PicasaWeb specs note a maximum video of 1GB, but that's only when uploaded via Google's own Picasa desktop software; third-party apps are limited to 100MB.)
There is one caveat about this new support in the PicasaWeb plugin: due to one bug that I was not able to work around, the new support is used only for initial uploads, and not for image replacements (such as when republishing changes). This is unfortunate, but not as dire as it would be if Google allowed videos to be replaced. Google does not allow videos to be replaced, so the plugin continues to quietly ignore video updates.
I'll be pushing support in my other plugins soon, but I wanted to write this post right away to share a small slice of my geeky photographer plugin-developer joy.
Nikon D700 + Voigtländer 125mm f/2.5 — 1/400 sec, f/2.5, ISO 320 — map & image data — nearby photos
Alone Together
Another Important Lr4.1 Update: Respecting “Private” Locations
Lightroom 4's new map module lets you put a circle around an area and assign it a name and, optionally, mark that this location is “private”. When you export images located within that private location, the geoencoded coordinates are removed from the exported copy, thus, in theory, preserving your privacy.
However, with some photo-hosting sites (PicasaWeb, SmugMug, and Flickr come to mind), the uploading app explicitly tells the site the image's map coordinates while uploading, and herein lies the problem: in Lr4.0, a plugin could always see a photo's location, but had no idea whether the user had marked it as “private”. Thus, private locations were removed from the copy uploaded, but the location was still sent along with the image to be displayed to the world. So much for “private”.
Lr4.1 includes infrastructure support that allows a plugin to inspect whether a photo's location is “private”, so, my plugins will be updated soon to support this. The PicasaWeb plugin already does, sporting new options to control if/when location data is sent to Google.
Nikon D700 + Voigtländer 125mm f/2.5 — 1/400 sec, f/2.5, ISO 640 — map & image data — nearby photos
Teamwork
The percentage indicator is very welcome! I’ve recently purchased a used D3s and the first (and only) time I uploaded a video on Picasa I thought your plugin did not supported it… I was (strangely) patient enough to understand that it was more a rendering/upload problem rather than a fault in your plugin. I’ll test the new indicator right away 😉
The render step is unchanged… just like the user, the plugin must just sit and wait, so no progress bar there. It’s the upload (which could potentially take much longer than the render) that is now reported in smooth, real-time detail. —Jeffrey
For what it’s worth I, at least, totally understand that giddyness. Sometimes I think the most depressing thing about doing software development is that the most gratifying things you do are somewhere between invisible and taken for granted by the end user. “You made it as efficient as possible in doing that thing? Well sure, why wouldn’t you?” Nobody knows our pain!
Okay maybe that last bit was a bit over the top. But I get you; those little touches are what really makes stuff like your plugins a pleasure and they’re well appreciated.
Now where’s your 500px plugin? 🙂
Wow, you are fast to adapt when you ask for comments:) I see the option for the size of the pictures. Thanks for sharing this, I know how you must be proud of you. I use technology to help people with disability and when I can get around a problem I’m pretty proud of the finding. I wish I had your knowledge of technology to get around stuff (but I’m not willing to learn and put all the hard work you must have put in it).
It’s a real pleasure to read your blog. I wish you a wonderful day.
I’m from Québec, Canada.
Dear Jeffrey Friedl,
I want to say thank you for this post and for all of your hard work. There are two things here that really please me:
1. I love the way that you push Lightroom and the engineering team further. Your personal quest to make this software better and better plus your willingness to share the results is awesome. I haven’t tried using Lightroom to upload large video files but I have encountered the “frozen progress bar” trouble when uploading large blocks of images to Flickr, Picasssa, etc. So a personal thanks for tackling this challenge and helping me out.
2. I think that you and the Adobe engineering team deserve a big thanks from all of us for your willingness to pour endless hours into making our lives easier. It’s great to know that some of your API requests and suggestions do get filled eventually! Better uploads via Lightroom plugins probably won’t help Adobe’s bottom line this quarter but its great to know that there are folks like you who care and who try!
Best regards,
—
David Marx
What a fun post. Nerd excitement is some of the best there is to witness. I’ve always enjoyed NASA and other engineers and scientists’ moments of success. Thanks.
I’m currently living in Colorado, just North of Boulder, but I was born in Tokyo in 1967.
Congrats! Geeky joy is perfectly fine with me… 😉
Mike.
Hi Jeffrey,
Nice blog post and thank you for keeping us posted on the behind the scene work!
Just wondering about the private data handling. It is kind of cute if a flag indicates that GPS data is private, but if a plugin doesn’t respect this flag… Shouldn’t Lr asked the user if a plugin may access the private location data like it is done in a smart phone? Just wondering… Anyway, this is something for Adobe to think about…
Keep up the good work!
– Daniel
I understand what you’re saying, but since the plugin has complete reign to inspect/delete any file on your entire system (any file, at least, that the user running Lightroom can access), little gates like you suggest aren’t really very meaningful. Plugins are full partners with Lightroom, limited only by what hooks Adobe has had the resources to build. (Those limitations are large, unfortunately, but they’re borne from a lack of manpower at Adobe). —Jeffrey
No giddyness over the release of the SDK? That’s my source of giddyness tonight. 😉
Also, with regards to the changes to the location support, I’m inclined to agree with Daniel – If a location is private, Lightroom shouldn’t really be giving the plugin a private location by default. I’d want plugin developers to have to explicitly request it (and implicitly acknowledge that the user’s requested that the location is private) rather than blindly take all the info possible.
Of course, it’s entirely possible to get around as you mention (extract data from the XMP sidecar, and so on), but that would be kludgey, and likely prone to breakage, which given that I’d want to make it hard for a plugin to get at private location data, is possibly a good thing.
By all means, submit a request to Adobe about it. Privacy issues are always given high priority, so they may well listen. I’ll also mention the concern to my contacts. —Jeffrey
Wonderful work Jeff. I am wondering if I missed it or not. do any of your Plugins work for
Google +?
Thanks,
John
There’s no specific G+ API yet, so the best we can do at this point is to upload to Google’s PicasaWeb (sometimes referred to by Google as “Google Photos”). David Marx made a series of videos detailing how to do this with my PicasaWeb plugin. —Jeffrey