The Voodoo of Elevation Gain and Strava (and How I Get Around It)

A Light Mountain Climb After Lunch
340m climb from the village of Kurodani, Takashima City, Shiga Japan
高島市黒谷からの山登り

In the screenshot of Google Earth above, you can see the route that I rode my bicycle last Wednesday... the little bit of green coming in from the right is a downhill section of road, followed by the long red uphill section through the tiered rice paddies of the village, then up the mountain on an old road no longer open to traffic.

I'd ridden 55km (34mi) to get to that point, and would end up riding 72km (44mi) farther before I got home, but ever since my first real ride in February, I've not been so interested in how much distance I've covered, but rather, in how much mountain I've conquered. This means the important statistic for me is how much vertical climb or elevation gain I did during a ride, the sum of all the rises/climbs/ascents on the route.

And it's not just me... vertical climb is often used as a badge of achievement for a ride, and seems to get more talk than distance. It's a strong reflection for how tough a route is.

The problem is that it's difficult to figure out the vertical climb for a particular route, and commonly-used methods are often wildly inaccurate.

Cyclers I've met record the ride with a GPS unit of some sort (their iPhone, a bike computer, a runner's watch, etc.) and upload the data to Strava, a service that allows you to analyze your activity and share info about it with others. It's very popular and mostly very well done.

For example, I mentioned it in my previous post about a long mountain bikeride that I did on April 30th, where Strava had informed me that I had beaten my record for a particularly tough mountain climb:


Data About My Rides
Courtesy of Strava.com

Footnote: I've since beat it again, but by only 10 seconds.

When you upload your data to Strava, they calculate your vertical climb for the activity, and display it prominently. For example, the aforementioned first real mountain ride shows some impressive stats:

Since it was my first real ride, I didn't know any better but to be impressed by the 3.3 kilometers (2 miles!) of vertical climb that I'd done. I was amazing. I was Superman.

Unfortunately, it's all a lie.

Over the course of several big mountain rides, I'd started to feel that the numbers reported on Strava were quite a bit inflated, but I couldn't put my finger on exactly how.

A week or so ago I posed about another ride, Short but Intense Bike Ride in Northern Kyoto, where someone commented about the apparent discrepancy between reality and what Strava reported (and hence what I claimed). This spurred me into investigating, and I've spent the last week digging in, writing tracklog-analysis software, doing tests, etc.

So, when I went out on a long ride last Wednesday, I brought an arsenal of equipment with me, recording five separate logs with three different GPS/GLONASS-equipped devices.

The trip ended up being my longest ride ever, about 125km (78mi), and also had the most vertical climb I'd ever done in one day. Among the many ways I recorded the trip's data, I used Strava's mobile app on my iPhone 6+, and when I was done I had the app upload all the data to Strava.

Here's what Strava reported about that trip:


Impressive!
but wrong

The elevation graph at the bottom is an accurate representation of the 11-hour ride, with four separate major mountain peaks transversed, along with many little ones (though they each didn't seem little at the time!).

Strava informs me that the trip encompassed an impressive 3,627m (11,900') of vertical climb. It looks like I started at about 50m and initially climbed to about 500m, and from there went down and up quite a bit — and believe me, I did — but looking at the graph it's difficult to come up with a 3,637m sum for all the up parts.

In reality, I believe the vertical climb on the trip was about 2,225m (7,300').

First, let's look at why Strava's numbers are so wrong, then let's look at how to get a better answer.

There are various ways a unit can record your elevation as you travel, and all of them are generally untrustworthy. Modern units like an iPhone or a dedicated unit like a Bad Elf GPS Pro+ have three methods to figure altitude: GPS, GLONASS, and a barometric altimeter.

GPS/GLONASS can be fairly accurate with your latitude and longitude, as I've discussed in prior posts, such as Informal GPS Logger Test: iPhone 4s GPS is Shockingly Good and Another Informal Location-Logger Test. It's fantastically-amazing technology that is perhaps the only time we come in contact with something that must actually take into account Einstein's general theory of relativity. But it's not perfect, and of the three dimensions, elevation is by far the least accurate.

Consider these views of three GPS/GLONASS units' tracklogs plotted in Google Earth, of various sections of last Wednesday's ride:

The three units rarely agreed with each other on the elevation, nor any of them with Google data as seen in Google Earth, and they each in their own way tended to wander off wildly at times. (The tracks, when viewed directly from above, all matched up with each other and the road quite well... it's only the elevation data that was, so to speak, all over the map.)

When wishing to track vertical climb, the absolute accuracy of a unit is not particularly important, it's the relative accuracy and stability that are important, and it seems that barometric altimeters provide better results.

My first experience with a barometric altimeter, on a Garmin GPS unit a decade ago, was very bad. If there was a breeze or you walked with the unit, it would record a non-trivial change in elevation (up or down, depending on how the unit was physically angled relative to the air movement). It was ridiculous.

Things seem better now. On last Wednesday's trip, I had two barometric altimeters with me, the one in my iPhone 6+, and another on a Bad Elf GPS+. Here's their plots for the last section seen above:

Anyway, a naïve calculation of vertical climb looks at each data point and, if the elevation of a point is higher than the previous point, add the difference to the total. Minor errors that result in small fluctuations over the short term result in wildly-inflated overall results, but one can smooth the raw data (e.g. with a moving average) to get more realistic results. That's apparently what Google Earth does, so let's look at what it calculates for each of my five tracklogs:

UnitTypeGoogle-Earth Vertical-Climb Calculation
BadElf GPS+ GPS/GLONASS3,479m11,414'
BadElf GPS+ Barometric 2,174m7,133'
Garmin eTrex 20GPS/GLONASS3,265m10,712'
Apple iPhone 6+GPS/GLONASS2,215m7,267'
Apple iPhone 6+Barometric 2,009m6,591'

That's quite a range of results from the very same trip. Which is correct? Frankly, probably none of them.

As we'll see later, there are techniques we can use to eke out some fairly reasonable results, but first let's see what Strava actually does...

What Strava Does

Strava's iPhone app has access to both iPhone altimeters (barometric and GPS/GLONASS), but in the end it uses neither. Rather, it throws away that elevation data and instead relies on its own database of elevation data (likely the same data that Google uses to show 3D elevation in Google Earth).

Strava's approach may well be fine in generally-flat areas, but it produces ridiculously-overinflated results in the mountainous areas I've checked, and it's easy to see why...

I created some software to analyze the tracklogs, smooth local irregularities, and present uphill sections in red and downhill sections in green. Let's look at the mountain shown in this article's lead photo, a long climb followed by a long, enjoyable descent down the other side...

Reality red means uphill, green means downhill
Reality
red means uphill, green means downhill

Both the climb and the descent were quite straightfoward; the climb was a solid climb without any downhill breaks, and the descent was a solid descent without any uphill sections. That's why the ascent is a solid red line above, and the descent is a solid green line.

Now, let's look at how Strava sees it, as well as all my data sources...

from Strava's elevation data
from Strava's elevation data
Strava  -  BE (baro)  -  Garmin (GPS)  -  BE (gps)  -  iPhone (GPS)  -  iPhone (baro)

mouseover a button to see that image

The Strava data shows many ups and downs on both sides of the mountain... ups and downs that simply weren't there. It's easy to see why when you look at the mountain in Google Earth (which I believe uses the same elevation data) from the side. Here's the side of the mountain that I rode down:


very-3D mountain pressed into a mostly-2D face

The road appears undulate up and down as it makes overall progresses lower, and to Strava, the ups count as vertical climb:


Strava detects many false climbs (216m/700' just in this view alone)

Of course, the road doesn't undulate up and down (it progresses monotonically down), but it does undulate side to side around ravines cut into the side of the mountain by eons of water and wind. You can sort of see the proper undulation in this path created with a barometric altimeter:


descending around ravines that do exist, but which aren't presented in Google Earth

I should be clear that I'm not complaining about Google... the resources they've freely gifted to the world in Google Earth and Google Maps continues to amaze me, and I'm extremely thankful for them.

I'm not quite sure that I'm complaining about Strava. Perhaps these easily-avoidable inflated results are particular to my area of Japan, and the rest of the world enjoys added accuracy due to Strava's policies. I don't know. But I do know that a lot of folks around the world complain about Strava's inflated elevation claims, so at least I know that I'm not alone.

So How Can We Do Better

First of all, I should note that Strava does respect the recorded elevation in some situations where it recognizes that the data was recorded with a barometric altimeter. Their own iPhone app, however, either doesn't use the barometric altimeter or doesn't notice that it does, so users get stuck with Strava's wild elevation data.

Where Strava does accept a device's elevation data, they offer the user a way to get corrected data, whereby Strava throws away the user's data and replaces it with their inaccurate elevation-database data. This results in a much less accurate result for many folks, but they offer no way to undo this action, so if you bump the link by accident, you're hosed.

They also don't allow you to toggle between your device elevation data (barometric or not) and their data. It seems that they really want to push their data, and who can blame them? Folks like me early on, ignorant of the hyper-inflated vertical-climb data, get an ego boost from seeing the big numbers. I suppose boosted egos help their business interests.

(I'm thankful for their service, which I have enjoyed without cost. I just wish this aspect were better.)

(Update: Strava has a lot of problems as a company in how they respect their users, but in the end I appreciate the tools that do seem to work, so I've signed up as a paying customer.)

Luckily, one crack in their anti-reality armor is that they allow geeky types like me to create tracklogs with other devices, then modify the tracklog to force Strava to accept the elevation data as is, and upload that.

Massaging Tracklog Data

I've spent the last few days writing software to analyze tracklog data, to smooth it out, and to derive meaningful information from it.

The first step is to identify when you're stopped, because if you're not moving, you should have no elevation gain or loss, no speed... nothing. You're stopped.

Unfortunately, GPS receivers tend to record drifting locations when stationary, such as the hour I spent visiting someone (this sweet lady) in the mountains along the way on Wednesday's long ride:


this is what an hour's complete lack of movement looks like

My analysis figured out that all that movement didn't really get me anywhere, so it filters it out and marks the stopped location with a little blue ring:


pause detected

I have my software note the length of the pause in the KML label, like this:

My pause detection seems to be highly accurate, and I compared its results with the reality that I remembered on the trip, and it's pretty much perfect. It even accurately detected a 10-second pause at an intersection waiting for a few cars to clear before I crossed over:


10-second pause for traffic

45-minute pause wandering in and around a convenience store
while waiting for a friend to join me on the trip

Then my software goes through the entire trip to find the highest peak and the lowest valley, and pin the elevation at those locations. That divides the whole trip into four parts, each of which is subject to the same analysis, and this analysis recurses (repeats onto itself) over and over until the difference between a local peak and a local valley is almost within the range of noise.

Once all these local peaks and valleys are found, the elevation of all the points in between are smoothed using a moving average. The elevation of each peak/valley point is left undisturbed, and it's the elevation rise between them that adds up to the trip's vertical climb.

After all this analysis, the software prepares a KML file that shows the trip, with red lines for climbs, green for descents, and white for flat sections (and, optionally, the blue circles for pauses). Here's one view of the second half of my trip:


Looking North up the eastern side of Lake Biwa

The red line at the far right ends at the Shirahige Shrine (seen here in the report on a ride two weeks ago), which we visited briefly after having come over the mountain, before turning south to ride home.

The distance from the shrine in the background to the spot in the foreground where the route turns 90 degrees and pitches up into the mountains is 30km (18mi) as the crow flies, which I mention for scale.

Here's the whole 125km (78mi) trip:

If you'd like to see it yourself, you can download the KML file (with pauses / without pauses) to view in Google Earth.

The with pauses view is best for me when I want to relive the experience, and I can see, for example, that I stopped for half an hour — 31:26 — to eat lunch (at this place that I last visited seven years ago), and then progressed up the mountain stopping at intervals for photos, to check directions, or to fix a popped chain.

That view is pretty cluttered if all I want to do is analyze whether the vertical-climb data seems reasonable, so for that the without pauses version is better:

The duration includes all the intervening pauses, rendering it fairly meaningless, so I should probably simply remove it, but at the moment it's still there.

Anyway, I did this analysis on all my tracklogs, and found that the one produced by the Bad Elf GPS+ with barometric altimeter seemed to most match reality, so that's the one I consider, relatively speaking, correct.

And after all this analysis and cleaning and smoothing, I'm left with a tracklog that I (relatively speaking) trust, so I prepare it so that Strava won't override the elevation data, and upload it. Strava doesn't know that I've grooomed it, so it does its standard processing, and comes up with this:

They somehow came up with an extra 100m of vertical gain, but since I truly don't know what the actual value is, I can't say that they're any less wrong than I am.

So, now that I have the ability to derive a realistic vertical-climb value, to be honest with myself I must go back and correct prior trips:


Date

Event
Vertical Gain
Claimed at the Time
Vertical Gain
Realistic Value
Strava
Inflation
(in feet)
Jan 24 My first tentative outing on bike N/A 685m             → 2,090'
Feb 7 My first real mountain ride 3,850m 1,517m 274% 12,630' → 4,620'
March 3 Kuyataki Waterfall with Manseki 762m 540m 141% 2,500' → 1,650'
March 4 Group ride southeast of Kyoto 1,787m 1,282m 140% 5,860' → 3,910'
April 16 Highly Visible mountain ride 1,607m 1,180m 136% 5,270' → 3,600'
April 23 Pleasant 105km lake ride 1,300m ? 4,200' →          
April 29 Coffee at Cowbell 1,720m 991m 174% 5,600' → 3,020'
April 30 Revisiting my first big mountain ride 3,159m 1,609m 196% 10,360' → 4,900'

I didn't have a good barometric-altimeter tracklog for one of the rides.

The amount of Strava inflation is shocking. If I could solve this problem for myself in a couple of days, you'd think a real company with presumably smart folks could come up with something.

Anyway, I'll need to find a better app for keeping tracklogs on the iPhone, one that lets me record both the GPS/GLONASS elevation and the barometric elevation, and also my heartrate. As it is now, I'll use a klugy mixture of various devices and combine all the data myself, massage it, smooth it, and then send it to Strava.

If they ever get around to addressing this, I suppose that would be a good time to bring up the next issue: they ignore elevation changes when calculating distance. They correctly note that the difference is perhaps at most 2%, but on a 100km ride that can still be over a mile wrong.

Overall, I guess Strava is just not as anal about statistics as I am...

Continued here...


The 30 most-recent comments (out of 37; see all), most recent last...

From California:

1) is it possible you’d share some of the code you developed to do this analysis?

2) accurate GPS positioning requires satellites to be widely separated across the sky. In mountains, where you’d really like the altitude and lat/long to be most accurate, you’re least likely to get good solutions without some serious gear – gear you won’t be carrying on your bike for sure. Surveyor friends tell me good GPS antenna cost in the range of $10,000 which I find amazing.

Yes, I’ll be happy to share the code after I tidy it up a bit… it sort of grew organically (like fertilizer) over the course of a few days, so it’s not too pretty. My brother pointed me at Tribmle, who are apparently the makers of high-end professional stuff that governments use…. with a good antenna, good receiver, and some post processing, they can get millimeter horizontal accuracy, which is indeed as amazing as the price. Not sure about vertical accuracy. —Jeffrey

— comment by Adam Bridge on May 12th, 2015 at 11:24am JST (2 years, 6 months ago) comment permalink

Interesting stuff! 🙂

You could further weight the points with their error to get more accurate tracks (or, with a weight of zero, discard inaccurate points). GPS receivers can sometime give you a measure of that error. For Garmin receivers look for PGRME in the NMEA data. There’s also an interesting wiki about the H/VDOP GPS errors and how to evaluate them (some receivers must provide those estimates since it can be used by the command line tool GPSbabel)

— comment by Damien on May 14th, 2015 at 7:30pm JST (2 years, 6 months ago) comment permalink

I’m not sure what your barometric ascent would’ve been on your ride, but I noticed that threw off my ride ascent was that the STRM elevation data I was using with MyTourBook was not very detailed for Japan. The narrow passes block GPS reception and the resulting errors bounce over low resolution elevation maps giving us credit for riding a sawtooth shaped road.
There is a Japanese DEM database with detailed elevations but maybe Strava doesn’t use this and relies on world SRTM data instead? I didn’t have this problem in Canada where SRTM data is detailed or on large open mountain roads in Japan where resolution isn’t so important. Or is it mainly the GPS lat/lon errors that throw the ascent off?

In the Strava/Japan case, it seems to me that the problem I saw here was not my tracklog lat/lon, but rough/mismatched data on Strava’s side. If my tracks were inaccurate due to trees or narrow passages, that would only compound the error. There is quite accurate data for Japan (at least for Kyoto)… see “More GPS Cycling Tests“, which I’ve just posted.

I didn’t mention it there, but I took the best track I had from that short trip (from among nine tracks, all of which were fairly bad in actually tracking the road), and remapped the elevation from the Japanese government website mentioned on the post. The result was not the 700m that I believe is correct, but instead was 789m because it mapped the meandering path that the GPS recorded in the deep mountains, instead of magically mapping the path I physically took. So even if Strava were to use this great data, it faces an uphill battle in ignoring a user’s elevation data. —Jeffrey

— comment by Tomas Svab on May 15th, 2015 at 12:05am JST (2 years, 6 months ago) comment permalink

Great content. (ps: I came here from dcrainmaker).

So this is something we’ve had to deal with in SportTracks for a decade or so (much longer than Strava!). We’ve also got the benefit of being a hacker/toolkit approach where users can tweak stuff, including a API plugin architecture… much more than Strava which is a mass-market app that has to “just work”, out of the box.

There are two challenges to deal with: 1) Is the data reliable? 2) What is “considered” an ascent?

The solution we took in the PC app to address these challenges was twofold:

1. You’ve got the option to overwrite your recorded elevation data with SRTM data using the “Elevation Correction” plugin. You can swap between the two samples (eg: “undo” correction), and well… you can fully edit your elevation track too if you *REALLY* care. (I think you can even upload custom data elevation models – so you could use higher resolution sampling for Japan).

2. We only consider ascend / descend totals where the effort is continuously up or down. There are some minimum distance requirements for this that are hardcoded, but more importantly – we let the user CHOOSE what grade % is considered a “hill” – both for ascent and descent. They can literally go into their settings and say – only count ascents if the hill is over 1%, or 2%, or 5%.

Problem 1 is solvable – if sometimes requiring a lot of work with data sources.

Problem 2 is not really “solvable”. Consider an undulating route over rolling hills. Even if the data source is perfect, and each hill can be detected, at the end of the day we still need to decide – should each hill be counted separately and accumulate to the total ascent, or is the total ascent zero? Does it matter how big the hills are? Or only how steep they are? At the end of the day, the question is somewhat subjective to the goals of the user, right?

You’ve got somewhat obvious data – big, obvious hills – but A LOT of our user’s data doesn’t look like this. And so… well… it’s a bit subjective.

Interestingly in our webapp, which was completely new technology – so far we’ve not addressed this. We take exactly what the GPS records and show it – as/is. BUT!! Users are clamoring for elevation correction (it’s now floated to the #1 requested enhancement), and so… we’re going to tackle it soon. In light of everything you’ve pointed out, in light of what we’ve learned on our PC app the last 10 years, and the expectation that apps now should “just work” – we’ve got a bit of research to do. Throwing a ton of settings that the user can tweak and override like we did in our PC app is probably *not* the right solution.

I expect the ideal solution will incorporate knowledge about the sport (running and biking are different!), the sensor (GPS or baro?), and possibly some secret sauce which evaluates the effort you’re putting out in relation to the grade. This is where the magic happens – at the end of the day you should be thinking about hills because of how they effect the effort you’re putting out. If you’re just rolling over small ups/downs it’s really not a “hill” in any sense of the word. In theory this should be reflected in the speed/heartrate/power data, and we could use that as a signal to detect what are legitimate hills.

Thanks for listening. 🙂

Thanks for the thoughts. Problem 1 is a tough one, exacerbating Problem 2. I took the liberty of embedding the link to SportsTrack, since it seems directly relevant to the discussion at hand. —Jeffrey

— comment by Aaron on May 18th, 2015 at 5:13am JST (2 years, 6 months ago) comment permalink

Found this via DC Rainmaker. Very nice writeup and I would be interested it trying out your software if its every made available.

I noticed long ago that Strava had serious elevation problems. I thought it was mostly issues with bad XY data from the device. I though Strava plotted that XY data and calculated known elevation data from elevation maps. Therefore if you were riding a long side a hill or drop off and the GPS put you up on a hill or down in a ravine Strava would interpret it as you going up/down the hill. Overtime this would add a lot of vertical feet to a ride. Other issues are when you go over a bridge. Strava would use the elevation of the land, not the bridge. You could imagine what going over a huge ravine would do to elevation.

Strava has so many issues as a company right now, its ridiculous. I bet that even if you handed them the solution, they would refuse to fix them problem.

For Example: When I learned that my Samsung Galaxy SIII had a barometric sensor and could support ANT+ (via dongle). I asked about supporting both features on their Android app. Their rep had no idea that phones could support these features (although the Sony Xperia Active did it long before). An Android ANT+ support topic was started 3+years ago – the same time they announced ANT+ support on iOS. The difference between Android and iOS at the time? A handful of Android phones ACTUALLY had ANT+ chips but iPhones required a dongle. In three years someone could learn to program for Android and write a cycling app that support barometric and ANT+ sensors. Ifor Powel, an individual, created IpBike (also referenced many times in that support thread) which supported ANT+ and Barometric (and later BTLE) sensors so you would think that an actual company with an “Android development TEAM” would have no problems adding proper support. Strava’s repeated response was that it wasn’t on their road map. Which of course infuriated people. Then after three years of hearing the same BS, without any warning, they opened a beta for ANT+ support on Android. Seriously WTF?

I actually learned how to edit the .TCX file to spoof a Garmin EDGE 500 so Strava would use my phone’s elevation data. Later IpBike added native support to spoof the EDGE 500. I now own an EDGE 500 and fēnix2 so I don’t have to worry about Stava’s crappy algorithm or history of not listening to user advice. Barometric sensors do have their own issues though. Mostly with longer rides and change in temperature throughout the day but still better than erroneous GPS data.

– sorry about the long comment. Keep up the cool and interesting work.

— comment by Derick on May 18th, 2015 at 7:36am JST (2 years, 6 months ago) comment permalink

Awesome post Jeff!

I’m just wondering if you have found an app that actually records both the GPS/GLONASS elevation and the barometric elevation? I don’t need heart rate, but it would be great if it records that as well. I did a quick search on Google without any success. If you know or anyone know of such an app, please let me know!

It’s pretty disappointing that Runmeter doesn’t produce a 1hz track log. IIRC, I think that’s one of the better running apps for iOS. Do you have anymore details to provide about your conversation with the developer? I’m also interested in their Cyclemeter app, but I might avoid using it if that also doesn’t produce a 1hz track log. Will they eventually add the ability to create a 1hz log?

Runmeter and Cyclemeter are identical (except for the name/icon), and it’s unlikely they’ll add a 1Hz feature because they think they’re already recording as quickly as they can. Maybe they are, but are just throwing away points? I don’t know of any app that records both kinds of elevation, though the author of Geo Tagr was receptive to the idea. I don’t know whether iOS even allows it, though. —Jeffrey

— comment by JJ Lee on May 18th, 2015 at 7:57pm JST (2 years, 6 months ago) comment permalink

Thanks for the response. I don’t see a way to respond to your post so I’m going to have to create a new post.

I decided to contact the developer of Cyclemeter and expressed my concerns base on your findings. Hopefully this will create some pressure and get them to do something about it. Accuracy is very important to me!

So have you contacted a bunch of track log creating software developers to see if they can add such a feature to their app? Or it’s just Geo Tagr? I guess if anyone knows, it’ll be the software developers. I’ll contact Geo Tagr myself and any other software developer that you can think of to make such a feature request!

I’ve contacted only Runmeter/Cyclemeter and Geo Tagr so far. I should probably just develop my own app (with my copious free time). —Jeffrey

— comment by JJ Lee on May 18th, 2015 at 8:30pm JST (2 years, 6 months ago) comment permalink

Runmeter may not be too off when they say they’re already recording as fast as iOS allows …

iOS sends position updates to an app “whenever it feels like it”. This could be 10 times a second, it could be once every minute. There is NO guarantee for position polling frequency provided by iOS for an app. The overall operating system determines what _it_ thinks is a significant enough change since the last position update (or that it thinks enough time has passed that it will send along another update just for good measure).

OK, now that that’s said … why do some of the other apps have more frequent updates and seem to record at a 1Hz rate when Runmeter seems to record more slowly? One possible explanation (and I have no inside information here about Runmeter or the other apps, this is just a guess) is that Runmeter is actually recording what the OS gives it, nothing more, while the other apps are interpolating data points to ensure at least a 1Hz recording rate even if they don’t get updates from the OS that quickly.

For example, the OS supplies the position updates of t = 0, x = 0, t = 1, x = 1, t = 3, x = 3. Maybe Runmeter is just recording those three specific updates with no processing of the data while the other apps are going in and “creating” a point of t = 2, x = 2 as an interpolation between the actually given data points?

Again, just a guess and I could be completely off.

— comment by Will on May 18th, 2015 at 9:39pm JST (2 years, 6 months ago) comment permalink

I have had the same experience – a particularly bad error creep occurs when holidaying on the Amalfi Coast. The coastal road from Sorrento to Amalfi is as level as the civil engineers could get away with, but still undulating.

GPS error in the forward/backward direction pretty much comes out in the wash, and GPS error laterally doesn’t add much overall distance. However, given you’re cycling along the side of a very steep piece of terrain, even small lateral displacements result in large elevation discrepancies. One minute you’re atop the cliffs, the next you’re in the sea.

It certainly helps you unlock the climbing challenges though 😉

— comment by Dan Phillips on May 19th, 2015 at 1:28am JST (2 years, 6 months ago) comment permalink

It might be worthwhile having a chat to the guys from Wahoo Fitness. I find it’s the best iPhone app for recording an activity, and then uploading to other services incl. Strava from there.

I had a look at their exports; here’s a CSV from a recent run: https://www.dropbox.com/s/dsi10qqnd8qszsf/2015-05-18-0853_Running_WF.csv

They might be open to logging barometric as well as GPS altitude?

I’ve often thought that the state of altitude recording is woeful; am a trail runner so it matters a lot to me too. I come from a background in robotics, and have a number of ideas on how to integrate the two altitude readings to give a proper real-time altitude; you’ve touched on a few of them already.

I didn’t look at this app originally because I thought it was specific to their HR devices, but just gave it a try and it works with my non-Wahoo HR monitor just fine. A quick test shows it uses only the GPS altitude, but otherwise it seems great. A short test yielded a 1Hz tracklog, so this bodes well. Thanks for the pointer. Adding support for the barometric altitude would be fantastic. —Jeffrey

— comment by Josh M on May 19th, 2015 at 9:55am JST (2 years, 6 months ago) comment permalink

So I heard back from the developer behind Runmeter. This is what they told me after asking them a bunch of questions.

——
Our app has the GPS set to give us as many points as it can, as fast as it can, with the highest accuracy it can. Once it does, however, we remove points that are nearly collinear, making sure that we gather a point at least every 50 meters or so. (We never add or move GPS points.) We do this for two reasons, one being that it makes for a smaller storage size, but also, it reduces the memory we need to use in the iPhone while displaying information in maps and graphs. We work very hard to be sure that key data, like splits and intervals and maximums and such have the highest fidelity possible. Serious cyclists we work with tell us our data collecting and import/export is the among the best for iPhone apps.

If you use a sensor such as speed or a heart rate, we gather that information with one reading ever 2 seconds. In our experience, gathering at a higher rate doesn’t appreciably increase accuracy or precision, and it nearly doubles the data that needs to be collected. However, many people perceive that 1hz monitoring leads to higher accuracy, and we respect those perceptions.
——

Not sure if this information is of any use to anyone here, but I figure I would post it.

Jeff, it would be great if you can make such an app. I’d definitely buy it! With your skills, I’m sure it won’t take that much time. Plus, if you start now, you can finally figure out if it’s possible for the IP6+ to even record both the GPS/GLONASS and barometric elevation data.

I’m disappointed that they throw away data. I can understand simplifying it for display, but I wish they’d include it in the exported GPX because people exporting the GPX may well be wanting to use it for something that Runmeter can’t possibly have taken into account (such as segment timing at Strava, or geoencoding photos after the fact). Given what they’ve said you’d be crazy to use their app with Strava. —Jeffrey

— comment by JJ Lee on May 19th, 2015 at 5:50pm JST (2 years, 6 months ago) comment permalink

I’ve contacted them and suggested it; they’ve “passed it on to their developers.” I already am an authorised developer for their SDK (am developing a HW product as a hobby) which gives you all the functionality of the app; I just need some motivation to get the app started… would basic location + both altitudes + HR do the trick?

That would be awesome. Add in there fetching the outside temperature for the current location every so often (every 15 min), and it’d be killer. —Jeffrey

— comment by Josh M on May 20th, 2015 at 9:29am JST (2 years, 6 months ago) comment permalink

Hi Jeffrey,
I use my Xperia z3c with ipBike and upload to Strava. On my last few rides along some regular routes I’ve noticed a fairly consistent altitude difference between the two apps – ipBike generally reports about 55-57m less overall altitude gain than Strava. Before I enable ipBike’s “hey Strava, don’t muck with elevation” feature I thought it would be interesting to see what I could extract from the gpx data and try to match it against an external source, a bit like what you’ve done – but probably not so polished!

I almost always ride in urban areas of Brisbane, Australia, so there’s almost always a good GPS signal. Perhaps 95+% of my rides are on bike paths or roads with known GIS data but occasionally I’ll get the mtb dusted off and go bush bashing.

About 2 weeks ago I had a great 40km ride on one of those regular routes which both ipBike and Strava reckoned was only 100m long – despite the ipBike summary recording the correct distance. That was infuriating enough that I dug into lxml.etree to do some recovery work. https://www.jmcpdotcom.com/blog/2015/06/01/fixing-up-gpx-ride-data-with-lxml-etree/ is where I rambled on about what I did. I have very little experience in data smoothing, so if you’ve got some handy pointers to algorithms and/or best practices I’d really appreciate the tip.

As an aside, I wonder what you process your photos with?

James

I don’t quite understand… you had a long ride that both apps summed up as only 100m? What did the track look like? I use Adobe Lightroom to process my photos. —Jeffrey

— comment by James on June 8th, 2015 at 4:29pm JST (2 years, 6 months ago) comment permalink

The track looked quite straight – from my home to 100m up the street was where the unique lat/lon elements ended. I had 6400 or so track points allegedly at the same point on the road, just with different elevations, cadence, hr and speed! It was an incredibly overcast day – in your experience can that make a difference to how well GPS signals are received by handheld units?

James

Overcast doesn’t matter, I think. Seems like a bug… in technical terms, your GPS unit got “wedgied”. /-: —Jeffrey

— comment by James on June 8th, 2015 at 10:26pm JST (2 years, 6 months ago) comment permalink

Very interesting and eye-opening article.

I live in the north of Spain, in a pretty abrupt area with lots of mountains and lots of narrow and deep valleys. I’ve been using Strava for 3 years already, and the funny thing is that for the first few months, both the elevation gains and profiles seemed fine or at least believable. I guess that, at that time, the reported elevation was being computed directly from the recorded GPS data. After a few months I started experiencing the very same issues you describe in the article, and it became quite frustrating to get these “spiky” elevation profiles. At first, I thought it was my phone, or even the rather old OS I was using, but I see it has nothing to do with the actual device you use.

What I don’t really understand is why they push their DB data over the GPS elevation data. The latter might not be the most accurate but perhaps they could try to smooth it out by averaging routes that go over the same places. Or, as someone in the comments suggested, combine the GPS data with barometric data if the device has that type of sensor.

It seems that what they do is quite consistent (though often inappropriate): if the device is known to them to have a barometric altimeter, they use the device’s data. Otherwise they use their data. But at least they provide a way to spoof a barometric altimeter, so you an force them to use the data you provide. I’ve lately developed a system whereby I snap my tracklog path to a road, and use known altitudes for points along the road to compute my altitude, so my results are now extremely accurate, but it’s a fairly manual process each time I ride a new road to set the elevation points for the road. But over time I’ll have created a map of Kyoto that I can snap my tracklogs to, and it’ll be easy to get honest data. I’m currently in the process of computing all this for my prior rides, so that I can ask Strava to update my prior-ride data to reflect reality. I hope they’ll do it. —Jeffrey

— comment by Juan on July 8th, 2015 at 5:50am JST (2 years, 5 months ago) comment permalink

I have an oldish Garmin speed/cadence sensor to go with my 800 GPS and when it detects the sensor it calibrates it (presumably using gps speed – no need for wheel size to be entered) and then uses that for speed. When I’m stood still at a junction the display reads 0.0km/h. Without the sensor it floats around the 0-1km/h range.

On occasion I’ve pressed the “correct” button on Strava only to regret it given the whacky numbers it’s provided so I’ve just removed it and uploaded it again.

Having never used a speed/cadence sensor, I’ve wondered how they handled the wheel-size issue, such as how it shrinks as pressure in the tire goes down. (I just bought one last week while I’m in The States visiting family, but won’t be able to actually use it until I return to Kyoto.) I’d wonder how accurate a GPS sync would be over the short term… I have to worry that they’re not as picky about accuracy as I am. The zero-speed case is uniquely easy (no tire movement is 0), of course. We’ll see… tests will certainly ensue when I return to Kyoto! —Jeffrey

— comment by PhillipA on August 17th, 2015 at 3:42am JST (2 years, 3 months ago) comment permalink

Sending you a standing ovation from Arapahoe, Pamlico County, NC . That was the most enjoyable technical analysis I have ever read and the aside about Lady making Tofu… superb man!

— comment by Will on September 23rd, 2015 at 1:41pm JST (2 years, 2 months ago) comment permalink

Jeffrey – thanks for shedding light on these discrepancies! I shared your interesting blog on our FB group “Mountain biking around Ubud/Bali”.

Happy riding!

— comment by Werner on October 20th, 2015 at 2:34pm JST (2 years, 1 month ago) comment permalink

Hi Jeff

I usually use your lightroom plugins but the blogged popped up in a search result. We’re in Okayama for two weeks and I went for a walk and runkeeper shows strong gps signals but would not track.

Anyhow, this was a very interesting read. Im off to reasearch why my runkeeper is not working here.

Have a great day.

Reagards,
Paul

— comment by Paul on November 14th, 2015 at 6:58am JST (2 years ago) comment permalink

Stumbled across your (excellent) post.

My question is : what would you suggest to mitigate the impact of tunnels/bridges and extreme terrain on assessing a course’s gradient?

I’ve been using gpsVisualizer’s “best source” elevation data to compare potential courses in preparation of a bike tour. I use point-to-point elevation data to compute 300M and 1KM gradient moving averages. As we travel with kids, I am reluctant to plan a course with a sustained (1km) gradient approaching 8% or a 300M gradient of more than 15%. A more careful inspection of a supposedly “easy” course showing a very steep 300M gradient (75%! 🙂 made me realize that elevation data applies to terrain, not to roads, such that gorges and mountains where civil engineers usually build roads with bridges and tunnels appear more daunting than they actually are.

Yeah, this is a real problem with a simplistic “map this course to elevation data” approach, even if the course data and elevation data are precise. Strava has the same problem (multiplied by the course data and elevation data not being precise). To get around it, of course, you need to know where there are tunnels and bridges, or use data that takes them into account. One way would be to follow the route on Google Earth and make the mental adjustments needed for what you see. Another idea might be to look at segments in Strava and find folks who have ridden the route with what seems to be a reasonable barometric barometer, and look at their elevation profile. No easy answers here that I know of. —Jeffrey

— comment by gauvins on December 21st, 2015 at 3:39pm JST (1 year, 11 months ago) comment permalink

Thank you Jeffrey for this very informative article! I am trying to plan multi-day mountainbike tours here in Albania and should have some reliable information which elevations expect me before starting off. No solution yet after this article, but at least I start to understand & accept that this whole pre-planning of elevation is pretty inaccurate, so I dont expect to much but instead just go out and test. Thanks again and best regards!

— comment by Tobi on January 28th, 2016 at 8:51pm JST (1 year, 10 months ago) comment permalink

(From New Mexico)

Wonderful post! I love the plots of how all the track logs differ, as well as the detailed analysis.

I’ve struggled with this problem for my own silly little Python script that plots elevation profiles from GPX files (ellie, in the GitHub repo “pytopo” if anybody cares). I’ve tried smoothing the data various ways but I always come out with numbers I know are badly inflated. That’s true regardless of whether I use barometric altitude (from an old Garmin) or GPS altitude (from a Galaxy S4 running OsmAnd) though of course it’s a lot worse with non-barometric data.

In your reply to Adam Bridge you said you wouldn’t mind sharing your code that cleans up the KML data and handles lingering at one place — did you ever get a chance to upload it somewhere? I’d love to see how you solved the problem since you obviously have a good handle on issues.

I actually moved away from trying to process my own elevation data… I just haven’t had the time to write up about it (much less tidy the software for release). What I do now doesn’t scale to anyone else’s use anyway… I combine high-resolution road and elevation data from the Japanese government with manually-selected points along each road, to “snap” my route. Thus, I totally throw away elevation data from the GPS unit. Every time I ride a road for the first time, I have to manually go through the route on the computer looking for local peaks and valleys, marking the location and exact elevation. It’s time consuming. Then I run my software that snaps the tracklog to the known elevation points, extrapolating in between to come up with a nice smooth elevation curve. The exact elevation in between may not match the road perfectly (the extrapolated elevation is linear, whereas the road may have variable elevation change rates), but the total change in elevation is accurate. It’s much more work than it’s worth, but that’s the geek in me. —Jeffrey

— comment by Akkana on March 6th, 2016 at 11:47am JST (1 year, 9 months ago) comment permalink

From Spain, Strava inflation is here too… just did a bike ride competition with oficial 3000m elevation gain, my sigma rox10 (with barometric) stated nearly 3000m gain, but Strava recorded 4500m gain!!
I’ve found that sistematically I have to trim 33% (a third) of the elevation gain stated in strava, well, not only strava but also any other app using gps to calculate elevation gains

good article!!! thanks

— comment by danoko on May 5th, 2016 at 12:05am JST (1 year, 7 months ago) comment permalink

Great article Jeffrey though a little over my tiny brain, very similar to my experience using strava on my android phone. I previously used “ride with gps” and found they elevation gain data more accurate although probably still a little inflated but as soon as I exported it to strava it would whack the elevation up to near double sometimes.

— comment by pepe on May 5th, 2016 at 6:33am JST (1 year, 7 months ago) comment permalink

Excellent Post!

I just saw this post while pulling my self off the trap!
I recently wrote a Chrome Extension which read the TCX file from the user input and output the Elevation Profile of the course. I use the exact same approach as yours to calculate the Elevation Gain and Loss along the course. However, the calculated results are quite different from what have been reported on Garmin’s website. I noticed that there are only less than 2,000 data points in the TCX file so I was thinking that maybe Garmin had “hidden” some data points. So I use Google MAPs API to get its data. I thought the data points from Google are more reliable. However, the Elevation Gain and Loss are still quite different. So I think your method of using Moving Average is what Garmin did on the back-end server. I applied the moving average to the points. The curve is indeed more smooth and the Elevation Gain reduces a lot, but still seems inflated. My question is how did you set the moving window? I am a runner and mainly focus on Half/Full Marathon so the typical distances are 13mi/26mi, much shorter than a typical cyclist’s distance. Thanks.

I don’t think there’s a good solution… it’s trying to derive accurate information from data that’s inaccurate to an unknown extent. In the solution I outlined in this post (a solution I no longer use in deference for known, hard-coded elevation points in my local area) I came up with a heuristic that seemed to work where I looked for local minimum/maximum over distances as short as 100m. For the moving average, I think I settled on 35m on either side of the current point. I don’t remember the details too well because it’s just a quagmire of guesses, and even if you come up with an approach that seems to work for a bunch of activities that you check, it seems you’ll invariably come across an activity that totally doesn’t work with the system, yet you can’t really see how it’s different from the others. It’s depressing to a data-perfectionist like me. —Jeffrey

— comment by Peilin on May 20th, 2016 at 3:58am JST (1 year, 6 months ago) comment permalink

Thanks for that, best analysis and explanation i’ve seen

— comment by Phil on June 28th, 2016 at 9:45pm JST (1 year, 5 months ago) comment permalink

Fantastic article, thanks so much! Have you released your software yet? If not, it would be fantastic if you could.

Do you have any suggestions for those of us who don’t have access to accurate laser-derived elevation data or the time to manually massage each file? What about the accuracy of the altimeter? Is relying on that a reasonable quick-and-easy workaround, assuming fairly stable air pressure?

Also, what about Garmin Connect’s elevation correction feature? Presumably it makes sense to keep this disabled? I wonder if enabling it will destroy any altimeter data when exporting activities from it into a FIT or TCX or GPX file? One would hope not…

Finally, since it’s over a year since you wrote this, I don’t suppose Strava have since discovered your blog, seen the idiocy of their ways, and shown any inclination to fix things? Thanks again!!

I had dinner with a Strava engineer, who said that the elevation-correction stuff was pretty accurate in many areas, so he was surprised by my complaint. The whole bridge/tunnel thing must remain an issue everywhere, though, so perhaps it just wasn’t his area at the company. I don’t know anything about Garmin Connect’s elevation features, sorry. I’d guess that the best off-the-shelf system would be something with a barometric altimeter…. I’ve been testing with a Garmin 820 of late, with GPS, GLONASS, and a barometric altimeter. In the few tests I’ve done, it’s been pretty consistent in under-reporting elevation gain by about 10%, which perhaps sounds horrible but it’s not that bad (and is a heck of a lot better than Strava’s automatic elevation stuff, at least in my area). —Jeffrey

— comment by Adam Spiers on September 4th, 2016 at 1:58am JST (1 year, 3 months ago) comment permalink

For some reason (e.g. stupidity, or expecting you to be older or have a good old UNIX wizards beard) I’ve only just managed to put 2 (regex.info) and 2 (your name) together and realise that you’re THE Jeffrey Friedl who wrote the famous regex book! Very cool.

Anyway, thanks for the reply. Any chance of an answer to my question about whether you’re going to release your software? 🙂

I think an (ideally Free Software) tool for cleaning GPS track data would be quite an effective weapon against these braindead proprietary systems. If it got popular enough (e.g. via DC Rainmaker), I would expect word about the issue to spread around at Strava, Garmin etc., and who knows, they might actually grow sufficiently motivated to fix it.

BTW I also bought an Edge 820 recently, and noticed that a) the altimeter calibration option is quite hard to find, and b) when it records tracks which partially go under sea level (which may or may not be due to lack of correct calibration), both the Garmin and Strava Android apps get hopelessly confused. I guess some of their software engineers made dumb assumptions that elevation data would always be positive. So presumably that’s bad news for folks in the Netherlands, or other areas of the world which genuinely dip below sea level. I reported the bugs to both companies, and got no response from Garmin, and the most braindead series of replies from a Strava support guy who got fixated on the calibration of my altimeter and couldn’t see the wider problem relating to negative elevation readings.

P.S. your blog didn’t automatically email me via email when you replied to either of my comments, although it *did* email me about follow-up comments from other people.

I don’t really know much about barometric altimeters, but isn’t the calibration only relevant for absolute altitude, and not really important for relative altitude (over the short span of a day). Of course the air pressure changes with the sun, but re-calibrating every so often (every hour?) would introduce a sudden jump, which perhaps is worse than nothing. Or maybe not. I moved on from using the altitude derived on any of the devices, leaving the code I talked about here to rot. It was horrendous, since it developed over time in parallel with my understanding of the realities. I’d love to be able to release something useful, but the code left lying around here is not that, sorry. )-: One thing that stops me from revisiting it is that the underlying source (the barometric altimeter in every unit I’ve tried) is essentially garbage. It might work nicely some days, but other days is inexplicably crazy. Garbage in, Garbage out, so why bother.—Jeffrey

— comment by Adam Spiers on September 19th, 2016 at 5:43pm JST (1 year, 2 months ago) comment permalink

From time to time I film my routes to get possibility to repeat it “indoors” (on the turbo trainer). I convert GPS track and video into something called Real Life Video. I’m not happy with these, in the profile there are strange issues.
Several months ago I’ve found nice method of “computing” CdA. The method proposed by dr R.Chung. This method rely on power meter and speed. Yes.. the only issue is to have power meter.
Does it sound enough irrelevant? I mention it.. because the intermediate result of the method is called “virtual elevation”. With very simple maths you can get nice altitude diagram. In very custom conditions (eg. no wind, same surface, same riding position, no brakes, etc) “virtual elevation” might (should?) mean “real”, or at least it don’t have same issues as it was taken from GPS or (even) barometer!
I rode some short routes several times (to be honest.. most often my way from home to work, about 20km), and it works almost perfectly! 180m+-10 (accuracy about 6%)! If I had better conditons (no wind..)
Next year I plan to measure my favourite “long” route (~200km).. GPS says that this is about 1400m, while barometer suggest 1000-2000m (with average 1600). I’m courious if the method works for longer routes as well, and which gives “best” results 🙂

— comment by Pajak on November 19th, 2016 at 7:05am JST (1 year ago) comment permalink

..Around 2% difference between horizontal run and actual distance. Um.. Not so when trail running where you regularly meet slopes in excess of 20 degrees (short by 6%), some are 30 (short by 13%), and can even reach 45% (short by 30%) at times. Now THAT is a long way out.

— comment by tim on November 6th, 2017 at 3:16pm JST (2 weeks, 4 days ago) comment permalink
Leave a comment...


All comments are invisible to others until Jeffrey approves them.

Please mention what part of the world you're writing from, if you don't mind. It's always interesting to see where people are visiting from.


You can use basic HTML; be sure to close tags properly.

Subscribe without commenting