Runkeeper and my Garmin GPS

The problem

I am an avid runkeeper.com user. I log every hike, every bike ride, everything. I have used two devices: my android phone, and a Garmin Vista hcx GPS. The later is particularly useful for my long hikes: It is sturdy, waterproof and batteries last very long.

My problem with my Garmin GPS is that something in the way it logs my tracks makes Runkeeper assume I am doing constant pauses, and resuming a few meters later. This takes all calculations (distance, total climb) very far off what I get from what I get from Mapsource or Google Earth. And this cannot happen in Runkeeper where I expect to have every possible statistic of my activities!

This has been happening on my activities for the last 2 or 3 years of Runkeeper usage...

Why now?

Long ago (Aug 2011) I reported this issue to the support team. This was their reply:
From: Jake
Subject: Problem in climb calculation

Hernan, no known issues with Climb calculation, it appears you have a large number of pause/resume points in this activity, which may be what accounts for the discrepancy.  It looks like there were gaps between pause points that may have accounted for large elevation change.  If you track an activity continuously, do you notice the same discrepancy?

Here'e how we calculate climb....http://support.runkeeper.com/kb/troubleshooting/how-does-runkeeper-calculate-elevation-and-climb

View this Discussion online: http://support.runkeeper.com/discussions/problems/10363-problem-in-climb-calculation

Unfortunately the link is now broken -- they have moved their forums, and apparently, garbage-collected old tickets. A total shame :(

This is why I have been using my phone more and more, and my GPS unit less. But after a 3-day hike where I really had to use the GPS, I decided to fix the issue...

The hack

The idea is really simple: Just make the tracks I upload "smoother", by adding points whenever two points differ in time (or distance, or elevation?) beyond what Runkeeper would consider a 'break'.

Normally, I load all my tracks to a .gdb file (from MapSource), and run scripts to export each track to all .gpx, .tcx and .kml -- just in case! Then I upload the .gpx or .tcx to Runkeeper. So I decided to do some post-processing of these files.

I found gpxpy, a Python library to read and write GPX format. One of the examples in the github page already parses out a file, and even prints some statistics -- almost exactly what I need. So what was left to me to do was to complete this, and I came up with the python script I am attaching at the end.

Results

I have uploaded and re-uploaded my 3-day hike, and now the numbers look much more accurate. There are still some pauses, but they may be totally true -- these are 10-hour hikes, and they do have pauses -- just not every minute.

Here is an image of my activity before fixing the track:

and this is the same activity, after the fix:

note the improvements: 1608m climbed (against 439), 12.95km instead of 11.07km, and a much cleaner curve!

I would love to receive feedback from other users -- as well as Runkeeper staff! For me, it is not a big deal if my GPS is too old or incapable or producing the correct input formats for them. I will keep loving and using Runkeeper!

If you found this article useful, or need help trying to use the code yourself, please leave a comment!

PS: The source code

Ubuntu repo problem around ubuntu-release-upgrader-core?

I ran an apt-get upgrade I have delayed for months, the last weekend (I am still in Ubuntu Quantal, 12.04 LTS) -- about 1000 packages to upgrade. It seemed to go fine, except that I did not notice there was a conflict leaving many un-configured packages. This is a "production" machine, the one I used to work daily.

First bad symptom on Monday was some fonts disappearing (!!) when windows were given focus after being in background -- happening in Eclipse editors, Chrome pages and some toolbars too. The problem repeated a while after rebooting. So I decided to go for apt-get dist-upgrade, as maybe the most recent update would help.

That was just where problems really started. Conflicts, and packages unconfigured. Booting led to the graphical loading screen, without ever reaching a login, and no network connection either!! (Total panic at this point).

Doing some apt-get -f install , and dpkg --configure -a, did help finishing to install some packages. There was one big problem left:

(Reading database ... 441700 files and directories currently installed.)
Unpacking ubuntu-release-upgrader-core (from ubuntu-release-upgrader-core_1%3a0.190.8_all.deb) ...
dpkg: error processing ubuntu-release-upgrader-core_1%3a0.190.8_all.deb (--install):
 trying to overwrite '/etc/update-manager/release-upgrades', which is also in package update-manager-core 1:0.174.4

I have no idea where this came from. I was unable to solve it cleanly. The last hack was to install the .deb manually (dpkg -i), with a --force flag to ignore the overwritten file! This IS WRONG, but allowed me to continue and finish upgrading the rest of the packages left behind.

With packages finally updated, the machine still did not show a login screen. Networking did work, on consoles. No errors anywhere. I believe there was no greeter configured... so I ran dpkg-reconfigure lightdm. 

After some reconfiguration, and choosing lightdm as greeter, I was able to log back in to the desktop. I lost my old greeter (I believe I was running gdm) but I cannot even boot now. I will have to continue debugging later... this week this machine just needs to work. (Which raises the question: Why on earth did I start all the updates this last weekend?).

I am back - well, maybe.

After 2 years of no blogging, I have been thinking of start again. I have a place, and there is always content I could create but do not blog... so I thought I would start doing it again....