Runkeeper and my Garmin GPS

The problem

I am an avid runkeeper.com user. I log every hike, every bike ride, everything. I have used two devices: my android phone, and a Garmin Vista hcx GPS. The later is particularly useful for my long hikes: It is sturdy, waterproof and batteries last very long.

My problem with my Garmin GPS is that something in the way it logs my tracks makes Runkeeper assume I am doing constant pauses, and resuming a few meters later. This takes all calculations (distance, total climb) very far off what I get from what I get from Mapsource or Google Earth. And this cannot happen in Runkeeper where I expect to have every possible statistic of my activities!

This has been happening on my activities for the last 2 or 3 years of Runkeeper usage...

Why now?

Long ago (Aug 2011) I reported this issue to the support team. This was their reply:
From: Jake
Subject: Problem in climb calculation

Hernan, no known issues with Climb calculation, it appears you have a large number of pause/resume points in this activity, which may be what accounts for the discrepancy.  It looks like there were gaps between pause points that may have accounted for large elevation change.  If you track an activity continuously, do you notice the same discrepancy?

Here'e how we calculate climb....http://support.runkeeper.com/kb/troubleshooting/how-does-runkeeper-calculate-elevation-and-climb

View this Discussion online: http://support.runkeeper.com/discussions/problems/10363-problem-in-climb-calculation

Unfortunately the link is now broken -- they have moved their forums, and apparently, garbage-collected old tickets. A total shame :(

This is why I have been using my phone more and more, and my GPS unit less. But after a 3-day hike where I really had to use the GPS, I decided to fix the issue...

The hack

The idea is really simple: Just make the tracks I upload "smoother", by adding points whenever two points differ in time (or distance, or elevation?) beyond what Runkeeper would consider a 'break'.

Normally, I load all my tracks to a .gdb file (from MapSource), and run scripts to export each track to all .gpx, .tcx and .kml -- just in case! Then I upload the .gpx or .tcx to Runkeeper. So I decided to do some post-processing of these files.

I found gpxpy, a Python library to read and write GPX format. One of the examples in the github page already parses out a file, and even prints some statistics -- almost exactly what I need. So what was left to me to do was to complete this, and I came up with the python script I am attaching at the end.

Results

I have uploaded and re-uploaded my 3-day hike, and now the numbers look much more accurate. There are still some pauses, but they may be totally true -- these are 10-hour hikes, and they do have pauses -- just not every minute.

Here is an image of my activity before fixing the track:

and this is the same activity, after the fix:

note the improvements: 1608m climbed (against 439), 12.95km instead of 11.07km, and a much cleaner curve!

I would love to receive feedback from other users -- as well as Runkeeper staff! For me, it is not a big deal if my GPS is too old or incapable or producing the correct input formats for them. I will keep loving and using Runkeeper!

If you found this article useful, or need help trying to use the code yourself, please leave a comment!

PS: The source code

Ubuntu repo problem around ubuntu-release-upgrader-core?

I ran an apt-get upgrade I have delayed for months, the last weekend (I am still in Ubuntu Quantal, 12.04 LTS) -- about 1000 packages to upgrade. It seemed to go fine, except that I did not notice there was a conflict leaving many un-configured packages. This is a "production" machine, the one I used to work daily.

First bad symptom on Monday was some fonts disappearing (!!) when windows were given focus after being in background -- happening in Eclipse editors, Chrome pages and some toolbars too. The problem repeated a while after rebooting. So I decided to go for apt-get dist-upgrade, as maybe the most recent update would help.

That was just where problems really started. Conflicts, and packages unconfigured. Booting led to the graphical loading screen, without ever reaching a login, and no network connection either!! (Total panic at this point).

Doing some apt-get -f install , and dpkg --configure -a, did help finishing to install some packages. There was one big problem left:

(Reading database ... 441700 files and directories currently installed.)
Unpacking ubuntu-release-upgrader-core (from ubuntu-release-upgrader-core_1%3a0.190.8_all.deb) ...
dpkg: error processing ubuntu-release-upgrader-core_1%3a0.190.8_all.deb (--install):
 trying to overwrite '/etc/update-manager/release-upgrades', which is also in package update-manager-core 1:0.174.4

I have no idea where this came from. I was unable to solve it cleanly. The last hack was to install the .deb manually (dpkg -i), with a --force flag to ignore the overwritten file! This IS WRONG, but allowed me to continue and finish upgrading the rest of the packages left behind.

With packages finally updated, the machine still did not show a login screen. Networking did work, on consoles. No errors anywhere. I believe there was no greeter configured... so I ran dpkg-reconfigure lightdm. 

After some reconfiguration, and choosing lightdm as greeter, I was able to log back in to the desktop. I lost my old greeter (I believe I was running gdm) but I cannot even boot now. I will have to continue debugging later... this week this machine just needs to work. (Which raises the question: Why on earth did I start all the updates this last weekend?).

I am back - well, maybe.

After 2 years of no blogging, I have been thinking of start again. I have a place, and there is always content I could create but do not blog... so I thought I would start doing it again....

LibreOffice on Maverick

I installed LibreOffice a few weeks ago, leaving OpenOffice in the past -- it was a great thing, but it is going to change I opt for LibreOffice as it keeps its spirit of freedom.

These instructions include installing from an official PPA, so updates will come from there until LibreOffice makes it into the main ubuntu repos. This comes from novatillasku: In short,

• sudo apt-get purge "openoffice*.*"
• sudo add-apt-repository ppa:libreoffice/ppa
• sudo apt-get update
• sudo apt-get install libreoffice libreoffice-gnome libreoffice-l10n-es

You might want to use libreoffice-kde instead of the -gnome version, and a different language pack instead of -es as well.

Bits from the past

A few weeks ago my father found in a backup disk some old texts he had written years ago: composed in WordStar for DOS; likely in our first computer, a PC XT (8-12Mhz!). Digging these files we concluded they were from about 1991 or 1992.... Too long ago! All I can say in my defense is that... I was younger? A child? The fact is that I used that WordStar version a lot.

Unfortunately these files were unreadable by any word processor I use and tried. Import filters promised a lot, a none of them worked for me. So it deserved some deeper digging.... First of all, for nostalgy sake, this is how WS looked like (now within a "modern" Win XP, running virtualized in my linux):

Now back to importing these files: I found the site wordstar.org with plenty of information, but most downloads were for Windows, and also most were not free. Here is a list of downloads. I tried some of them (under Wine), like: WS-Con, WSRTF and a few more. None of them were fully working; most have problems with accents or whatever encoding these files used.

Fortunately I found a text from in site, which describes the file format. This format is quite simple, and has a nice design allowing to extract "most" of the text by just looking the lowest 7 bits of each bytes, and discarding everything with the 8th bit set. If you want formatting, you would have to interpret those high bytes, though they are not too complex and we used very few in our old texts.

So I read this and wrote a python script to process them... The first few attempts were mangling, again, all my accents and the 'ñ' character (these are in Spanish) so I had to start digging at ascii codes. I have almost tatooed in my memory that 'ñ' = 164, after typing "Alt-1-6-4" so many times in DOS. (There were only US keyboards by that time... and I still use them). But a character 164 means something else in python or in nowadays encodings... sometimes as bad as:

>>> chr(164)
'\xa4'
>>> chr(164).encode('utf-8')
Traceback (most recent call last):
  File "", line 1, in
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 0: ordinal not in range(128)

while the 'ñ' has different codes today:

>>> 'ñ'
'\xc3\xb1'   ### This is UTF-16 I think
>>> 'ñ'.decode('utf-8')
u'\xf1'      ### Its UTF-8 encoding

so which was the correct encoding? There is a small note in this page, saying "how to type in microsoft windows", and later a note saying DOS was using "codepage 437". That's cool, I had already found the list of all encodings python was bundled with in /usr/lib/python2.6/encodings (and there is indeed a file cp437.py)

So the real key was to do something like chr(164).decode('cp437'). That returns the unicode string u'\xf1', which is the "real" 'ñ'. That made the trick, and the script was done.

As a side note: I found some more characters I could not filter out initially: Double-byte codes like ESC-'4' or ESC-'5' around words... what was that? Some sleepy neuron remembered that we use to have a Star NX-1001 (multifont!), and I suspected that was a printer code. In fact, the manual (which still exists! go Star!) says those are the start of italicized text, and the return to normal font face. So that's not part of WordStar format, it's another problem - the way we handled formatting in our printer. (It was good to remember that, our best printer, again!).

Now if you read this far you must be a nostalgic looking for old memories, of really looking to translate a wordstar file. I uploaded the script to http://pastebin.com/pfY8Dbgv - it converts to plain text or a basic HTML. I hope it helps you!

Automatically set VGA output as "primary" in Gnome

I always liked gnome-display-properties in that by just clicking its tray icon it will enable (or disable) the VGA output of my laptop if my "secondary" LCD is connected or not. I switch to laptop-LCD only to laptop-plus-external-LCD daily. Recently I started using this external monitor as "primary", using also a external keyboard and mouse, and wanted the gnome panels (menu, taskbar, etc.) to appear there.

However, gnome-display-properties assumes the monitor at the "left" is primary, so menus are always displayed there. It has no UI, no gesture to configure it differently. Fortunately, xrandr (which I think gnome-display-properties uses) allows to freely configure evertyhing related to displays: enable/disable, geometry, layout, etc. After experimenting a little I came up with this little script:

OUT=`xrandr|grep connected|grep -v disconnected| wc -l`
if [ $OUT -gt 1 ]; then
  xrandr --output LVDS1 --auto --output VGA1 --auto --right-of LVDS1 --primary
else
  xrandr --output LVDS1 --auto --output VGA1 --off
fi

which enables my VGA output if a monitor is connected, but setting it as primary (the command line is self-explanatory). I bound this script to an unused key (the blue "ThinkVantage" key on my thinkpad) and now I switch configuration exactly as I wanted: with one key press.

Note: I played a little with XFCE4.8 and I found a different (though related) issue: panels not moving from one to the other display automatically. I asked in the forums and this is unfortunately not supported -- though it was scheduled for 4.10 as soon as I posted a ticket, as Nick from XFCE recommended.

Trouble using Eclipse 3.4.x on Ubuntu Karmic

I was using Eclipse 3.5+ last months, but fo a project I needed to use 3.4. I downloaded three versions so far and they all die after trying to open or create a workspace: After the splash screen, an empty dialog appears which you can't even close.

I found the solution here: It appears that I had to add -Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib/xulrunner to eclipse.ini This is odd, since my /usr/lib/xulrunner is empty (except for a plugins folder, which has the flash plugin only?), and my xulrunner lives actually in /usr/lib/xulrunner-1.9.1.9. Anyway, it works with /usr/lib/xulrunner but it does not work with /.../...-1.9.1.9 -- for now!

(Another suggestion on that post is to add "-vm /path/to/your/bin/java" but I do not seem to need this).