Ubuntu repo problem around ubuntu-release-upgrader-core?

I ran an apt-get upgrade I have delayed for months, the last weekend (I am still in Ubuntu Quantal, 12.04 LTS) -- about 1000 packages to upgrade. It seemed to go fine, except that I did not notice there was a conflict leaving many un-configured packages. This is a "production" machine, the one I used to work daily.

First bad symptom on Monday was some fonts disappearing (!!) when windows were given focus after being in background -- happening in Eclipse editors, Chrome pages and some toolbars too. The problem repeated a while after rebooting. So I decided to go for apt-get dist-upgrade, as maybe the most recent update would help.

That was just where problems really started. Conflicts, and packages unconfigured. Booting led to the graphical loading screen, without ever reaching a login, and no network connection either!! (Total panic at this point).

Doing some apt-get -f install , and dpkg --configure -a, did help finishing to install some packages. There was one big problem left:

(Reading database ... 441700 files and directories currently installed.)
Unpacking ubuntu-release-upgrader-core (from ubuntu-release-upgrader-core_1%3a0.190.8_all.deb) ...
dpkg: error processing ubuntu-release-upgrader-core_1%3a0.190.8_all.deb (--install):
 trying to overwrite '/etc/update-manager/release-upgrades', which is also in package update-manager-core 1:0.174.4

I have no idea where this came from. I was unable to solve it cleanly. The last hack was to install the .deb manually (dpkg -i), with a --force flag to ignore the overwritten file! This IS WRONG, but allowed me to continue and finish upgrading the rest of the packages left behind.

With packages finally updated, the machine still did not show a login screen. Networking did work, on consoles. No errors anywhere. I believe there was no greeter configured... so I ran dpkg-reconfigure lightdm. 

After some reconfiguration, and choosing lightdm as greeter, I was able to log back in to the desktop. I lost my old greeter (I believe I was running gdm) but I cannot even boot now. I will have to continue debugging later... this week this machine just needs to work. (Which raises the question: Why on earth did I start all the updates this last weekend?).

I am back - well, maybe.

After 2 years of no blogging, I have been thinking of start again. I have a place, and there is always content I could create but do not blog... so I thought I would start doing it again....

LibreOffice on Maverick

I installed LibreOffice a few weeks ago, leaving OpenOffice in the past -- it was a great thing, but it is going to change I opt for LibreOffice as it keeps its spirit of freedom.

These instructions include installing from an official PPA, so updates will come from there until LibreOffice makes it into the main ubuntu repos. This comes from novatillasku: In short,

• sudo apt-get purge "openoffice*.*"
• sudo add-apt-repository ppa:libreoffice/ppa
• sudo apt-get update
• sudo apt-get install libreoffice libreoffice-gnome libreoffice-l10n-es

You might want to use libreoffice-kde instead of the -gnome version, and a different language pack instead of -es as well.

Bits from the past

A few weeks ago my father found in a backup disk some old texts he had written years ago: composed in WordStar for DOS; likely in our first computer, a PC XT (8-12Mhz!). Digging these files we concluded they were from about 1991 or 1992.... Too long ago! All I can say in my defense is that... I was younger? A child? The fact is that I used that WordStar version a lot.

Unfortunately these files were unreadable by any word processor I use and tried. Import filters promised a lot, a none of them worked for me. So it deserved some deeper digging.... First of all, for nostalgy sake, this is how WS looked like (now within a "modern" Win XP, running virtualized in my linux):

Now back to importing these files: I found the site wordstar.org with plenty of information, but most downloads were for Windows, and also most were not free. Here is a list of downloads. I tried some of them (under Wine), like: WS-Con, WSRTF and a few more. None of them were fully working; most have problems with accents or whatever encoding these files used.

Fortunately I found a text from in site, which describes the file format. This format is quite simple, and has a nice design allowing to extract "most" of the text by just looking the lowest 7 bits of each bytes, and discarding everything with the 8th bit set. If you want formatting, you would have to interpret those high bytes, though they are not too complex and we used very few in our old texts.

So I read this and wrote a python script to process them... The first few attempts were mangling, again, all my accents and the 'ñ' character (these are in Spanish) so I had to start digging at ascii codes. I have almost tatooed in my memory that 'ñ' = 164, after typing "Alt-1-6-4" so many times in DOS. (There were only US keyboards by that time... and I still use them). But a character 164 means something else in python or in nowadays encodings... sometimes as bad as:

>>> chr(164)
'\xa4'
>>> chr(164).encode('utf-8')
Traceback (most recent call last):
  File "", line 1, in
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 0: ordinal not in range(128)

while the 'ñ' has different codes today:

>>> 'ñ'
'\xc3\xb1'   ### This is UTF-16 I think
>>> 'ñ'.decode('utf-8')
u'\xf1'      ### Its UTF-8 encoding

so which was the correct encoding? There is a small note in this page, saying "how to type in microsoft windows", and later a note saying DOS was using "codepage 437". That's cool, I had already found the list of all encodings python was bundled with in /usr/lib/python2.6/encodings (and there is indeed a file cp437.py)

So the real key was to do something like chr(164).decode('cp437'). That returns the unicode string u'\xf1', which is the "real" 'ñ'. That made the trick, and the script was done.

As a side note: I found some more characters I could not filter out initially: Double-byte codes like ESC-'4' or ESC-'5' around words... what was that? Some sleepy neuron remembered that we use to have a Star NX-1001 (multifont!), and I suspected that was a printer code. In fact, the manual (which still exists! go Star!) says those are the start of italicized text, and the return to normal font face. So that's not part of WordStar format, it's another problem - the way we handled formatting in our printer. (It was good to remember that, our best printer, again!).

Now if you read this far you must be a nostalgic looking for old memories, of really looking to translate a wordstar file. I uploaded the script to http://pastebin.com/pfY8Dbgv - it converts to plain text or a basic HTML. I hope it helps you!

Automatically set VGA output as "primary" in Gnome

I always liked gnome-display-properties in that by just clicking its tray icon it will enable (or disable) the VGA output of my laptop if my "secondary" LCD is connected or not. I switch to laptop-LCD only to laptop-plus-external-LCD daily. Recently I started using this external monitor as "primary", using also a external keyboard and mouse, and wanted the gnome panels (menu, taskbar, etc.) to appear there.

However, gnome-display-properties assumes the monitor at the "left" is primary, so menus are always displayed there. It has no UI, no gesture to configure it differently. Fortunately, xrandr (which I think gnome-display-properties uses) allows to freely configure evertyhing related to displays: enable/disable, geometry, layout, etc. After experimenting a little I came up with this little script:

OUT=`xrandr|grep connected|grep -v disconnected| wc -l`
if [ $OUT -gt 1 ]; then
  xrandr --output LVDS1 --auto --output VGA1 --auto --right-of LVDS1 --primary
else
  xrandr --output LVDS1 --auto --output VGA1 --off
fi

which enables my VGA output if a monitor is connected, but setting it as primary (the command line is self-explanatory). I bound this script to an unused key (the blue "ThinkVantage" key on my thinkpad) and now I switch configuration exactly as I wanted: with one key press.

Note: I played a little with XFCE4.8 and I found a different (though related) issue: panels not moving from one to the other display automatically. I asked in the forums and this is unfortunately not supported -- though it was scheduled for 4.10 as soon as I posted a ticket, as Nick from XFCE recommended.

Trouble using Eclipse 3.4.x on Ubuntu Karmic

I was using Eclipse 3.5+ last months, but fo a project I needed to use 3.4. I downloaded three versions so far and they all die after trying to open or create a workspace: After the splash screen, an empty dialog appears which you can't even close.

I found the solution here: It appears that I had to add -Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib/xulrunner to eclipse.ini This is odd, since my /usr/lib/xulrunner is empty (except for a plugins folder, which has the flash plugin only?), and my xulrunner lives actually in /usr/lib/xulrunner-1.9.1.9. Anyway, it works with /usr/lib/xulrunner but it does not work with /.../...-1.9.1.9 -- for now!

(Another suggestion on that post is to add "-vm /path/to/your/bin/java" but I do not seem to need this).

Google Codejam 2010 participants feeds

On May 7th the Google CodeJam 2010 started, with about 15,000 contestants in the Qualification round! If you are competing, you surely hit this blog looking for some stats.

I have not found any feed (yet) about participants. All I found was this thread from the google group: where Igor N. from Google said: we still have the AJAX requests that the scoreboard page uses to grab 20 rows at a time.

So I went ahead and grabbed, 20 rows at a time.... One bash command line that iterates and gets 15000/20 small json files, and a few python lines to parse and pretty print them. Here is how my output currently looks like (the scores for each problem overflow to the right):

## Country: Argentina
#210 (99): ged       [ 4:42:08]  [s 2:16:12 L 2:16:57]  [s 4:37:31 L 4:38:08]  [s 3:24:24 L 3:25:22]
#326 (99): lrearte   [ 6:47:22]  [s 6:46:17 L 6:47:22]  [s 6:43:08 L 6:44:32]  [s 6:34:26 L 6:36:44]
#408 (99): aurinegro [ 8:29:45]  [s 0:43:37 L 0:44:24]  [s 1:09:52 L 8:25:45]  [s 2:12:08 L 2:13:43]
#454 (99): axelbrz   [ 9:16:12]  [s 4:35:26 L 4:41:16]  [s 9:04:06 L 9:08:12]  [s 2:54:08 L 2:58:50]
#466 (99): dodi      [ 9:28:48]  [s 5:34:01 L 5:37:01]  [s 9:20:20 L 9:28:48]  [s 7:27:13 L 7:31:59]
#709 (99): fidels    [12:40:11]  [s 0:12:58 L 0:15:09]  [s 3:35:34 L 3:39:19]  [s12:37:54 L12:40:11]
(....)

There are more fields to show; basically everything shown in the scoreboard is obtained from these feeds so you can look for the extra data in the json files.

Feel free to use your script for yourself. I would love to hear similar pages or your own usage of these feeds.


Here is the python script (note that the bash command line to get the results is in a comment)

#!/usr/bin/python
import os
import json

## Run first:
## for i in `seq 1 20 10500`; do wget -O $i.json "http://code.google.com/codejam/contest/scoreboard/do?cmd=GetScoreboard&contest_id=433101&show_type=all&start_pos=$i&views_time=1&views_file=0&csrfmiddlewaretoken="; done
def time(pty):
  return "--:--:--" if pty<0 else "%2d:%02d:%02d" % (pty//60//60, (pty//60)%60, pty%60)
def problems(arr):
  s = ""
  c = 's'
  for t in arr:
    s += '   [' if c=='s' else ' '
    s += c+time(t)
    if c=='L': s += ']'
    c = 'L' if c=='s' else 's'
  return s

country='Argentina'
print "## Country: "+country

i=1
while True:
  try:
    f = open(str(i)+".json")
  except:
    exit
  if (f.closed): 
    break
  s = f.read();
  f.close()

  obj = json.loads(s)
  for coder in obj['rows']:
    if coder['c']==country:
      print "#%5d (%3d): %-30s  [%s]    %s" % (coder['r'], coder['pts'], coder['n'], time(coder['pen']), problems(coder['ss']))

  i += 20