Thursday, September 3, 2015

Java: "unmappable character for encoding ASCII"

I hit a strange compile error today: a project that compiled in several environments was failing to build in a build server. The error was in the "copyright" symbol in all .java files, something like...

...../.../MyFile.java:8: error: unmappable character for encoding ASCII
 *  ?? Copyright XX All Rights Reserved

     ^

Note that "file MyFile.java" showed UTF8, and this was indeed its encoding. Turns out Java was deciding or guessing it was ASCII, incorrectly.

One solution was to force the encoding, like this:
export JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8  
And even better, since I use Gradle, was to use compileJava.options.encoding. In my top-level gradle file:
subprojects {
    apply plugin: 'java'    compileJava.options.encoding = 'UTF-8'
... 

I hope this helps someone else!

Monday, December 15, 2014

Happy Birthday, dad

This is how it all started for me... The notes below are from a old notebook, 22 years ago. It was 1992 and I was coding my first programs: my father and I were learning to code together. He, as always, was able to spot and encourage my passions. Programming became my passion, profession and way of living.

I do not have digital records of that age, and had no printer either, so these handwritten listings are all that was left from that BASIC code. I can see my coding style changed, too...


He would turn 75 years old today. Just he left exactly weeks ago -- now we know he is free from his damn awful disease.



I miss you, Papá.

Wednesday, June 11, 2014

XBMC in Odroid U3 (cont'd)

I think I owe an update on the Odroid U3 topic I started months ago.

First, I should say I quickly abandoned the Android image: for the reasons described in the previous post. I moved to some "Game Station Turbo" image, linux-based, that I have used for months. Performance was decent, but it was limited to 720p and even playing videos in this resolution was not totally smooth. Some videos would hang, also (video freezing, but audio continuing -- workaround was to press Stop, then play again the video resuming from the position I just stopped it). I never used the gaming features of that image, by the way -- they did not work for me (I only tried some ROMs but was not that much interested in doing further research). Asking in the forums, they said the 720p limitation was a problem in the drivers, that should "get solved in a few months". So I waited.

More recently I went to the forums again looking for updates. I tried a couple of different images, until I found this one, based on Ubuntu 13.10 and LXDE. Driver problems are fixed, or replaced by some other drivers... and the image boots from install in 1080, and plays 720p videos absolutely smoothly! And 1080p seemed smooth at the first try, though I need to play an entire movie to be sure.

(For some technical note: Installing is similar as other images: "xz -cd ((image name.xz)) | pv > /dev/mmcblk0", and then extend the last partition to the end of the SD card space).

Seems they are still doing updates on the drivers, so instructions to re-download, compile and install them are in that forum thread. The image still uses XBMC 12.3, but a v13 will come soon (when there is a final release; now xbmc 13 is still beta). I can't wait! EDIT: mdrjr confirms one should not need to update drivers -- they are part of the image already.

I am very thankful to memeka, mdrjr and the other folks from Odroid community who are working on this. This is a major update for those using Odroid as XBMC media center, as I do, and now it makes the Odroid absolutely worth every cent!


Thursday, February 20, 2014

Unlocked my Motorola Defy

Locked phone!
I have a 3yr-old Motorola Defy, and recently upgraded to another model. This Defy came from Personal (carrier in Argentina), and I wanted to use it with a SIM from a different carrier (Claro). When I put that SIM card... a message would ask for "Pin de desbloqueo de red de la tarjeta SIM".

The phone was usable (apps, wifi), but no mobile signal was picked.


Tried PUK code, known short PINs, asked the subsidy password to my carrier, and nothing worked.

(Interestingly enough: Unlike other people's experiences, I phoned Personal and after verifying I am the owner of the phone, they just gave me the subsidy password in a couple of minutes. Unfortunately, it did not work with my phone).

I suspect the code did not work out of the box because my phone was re-flashed long ago, I had installed Cyanogenmod 9 to get Android 4.x.

System Settings / version
I tried some "baseband selector" but this did not help. I think this option is the same thing available in Cyanogenmod options menu -- you just select the frequencies, but the phone is still locked.

So after looking for hours (days), I found one solution based on installing some update from the recovery menu. I had not used this for a long time so it was all fun and scary... again.

The source is this 51-pages forum in XDA. The solution uses a file named ICS_Defy_softunlock_v1.3(EPU_U_00.59.01).zip, and it is called a "soft unlock". I am documented what worked for me:

Pre-requisites:
  • To flash CM9, I had already rooted and installed 2nd-init. I did this long ago, and it is the only actual requisite.
  • I had CM9 previously installed - so the "ICS" version of the patch worked for me (and not GB for Gingerbread). I believe both CM7 and CM9 work.
  • As shown in my System Settings page, my base band version is EPU93_U_00.59.01, exactly what this soft unlock requires. If yours is different, you may need to install the baseband update as well (from the same XDA forum).
Steps:
  1. Reboot in Recovery mode
    Shutdown menu / recobe
  2. Select "Install zip from SD card" and "Choose zip from sdcard"

  3. Pick the ICS_Defy_softunlock_v1.3(EPU_U_00.59.01).zip file and confirm.
  4. Reboot your phone without SIM card (or with the original one -- not the target carrier SIM!)
  5. Once fully rebooted, power off, insert new SIM card, and start the phone again.

    It worked for me! The screenshot shows the phone in Claro network now.
Working with new SIM
I read that this, being a "software unlock", is not persistent across ROM changes. So if I ever replaced the current CM9 in this phone, I might need to do this patching again.... Hopefully this blog post will help me at that time!

Thanks xdadevelopers!

Friday, February 14, 2014

ODroid U3: Hello Android

I got an Odroid U3 , a tiny, low consumption and yet powerful computer. I want to replace my dead Raspberry Pi which acted as mediacenter, and add some gaming to it.

It turns out that knowing what to install is not totally trivial. There is a myriad of images, different linux and android versions. I am documenting what I started doing:

  • There is a big collection of image links here.
  • I downloaded an android Ice Cream Sandwich, 4.0.3. (The 4.1 and 4.2 versions were said to not work from SD cards; which is what I bought (eMMC is more expensive)).
  • The actual installation steps are here. Some clarifications below...
  • First, one needs to burn a .xz image:
    xz -cd ./IceCreamSandwich-4.0.3_CouchPotato-U2-HDMI-SD.20140101.img.xz > /dev/mmcblk0
  • This image is from a small SD, and I needed to resize to my 16Gb card. The partition to enlarge is some FAT32 in the middle, so there are quite a few steps involving backing up contents and copy them back. Details here.
    • What takes time is (1) to burn the .xz, initially. The backup is just about 250Mb in size

My first impressions:
  • Android runs smooth, some games are nice too. But I already found some cons...
  • I cannot have my USB external HDD get recognized. This one has two ext4 partitions (full of movies I was using with my old Raspbmc). Some "StickMount" and "Paragon NTFS" are installed, and I tried others, but no luck yet.
  • XBMC was slow to play a 720mb from a pen drive (fat32, it did work). Another player pre-installed in this image worked ok, with hardware decoding.
  • Three times already, Google Play was stuck and unable to download new stuff from the android market (some error shown in the notifications). The only solution I found from the forums is to wipe Play's data from android Settings; then remove my gmail account, reboot, and start over adding my account and re-downloading stuff.
  • The whole UX is far from ideal. The OS thinks it is a tablet. All UI is oriented to dragging, pinching, using huge amount of scrolls -- as if your fingers were on a touch screen, and not moving a cursor on a big TV. Totally awkward on a small touchpad from my wireless keyboard. Games expect you to touch everywhere too... not to press keys. I guess this is android!
I just read there is a GameStation Turbo including XBMC with CEC support (for my TV remote) and some games too.... It seems to be linux, too, which I will like more. Going to try out the next few days!

Monday, February 10, 2014

Cut my SIM to MicroSIM size

I bought a new phone, with the surprise that my SIM ("Mini-SIM", formally) would not fit: This and other new phones use Micro-SIM (or even Nano-SIM) format. The chip contacts are the same, just with less plastic around.

You mobile carrier company should be able to swap your SIM card or just cut it with one of the many automatic clippers around. Some of them charge for this -- I am sure they would, in my country. And, there is the hackish way which I am of course always up to: cut it yourself!

There are many posts about this on the web. I just picked one of the mostly linked templates, and cut it. Here is the before:

And the after:
(I sorry - this is not the card after cutting - just the leftovers: the card is in my phone now!)

If you are careful, I think it should just work. It was not such a complicated or exact work to me.

My only advice, maybe a refinement on what I read, is that you mark the cuts with a cutter (and a metal ruler) before really cutting, which I did with regular scissors. The lines you "draw" with the cutter are perfectly thin, better than using a marker or pen.

This was no software hacking. Tangible stuff this time!

Thursday, December 5, 2013

Runkeeper and my Garmin GPS

The problem

I am an avid runkeeper.com user. I log every hike, every bike ride, everything. I have used two devices: my android phone, and a Garmin Vista hcx GPS. The later is particularly useful for my long hikes: It is sturdy, waterproof and batteries last very long.

My problem with my Garmin GPS is that something in the way it logs my tracks makes Runkeeper assume I am doing constant pauses, and resuming a few meters later. This takes all calculations (distance, total climb) very far off what I get from what I get from Mapsource or Google Earth. And this cannot happen in Runkeeper where I expect to have every possible statistic of my activities!

This has been happening on my activities for the last 2 or 3 years of Runkeeper usage...

Why now?

Long ago (Aug 2011) I reported this issue to the support team. This was their reply:
From: Jake
Subject: Problem in climb calculation

Hernan, no known issues with Climb calculation, it appears you have a large number of pause/resume points in this activity, which may be what accounts for the discrepancy.  It looks like there were gaps between pause points that may have accounted for large elevation change.  If you track an activity continuously, do you notice the same discrepancy?

Here'e how we calculate climb....http://support.runkeeper.com/kb/troubleshooting/how-does-runkeeper-calculate-elevation-and-climb

View this Discussion online: http://support.runkeeper.com/discussions/problems/10363-problem-in-climb-calculation

Unfortunately the link is now broken -- they have moved their forums, and apparently, garbage-collected old tickets. A total shame :(

This is why I have been using my phone more and more, and my GPS unit less. But after a 3-day hike where I really had to use the GPS, I decided to fix the issue...

The hack

The idea is really simple: Just make the tracks I upload "smoother", by adding points whenever two points differ in time (or distance, or elevation?) beyond what Runkeeper would consider a 'break'.

Normally, I load all my tracks to a .gdb file (from MapSource), and run scripts to export each track to all .gpx, .tcx and .kml -- just in case! Then I upload the .gpx or .tcx to Runkeeper. So I decided to do some post-processing of these files.

I found gpxpy, a Python library to read and write GPX format. One of the examples in the github page already parses out a file, and even prints some statistics -- almost exactly what I need. So what was left to me to do was to complete this, and I came up with the python script I am attaching at the end.

Results

I have uploaded and re-uploaded my 3-day hike, and now the numbers look much more accurate. There are still some pauses, but they may be totally true -- these are 10-hour hikes, and they do have pauses -- just not every minute.

Here is an image of my activity before fixing the track:
and this is the same activity, after the fix:
note the improvements: 1608m climbed (against 439), 12.95km instead of 11.07km, and a much cleaner curve!

I would love to receive feedback from other users -- as well as Runkeeper staff! For me, it is not a big deal if my GPS is too old or incapable or producing the correct input formats for them. I will keep loving and using Runkeeper!

If you found this article useful, or need help trying to use the code yourself, please leave a comment!

PS: The source code



Tuesday, November 19, 2013

Ubuntu repo problem around ubuntu-release-upgrader-core?

I ran an apt-get upgrade I have delayed for months, the last weekend (I am still in Ubuntu Quantal, 12.04 LTS) -- about 1000 packages to upgrade. It seemed to go fine, except that I did not notice there was a conflict leaving many un-configured packages. This is a "production" machine, the one I used to work daily.

First bad symptom on Monday was some fonts disappearing (!!) when windows were given focus after being in background -- happening in Eclipse editors, Chrome pages and some toolbars too. The problem repeated a while after rebooting. So I decided to go for apt-get dist-upgrade, as maybe the most recent update would help.

That was just where problems really started. Conflicts, and packages unconfigured. Booting led to the graphical loading screen, without ever reaching a login, and no network connection either!! (Total panic at this point).

Doing some apt-get -f install , and dpkg --configure -a, did help finishing to install some packages. There was one big problem left:

(Reading database ... 441700 files and directories currently installed.)
Unpacking ubuntu-release-upgrader-core (from ubuntu-release-upgrader-core_1%3a0.190.8_all.deb) ...
dpkg: error processing ubuntu-release-upgrader-core_1%3a0.190.8_all.deb (--install):
 trying to overwrite '/etc/update-manager/release-upgrades', which is also in package update-manager-core 1:0.174.4

I have no idea where this came from. I was unable to solve it cleanly. The last hack was to install the .deb manually (dpkg -i), with a --force flag to ignore the overwritten file! This IS WRONG, but allowed me to continue and finish upgrading the rest of the packages left behind.

With packages finally updated, the machine still did not show a login screen. Networking did work, on consoles. No errors anywhere. I believe there was no greeter configured... so I ran dpkg-reconfigure lightdm. 

After some reconfiguration, and choosing lightdm as greeter, I was able to log back in to the desktop. I lost my old greeter (I believe I was running gdm) but I cannot even boot now. I will have to continue debugging later... this week this machine just needs to work. (Which raises the question: Why on earth did I start all the updates this last weekend?).

I am back - well, maybe.

After 2 years of no blogging, I have been thinking of start again. I have a place, and there is always content I could create but do not blog... so I thought I would start doing it again....

Monday, March 7, 2011

LibreOffice on Maverick

I installed LibreOffice a few weeks ago, leaving OpenOffice in the past -- it was a great thing, but it is going to change I opt for LibreOffice as it keeps its spirit of freedom.
These instructions include installing from an official PPA, so updates will come from there until LibreOffice makes it into the main ubuntu repos. This comes from novatillasku: In short,
  • sudo apt-get purge "openoffice*.*"
  • sudo add-apt-repository ppa:libreoffice/ppa
  • sudo apt-get update
  • sudo apt-get install libreoffice libreoffice-gnome libreoffice-l10n-es
You might want to use libreoffice-kde instead of the -gnome version, and a different language pack instead of -es as well.

Wednesday, February 23, 2011

Bits from the past

A few weeks ago my father found in a backup disk some old texts he had written years ago: composed in WordStar for DOS; likely in our first computer, a PC XT (8-12Mhz!). Digging these files we concluded they were from about 1991 or 1992.... Too long ago! All I can say in my defense is that... I was younger? A child? The fact is that I used that WordStar version a lot.

Unfortunately these files were unreadable by any word processor I use and tried. Import filters promised a lot, a none of them worked for me. So it deserved some deeper digging.... First of all, for nostalgy sake, this is how WS looked like (now within a "modern" Win XP, running virtualized in my linux):

Now back to importing these files: I found the site wordstar.org with plenty of information, but most downloads were for Windows, and also most were not free. Here is a list of downloads. I tried some of them (under Wine), like: WS-Con, WSRTF and a few more. None of them were fully working; most have problems with accents or whatever encoding these files used.

Fortunately I found a text from in site, which describes the file format. This format is quite simple, and has a nice design allowing to extract "most" of the text by just looking the lowest 7 bits of each bytes, and discarding everything with the 8th bit set. If you want formatting, you would have to interpret those high bytes, though they are not too complex and we used very few in our old texts.

So I read this and wrote a python script to process them... The first few attempts were mangling, again, all my accents and the 'ñ' character (these are in Spanish) so I had to start digging at ascii codes. I have almost tatooed in my memory that 'ñ' = 164, after typing "Alt-1-6-4" so many times in DOS. (There were only US keyboards by that time... and I still use them). But a character 164 means something else in python or in nowadays encodings... sometimes as bad as:

>>> chr(164)
'\xa4'
>>> chr(164).encode('utf-8')
Traceback (most recent call last):
  File "", line 1, in
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 0: ordinal not in range(128)
while the 'ñ' has different codes today:

>>> 'ñ'
'\xc3\xb1'   ### This is UTF-16 I think
>>> 'ñ'.decode('utf-8')
u'\xf1'      ### Its UTF-8 encoding
so which was the correct encoding? There is a small note in this page, saying "how to type in microsoft windows", and later a note saying DOS was using "codepage 437". That's cool, I had already found the list of all encodings python was bundled with in /usr/lib/python2.6/encodings (and there is indeed a file cp437.py)

So the real key was to do something like chr(164).decode('cp437'). That returns the unicode string u'\xf1', which is the "real" 'ñ'. That made the trick, and the script was done.

As a side note: I found some more characters I could not filter out initially: Double-byte codes like ESC-'4' or ESC-'5' around words... what was that? Some sleepy neuron remembered that we use to have a Star NX-1001 (multifont!), and I suspected that was a printer code. In fact, the manual (which still exists! go Star!) says those are the start of italicized text, and the return to normal font face. So that's not part of WordStar format, it's another problem - the way we handled formatting in our printer. (It was good to remember that, our best printer, again!).

Now if you read this far you must be a nostalgic looking for old memories, of really looking to translate a wordstar file. I uploaded the script to http://pastebin.com/pfY8Dbgv - it converts to plain text or a basic HTML. I hope it helps you!

Sunday, February 13, 2011

Automatically set VGA output as "primary" in Gnome

I always liked gnome-display-properties in that by just clicking its tray icon it will enable (or disable) the VGA output of my laptop if my "secondary" LCD is connected or not. I switch to laptop-LCD only to laptop-plus-external-LCD daily. Recently I started using this external monitor as "primary", using also a external keyboard and mouse, and wanted the gnome panels (menu, taskbar, etc.) to appear there.

However, gnome-display-properties assumes the monitor at the "left" is primary, so menus are always displayed there. It has no UI, no gesture to configure it differently. Fortunately, xrandr (which I think gnome-display-properties uses) allows to freely configure evertyhing related to displays: enable/disable, geometry, layout, etc. After experimenting a little I came up with this little script:
OUT=`xrandr|grep connected|grep -v disconnected| wc -l`
if [ $OUT -gt 1 ]; then
  xrandr --output LVDS1 --auto --output VGA1 --auto --right-of LVDS1 --primary
else
  xrandr --output LVDS1 --auto --output VGA1 --off
fi
which enables my VGA output if a monitor is connected, but setting it as primary (the command line is self-explanatory). I bound this script to an unused key (the blue "ThinkVantage" key on my thinkpad) and now I switch configuration exactly as I wanted: with one key press.

Note: I played a little with XFCE4.8 and I found a different (though related) issue: panels not moving from one to the other display automatically. I asked in the forums and this is unfortunately not supported -- though it was scheduled for 4.10 as soon as I posted a ticket, as Nick from XFCE recommended.

Wednesday, June 2, 2010

Trouble using Eclipse 3.4.x on Ubuntu Karmic

I was using Eclipse 3.5+ last months, but fo a project I needed to use 3.4. I downloaded three versions so far and they all die after trying to open or create a workspace: After the splash screen, an empty dialog appears which you can't even close.

I found the solution here: It appears that I had to add -Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib/xulrunner to eclipse.ini This is odd, since my /usr/lib/xulrunner is empty (except for a plugins folder, which has the flash plugin only?), and my xulrunner lives actually in /usr/lib/xulrunner-1.9.1.9. Anyway, it works with /usr/lib/xulrunner but it does not work with /.../...-1.9.1.9 -- for now!

(Another suggestion on that post is to add "-vm /path/to/your/bin/java" but I do not seem to need this).

Sunday, May 9, 2010

Google Codejam 2010 participants feeds

On May 7th the Google CodeJam 2010 started, with about 15,000 contestants in the Qualification round! If you are competing, you surely hit this blog looking for some stats.

I have not found any feed (yet) about participants. All I found was this thread from the google group: where Igor N. from Google said: we still have the AJAX requests that the scoreboard page uses to grab 20 rows at a time.

So I went ahead and grabbed, 20 rows at a time.... One bash command line that iterates and gets 15000/20 small json files, and a few python lines to parse and pretty print them. Here is how my output currently looks like (the scores for each problem overflow to the right):
## Country: Argentina
#210 (99): ged       [ 4:42:08]  [s 2:16:12 L 2:16:57]  [s 4:37:31 L 4:38:08]  [s 3:24:24 L 3:25:22]
#326 (99): lrearte   [ 6:47:22]  [s 6:46:17 L 6:47:22]  [s 6:43:08 L 6:44:32]  [s 6:34:26 L 6:36:44]
#408 (99): aurinegro [ 8:29:45]  [s 0:43:37 L 0:44:24]  [s 1:09:52 L 8:25:45]  [s 2:12:08 L 2:13:43]
#454 (99): axelbrz   [ 9:16:12]  [s 4:35:26 L 4:41:16]  [s 9:04:06 L 9:08:12]  [s 2:54:08 L 2:58:50]
#466 (99): dodi      [ 9:28:48]  [s 5:34:01 L 5:37:01]  [s 9:20:20 L 9:28:48]  [s 7:27:13 L 7:31:59]
#709 (99): fidels    [12:40:11]  [s 0:12:58 L 0:15:09]  [s 3:35:34 L 3:39:19]  [s12:37:54 L12:40:11]
(....)
There are more fields to show; basically everything shown in the scoreboard is obtained from these feeds so you can look for the extra data in the json files.

Feel free to use your script for yourself. I would love to hear similar pages or your own usage of these feeds.


Here is the python script (note that the bash command line to get the results is in a comment)
#!/usr/bin/python
import os
import json

## Run first:
## for i in `seq 1 20 10500`; do wget -O $i.json "http://code.google.com/codejam/contest/scoreboard/do?cmd=GetScoreboard&contest_id=433101&show_type=all&start_pos=$i&views_time=1&views_file=0&csrfmiddlewaretoken="; done
def time(pty):
  return "--:--:--" if pty<0 else "%2d:%02d:%02d" % (pty//60//60, (pty//60)%60, pty%60)
def problems(arr):
  s = ""
  c = 's'
  for t in arr:
    s += '   [' if c=='s' else ' '
    s += c+time(t)
    if c=='L': s += ']'
    c = 'L' if c=='s' else 's'
  return s

country='Argentina'
print "## Country: "+country

i=1
while True:
  try:
    f = open(str(i)+".json")
  except:
    exit
  if (f.closed): 
    break
  s = f.read();
  f.close()

  obj = json.loads(s)
  for coder in obj['rows']:
    if coder['c']==country:
      print "#%5d (%3d): %-30s  [%s]    %s" % (coder['r'], coder['pts'], coder['n'], time(coder['pen']), problems(coder['ss']))

  i += 20

Wednesday, April 21, 2010

Add a Java runtime to Ubuntu using update-alternatives

I just installed a Java JDK not using the package system. I wanted all users in this machine to be "aware" of this java runtime so the "java" command defaults to this implementation.

Ubuntu offers the "update-alternatives" utility to do this. This is both used to configure which of your (in this case) java implementations to use, but it can also be used to add a new one to the list. Here is the recipe (run as root, or sudo all commands):
# update-alternatives --install /usr/bin/java java \
  /opt/jdk1.6.0_14_x64/bin/java 100
# update-alternatives --config java

There is 1 program that provides 'java'.

  Selection    Command
-----------------------------------------------
*+ 1           /opt/jdk1.6.0_14_x64/bin/java

Enter to keep the current selection[+], or type selection number: 1
# java -version
java version "1.6.0_14"
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) 64-Bit Server VM (build 14.0-b16, mixed mode)
Note that in my case, since this was the only Java installed, there was no need to configure it via --config. I just ran it to make sure this one was the current choice. Also, if you do not want the interactive prompt you can just run:
sudo update-alternatives --set java /opt/jdk1.6.0_14_x64/bin/java

Thursday, April 8, 2010

Skype video on Ubuntu 9.10

I had solved this long ago but reappeared in my relatively new Ubuntu 9.10 install: Skype (2.1.x) does not show video from my webcam; a usb Logitech QuickCam Pro 5000:
$ lsusb
Bus 001 Device 002: ID 046d:08c5 Logitech, Inc. QuickCam Pro 5000
I tried a few things and it turned out that the solution was something I had found that was supposed to aim a problem with 64bit systems and 32bit libraries. One needs to set the variable LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so. (That's the path in my Ubuntu, it may vary in other system).

Then, to launch Skype, I am using a small script:
#!/bin/bash
LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so skype
for which I also created a menu icon.

Sunday, March 28, 2010

Odd 1366x768 LCD res on Nvidia

I bought a LCD monitor (LG Flatron W1943S) months ago for my mother. This desktop PC runs Kubuntu 9.04 with an NVidia 6600 card. I had never been able to fix a problem: about 60 pixels of overscan to the right, which is very awful and annoying specially when you need to move scrollbar...

(My mother with infinite patience has moved too many windows to the left to pick these missing buttons... I would have already thrown the monitor out the window.)

Some further debugging today let me find this:

First, the native resolution 1366x768 is not supported by nvida (the card? the driver?). You need exact 16:9, or multiples of 8, I igore it -- the point is that you need to get the modeline for 1368 horizontal pixels; I used this to generate it and put it on xorg.conf:

$ gtf 1366 768 60
# 1368x768 @ 60.00 Hz (GTF) hsync: 47.70 kHz; pclk: 85.86 MHz
Modeline "1368x768_60.00" 85.86 1368 1440 1584 1800 768 769 772 795

(In Screen section, under a Display subsection, added Modes "1368x768_60.00")
It also works with 1360x768, I still don't know which one is better.

Then, the most important part was in the monitor itself: I read in a forum (about different hardware I think) that there was a "reset to factory defaults" in the monitor. So, afer setting this resolution, I navigated through the unintuitive OSD monitor menu, under Setup - in the last screen - last option is Factory Reset. That simply accomodated all the pixels within the screen! And easier: There is an "auto/set" button that appears to be the same! All this time I was looking for a xorg vs. drivers vs. monitor configuration issue, I never thought it would be that simple.

I am still trying to get a real 1:1 sampling, I see some pixels appearing as "blurred" horizontally. Suggestions are welcome...

Tuesday, March 23, 2010

Rosegarden, Lilypond, Timidity and my sax lessons

I started trying to write some decent music sheets, for my sax lessons. I wanted to find some free, opensource linux alternatives to programs like Sibelius, and indeed I found them!

Fortunately, all of them are in Karmic repositories, so I just apt-get install rosegarden and this installed Lilypond, a latex-based music-typesetting (if that's a word) system, and Rosegarden, a very good stand alone GUI that not only serves to write the music, but also allows listening via MIDI (something I could do yet, but I will...)

So this post is actually about how to use Rosegarden to save a MIDI file that Timidity can play (my Timidity install just worked, while Jackd soundsystem did not). So go ahead and type some music in Rosegarden. Then export it as a .mid file. Then, timidity file.mid should already work.

Sound fonts But, the default piano sound of my cheap soundcard is far from nice. I wanted to install some sound banks ("sound fonts"). There are some of them here.

Unpacking? I downloaded a file 1276-Soft Tenor Sax.sfpack. Now, how to open it? First problem, there is no sfpack-unpacker for linux. There is a small DOS utility here that runs fine under Wine, so I used it: run wine SFPACK.EXE, click Add Files, select the .sfpack file, and click Go. This unpacks the file 1276-Soft Tenor Sax.sf2, which is something the sound card (or the midi system) can use.

Installing. To use the soundfont, you must give Timidity a config file. There are many config files for several fonts here, but my file was not there. So I created a very raw configuration for it. First, comment the enabled "source" line in /etc/timidity/timidity.cfg, and add a new line source /etc/timidity/my.cfg. Then, edit that new file /etc/timidity/my.cfg, adding just this content:
bank 0
0 %font "1276-Soft Tenor Sax.sf2" 0 0 tune=-14 amp=31 pan=0
Note: I am using "-12", not "-14", so other MIDIs are played on tune as well... but the reasoning is still useful so I am leaving this version in this article (See update at the end)

what this does is creating a single program "0" on bank "0", using the .sf2 file given. (Note: I put that file also in /etc/timidity, otherwise type the full path in there). The tune parameter is explained below (you can omit it). The amp tells how much to amplify, and the pan can be used to balance it to left/right (I use it in another example to send different instruments to left and right channels).

Save it wav/mp3. I wanted this as an easily playable sound file, so I ran timidity input.mid -Ow -o output.wav to convert it. My music file was just one instrument, number 0, bank 0, so it now vaguely resembles a sax (better than the original piano, at least). I later converted it to mp3 and put it on my player to study listening to it.

Tune to Bb! One caveat: I typed whas was supposed to be in C, while my tenor sax is Bb. I could have exported to Midi one tone down from Rosegarden, but I missed it. Also, the way it was originally exported it was one octave higher than the way I play that music in my sax. That's the reason of the tune=-14 parameter in my .cfg file: 14 semitones down (12 semitones = shift 1 octave down; and another tone to tune it in Bb). Note: There is an alternate, better method -- see the update at the end

Swing! One more thing -- the song I am practicing was supposed to be played with swing rithm -- that is, octaves lasts 2/3 and 1/3 of the time of a half -- but we just type them all the same way, as regular octaves. I imagined there should be an easy formula or script to apply to a Lilypond file, that could convert a regular rythm to swing... and there is! In Rosegarden, use the Quantize option (after selecting the whole file), use type=grid, base=eight, swing=100%, and make sure to check "quantize durations". That's it! I did not saved the result, but only exported it to .mid, but it sounds perfectly :)


That was my experience from last night -- while children sleep I cannot play the sax, so doing this was a funny alternative!

Update (Apr 20th): While tuning 1 tone down im timidity.cfg is possible, it does of course has side effects. Every MIDI will be shifted down... quite undesirable. Later I found that Rosegarden allows to shift instruments, so most of them in my partiture (written fo four saxophones) just appear as "-2" (one tone down) or "10" (one octave higher, but still 1 tone down) to be played in Bb.

Thursday, February 25, 2010

Install Ubuntu from Alternate CD image; booting from USB flash drive

I recently tried a couple of ubuntu versions, installing them on a partition to play a few hours. In the process I re-discovered unetbootin, a wonderful tool for this: I burned the CD ISO images in a USB flash drive, instead of burning CDs (even if you use rewritable, it is slower than a USB key). UNetbootin let's you make a bootable pendrive from any ISO image; you can even download your favorite distro CD from there.

The main comment of this post is, however, not just the unetbootin process, but two tips that solved issues with these images.

First, the simpler one: You have to mark the partition in the USB drive as bootable, or you will not be able to boot... To do this, run fdisk /dev/sdb (or whichever your drive's device is), and add its boot flag.

The second on is subtler. With the Ubuntu 9.10 Desktop CD everything worked perfectly; I was able to boot. But later I tried the Alternate CD, which after booting it could not find the contents of the "CD" it was installing from. Somehow it missed the point it had boot from USB; so there would not be any contents on CD-ROM to read (in fact, that machine does not even have a CD drive). What I had to do was to add a flag to the kernel boot options: When UNetbootin starts, before pressing Enter on the "Default" option, press Tab to edit it. Edit the kernel line, to make it look like this one:
/ubnkern initrd=/ubninit vga=normal cdrom-detect/try-usb=true -- quiet

(Everything should be in a single line). Your line might not be identical; but the important part is to add "cdrom-detect/try-usb=true" before the "--". With this option the installer found the contents and was able to continue the rest of the install process.

Wednesday, February 24, 2010

Resizing a VirtualBox bootable WinXP drive

I have a couple of Windows images running in VirtualBox machines. (Yes I only boot linux, but still have a couple of windows images confined to their VMs). They ran out of diskspace; I created too small disks. I had attempted to solve this by adding increasingly larger secondary disks; but Windows does not like small "C:" partitions; sooner or later problems arise.

I found tips from several places about how to do this so I am going to compile my experience here. Since VDI images cannot be just resized (disk geometry can be altered), one needs to move the data to another VDI disk. The procedure is then: 1) Create a new (bigger) disk, 2) Clone the old disk into the new disk, 3) replace the disk in the virtual machine.

So far, that's more or less simple if the disk is not the primary, bootable disk. But I tried cloning with Acronis TrueImage, GParted and a couple of other tools, and they all resulted in an unbootable disk with "Disk reader error, press Ctrl+Alt+Del" message when trying to boot. I tried using the windows recovery console rebuild the MBR (fixmbr.exe) or the boot sector (fixboot.exe) but none of them worked. The extra tip was touching a parameter in the boot sector of the disk... find the details below.

Before you start, you might want to backup your entire VDI image; just in case (none of these steps will modify it... but there is always the case of selecting the wrong option).

All the steps then look like:
  1. Create the disk

    1. Shut down the virtual Windows machine
    2. From VirtualBox disk manager, create a new disk (bigger!), and attach it as secondary to that machine.

  2. Copy data. Use some cloning tool to copy the entire disk from the old disk to the new one. In my case, I found the simpler way was:

    1. Download a GParted LiveCD,
    2. Add it to the list of VirtualBox ISO images,
    3. Mount it on the virtual machine, and make sure the CDROM is first in boot order sequence.
    4. Boot GParted. Select the old disk/partition, click "Copy". Select the new disk, create on "Create Partition Table" (type "MSDOS"), then click "Paste" to bring the new data. In the resizing dialog, you can extend the partition to the end of the disk. Click "Apply" and it will take a while until the partition is copied. Then go to "Manage Flags" and enable the "Boot" flag.
    5. Shutdown the virtual machine; remove the ISO cd image.

  3. Enable the new disk (You might want to try seeing if the new disk just boots -- If you do not get a "Disk Read Error" message, skip next steps and you are done!).

    1. Start the virtual machine (still with the old disk as primary). You should be able to view the bigger, secondary disk, with the same contents as the old one.
    2. From within Windows, download Roadkil's Boot Builder. Then a) Read the boot sector of the NEW disk, b) change the "Heads" parameters to 255, c) write the sector back to the NEW disk. (Make sure not to screw up disk names! your old disk is still the backup.). See screenshot.
    3. Shutdown Windows. From VirtualBox, remove the old (smaller) disk, and leave only the new (larger) disk. Start up the VM, it will nicely boot up. (Well, Windows will boot up...)

I have no idea why that Heads parameter needs to be changed; but changing it from 128 to the magic 255 value worked for me (enlarging a 3Gb disk to 10Gb one) and others as well.

Happy cloning!