Quick DVD authoring, with subtitles

Let's say you have a DivX movie (movie.avi) and a subtitles file (subtitles.srt) and you want to burn this into a DVD, which any player (non divx-capable) can play.

This is called DVD authoring, and there are many tools to do the job (though I have not found one that I really like, yet). For such a simple task, a few commands on the console are the simplest option.

1. Transcode move to MPEG-2

See the mencoder documentation at the end - it contains examples for other formats.

mencoder -oac lavc -ovc lavc -of mpeg -mpegopts format=dvd:tsaf \
 -vf scale=720:576,harddup -srate 48000 -af lavcresample=48000 \
 -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=5000:\
keyint=15:vstrict=0:acodec=ac3:abitrate=192:aspect=16/9 -ofps 25 \
 -o movie.mpg movie.avi

2. Render and multiplex subtitles into movie

Create a file like this one, and save it as sub.xml:

<subpictures>
  <stream>
    <textsub filename="subtitles.srt" characterset="ISO-8859-1"
         fontsize="22.0" font="devedesans.ttf"
horizontal-alignment="center"
         vertical-alignment="bottom" left-margin="60"
right-margin="60"
         top-margin="20" bottom-margin="40" subtitle-fps="25"
         movie-fps="25" movie-width="720" movie-height="574"/>
  </stream>
</subpictures>

Run spumux to create a new mpeg file with the subtitles stream:

spumux -s0 sub.xml < movie.mpg > m_subs.mpg

3. Author the DVD

Finally, we need to tell dvdauthor how to create a dvd. Create a file dvd.xml like this one:

<dvdauthor>
    <vmgm>
    <titleset>
        <titles>
            <subpicture lang="es">
            <pgc>
                <vob file="m_subs.mpg">
            </pgc>
        </titles>
    </titleset>
</dvdauthor>

And run dvdauthor:

dvdauthor -o thedvd -x dvd.xml

4. Round up

Voila! The directory thedvd will contain the DVD structure (not an ISO image). You can test the results with mplayer:
mplayer dvd:// -dvd-device thedvd

and if it looks ok, just burn it into a dvd - k3b can burn this (use new Project -> movie DVD)

At the end you can delete *.mpg files, and the thedvd directory.

References:

• mencoder on dvd authroing: http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-vcd-dvd.html
  
• spumux: http://dvdauthor.sourceforge.net/doc/spumux.html
  

thunderbird: very small fonts, and links not opening in browser

I recently installed and new kubuntu on my mother's PC (yes!! finally, wiped out Windows completely!!), and of course I am the help desk support since then...

A few days ago I had the chance to solve to curious issues she was having with thunderbird. I had to google a little for both solutions:

Very small fonts: Almost everything in thunderbird had a very, very small font size: menus, message list, etc. I tried changing DPI resolution in KDE, setting global font preferences por KDE and GTK applications, with no avail. Finally the solution was to tweak a property within thunderbird:

layout.css.cpi = 0

(the default value was unset, or -1, I can't remember). So simple! I don't know what is the meaning of the property, but solved the issue and now all fonts are just like in any other application.

Clicking links does not open a browser: When reading an email with a URL, clicking on them did just nothing. I browsed every possible option in the Preferences windows, but there was nothing that fixed it. The solution, again, was hidden in the advanced preferences. I had to set a few variables:

network.protocol-handler.app.http = /usr/bin/firefox
network.protocol-handler.app.https = /usr/bin/firefox

(that can be done for ftp as well -- and any other protocol if needed).

Note: To edit the "advanced" preferences, go to Edit > Preferences > Advanced > Config Editor. Be aware that you can misconfigure it very easily from there!

Both are things far from what a regular end user could change. I wonder why the default install left things that way....

so many wifi clients!

Last night I ran a apt-get upgrade after some time updating anything -- over a month. It looked like mostly an update for many, many KDE packages (I think I got 4.3.1). Just in case, I rebooted -- I always do it after upgrading many packages in case something goes wrong, to make it evident. Well, something went wrong: the knetworkmanager would not connect again to my WPA-protected home wifi. As a result, I lost a couple of hours of sleep time last night... things are working again, after some fight.

I thought it would be interesting to remember how many wifi clients I used over the last year or so:

• network manager from KDE3: I used this one for a long time. Even after starting using KDE4 (in those scary days of betas of the 4.0.x branch, where virtually nothing worked as expected), this rock-solid tool kept me connected to my wireless network. Its biggest problem was just that it was slated to die; it had the KDE3 look, and would be vanished soon. (Image from softpedia)
  
  
  
• At some point, a plasmoid appeared as a replacement. This one was a nightmare at the beginning, but after some time it started to work. It never worked 100%, there where some inconsistencies (when editing networks' properties, or when showing the current state, some things did not reflect correct status). It worked for some time. (Image taken from here)
  
  
  
• Just a couple of months ago (from KDE 4.3 betas?) the plasma applet changed is icon to something really ugly, and a tooltip message said that its development was incomplete and suggested to use "knetworkmanager" instead. So I moved... and well, this one worked nicely! I thought it was the end of the journey. Well integrated in KDE4, properties appearing in control panel too... some tiny features where appearing from time to time after upgrades. Very nice.
  Sadly, after yesterday's update, it did not connect anymore. I can click the network names I created, and nothing happens. Just that. I don't know how to debug it.
  
  
  
• I should also mention that when these things fail, I also resort to wpa_supplicant, connecting from command line with a config file I already have pre-configured and a command line saved right there as I always, always forget the parameters. But again, this is working 100%. It keeps disconnecting, it shows a lot of errors in the console... It kind of works, and saved me sometimes. Last night it was not the exception, so I did not want to continue using it for work over weeks until finding a solution.
• Looking for solutions last night I found wicd. I just thought 'yet another program to try'... and worse, it depended on gnome libraries so it looked like some gnome-only applet. But comments were very positive, so I tried it. Installing the package also uninstalled knetworkmanager, good job! The package also added a daemon, perfect.... and all I can say is it just works. Ok, the UI is not KDE4-ish. But I can configure everything I need, and it worked perfectly after a couple of reboots. (As a side comment: Ubuntu's repositories had a 1.5.x version, and after installing it and knowing it worked, I moved to wicd's repos which brought 1.6.2 version, which is even nicer).
  
  
  

This can be read as a rant against wifi support on linux. And it is, in part. ((Wireless connections have been more of a problem for me than anything else -- even more than 3d acceleration which has settled down reasonably many months (or years) ago for me. Wireless is still... a pain.)) The web is too confusing as it becomes quickly outdated; when you run a search you will find results for hundreds of methods, drivers, hardware, linux distribution, even different ubuntu versions, and none of them except one will work on your scenario.

My intention is to let others know wicd, and to receive feedback about these things, like: Am I too dumb, that I break my networking so often? Is this regular fight intended for every end user that just intends to keep his ubuntu updated? (I know I play beyond end-user-ness, and I enjoy it, but I also think ubuntu must be something an end user must be able to manage by itself).
Meanwhile... I will happily use wicd -- as long as it works :)

ffmpeg with AMR (.amr/.3gp) and x264 on Ubuntu

A few days ago I needed to convert some AMR audio files (recorded in a phone) to a more common format -- like mp3. I found out this would be difficult since:

• the ffmpeg from Ubuntu repositories does not suppport AMR
• ffmpeg dropped support of libamr as it used to have - now they use lib-opencore-amr

Fortunately I found a LOT of help in Ubuntu forums, thanks mc4man for that! I am going to summarize here what I did to finally decode amr files with ffmpeg. Here we will compile ffmpeg and x264 from source code, and install them as packages so we can remove them at any time later.

Following this how-to by FakeOutdoorsman, I downloaded x264 sources (why not? let's compile to support as many formats as possible), compiled it, and downloaded ffmpeg source code:

1. Make sure you don't have any of this installed from ubuntu:

sudo apt-get purge ffmpeg x264 libx264-dev

2. Download and compile x264: (You may need to install some build tools -- see the original how-to if you don't have them)

cd
git clone git://git.videolan.org/x264.git
cd x264
./configure
make
sudo checkinstall --fstrans=no --install=yes --pkgname=x264 --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" --default

3. Get libopencore-AMR-NB and libopencore-AMR-WB from DebianMultimedia. These are the urls for i386 and amd64. Download the four files libopencore-amr*, and install them with dpkg -i .

4. Download, configure and build ffmpeg. Note that the ./configure step is different than the one in the mentioned how-to -- it adds the flags for AMR.

cd
svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libxvid --enable-x11grab --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-version3
make
sudo checkinstall --fstrans=no --install=yes --pkgname=ffmpeg --pkgversion "3:0.svn`date +%Y%m%d`-12ubuntu3" --default

And it's done! I am now able to transcode my files to mp3:

ffmpeg -i test.amr -f avi -vcodec xvid -acodec libmp3lame -ar 22050 test.mp3

Now if later you want to remove these packages, remove ffmpeg, x264 and the four libopencore-amr* we have installed. And also, if you ever want to upgrade with a newer source code, see the original post; they instructions are there (I have not tested that yet).

that ol'university times...

Here is another offtopic. I found this beatiful piece of software, a kind of 'norton commander' -- ok, just a file browser for MS-DOS.... It features a recursive calculator of directory sizes, and a magnificent hex file viewer. Wow! The amazing part is that this fully written on assembler, , for an assignment a group of four friends (Abel Valente, Juan Francisco Pertino, Alejandro Miguez & me) did for the course "Taller de Arquitectura", during our 3rd year of Licenciatura en Informatica, Faculty of Informatics, University of La Plata, Argentina. One of them and me, in parallel, just remembered this work a couple of days ago: almost exactly 10 years after writing it!

Here are a few screenshots. The main screen with the directory size shown (it surprises me that Windows directory size does not overflow...):


Here, the hex viewer!


And finally, the output when exiting, with the date.... August 1999!


One more: source code.... Over 120kb and assembly, sweat and tears:


So many things learned... We will never forget the conditional jumps from the end of a loop.... to the beginning of a loop in a different procedure! And the magic trick of PUSHing and POPing all the flags to make the problem disappear.... ;)

I still can't believe we did it! Those were one of the best 34,687 bytes we ever produced! It's also hard to believe 10 years passed by. I must be getting old.

Opening an iptables firewall port on RHEL 5

I started a new service on a host running RHEL 5.1... and discovered that the port I needed to connect remotely was closed, the firewall would not let me in.

The error I got was somewhat misleading,

telnet [my host]  12345
Trying [ip address]...
telnet: Unable to connect to remote host: No route to host

however the same command from the same host worked; si I knew it was a firewall problem.

The firewall is iptables, and this tutorial saved my day. I just ran iptables -L to show current rules, verifying there were a few open ports (not including mine) and a last rule to drop every other incoming connection. So I ran iptables -I INPUT -p tcp --dport 12345 -j ACCEPT and that's it... the port is now open.

That's good for testing. Now, to make this persistent, go edit /etc/sysconfig/iptables, and add a line like this:

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 12345 -j ACCEPT

and run /sbin/service/iptables restart

Speed up ubuntu boot time

My notebook has run linux for a couple of years already, upgrading from kubuntu version to version, and having installed and uninstalled countless packages. This impacts in, for example, boot time.... I feel it takes too long to boot.

Here are a few things I have done last night to improve boot time. Some may or may not apply to you. I am running Kubuntu 9.04 Jaunty. Hopefully, you might even suggest more improvements.

Bootchart

First, to analyze booting time, install bootchart. It is a wonderful service that renders a graph of how startup scripts and services are run. It might add a little overhead to your load time, but it will also tell you exactly where and how time is spent. On Ubuntu: apt-get install bootchart. Mine has been there for months, but if you are just installing then boot your linux once so you get at least one chart: check out /var/log/bootchart -- a PNG image and few reports in a .tgz are created. See the image for an example: my boot took 46 seconds.

I am going to list a couple of things I did to cut boot time:

Remove services: You can see in the chart there are a LOT of services being started. I use many of them; I stopped using a few of them and others I seldom use. So I could remove some of them from my startup -- with the risk of having a few things stop working. In other cases, I will just know they are stopped, and will start it when I needed (For example: For work, there are weeks were I use a local apache2 daily; but there are periods where I don't use it -- In this case I am going to start it manually when needed). To remove a process from the startup, run (every command listed here is to be run as root): update-rc.d -f remove. If my notes are correct, these are those I have removed:

• apache2: I don't use it daily
• mysql (3 services starting with 'mysql'): I don't use it daily, either
• tomcat5.5: the same
• bluetoothd: I don't have bluetooth hardware!
• kdm-kde4: This is a leftover from last year (was it in Gutsy? Hardy?) that I installed a KDE4 pre-release, and had the option to choose between KDE 3.x and 4.x. The service for my desktop manager is "kdm" (it appears in the chart), not "kdm-kde4" which just gives an error while running.
• winbindd: used for samba if I am correct - I had a samba server, but not anymore
• firestarted: a software firewall. I am behind a firewalling router already. Sometimes I use firestarter for specific things... let's start it manually when needed.
• exim4: a mail server? I have never used it.
• timidity: a daemon for MIDI emulation... I think I installed it and used it once in a year.
• virtualbox-ose: Another leftover -- I used to run VirtualBox OSE (Open Source Edition), but now I am using the "closed" one. The latter has "vboxdrv" and "vboxnet" services; but virtualbox-ose is some leftover from the older virtualbox.
• apport: A tool for crash reports. I don't know if I should removed this... but I have never filled a crash report.
• avahi-daemon: "discovers network resources".... uhm, I don't know what this is about. I removed it, and will re-add it if needed.
• wclientd: this is for a VPN client I have stopped using some time ago. I have other vpn services (agn*) running, this one is not needed.

I think those are most of the services I have removed. After removing, just booted once... and from the 46s original seconds I am down to about 36s!

View and remove services, graphically: There is a nice tool that lists your startup services, and you can choose to start/stop them for current session or remove them from startup sequence: "BUM", Boot Up Manager. I mostly listed manually my /etc/rc?.d directories and viewed bootchart graphs, but this one is also useful too (and it allows to remove services with a click).

Readahead

Since a few versions ago "readahead" is included by default in ubuntu installs. It is meant to read in advance all or most the files needed to boot, in the best order to maximize disk access speed, and avoid too many random seeks in the hard disk as services are started. In most installes, it uses a prebuilt list of files that are read --in a default ubuntu install--, but if you have upgraded versions, installed packages, or just let your install grow up with time, this list will be likely outdated.

To update the readahead-list, do the following: Boot your machine, and in the GRUB menu press "e" to edit the kernel parameters. Select your kernel, press "e" again and append the word "profile" (separated with a space from the last parameter written there), press Enter to accept this change and press "b" to boot. This boot will take MUCH longer than usual; it will record all disk activity. The good thing is that all the files will be listed to file /etc/readahead/boot, sorted for optimal access. Once linux booted, just reboot (you don't need to change anything in grub, the "profile" thing was a setting for boot only one time).

The original process is shown in the example chart, as taking about 10 seconds. This is actually good: if we don't do this, it would most likely take even more time to read all files needed for services to load. After profiling my boot, this time increased to about 15 seconds. But the overall time was down to 30s! (From the about 36s before readahead). This is because after the length readahead-list operation, all services load much faster.

Note that some people get benefit from readahead, and others do not. You can check if it works for you by disabling it, just emptying the file /var/readahead/boot. Measure the next load to see if readahead is useful for you. For me, it was!

Summary

I just did the two things above: removing services I don't need, and re-profiling to customize my readahead-list. This made my linux boot in 30s, from the 46s that was using -- a big gain! Yet, I am not completely happy - I know it can be done better. So I would love to hear if you have more tips!

Also, once you are done with bootchart you might want to remove it from boot. (I would not uninstall the package, you may want to use it from time to time). Just do "update-rc.d -f bootchart remove" and "update-rc.d -f stop-bootchart remove". To re-add it, you may re-install the package or add it to runlevels 2-3-4-5: update-rc.d stop-bootchart start 99 2 3 4 5

For a couple of alternatives:

• Some people report that the splash screen (with the progress bar, while linux loads) can be expensive and slow down things. You can remove the "splash" parameter from your kernel boot (try it out in the same way "profile" was added, as described above) and see if it helps. It did not help in my case. If it does for you, then you need to go change your "defoptions" in /boot/grub/menu.lst and reinstall grub to get the 'splash' parameter removed permanently.
  
• There is an endless thread (3 years old already) here with information about readahead, and if you read it thoroughly, with a lot of information and other tips.
  

As conclusion: I spent over 2 hours last night (not including the time to write this post) to gain 15 seconds during bootup. So I will need to boot my notebook 480 times to recover the initial investment! :) But I don't regret, it was funny and I learned a lot in the process, I think. I would love to learn more tips about it -- it would be much nicer to boot in 20s or so...

Passwordless SSH

I've done this a million times already -- but nevertheless I need to look at a reference every time I do it.

The idea is to be able to ssh from user1@machine1 to user2@machine2, without being asked for a password.

To do it, run the following commands (all of them as user1@machine1):

1. Create a ssa key on machine1:
   ssh-keygen -t rsa
2. Copy the public key over to machine2:
   cat ~/.ssh/id_rsa.pub | ssh user2@machine2 "cat - >> ~/.ssh/authorized_keys"
   (you may need to accept the server's fingerprint (say Yes), and will need to type your password this time).

It's done. Now just ssh to machine2 and it won't ask for a password.

Note: This also works for doing ssh to the same machine, under a different user. Just make sure user1's id_rsa.pub key is added to user2's ~/.ssh/authorized_keys file.

Note2: SSH is very strict with respect to permissions. The .ssh dir MUST have "700" permissions mask, and authorized_keys must be a 600. Anything different (and if you create authorized_keys, it will likely be created with a different access mask) will make this authentication method fail, and move on to the next one (ie. ask for a password).

Installed Python module (thrift) not being picked up

I have been using thrift for some time (compiled from source code), and a recent upgrade of my linux to Jaunty broke something --- I think.

My symptom was trying to run a script that imported Thrift, receiving this message:

Traceback (most recent call last):
 File "./Cassandra-remote", line 11, in <module>
   from thrift.transport import TTransport
ImportError: No module named thrift.transport

However that module file exists in my disk: /usr/lib/python2.6/site-packages/thrift/transport/TTransport.py

It took a while to realize that this path was not being inspected by python:

$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['',
'/usr/lib/python2.6',
'/usr/lib/python2.6/plat-linux2',
'/usr/lib/python2.6/lib-tk',
'/usr/lib/python2.6/lib-old',
'/usr/lib/python2.6/lib-dynload',
'/usr/lib/python2.6/dist-packages',
'/usr/lib/python2.6/dist-packages/Numeric',
'/usr/lib/python2.6/dist-packages/PIL',
'/var/lib/python-support/python2.6',
'/var/lib/python-support/python2.6/gtk-2.0',
'/usr/local/lib/python2.6/dist-packages']

(note that "site-packages" is not in the list). I assume that I configured my thrift code when the paths were different in my system, and some upgrade changed python directories. So when I "make install" thrift, they are still copied into site-packages. Now, having thrift already installed there, it is a matter of adding this dir to the search path. This is done with sys.path.append, according to python's documentation:

>>> sys.path.append('/usr/lib/python2.6/site-packages')
>>> sys.path
['',
'/usr/lib/python2.6',
(...)
'/usr/local/lib/python2.6/dist-packages',
'/usr/lib/python2.6/site-packages']
>>> from thrift.transport import TTransport
(No error message now)

The "sys.path.append" worked, by I need to persist this change. This can be done by changing the variable PYTHONPATH, I am adding this line to my .bashrc:

export PYTHONPATH=/usr/lib/python2.6/site-packages

And that's it. Any new console (on my user at least) gets this path and I can import the thrift module now.

I wonder how to make this change available to all users -- where is the "default" PYTHONPATH defined?

Broken packages trying to get Amarok 2.1 to work on Jaunty

This is not a nice post with solutions to anyone else... only the partially successful attempts to fix my somewhat broken system.

I recently upgraded from Kubuntu 8.10/Intrepid to 9.04/Jaunty. Shortly after I installed Amarok 2.1, which was one of the few things I really wanted from Jaunty. Sadly, amarok did not play any local mp3 file. Googling around, I found that I could be missing libxine1-ffmpeg (or libxine1-allcodecs or libxine1-plugins, which depend on libxine1-ffmpeg). But I was unable to install it:

$ sudo apt-get install libxine1-ffmpeg
(...)
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libxine1-ffmpeg: Depends: libavcodec52 (>= 3:0.svn20090303-1)
but it is not going to be installed or
libavcodec-unstripped-52 (>= 3:0.svn20090303-1) but it is not
going to be installed
E: Broken packages

This is where I got swamped in package versions. Trying to get libavcodec52 to work:

$ sudo apt-get install libavcodec52
(...)
The following packages have unmet dependencies:
 libavcodec52: Depends: libavutil49 (> 3:0.svn20090303-99) but 3:20080706-0.3lenny1 is to be installed or
                        libavutil-unstripped-49 (> 3:0.svn20090303-99) but it is not going to be installed

Let's try with libavutil49:

$ sudo apt-get install libavutil49
(...)
libavutil49 is already the newest version.
libavutil49 set to manually installed.

So libavutil is already installed... but libavcodec52 does not install! Let's try with the other dependency: install libavutil-unstripped-49:

$ sudo apt-get install libavutil-unstripped-49
(...)
The following packages will be REMOVED:
dvdrip ffmpeg k9copy ksubtitleripper libavcodec51 libavdevice52
libavformat52 libavutil49 libmjpegtools-1.9 libmjpegtools0
libpostproc51 libquicktime1 libswscale0 mjpegtools qdvdauthor
subtitleripper transcode videotrans
The following NEW packages will be installed:
libavutil-unstripped-49
(...)
Setting up libavutil-unstripped-49 (3:0.svn20090303-1ubuntu2+unstripped1) ...

At last, some package was installed. But I removed many packages on the way!! I hope I don't find I needed them...

Going backwards, I was able to install now libavcodec52, but this still does not help for libxine1-ffmpeg:

$ sudo apt-get install libavcodec52
...
Setting up libavcodec52 (3:0.svn20090303-1ubuntu6)

$ sudo apt-get install libxine1-ffmpeg
(...)
The following packages have unmet dependencies:
libxine1-ffmpeg: Depends: libavcodec52 (>= 3:0.svn20090303-1) but it is not going to be installed or
libavcodec-unstripped-52 (>= 3:0.svn20090303-1) but it is not going to be installed
E: Broken packages

So I replaced libavcodec52 by libavcodec-unstripped-52:

$ sudo apt-get install libavcodec-unstripped-52
...
The following packages will be REMOVED:
libavcodec52
The following NEW packages will be installed:
libavcodec-unstripped-52
...
Setting up libavcodec-unstripped-52 (3:0.svn20090303-1ubuntu2+unstripped1) ...

Now I can finally install libxine1-ffmpeg:

$ sudo apt-get install libxine1-ffmpeg
...
Setting up libxine1-ffmpeg (1.1.16.3-0ubuntu1) ...

(And I also installed libxine1-plugins just in case).

Now Amarok 2.1.1 can play my mp3's. But it was a painful way and I am pretty sure I have broken other things. So I wonder, if any reader got so far:

• why libavcodec52 on my Jaunty wanted to install libavutil49 versin 3:20080706-0.3lenny1 ?
  
• what's the difference between libavcodec52 and libavcodec-unstripped-52?
• how did I get to the 'broken packages' state? did I do something wrong, or are there still some packages in Jaunty with broken dependencies?
  

Install VMWare in Ubuntu

I did this a couple of times already and there seems to be no repository with .debs for Ubuntu as today. So here is how to install it with (some) package and dependency support:

1. First download from here. Pick the RPM file, in my case it was named like this: VMware-Player-2.5.2-156735.i386.rpm
2. Convert the RPM to a friendly DEB:

   $ sudo alien VMware-Player-2.5.2-156735.i386.rpm --scripts
   vmware-player_2.5.2-156736_i386.deb generated
   

   
   
3. Install the DEB. Note I had a few warnings:

   $ sudo dpkg -i vmware-player_2.5.2-156736_i386.deb
   Selecting previously deselected package vmware-player.
   (Reading database ... 276046 files and directories currently installed.)
   Unpacking vmware-player (from vmware-player_2.5.2-156736_i386.deb) ...
   Setting up vmware-player (2.5.2-156736) ...
   Extracting VMware Installer...done.
   Installing VMware Installer 1.0
   Copying files...
   Configuring...
   Installing VMware Player 2.5.2
   Copying files...
   Configuring...
   Installing VMware Player 2.5.2
   Copying files...
   Configuring...
   
   Traceback (most recent call last):
    File "/tmp/vmis.c32339/install/vmware-installer/python/lib/logging/handlers.py", line 73, in emit
      if self.shouldRollover(record):
    File "/tmp/vmis.c32339/install/vmware-installer/python/lib/logging/handlers.py", line 146, in shouldRollover
      msg = "%s\n" % self.format(record)
    File "/tmp/vmis.c32339/install/vmware-installer/python/lib/logging/__init__.py", line 630, in format
      return fmt.format(record)
    File "/tmp/vmis.c32339/install/vmware-installer/python/lib/logging/__init__.py", line 421, in format
      s = self._fmt % record.__dict__
   UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 62: ordinal not in range(128)
   Installation was successful
   

   
   

I admit I don't know if these errors are critical or not -- let me know if you do. But after installing it this way I am able to use it perfectly (it's been a few days of usage already and I think everything works) .

(The advantage of this method over just downloading a .bin and installing it is that this gets registered in your package manager, and therefore you can uninstall it or cleanly update it later)

Drawing coordinates from a Twitter feed in Google Maps

A friend of mine is in vacations and he started posting coordinates of the places he was visiting on Twitter. It would be nice to show this path on a map, I thought, and that is what I did in one hour last night.

I used the "write less, do more" Javascript library jQuery, along with jTwitter, a utility on twitter that works as a jQuery plugin, in a mashup with google maps.

You can see the result here. Type in the twitter username of someone that is posting coordinates (strings like "(-23.748504,-65.500402)" will be parsed out from any tweet text), and that's it!



Following is most of the code for this. I added a couple of lines to jTwitter to allow to (a) call a custom function after each tweet is rendered, and (b) call another function when everything is done. Then we do this when pressing the Go! button:

 var matches, u = $("input#user").val(), a = [], s = [];
 var fitem = function() { // Called on each tweet. "this" is the tweet object 
  if (matches = this.text.match(/\(-?[0-9]+\.[0-9]+,-?[0-9]+\.[0-9]+\)/g)) {
   for (var i=0; i<matches.length; i++) {
    a.push(matches[i]);
    s.push(this.text);
   }
  }
 };
 var fload = function() { // Called when all tweets were read and written to html
  draw(a, s);
 }
 $("#target").jTwitter({ username: u, onitem: fitem, onload: fload });

And this for drawing the coordinates we just parsed:

function draw(coords, texts) {
 // All coords are texts -- Map them into GLatLng objects
 var points = [];
 for (var i=0; i<coords.length; i++) {
  var m = coords[i].substr(1, coords[i].length-2).split(",");
  points.push(new GLatLng(parseFloat(m[0]), parseFloat(m[1])));
 }
 // Create a map, and the GPolyLine from the points
 var line = new GPolyline(points, "#FF0000", 2.0, 0.8);
 var map = new GMap2(document.getElementById("map"));
 // Center map and set zoom level to show our points
 var bounds = line.getBounds();
 var zoom = map.getBoundsZoomLevel(bounds);
 if (bounds) map.setCenter(bounds.getCenter(), zoom);
 // Render each point as a separate marker (to get a tooltip), and draw the polyline 
 for(var i=0;i<points.length;i++) {
  map.addOverlay(new GMarker(points[i], {
   title: texts[i]
  }));
 }
 map.addOverlay(line);

 // More mapping stuff
 map.addControl(new GMapTypeControl());
 map.addControl(new GScaleControl());
 map.addControl(new GLargeMapControl());
 map.addMapType(G_PHYSICAL_MAP);

It is a toy and with very limited functionality... but it was a nice exercise. I am planning to add features, I think it is a cool idea. I would love to hear if anyone has ideas around this.

Obtain a Yamaha saxophone's birth date, from its serial number

Note to programmers: This is WAY offtopic!

Anyone owning a musical instrument may wonder which is its manufacturing date. I own a saxophone, and sax players over the web seem to be proud of knowing when their instruments were born!

Mine is a Yamaha YTS-25, and as with any Yamaha sax, there is no way to know the manufacturing date given the serial number. They are not sequential. People made some tables of known numbers.... but I found one interesting and suggestion in the web: Just ask customer service for it! So I wrote them (imail1 /at/ yamaha /dot/ com) last Thursday, and got a very kind letter on Monday telling me the month and year it was manufactured.

Now I can say I am happy to play a 16 year old saxophone :) Thanks Yamaha!

Badware report! Hacked?

My blog is about the little hacks I do every day. Like wikipedia's definition #5 of the verb hacking:

• #5. (computing) To accomplish a difficult programming task. He can hack like no one else and make the program work as expected.

Yes, that's my hacking. But today I think I was hacked, sadly with the definition #4 of this term:

• #4. (slang, computing) To hack into; to gain unauthorized access to a computer system (e.g. website) or network by manipulating code; a crack.

I received an hour ago an email starting like:

Dear site owner or webmaster of (mysite),

We recently discovered that some of your pages can cause users to be infected with malicious software. We have begun showing a warning page to users who visit these pages by clicking a search result on Google.com.

...

In fact, if I do a google search that lists my site, it says below "This site can harm your computer". And if you are brave and click the result, an awful page with a big and red Warning! message stops you explaining my site is going to harm you...

It turned out that somehow, I got injected a couple of script tags in some pages. They were an obfuscated javascript code, starting like this:

FBE38="parseI";FBE38+="n";FBE38+="t";DD6EDCFB9FEE="St";
DD6EDCFB9FEE+="rin";DD6EDCFB9FEE+="g.";DD6EDCFB9FEE+="fro";
DD6EDCFB9FEE+="mC";DD6EDCFB9FEE+="harCode";
function F5304B9279(C5F40D2E4C)
....

Well, a fun hacking (definition #5, for us good hackers) exercise is to decode that. I am not going to post the entire code here. It did some wrapping around parseInt(_,16) and String.fromCharCode to build a kind of evaluator of a piece of javascript code, encoded as a very long hexadecimal string. Then, when evaluated, it inserted an iframe linking to a place hosted in hostsads.cn domain. I don't know what they inserted in my page. It might be invisible, it might be harmful and exploit some browser vulnerability.

Scripts removed, passwords reset, a form filled into google for my site to be reviewed and the warning removed.

But.... Did anyone have the same problem? Do you know what they do exactly with the injected iframe? I could find this problem by googling around -- I would love to hear if anyone have seen it.

NO_PUBKEY? During apt-get update

This happens to me pretty frequently, as I jump over different repositories to get ubuntu packages:

W: GPG error: http://ppa.launchpad.net jaunty Release: The following 
signatures couldn't be verified because the public key is not 
available: NO_PUBKEY 2836CB0A8AC93F7A
W: You may want to run apt-get update to correct these problems

Note this happens during apt-get update, so running it is not the solution!

Assuming this is a repository you trust on, let us just add this key to your ubuntu so you can download updates from there:

1. $ gpg --keyserver keyserver.ubuntu.com --recv 2836CB0A8AC93F7A
   gpg: requesting key 8AC93F7A from hkp server keyserver.ubuntu.com
   gpg: key 8AC93F7A: public key "Launchpad Kubuntu Updates" imported
   gpg: Total number processed: 1
   gpg:               imported: 1  (RSA: 1)
   

   (The number 2836CB0A8AC93F7A is the key listed as missing in the error message)
   



2. $ gpg --export --armor 2836CB0A8AC93F7A | sudo apt-key add -
   OK
   

   (Again, replace that value by your key)
   

Voila. Run apt-get update again and you should be able to receive that repositories' packages.

Update: palo points out you can do the same with just this line:

sudo apt-key adv --recv-key --keyserver keyserver.ubuntu.com 2836CB0A8AC93F7A

And that's even shorter -- thanks!

Recovering a lost Windows XP password

I have a computer that run linux 99% of the time... But I also boot windows XP once in a while (every year or so) to do something specific like trying out some 3d game. Don't ask why, but I tend to forget that password. It happened twice already, so I am documenting the workaround for the next time it happens!

Very quickly, here are the steps:

1. Download ophcrack: http://ophcrack.sourceforge.net/
2. Download also some rainbow tables. I have used xp_free_small (380mb). The smallest but not fastest or strongest table -- enough for my purposes.
3. ./configure && make
4. (I skip the make install -- I am going to use the binary that is left in src/)
5. I am running windows in the same computer, so I will just mount the windows partition: sudo mount /dev/sda1 /mnt/xp
6. Click Load -> Encripted SAM. Choose /mnt/xp/WINDOWS/system32/config . This will load all your password hashes directly from that Windows.
   
7. Select accounts you don't want to crack (e.g. you already know the password), to save cpu time, and click Delete on them.
   
8. Tables -> Install , add the one(s) that you downloaded.
9. Click Crack!

That's it. If you are lucky, just as I am, you will have your password shown in screen, in the clear, for anyone looking on your shoulders.

ophcrack has a lot of useful options, I strongly recommend reading at least the help (big Help button on top of main window).

happy cracking!

Trac and Subversion in Apache, on Ubuntu

I got tired of developing senderos.net.ar without source control. Yes, I know, I should have done all this before.

Anyway... These are the notes to setup a subversion repository and a trac server (to browse it and to keep an internal wiki for myself).

I am running Kubuntu 8.10.

These instructions are somewhat based on this tutorial and this blog post (and many other pages I read in between).

I am going to create

• a repository in /home/svnrepo,


• trac environment in /home/tracrepo,


• a single project 'senderos' (subprojects as children, then trunk/tags/branches structure)


• reuse my already working apache server


• use single http authentication
  

Create subversion server.

As root:

svnadmin create /home/svnrepo --fs-type fsfs
chown www-data:www-data /home/svnrepo -R      
vim /home/svnrepo/conf/svnserve.conf
   anon-access = none                        
   auth-access = write                       
   password-db = passwd                      
vim /home/svnrepo/conf/passwd                      
   herchu=myPassword

Test: Run svnserve -d , then stop it.

Apache setup for subversion

As root:

apt-get install libapache2-svn
/etc/init.d/apache2 restart
a2enmod dav_svn
sudo vim /etc/apache2/mods-enabled/dav_svn.conf
  <Location /svn>
    DAVsvn
    SVNPath /home/svnrepo
    AuthType Basic
    AuthName "Earthsea Subversion"
    AuthUserFile /etc/apache2/dav_svn.passwd
    require valid-user
  </Location>
htpasswd -cm /etc/apache2/svn_dav.passwd herchu
  New password: xxx
  Re-type new password: xxx
  Adding password for user herchu
/etc/init.d/apache2 restart

Testing subversion

svn info http://localhost/svn
  Authentication realm: <http://localhost:80> Earthsea Subversion
  Password for 'herchu': ....                                  
          ...                                                          
  Repository UUID: aabbccdd-1122-3344-5566-77889900aabb        
  Revision: 0                                                  
  ...                                                          

Upload my already crowded dirs:

From an empty dir: svn import . http://localhost/svn/senderos/web/trunk
From ~/mapsWeb: svn co http://localhost/svn/senderos/web/trunk .
Then svn add of the files I wanted, and svn ci.

Create the Trac server

Partially based on ubuntu howto

apt-get install trac
trac-admin /home/tracrepo initenv
  Project Name [My Project]> senderos
  Database connection string [sqlite:db/trac.db]> sqlite:db/senderos.db
  Repository type [svn]> svn
  Path to repository [/path/to/repos]> /home/svnrepo
sudo chown www-data:www-data /home/tracrepo/ -R

Test trac with manually loaded server

tracd --port 8000 /home/tracrepo

Open http://localhost:8000 and browse your svn repository and see things are working ok...

Run trac from within apache

This is not installed by default, and I need it:

apt-get install libapache2-mod-python
a2enmod python
sudo /etc/init.d/apache2 restart

More testing...

Temporarily add this in sites-enabled/default
<Location /mpinfo>
   SetHandler mod_python
   PythonInterpreter main_interpreter
   PythonHandler mod_python.testhandler
</Location>

and load http://localhost/mpinfo (If it works, remove this config part)

Finally, setup trac:
Add to sites-enabled/default:

<Location /trac>
   SetHandler mod_python
   PythonInterpreter main_interpreter
   PythonHandler trac.web.modpython_frontend
   PythonOption TracEnv /home/tracrepo
   #   PythonOption TracUriRoot /trac
      PythonOption PYTHON_EGG_CACHE /tmp
</Location>

<Location /trac/login>
   AuthType Basic
   AuthName "senderos"
   AuthUserFile /etc/apache2/dav_svn.passwd
   Require valid-user
</Location>

Make sure you restart your browser while restarting apache!! (you will mess with /login and sessions otherwise)

Check if:

• load http://localhost/trac - Check in "browse" button if the svn repository is browsable
  


• try editing a wiki page (requires to be logged in, you must see a "create" button on unexistent wiki pages)
  

good luck!

first post

testing first post...

I will regularly post any kind of activity to this log.