Quick DVD authoring, with subtitles

Let's say you have a DivX movie (movie.avi) and a subtitles file (subtitles.srt) and you want to burn this into a DVD, which any player (non divx-capable) can play.

This is called DVD authoring, and there are many tools to do the job (though I have not found one that I really like, yet). For such a simple task, a few commands on the console are the simplest option.

1. Transcode move to MPEG-2

See the mencoder documentation at the end - it contains examples for other formats.

mencoder -oac lavc -ovc lavc -of mpeg -mpegopts format=dvd:tsaf \
 -vf scale=720:576,harddup -srate 48000 -af lavcresample=48000 \
 -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=5000:\
keyint=15:vstrict=0:acodec=ac3:abitrate=192:aspect=16/9 -ofps 25 \
 -o movie.mpg movie.avi

2. Render and multiplex subtitles into movie

Create a file like this one, and save it as sub.xml:

<subpictures>
  <stream>
    <textsub filename="subtitles.srt" characterset="ISO-8859-1"
         fontsize="22.0" font="devedesans.ttf"
horizontal-alignment="center"
         vertical-alignment="bottom" left-margin="60"
right-margin="60"
         top-margin="20" bottom-margin="40" subtitle-fps="25"
         movie-fps="25" movie-width="720" movie-height="574"/>
  </stream>
</subpictures>

Run spumux to create a new mpeg file with the subtitles stream:

spumux -s0 sub.xml < movie.mpg > m_subs.mpg

3. Author the DVD

Finally, we need to tell dvdauthor how to create a dvd. Create a file dvd.xml like this one:

<dvdauthor>
    <vmgm>
    <titleset>
        <titles>
            <subpicture lang="es">
            <pgc>
                <vob file="m_subs.mpg">
            </pgc>
        </titles>
    </titleset>
</dvdauthor>

And run dvdauthor:

dvdauthor -o thedvd -x dvd.xml

4. Round up

Voila! The directory thedvd will contain the DVD structure (not an ISO image). You can test the results with mplayer:
mplayer dvd:// -dvd-device thedvd

and if it looks ok, just burn it into a dvd - k3b can burn this (use new Project -> movie DVD)

At the end you can delete *.mpg files, and the thedvd directory.

References:

• mencoder on dvd authroing: http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-vcd-dvd.html
  
• spumux: http://dvdauthor.sourceforge.net/doc/spumux.html
  

thunderbird: very small fonts, and links not opening in browser

I recently installed and new kubuntu on my mother's PC (yes!! finally, wiped out Windows completely!!), and of course I am the help desk support since then...

A few days ago I had the chance to solve to curious issues she was having with thunderbird. I had to google a little for both solutions:

Very small fonts: Almost everything in thunderbird had a very, very small font size: menus, message list, etc. I tried changing DPI resolution in KDE, setting global font preferences por KDE and GTK applications, with no avail. Finally the solution was to tweak a property within thunderbird:

layout.css.cpi = 0

(the default value was unset, or -1, I can't remember). So simple! I don't know what is the meaning of the property, but solved the issue and now all fonts are just like in any other application.

Clicking links does not open a browser: When reading an email with a URL, clicking on them did just nothing. I browsed every possible option in the Preferences windows, but there was nothing that fixed it. The solution, again, was hidden in the advanced preferences. I had to set a few variables:

network.protocol-handler.app.http = /usr/bin/firefox
network.protocol-handler.app.https = /usr/bin/firefox

(that can be done for ftp as well -- and any other protocol if needed).

Note: To edit the "advanced" preferences, go to Edit > Preferences > Advanced > Config Editor. Be aware that you can misconfigure it very easily from there!

Both are things far from what a regular end user could change. I wonder why the default install left things that way....

so many wifi clients!

Last night I ran a apt-get upgrade after some time updating anything -- over a month. It looked like mostly an update for many, many KDE packages (I think I got 4.3.1). Just in case, I rebooted -- I always do it after upgrading many packages in case something goes wrong, to make it evident. Well, something went wrong: the knetworkmanager would not connect again to my WPA-protected home wifi. As a result, I lost a couple of hours of sleep time last night... things are working again, after some fight.

I thought it would be interesting to remember how many wifi clients I used over the last year or so:

• network manager from KDE3: I used this one for a long time. Even after starting using KDE4 (in those scary days of betas of the 4.0.x branch, where virtually nothing worked as expected), this rock-solid tool kept me connected to my wireless network. Its biggest problem was just that it was slated to die; it had the KDE3 look, and would be vanished soon. (Image from softpedia)
  
  
  
• At some point, a plasmoid appeared as a replacement. This one was a nightmare at the beginning, but after some time it started to work. It never worked 100%, there where some inconsistencies (when editing networks' properties, or when showing the current state, some things did not reflect correct status). It worked for some time. (Image taken from here)
  
  
  
• Just a couple of months ago (from KDE 4.3 betas?) the plasma applet changed is icon to something really ugly, and a tooltip message said that its development was incomplete and suggested to use "knetworkmanager" instead. So I moved... and well, this one worked nicely! I thought it was the end of the journey. Well integrated in KDE4, properties appearing in control panel too... some tiny features where appearing from time to time after upgrades. Very nice.
  Sadly, after yesterday's update, it did not connect anymore. I can click the network names I created, and nothing happens. Just that. I don't know how to debug it.
  
  
  
• I should also mention that when these things fail, I also resort to wpa_supplicant, connecting from command line with a config file I already have pre-configured and a command line saved right there as I always, always forget the parameters. But again, this is working 100%. It keeps disconnecting, it shows a lot of errors in the console... It kind of works, and saved me sometimes. Last night it was not the exception, so I did not want to continue using it for work over weeks until finding a solution.
• Looking for solutions last night I found wicd. I just thought 'yet another program to try'... and worse, it depended on gnome libraries so it looked like some gnome-only applet. But comments were very positive, so I tried it. Installing the package also uninstalled knetworkmanager, good job! The package also added a daemon, perfect.... and all I can say is it just works. Ok, the UI is not KDE4-ish. But I can configure everything I need, and it worked perfectly after a couple of reboots. (As a side comment: Ubuntu's repositories had a 1.5.x version, and after installing it and knowing it worked, I moved to wicd's repos which brought 1.6.2 version, which is even nicer).
  
  
  

This can be read as a rant against wifi support on linux. And it is, in part. ((Wireless connections have been more of a problem for me than anything else -- even more than 3d acceleration which has settled down reasonably many months (or years) ago for me. Wireless is still... a pain.)) The web is too confusing as it becomes quickly outdated; when you run a search you will find results for hundreds of methods, drivers, hardware, linux distribution, even different ubuntu versions, and none of them except one will work on your scenario.

My intention is to let others know wicd, and to receive feedback about these things, like: Am I too dumb, that I break my networking so often? Is this regular fight intended for every end user that just intends to keep his ubuntu updated? (I know I play beyond end-user-ness, and I enjoy it, but I also think ubuntu must be something an end user must be able to manage by itself).
Meanwhile... I will happily use wicd -- as long as it works :)

ffmpeg with AMR (.amr/.3gp) and x264 on Ubuntu

A few days ago I needed to convert some AMR audio files (recorded in a phone) to a more common format -- like mp3. I found out this would be difficult since:

• the ffmpeg from Ubuntu repositories does not suppport AMR
• ffmpeg dropped support of libamr as it used to have - now they use lib-opencore-amr

Fortunately I found a LOT of help in Ubuntu forums, thanks mc4man for that! I am going to summarize here what I did to finally decode amr files with ffmpeg. Here we will compile ffmpeg and x264 from source code, and install them as packages so we can remove them at any time later.

Following this how-to by FakeOutdoorsman, I downloaded x264 sources (why not? let's compile to support as many formats as possible), compiled it, and downloaded ffmpeg source code:

1. Make sure you don't have any of this installed from ubuntu:

sudo apt-get purge ffmpeg x264 libx264-dev

2. Download and compile x264: (You may need to install some build tools -- see the original how-to if you don't have them)

cd
git clone git://git.videolan.org/x264.git
cd x264
./configure
make
sudo checkinstall --fstrans=no --install=yes --pkgname=x264 --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" --default

3. Get libopencore-AMR-NB and libopencore-AMR-WB from DebianMultimedia. These are the urls for i386 and amd64. Download the four files libopencore-amr*, and install them with dpkg -i .

4. Download, configure and build ffmpeg. Note that the ./configure step is different than the one in the mentioned how-to -- it adds the flags for AMR.

cd
svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libxvid --enable-x11grab --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-version3
make
sudo checkinstall --fstrans=no --install=yes --pkgname=ffmpeg --pkgversion "3:0.svn`date +%Y%m%d`-12ubuntu3" --default

And it's done! I am now able to transcode my files to mp3:

ffmpeg -i test.amr -f avi -vcodec xvid -acodec libmp3lame -ar 22050 test.mp3

Now if later you want to remove these packages, remove ffmpeg, x264 and the four libopencore-amr* we have installed. And also, if you ever want to upgrade with a newer source code, see the original post; they instructions are there (I have not tested that yet).

that ol'university times...

Here is another offtopic. I found this beatiful piece of software, a kind of 'norton commander' -- ok, just a file browser for MS-DOS.... It features a recursive calculator of directory sizes, and a magnificent hex file viewer. Wow! The amazing part is that this fully written on assembler, , for an assignment a group of four friends (Abel Valente, Juan Francisco Pertino, Alejandro Miguez & me) did for the course "Taller de Arquitectura", during our 3rd year of Licenciatura en Informatica, Faculty of Informatics, University of La Plata, Argentina. One of them and me, in parallel, just remembered this work a couple of days ago: almost exactly 10 years after writing it!

Here are a few screenshots. The main screen with the directory size shown (it surprises me that Windows directory size does not overflow...):


Here, the hex viewer!


And finally, the output when exiting, with the date.... August 1999!


One more: source code.... Over 120kb and assembly, sweat and tears:


So many things learned... We will never forget the conditional jumps from the end of a loop.... to the beginning of a loop in a different procedure! And the magic trick of PUSHing and POPing all the flags to make the problem disappear.... ;)

I still can't believe we did it! Those were one of the best 34,687 bytes we ever produced! It's also hard to believe 10 years passed by. I must be getting old.

Opening an iptables firewall port on RHEL 5

I started a new service on a host running RHEL 5.1... and discovered that the port I needed to connect remotely was closed, the firewall would not let me in.

The error I got was somewhat misleading,

telnet [my host]  12345
Trying [ip address]...
telnet: Unable to connect to remote host: No route to host

however the same command from the same host worked; si I knew it was a firewall problem.

The firewall is iptables, and this tutorial saved my day. I just ran iptables -L to show current rules, verifying there were a few open ports (not including mine) and a last rule to drop every other incoming connection. So I ran iptables -I INPUT -p tcp --dport 12345 -j ACCEPT and that's it... the port is now open.

That's good for testing. Now, to make this persistent, go edit /etc/sysconfig/iptables, and add a line like this:

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 12345 -j ACCEPT

and run /sbin/service/iptables restart

Speed up ubuntu boot time

My notebook has run linux for a couple of years already, upgrading from kubuntu version to version, and having installed and uninstalled countless packages. This impacts in, for example, boot time.... I feel it takes too long to boot.

Here are a few things I have done last night to improve boot time. Some may or may not apply to you. I am running Kubuntu 9.04 Jaunty. Hopefully, you might even suggest more improvements.

Bootchart

First, to analyze booting time, install bootchart. It is a wonderful service that renders a graph of how startup scripts and services are run. It might add a little overhead to your load time, but it will also tell you exactly where and how time is spent. On Ubuntu: apt-get install bootchart. Mine has been there for months, but if you are just installing then boot your linux once so you get at least one chart: check out /var/log/bootchart -- a PNG image and few reports in a .tgz are created. See the image for an example: my boot took 46 seconds.

I am going to list a couple of things I did to cut boot time:

Remove services: You can see in the chart there are a LOT of services being started. I use many of them; I stopped using a few of them and others I seldom use. So I could remove some of them from my startup -- with the risk of having a few things stop working. In other cases, I will just know they are stopped, and will start it when I needed (For example: For work, there are weeks were I use a local apache2 daily; but there are periods where I don't use it -- In this case I am going to start it manually when needed). To remove a process from the startup, run (every command listed here is to be run as root): update-rc.d -f remove. If my notes are correct, these are those I have removed:

• apache2: I don't use it daily
• mysql (3 services starting with 'mysql'): I don't use it daily, either
• tomcat5.5: the same
• bluetoothd: I don't have bluetooth hardware!
• kdm-kde4: This is a leftover from last year (was it in Gutsy? Hardy?) that I installed a KDE4 pre-release, and had the option to choose between KDE 3.x and 4.x. The service for my desktop manager is "kdm" (it appears in the chart), not "kdm-kde4" which just gives an error while running.
• winbindd: used for samba if I am correct - I had a samba server, but not anymore
• firestarted: a software firewall. I am behind a firewalling router already. Sometimes I use firestarter for specific things... let's start it manually when needed.
• exim4: a mail server? I have never used it.
• timidity: a daemon for MIDI emulation... I think I installed it and used it once in a year.
• virtualbox-ose: Another leftover -- I used to run VirtualBox OSE (Open Source Edition), but now I am using the "closed" one. The latter has "vboxdrv" and "vboxnet" services; but virtualbox-ose is some leftover from the older virtualbox.
• apport: A tool for crash reports. I don't know if I should removed this... but I have never filled a crash report.
• avahi-daemon: "discovers network resources".... uhm, I don't know what this is about. I removed it, and will re-add it if needed.
• wclientd: this is for a VPN client I have stopped using some time ago. I have other vpn services (agn*) running, this one is not needed.

I think those are most of the services I have removed. After removing, just booted once... and from the 46s original seconds I am down to about 36s!

View and remove services, graphically: There is a nice tool that lists your startup services, and you can choose to start/stop them for current session or remove them from startup sequence: "BUM", Boot Up Manager. I mostly listed manually my /etc/rc?.d directories and viewed bootchart graphs, but this one is also useful too (and it allows to remove services with a click).

Readahead

Since a few versions ago "readahead" is included by default in ubuntu installs. It is meant to read in advance all or most the files needed to boot, in the best order to maximize disk access speed, and avoid too many random seeks in the hard disk as services are started. In most installes, it uses a prebuilt list of files that are read --in a default ubuntu install--, but if you have upgraded versions, installed packages, or just let your install grow up with time, this list will be likely outdated.

To update the readahead-list, do the following: Boot your machine, and in the GRUB menu press "e" to edit the kernel parameters. Select your kernel, press "e" again and append the word "profile" (separated with a space from the last parameter written there), press Enter to accept this change and press "b" to boot. This boot will take MUCH longer than usual; it will record all disk activity. The good thing is that all the files will be listed to file /etc/readahead/boot, sorted for optimal access. Once linux booted, just reboot (you don't need to change anything in grub, the "profile" thing was a setting for boot only one time).

The original process is shown in the example chart, as taking about 10 seconds. This is actually good: if we don't do this, it would most likely take even more time to read all files needed for services to load. After profiling my boot, this time increased to about 15 seconds. But the overall time was down to 30s! (From the about 36s before readahead). This is because after the length readahead-list operation, all services load much faster.

Note that some people get benefit from readahead, and others do not. You can check if it works for you by disabling it, just emptying the file /var/readahead/boot. Measure the next load to see if readahead is useful for you. For me, it was!

Summary

I just did the two things above: removing services I don't need, and re-profiling to customize my readahead-list. This made my linux boot in 30s, from the 46s that was using -- a big gain! Yet, I am not completely happy - I know it can be done better. So I would love to hear if you have more tips!

Also, once you are done with bootchart you might want to remove it from boot. (I would not uninstall the package, you may want to use it from time to time). Just do "update-rc.d -f bootchart remove" and "update-rc.d -f stop-bootchart remove". To re-add it, you may re-install the package or add it to runlevels 2-3-4-5: update-rc.d stop-bootchart start 99 2 3 4 5

For a couple of alternatives:

• Some people report that the splash screen (with the progress bar, while linux loads) can be expensive and slow down things. You can remove the "splash" parameter from your kernel boot (try it out in the same way "profile" was added, as described above) and see if it helps. It did not help in my case. If it does for you, then you need to go change your "defoptions" in /boot/grub/menu.lst and reinstall grub to get the 'splash' parameter removed permanently.
  
• There is an endless thread (3 years old already) here with information about readahead, and if you read it thoroughly, with a lot of information and other tips.
  

As conclusion: I spent over 2 hours last night (not including the time to write this post) to gain 15 seconds during bootup. So I will need to boot my notebook 480 times to recover the initial investment! :) But I don't regret, it was funny and I learned a lot in the process, I think. I would love to learn more tips about it -- it would be much nicer to boot in 20s or so...