Opening an iptables firewall port on RHEL 5

I started a new service on a host running RHEL 5.1... and discovered that the port I needed to connect remotely was closed, the firewall would not let me in.

The error I got was somewhat misleading,

telnet [my host]  12345
Trying [ip address]...
telnet: Unable to connect to remote host: No route to host

however the same command from the same host worked; si I knew it was a firewall problem.

The firewall is iptables, and this tutorial saved my day. I just ran iptables -L to show current rules, verifying there were a few open ports (not including mine) and a last rule to drop every other incoming connection. So I ran iptables -I INPUT -p tcp --dport 12345 -j ACCEPT and that's it... the port is now open.

That's good for testing. Now, to make this persistent, go edit /etc/sysconfig/iptables, and add a line like this:

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 12345 -j ACCEPT

and run /sbin/service/iptables restart

Speed up ubuntu boot time

My notebook has run linux for a couple of years already, upgrading from kubuntu version to version, and having installed and uninstalled countless packages. This impacts in, for example, boot time.... I feel it takes too long to boot.

Here are a few things I have done last night to improve boot time. Some may or may not apply to you. I am running Kubuntu 9.04 Jaunty. Hopefully, you might even suggest more improvements.

Bootchart

First, to analyze booting time, install bootchart. It is a wonderful service that renders a graph of how startup scripts and services are run. It might add a little overhead to your load time, but it will also tell you exactly where and how time is spent. On Ubuntu: apt-get install bootchart. Mine has been there for months, but if you are just installing then boot your linux once so you get at least one chart: check out /var/log/bootchart -- a PNG image and few reports in a .tgz are created. See the image for an example: my boot took 46 seconds.

I am going to list a couple of things I did to cut boot time:

Remove services: You can see in the chart there are a LOT of services being started. I use many of them; I stopped using a few of them and others I seldom use. So I could remove some of them from my startup -- with the risk of having a few things stop working. In other cases, I will just know they are stopped, and will start it when I needed (For example: For work, there are weeks were I use a local apache2 daily; but there are periods where I don't use it -- In this case I am going to start it manually when needed). To remove a process from the startup, run (every command listed here is to be run as root): update-rc.d -f remove. If my notes are correct, these are those I have removed:

• apache2: I don't use it daily
• mysql (3 services starting with 'mysql'): I don't use it daily, either
• tomcat5.5: the same
• bluetoothd: I don't have bluetooth hardware!
• kdm-kde4: This is a leftover from last year (was it in Gutsy? Hardy?) that I installed a KDE4 pre-release, and had the option to choose between KDE 3.x and 4.x. The service for my desktop manager is "kdm" (it appears in the chart), not "kdm-kde4" which just gives an error while running.
• winbindd: used for samba if I am correct - I had a samba server, but not anymore
• firestarted: a software firewall. I am behind a firewalling router already. Sometimes I use firestarter for specific things... let's start it manually when needed.
• exim4: a mail server? I have never used it.
• timidity: a daemon for MIDI emulation... I think I installed it and used it once in a year.
• virtualbox-ose: Another leftover -- I used to run VirtualBox OSE (Open Source Edition), but now I am using the "closed" one. The latter has "vboxdrv" and "vboxnet" services; but virtualbox-ose is some leftover from the older virtualbox.
• apport: A tool for crash reports. I don't know if I should removed this... but I have never filled a crash report.
• avahi-daemon: "discovers network resources".... uhm, I don't know what this is about. I removed it, and will re-add it if needed.
• wclientd: this is for a VPN client I have stopped using some time ago. I have other vpn services (agn*) running, this one is not needed.

I think those are most of the services I have removed. After removing, just booted once... and from the 46s original seconds I am down to about 36s!

View and remove services, graphically: There is a nice tool that lists your startup services, and you can choose to start/stop them for current session or remove them from startup sequence: "BUM", Boot Up Manager. I mostly listed manually my /etc/rc?.d directories and viewed bootchart graphs, but this one is also useful too (and it allows to remove services with a click).

Readahead

Since a few versions ago "readahead" is included by default in ubuntu installs. It is meant to read in advance all or most the files needed to boot, in the best order to maximize disk access speed, and avoid too many random seeks in the hard disk as services are started. In most installes, it uses a prebuilt list of files that are read --in a default ubuntu install--, but if you have upgraded versions, installed packages, or just let your install grow up with time, this list will be likely outdated.

To update the readahead-list, do the following: Boot your machine, and in the GRUB menu press "e" to edit the kernel parameters. Select your kernel, press "e" again and append the word "profile" (separated with a space from the last parameter written there), press Enter to accept this change and press "b" to boot. This boot will take MUCH longer than usual; it will record all disk activity. The good thing is that all the files will be listed to file /etc/readahead/boot, sorted for optimal access. Once linux booted, just reboot (you don't need to change anything in grub, the "profile" thing was a setting for boot only one time).

The original process is shown in the example chart, as taking about 10 seconds. This is actually good: if we don't do this, it would most likely take even more time to read all files needed for services to load. After profiling my boot, this time increased to about 15 seconds. But the overall time was down to 30s! (From the about 36s before readahead). This is because after the length readahead-list operation, all services load much faster.

Note that some people get benefit from readahead, and others do not. You can check if it works for you by disabling it, just emptying the file /var/readahead/boot. Measure the next load to see if readahead is useful for you. For me, it was!

Summary

I just did the two things above: removing services I don't need, and re-profiling to customize my readahead-list. This made my linux boot in 30s, from the 46s that was using -- a big gain! Yet, I am not completely happy - I know it can be done better. So I would love to hear if you have more tips!

Also, once you are done with bootchart you might want to remove it from boot. (I would not uninstall the package, you may want to use it from time to time). Just do "update-rc.d -f bootchart remove" and "update-rc.d -f stop-bootchart remove". To re-add it, you may re-install the package or add it to runlevels 2-3-4-5: update-rc.d stop-bootchart start 99 2 3 4 5

For a couple of alternatives:

• Some people report that the splash screen (with the progress bar, while linux loads) can be expensive and slow down things. You can remove the "splash" parameter from your kernel boot (try it out in the same way "profile" was added, as described above) and see if it helps. It did not help in my case. If it does for you, then you need to go change your "defoptions" in /boot/grub/menu.lst and reinstall grub to get the 'splash' parameter removed permanently.
  
• There is an endless thread (3 years old already) here with information about readahead, and if you read it thoroughly, with a lot of information and other tips.
  

As conclusion: I spent over 2 hours last night (not including the time to write this post) to gain 15 seconds during bootup. So I will need to boot my notebook 480 times to recover the initial investment! :) But I don't regret, it was funny and I learned a lot in the process, I think. I would love to learn more tips about it -- it would be much nicer to boot in 20s or so...

Passwordless SSH

I've done this a million times already -- but nevertheless I need to look at a reference every time I do it.

The idea is to be able to ssh from user1@machine1 to user2@machine2, without being asked for a password.

To do it, run the following commands (all of them as user1@machine1):

1. Create a ssa key on machine1:
   ssh-keygen -t rsa
2. Copy the public key over to machine2:
   cat ~/.ssh/id_rsa.pub | ssh user2@machine2 "cat - >> ~/.ssh/authorized_keys"
   (you may need to accept the server's fingerprint (say Yes), and will need to type your password this time).

It's done. Now just ssh to machine2 and it won't ask for a password.

Note: This also works for doing ssh to the same machine, under a different user. Just make sure user1's id_rsa.pub key is added to user2's ~/.ssh/authorized_keys file.

Note2: SSH is very strict with respect to permissions. The .ssh dir MUST have "700" permissions mask, and authorized_keys must be a 600. Anything different (and if you create authorized_keys, it will likely be created with a different access mask) will make this authentication method fail, and move on to the next one (ie. ask for a password).

Installed Python module (thrift) not being picked up

I have been using thrift for some time (compiled from source code), and a recent upgrade of my linux to Jaunty broke something --- I think.

My symptom was trying to run a script that imported Thrift, receiving this message:

Traceback (most recent call last):
 File "./Cassandra-remote", line 11, in <module>
   from thrift.transport import TTransport
ImportError: No module named thrift.transport

However that module file exists in my disk: /usr/lib/python2.6/site-packages/thrift/transport/TTransport.py

It took a while to realize that this path was not being inspected by python:

$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['',
'/usr/lib/python2.6',
'/usr/lib/python2.6/plat-linux2',
'/usr/lib/python2.6/lib-tk',
'/usr/lib/python2.6/lib-old',
'/usr/lib/python2.6/lib-dynload',
'/usr/lib/python2.6/dist-packages',
'/usr/lib/python2.6/dist-packages/Numeric',
'/usr/lib/python2.6/dist-packages/PIL',
'/var/lib/python-support/python2.6',
'/var/lib/python-support/python2.6/gtk-2.0',
'/usr/local/lib/python2.6/dist-packages']

(note that "site-packages" is not in the list). I assume that I configured my thrift code when the paths were different in my system, and some upgrade changed python directories. So when I "make install" thrift, they are still copied into site-packages. Now, having thrift already installed there, it is a matter of adding this dir to the search path. This is done with sys.path.append, according to python's documentation:

>>> sys.path.append('/usr/lib/python2.6/site-packages')
>>> sys.path
['',
'/usr/lib/python2.6',
(...)
'/usr/local/lib/python2.6/dist-packages',
'/usr/lib/python2.6/site-packages']
>>> from thrift.transport import TTransport
(No error message now)

The "sys.path.append" worked, by I need to persist this change. This can be done by changing the variable PYTHONPATH, I am adding this line to my .bashrc:

export PYTHONPATH=/usr/lib/python2.6/site-packages

And that's it. Any new console (on my user at least) gets this path and I can import the thrift module now.

I wonder how to make this change available to all users -- where is the "default" PYTHONPATH defined?

Broken packages trying to get Amarok 2.1 to work on Jaunty

This is not a nice post with solutions to anyone else... only the partially successful attempts to fix my somewhat broken system.

I recently upgraded from Kubuntu 8.10/Intrepid to 9.04/Jaunty. Shortly after I installed Amarok 2.1, which was one of the few things I really wanted from Jaunty. Sadly, amarok did not play any local mp3 file. Googling around, I found that I could be missing libxine1-ffmpeg (or libxine1-allcodecs or libxine1-plugins, which depend on libxine1-ffmpeg). But I was unable to install it:

$ sudo apt-get install libxine1-ffmpeg
(...)
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libxine1-ffmpeg: Depends: libavcodec52 (>= 3:0.svn20090303-1)
but it is not going to be installed or
libavcodec-unstripped-52 (>= 3:0.svn20090303-1) but it is not
going to be installed
E: Broken packages

This is where I got swamped in package versions. Trying to get libavcodec52 to work:

$ sudo apt-get install libavcodec52
(...)
The following packages have unmet dependencies:
 libavcodec52: Depends: libavutil49 (> 3:0.svn20090303-99) but 3:20080706-0.3lenny1 is to be installed or
                        libavutil-unstripped-49 (> 3:0.svn20090303-99) but it is not going to be installed

Let's try with libavutil49:

$ sudo apt-get install libavutil49
(...)
libavutil49 is already the newest version.
libavutil49 set to manually installed.

So libavutil is already installed... but libavcodec52 does not install! Let's try with the other dependency: install libavutil-unstripped-49:

$ sudo apt-get install libavutil-unstripped-49
(...)
The following packages will be REMOVED:
dvdrip ffmpeg k9copy ksubtitleripper libavcodec51 libavdevice52
libavformat52 libavutil49 libmjpegtools-1.9 libmjpegtools0
libpostproc51 libquicktime1 libswscale0 mjpegtools qdvdauthor
subtitleripper transcode videotrans
The following NEW packages will be installed:
libavutil-unstripped-49
(...)
Setting up libavutil-unstripped-49 (3:0.svn20090303-1ubuntu2+unstripped1) ...

At last, some package was installed. But I removed many packages on the way!! I hope I don't find I needed them...

Going backwards, I was able to install now libavcodec52, but this still does not help for libxine1-ffmpeg:

$ sudo apt-get install libavcodec52
...
Setting up libavcodec52 (3:0.svn20090303-1ubuntu6)

$ sudo apt-get install libxine1-ffmpeg
(...)
The following packages have unmet dependencies:
libxine1-ffmpeg: Depends: libavcodec52 (>= 3:0.svn20090303-1) but it is not going to be installed or
libavcodec-unstripped-52 (>= 3:0.svn20090303-1) but it is not going to be installed
E: Broken packages

So I replaced libavcodec52 by libavcodec-unstripped-52:

$ sudo apt-get install libavcodec-unstripped-52
...
The following packages will be REMOVED:
libavcodec52
The following NEW packages will be installed:
libavcodec-unstripped-52
...
Setting up libavcodec-unstripped-52 (3:0.svn20090303-1ubuntu2+unstripped1) ...

Now I can finally install libxine1-ffmpeg:

$ sudo apt-get install libxine1-ffmpeg
...
Setting up libxine1-ffmpeg (1.1.16.3-0ubuntu1) ...

(And I also installed libxine1-plugins just in case).

Now Amarok 2.1.1 can play my mp3's. But it was a painful way and I am pretty sure I have broken other things. So I wonder, if any reader got so far:

• why libavcodec52 on my Jaunty wanted to install libavutil49 versin 3:20080706-0.3lenny1 ?
  
• what's the difference between libavcodec52 and libavcodec-unstripped-52?
• how did I get to the 'broken packages' state? did I do something wrong, or are there still some packages in Jaunty with broken dependencies?