Friday, September 18, 2009

iotop, your best friend when LED of HD stays on

The server is slow to hell, badly accessing remotely, a simple ls it’s enougth to take a lot of seconds running… openning a program, no way… lot of minutes, time to take a cup of coffee.
You, don’t understanding what’s goin on, go get a look at the server, sometimes wishing to reset it, then, surprisely see that the HD LED look’s like the POWER one, but, Red…

Then think: “Whata…”


In situations like that where are a lot of disk activities, come the question “What now?”, how to verify what is doing such a lot of disk access?

There are tools for that, like iostat and vmstat columns bi and bo

Iostat is a tool that’s is very handy in cases like this one, but his values interpretation and the way that the values are on screen aren’t frendly.

For that, there’s a very handy tool, iotop, it have a very nice CLI interface that show the I/O data. Utilize the very own values that kernel pass throught and show us in a very elegant style :)
iotop_small
Needs to have Python >= 2.5 and kernel Linux >= 2.6.20 with those options enabled
  • CONFIG_TASKSTATS
  • CONFIG_TASK_DELAY_ACCT
  • CONFIG_TASK_IO_ACCOUNTING

Pre-requisites

1.wget http://guichaz.free.fr/iotop/files/iotop-0.3.1.tar.bz2
2.tar jxvf iotop-0.3.1.tar.bz2 -C /usr/src
3.cd /usr/src/iotop-0.3.1/
To execute the program without installing it, just call via CLI:
1../iotop.py
In case that the following error comes:
01.Traceback (most recent call last):
02.  File "./iotop.py", line 8, in ?
03.    from iotop.ui import main
04.  File "/usr/src/iotop-0.3.1/iotop/ui.py", line 12, in ?
05.    from iotop.data import find_uids, TaskStatsNetlink, ProcessList
06.  File "/usr/src/iotop-0.3.1/iotop/data.py", line 12, in ?
07.    from iotop import ioprio, vmstat
08.  File "/usr/src/iotop-0.3.1/iotop/ioprio.py", line 1, in ?
09.    import ctypes
It’s because you haven’t the python installed os it isn’t the 2.5 version. It’s necessary to install it.

Installing Python 2.5 on Red Hat 5.3 / CentOS 5.3

Red Hat 5 / CentOS 5 comes with python 2.4 by default. It isn’t cool to simply install the new version over the default one, instead, I’m disponibilizing here the RPMs of python 2.5 for Red Hat 5 / CentOS 5 that will be installed in his own directory /usr/bin/python25, so, the scripts that Yum and others scripts made use of python 2.4 won’t get hurt.
The news is, always that you need to use the new version of python 2.5 you’ll have to change the interpretor PATH that call python25. It’s better than the whole system going unstable, isn’t it? ;)

Executing iotop.py

With python25 installed, it’s time to modify the script iotop.py and change the interpreter on the first line, from:
1.#!/usr/bin/python
to
1.#!/usr/bin/python25
Ok, time to run the script
1../iotop.py
If any of the pre-requisites won’t be safisfied, a message like this one will show informing what is missing
1.Could not run iotop as some of the requirements are not met:
2.- Python >= 2.5 for AF_NETLINK support: Found
3.- Linux >= 2.6.20 with I/O accounting support (CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT, CONFIG_TASK_IO_ACCOUNTING): Not found
Python 2.5 it’s OK, but kernel isn’t.
Simple, compiling kernel with those options:
To compile the new kernel, there’s a post here that explains it, here I’ll show you just which options to be considered, compile and install the kernel it’s with you now ;)
1.General setup  --->
2....
3.[*] Export task/process statistics through netlink (EXPERIMENTAL)
4.[*]   Enable per-task delay accounting (EXPERIMENTAL)
5.[*]   Enable extended accounting over taskstats (EXPERIMENTAL)
6.[*]     Enable per-task storage I/O accounting (EXPERIMENTAL)
7....
Screenshot-3

Kernel 2.6.30.5 (rpm) with process statistics ready to be used with iotop

To install
1.rpm -ivh kernel-2.6.30.5fogonacaixadagua-1.i386.rpm
in case you already installed this kernel before, will need to generate again an image initrd and use the flag --force with rpm -ivh --force command.

iotop running

Screenshot-root@localhost:-usr-src-iotop-0.3.1

Installing iotop

Into the directory you extracted iotop-0.3.1 run the command
./setup.py install
1.cd /usr/src/iotop-0.3.1
2../setup.py install
How it was installed python 2.5, if you run /usr/bin/iotop the following error will be gracefuly show to you
1.Traceback (most recent call last):
2.File "/usr/bin/iotop", line 4, in
3.import pkg_resources
4.ImportError: No module named pkg_resources
To resolve that one, it’s quite simple, into directory where you extracted iotop, in this case /usr/src/iotop-0.3.1 copy the entire directory iotop to /usr/lib/python-2.5
1.cd /usr/src/iotop-0.3.1
2.cp -Rp iotop /usr/lib/python2.5/
And after that, edit the file /usr/bin/iptop and modify the line of interpreter
from
1.#!/usr/bin/python
to
1.#!/usr/bin/python25
Ok, iotop running!

No comments:

Post a Comment