Thursday, July 24, 2014

Linux / Unix logtop: Realtime Log Line Rate Analyser

How can I analyze line rate taking log file as input on a Linux system? How do I find the IP flooding my Apache/Nginx/Lighttpd web-server on a Debian or Ubuntu Linux?

Tutorial details
DifficultyEasy (rss)
Root privilegesYes
Estimated completion timeN/A
You need to use a tool called logtop. It is a system administrator tool to analyze line rate taking log file as input. It reads on stdin and print a constantly updated result displaying, in columns in the following format: Line number, count, frequency, and the actual line

How do install logtop on a Debian or Ubuntu based system?

Simply type the following apt-get command:
$ sudo apt-get install logtop
Sample outputs:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
0 upgraded, 1 newly installed, 0 to remove and 3 not upgraded.
Need to get 15.7 kB of archives.
After this operation, 81.9 kB of additional disk space will be used.
Get:1 precise/universe logtop amd64 0.3-1 [15.7 kB]
Fetched 15.7 kB in 0s (0 B/s)
Selecting previously unselected package logtop.
(Reading database ... 114954 files and directories currently installed.)
Unpacking logtop (from .../logtop_0.3-1_amd64.deb) ...
Processing triggers for man-db ...
Setting up logtop (0.3-1) ...


The syntax is as follows:
logtop [OPTIONS] [FILE]
command | logtop
command1 | filter | logtop
command1 | filter | logtop [options] [file]


Here are some common examples of logtop.

Show the IP address flooding your LAMP server

Type the following command:
tail -f www.cyberciti.biz_access.log | cut -d' ' -f1 | logtop
Sample outputs:
Fig.01: logtop command in action
Fig.01: logtop command in action

See squid cache HIT and MISS log

tail -f cache.log | grep -o "HIT\|MISS" | logtop
To see realtime hit / miss ratio on some caching software log file, enter:
tail -f access.log | cut -d' ' -f1 | logtop -s 20000
The -s option set logtop to work with the maximum of K lines instead of 10000.

Monday, July 21, 2014

How to set up a highly available Apache cluster using Heartbeat

A highly available cluster uses redundant servers to ensure maximum uptime. Redundant nodes mitigate risks related to single points of failure. Here's how you can set up a highly available Apache server cluster on CentOS.
Heartbeat provides cluster infrastructure services such as inter-cluster messaging, node memberships, IP allocation and migration, and starting and stopping of services. Heartbeat can be used to build almost any kind of highly available clusters for enterprise applications such as Apache, Samba, and Squid. Moreover, it can be coupled with load balancing software so that incoming requests are shared by all cluster nodes.
Our example cluster will consist of three servers that run Heartbeat. We'll test failover by taking down servers manually and checking whether the website they serve is still available. Here's our testing topology:
Topology The IP address against which the services are mapped needs to be reachable at all time. Normally Heartbeat would assign the designated IP address to a virtual network interface card (NIC) on the primary server for you. If the primary server goes down, the cluster will automatically shift the IP address to a virtual NIC on another of its available servers. When the primary server comes back online, it shifts the IP address back to the primary server again. This IP address is called "floating" because of its migratory properties.

Install packages on all servers

To set up the cluster, first install the prerequisites on each node using yum:
yum install PyXML cluster-glue cluster-glue-libs resource-agents
Next, download and install two Heartbeat RPM files that are not available in the official CentOS repository.
rpm -ivh heartbeat-*
Alternatively, you can add the EPEL repository to your sources and use yum for the installs.
Heartbeat will manage starting up and stopping Apache's httpd service, so stop Apache and disable it from being automatically started:
service httpd stop
chkconfig httpd off

Set up hostnames

Now set the server hostnames by editing /etc/sysconfig/network on each system and changing the HOSTNAME line:
The new hostname will activate at the next server boot-up. You can use the hostname command to immediately activate it without restarting the server:
You can verify that the hostname has been properly set by running uname -n on each server.

Configure Heartbeat

To configure Heartbeat, first copy its default configuration files from /usr to /etc/ha.d/:
cp /usr/share/doc/heartbeat-3.0.4/authkeys /etc/ha.d/
cp /usr/share/doc/heartbeat-3.0.4/ /etc/ha.d/
cp /usr/share/doc/heartbeat-3.0.4/haresources /etc/ha.d/
You must then modify all three files on all of your cluster nodes to match your requirements.
The authkeys file contains the pre-shared password to be used by the cluster nodes while communicating with each other. Each Heartbeat message within the cluster contains the password, and nodes process only those messages that have the correct password. Heartbeat supports SHA1 and MD5 passwords. In authkeys, the following directives set the authentication method as SHA1 and define the password to be used:
auth 2
2 sha1 pre-shared-password
Save the file, then give it permissions of r-- with the command chmod 600 /etc/ha.d/authkeys.
Next, in, define timers, cluster nodes, messaging mechanisms, layer 4 ports, and other settings:
## logging ##
logfile        /var/log/ha-log
logfacility     local0hea

## timers ##
## All timers are set in seconds. Use 'ms' if you need to define time in milliseconds. ##

## heartbeat intervals ##
keepalive 2

## node is considered dead after this time ##
deadtime 15

## some servers take longer time to boot. this timer defines additional time to wait before confirming that a server is down ##
##  the recommended time for this timer is at least twice of the dead timer ##
initdead 120

## messaging parameters ##
udpport        694

bcast   eth0
## you can use multicasts or unicasts as well ##

## node definitions ##
## make sure that the hostnames match uname -n ##

Finally, the file haresources contains the hostname of the server that Heartbeat considers the primary node, as well as the floating IP address. It is vital that this file be identical across all servers. As long as the primary node is up, it serves all requests; Heartbeat stops the highly available service on all other nodes. When Heartbeat detects that that primary node is down, it automatically starts the service on the next available node in the cluster. When the primary node comes back online, Heartbeat sets it to take over again and serve all requests. Finally, this file contains the name of the script that is responsible for the highly available service: httpd in this case. Other possible values might be squid, smb, nmb, or postfix, mapping to the name of the service startup script typically located in the /etc/init.d/ directory.
In haresources, define to be the primary server, to be the floating IP address, and httpd to be the highly available service. You do not need to create any interface or manually assign the floating IP address to any interface – Heartbeat takes care of that for you: httpd
After the configuration files are ready on each of the servers, start the Heartbeat service and add it to system startup:
service heartebeat start
chkconfig heartbeat on
You can keep an eye on the Heartbeat log with the command tailf /var/log/ha-log.
Heartbeat can be used to for multiple services. For example, the following directive in haresources would make Heartbeat manage both Apache and Samba services: httpd smb nmb
However, unless you're also running a cluster resource manager (CRM) such as Pacemaker, I do not recommend using Heartbeat to provide mulitple services in a single cluster. Without Pacemaker, Heartbeat monitors cluster nodes in layer 3 using IP addresses. As long as an IP address is reachable, Heartbeat is oblivious to any crashes or difficulties that services may be facing on a server node.


Once Heartbeat is up and running, test it out. Create separate index.html files on all three servers so you can see which server is serving the page. Browse to or, if you have DNS set up, its domain name equivalent. The page should be loaded from, and you can check this by looking at the Apache log file in server1. Try refreshing the page and verify whether the page is being loaded from the same server each time.
If this goes well, test failover by stopping the Heartbeat service on The floating IP address should be migrated to server 2, and the page should be loaded from there. A quick look into server2 Apache log should confirm the fact. If you stop the service on server2 as well, the web pages will be loaded from, the only available node in the cluster. When you restart the services on server1 and server2, the floating IP address should migrate from the active node to server1, per the setup in haresources.
As you can see, it's easy to set up a highly available Apache cluster under CentOS using Heartbeat. While we used three servers, Heartbeat should work with more or fewer nodes as well. Heartbeat has no constraint on the number of nodes, so you can scale the setup as you need.

Friday, July 18, 2014

Counting lines of code with cloc

Are you working on a project and need to submit your progress, statistics or perhaps you need to calculate a value of your code? cloc is a powerful tool that allows you to count all lines of your code, exclude comment lines and white space and even sort it by programming language.

cloc is available for all major Linux distributions. To install cloc on your system simply install cloc package from system's package repository:
# apt-get install cloc
# yum install cloc
cloc work on per file or per directory basis. To count the lines of the code simply point cloc to a directory or file. Let's create my_project directory with single bash script:
$ mkdir my_project
$ cat my_project/ 

echo "hello world"
Let cloc to count the lines of our code:
$ cloc my_project/ 
       1 text file.
       1 unique file.                              
       0 files ignored. v 1.60  T=0.00 s (262.8 files/s, 788.4 lines/s)
Language                     files          blank        comment           code
Bourne Shell                     1              1              0              2
Let's add another file by this time with perl code and count the line of code by pointing it to the entire directory rather then just a single file:
$ cat my_project/

print "hello world\n"
$ ls my_project/
$ cloc my_project/
       2 text files.
       2 unique files.                              
       0 files ignored. v 1.60  T=0.01 s (287.8 files/s, 863.4 lines/s)
Language                     files          blank        comment           code
Perl                             1              1              0              2
Bourne Shell                     1              1              0              2
SUM:                             2              2              0              4
In the next example we will print results for each file separately on each line. This can be done by the use of --by-file option:
$ cloc --by-file my_project/
       2 text files.
       2 unique files.                              
       0 files ignored. v 1.60  T=0.01 s (149.5 files/s, 448.6 lines/s)
File                              blank        comment           code
my_project/                    1              0              2
my_project/                    1              0              2
SUM:                                  2              0              4

cloc can obtain count of all code lines also from a compressed file. In the next example we count code lines of entire joomla project, provided the we have already downloaded its zipped source code:
$ cloc /tmp/
count lines of code - compressed file
Count lines of currently running kernel's source code ( redhat/fedora ):
$ cloc /usr/src/kernels/`uname -r`
count lines of kernel source code
For more information and options see cloc manual page man cloc

Wednesday, July 16, 2014

How to check RPM package dependencies on Fedora, CentOS or RHEL

A typical RPM package on Red Hat-based systems requires all its dependent packages be installed to function properly. For end users, the complexity of such RPM dependency is hidden by package managers (e.g., yum or DNF) during package install/upgrade/removal process. However, if you are a sysadmin or a RPM maintainer, you need to be well-versed in RPM dependencies to maintain run-time environment for the system or roll out up-to-date RPM specs.
In this tutorial, I am going to show how to check RPM package dependencies. Depending on whether a package is installed or not, there are several ways to identify its RPM dependencies.

Method One

One way to find out RPM dependencies for a particular package is to use rpm command. The following command lists all dependent packages for a target package.
$ rpm -qR

Note that this command will work only if the target package is already installed. If you want to check package dependencies for any uninstalled package, you first need to download the RPM package locally (no need to install it).
To download a RPM package without installing it, use a command-line utility called yumdownloader. Install yumdownloader as follows.
$ sudo yum install yum-utils
Now let's check RPM depenencies of a uninstalled package (e.g., tcpdump). First download the package in the current folder with yumdownloader:
$ yumdownloader --destdir=. tcpdump
Then use rpm command with "-qpR" options to list dependencies of the downloaded package.
# rpm -qpR tcpdump-4.4.0-2.fc19.i686.rpm

Method Two

You can also get a list of dependencies for a RPM package using repoquery tool. repoquery works whether or not a target package is installed. This tool is included in yum-utils package.
$ sudo yum install yum-utils
To show all required packages for a particular package:
$ repoquery --requires --resolve

For repoquery to work, your computer needs network connectivity since repoquery pulls information from Yum repositories.

Method Three

The third method to show RPM package dependencies is to use rpmreaper tool. Originally this tool is developed to clean up unnecessary packages and their dependencies on RPM-based systems. rpmreaper has an ncurses-based intuitive interface for browsing installed packages and their dependency trees.
To install rpmrepater, use yum command. On CentOS, you need to set up EPEL repo first.
$ sudo yum install rpmreaper
To browser RPM dependency trees, simply run:
$ rpmreaper

The rpmrepater interface will show you a list of all installed packages. You can navigate the list using up/down arrow keys. Press "r" on a highlighted package to show its dependencies. You can expand the whole dependency tree by recursively pressing "r" keys on individual dependent packages. The "L" flag indicates that a given package is a "leaf", meaning that no other package depends on this package. The "o" flag implies that a given package is in the middle of dependency chain. Pressing "b" on such a package will show you what other packages require the highlighted package.

Method Four

Another way to show package dependencies on RPM-based systems is to use rpmdep which is a command-line tool for generating a full package dependency graph of any installed RPM package. The tool analyzes RPM dependencies, and produce partially ordered package lists from topological sorting. The output of this tool can be fed into dotty graph visualization tool to generate a dependency graph image.
To install rpmdep and dotty on Fedora:
$ sudo yum install rpmorphan graphviz
To install the same tools on CentOS:
$ wget
$ sudo rpm -ivh rpmorphan-1.14-1.noarch.rpm
$ sudo yum install graphviz
To generate and plot a dependency graph of a particular installed package (e.g., gzip):
$ -dot gzip
$ dot -Tpng -o output.png

So far in this tutorial, I demonstrate several ways to check what other packages a given RPM package relies on. If you want to know more about .deb package dependencies for Debian-based systems, you can refer to this guide instead.

Linux Terminal: inxi – a full featured system information script

Sometimes it’s useful to know which components you are using on a GNU/Linux computer or server, you can go with the long way, taking a look at the boot message for all the hardware discovered, use some terminal commands such as lsusb,lspci or lshw or some graphical tools such as hardinfo (my favourite graphical tool) or Inex/CPU-G.
But I’ve discovered on my Linux Mint, that, by default, I’ve now a new option: inxi
inxi it’s a full featured system information script wrote in bash, that easily will show on a terminal all the info of your system.

Inxi comes pre-installed with SolusOS, Crunchbang, Epidemic, Mint, AntiX and Arch Linux but as it is a bash script it works on a lot of other distributions. Although it is intended for use with chat applications like IRC it also works from a shell and provides an abundance of information, It is is a fork of locsmif’s largely unmaintained yet very clever, infobash script. inxi is co-developed, a group project, primarily with trash80 on the programming side.
Inxi works on Konversation, Xchat, irssi, Quassel, as well as on most other IRC clients. Quassel includes (usually an older version of) inxi.
Installation is as easy as downloading and chmoding a file.


Inxi is present in the default repository of most distros so you can install it (if you are missing it) with these commands:
# Ubuntu/Debian users
$ sudo apt-get install inxi
# CentOS/Fedora users
$ sudo yum install inxi
# Arch
$ sudo pacman -s inxi
If inxi is not present on your distro, then you can install it by following the instructions here

Basic Usage

Just open a terminal (with a normal user) and give the command inxi, this will show up the basic information of your system (in colors !!), something like this:
linuxaria@mint-desktop ~ $ inxi
CPU~Dual core Intel Pentium CPU G620 (-MCP-) clocked at 1600.000 Mhz Kernel~3.13.0-24-generic x86_64 Up~8:20 Mem~2814.4/7959.2MB HDD~644.1GB(16.8% used) Procs~221 Client~Shell inxi~1.8.4
Ok, interesting but what if you would like some more info ?
Don’t worry the commands it’s full of options, some are:
-A Show Audio/sound card information.
-C Show full CPU output, including per CPU clockspeed.
-D Show full hard Disk info, not only model, ie: /dev/sda ST380817AS 80.0GB. See also -x and -xx.
-F Show Full output for inxi. Includes all Upper Case line letters, plus -s and -n.
Does not show extra verbose options like -x -d -f -u -l -o -p -t -r unless you use that argument.
-G Show Graphic card information (card, x type, resolution, glx renderer, version).
-I Show Information: processes, uptime, memory, irc client, inxi version.
-l Show partition labels. Default: short partition -P. For full -p output, use: -pl (or -plu).
-n Show Advanced Network card information. Same as -Nn. Shows interface, speed, mac id, state, etc.
-N Show Network card information. With -x, shows PCI BusID, Port number.
And this is just a short list of all the options you can get, as alternatively you could use the -v (verbosity) flag:
-v Script verbosity levels. Verbosity level number is required. Should not be used with -b or -F
Supported levels: 0-7 Example: inxi -v 4
0 – Short output, same as: inxi
1 – Basic verbose, -S + basic CPU + -G + basic Disk + -I.
2 – Adds networking card (-N), Machine (-M) data, shows basic hard disk data (names only),
and, if present, basic raid (devices only, and if inactive, notes that). similar to: inxi -b
3 – Adds advanced CPU (-C), network (-n) data, and switches on -x advanced data option.
4 – Adds partition size/filled data (-P) for (if present):/, /home, /var/, /boot
Shows full disk data (-D).
5 – Adds audio card (-A); sensors (-s), partition label (-l) and UUID (-u), short form of optical drives,
standard raid data (-R).
6 – Adds full partition data (-p), unmounted partition data (-o), optical drive data (-d), full raid.
7 – Adds network IP data (-i); triggers -xx.
This is an example of output with -v 7
linuxaria@mint-desktop ~ $ inxi -v7 -c 0
System:    Host: mint-desktop Kernel: 3.13.0-24-generic x86_64 (64 bit, gcc: 4.8.2) 
           Desktop: Xfce 4.11.6 (Gtk 2.24.23) Distro: Linux Mint 17 Qiana
Machine:   Mobo: ASRock model: H61M-HVS Bios: American Megatrends version: P1.50 date: 11/04/2011
CPU:       Dual core Intel Pentium CPU G620 (-MCP-) cache: 3072 KB flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 10377 
           Clock Speeds: 1: 1600.00 MHz 2: 1600.00 MHz
Graphics:  Card: Advanced Micro Devices [AMD/ATI] Park [Mobility Radeon HD 5430] bus-ID: 01:00.0 
           X.Org: 1.15.1 drivers: ati,radeon (unloaded: fbdev,vesa) Resolution: 1920x1080@60.0hz 
           GLX Renderer: Gallium 0.4 on AMD CEDAR GLX Version: 3.0 Mesa 10.1.0 Direct Rendering: Yes
Audio:     Card-1: Intel 6 Series/C200 Series Chipset Family High Definition Audio Controller driver: snd_hda_intel bus-ID: 00:1b.0
           Card-2: Advanced Micro Devices [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300 Series] driver: snd_hda_intel bus-ID: 01:00.1
           Sound: Advanced Linux Sound Architecture ver: k3.13.0-24-generic
Network:   Card-1: Realtek RTL8101E/RTL8102E PCI Express Fast Ethernet controller 
           driver: r8169 ver: 2.3LK-NAPI port: d000 bus-ID: 03:00.0
           IF: eth0 state: down mac: bc:5f:f4:12:18:d3
           Card-2: D-Link DWA-125 Wireless N 150 Adapter(rev.A3) [Ralink RT5370] 
           driver: rt2800usb ver: 2.3.0 usb-ID: 2001:3c19
           IF: wlan0 state: up mac: 28:10:7b:42:3e:82
           WAN IP: IF: eth0 ip: N/A ip-v6: N/A IF: wlan0 ip: ip-v6: fe80::2a10:7bff:fe42:3e82 
Drives:    HDD Total Size: 644.1GB (16.8% used) 1: id: /dev/sda model: ST500DM002 size: 500.1GB serial: W2AGA8A2 
           2: id: /dev/sdb model: SanDisk_SDSSDP12 size: 126.0GB serial: 134736401617 
           3: id: /dev/sdd model: SD/MMC size: 2.0GB serial: 058F63646476-0:0 
           4: USB id: /dev/sdc model: DataTraveler_G3 size: 16.0GB serial: 001CC0EC30C8BAB085FE002F-0:0 
           Optical: /dev/sr0 model: N/A rev: N/A dev-links: cdrom
           Features: speed: 12x multisession: yes audio: yes dvd: yes rw: cd-r,cd-rw,dvd-r,dvd-ram state: N/A
Partition: ID: / size: 25G used: 5.1G (22%) fs: ext4 dev: /dev/sdb1 
           label: N/A uuid: 133f805a-3963-42ef-a3b4-753db11789df
           ID: /ssd size: 91G used: 24G (28%) fs: ext4 dev: /dev/sdb2 
           label: N/A uuid: 4ba69219-75e4-44cc-a2ee-ccefddb82718
           ID: /home size: 416G used: 60G (16%) fs: btrfs dev: /dev/sda6 
           label: N/A uuid: 20d66995-8107-422c-a0d9-f731e1e02078
           ID: /media/linuxaria/3634-3330 size: 1.9G used: 1.9G (99%) fs: vfat dev: /dev/sdd1 
           label: N/A uuid: 3634-3330
           ID: /media/linuxaria/KINGSTON size: 15G used: 11G (70%) fs: vfat dev: /dev/sdc1 
           label: KINGSTON uuid: 25B5-AD6B
           ID: swap-1 size: 4.00GB used: 0.00GB (0%) fs: swap dev: /dev/sda5 
           label: N/A uuid: 85e49559-db67-41a6-9741-4efc3f2aae1f
RAID:      System: supported: N/A
           No RAID devices detected - /proc/mdstat and md_mod kernel raid module present
           Unused Devices: none
Unmounted: ID: /dev/sda1 size: 50.00G label: N/A uuid: a287ff9c-1eb5-4234-af5b-ea92bd1f7351 
           ID: /dev/sr0 size: 1.07G label: N/A uuid: N/A 
Sensors:   System Temperatures: cpu: 38.0C mobo: N/A gpu: 52.0 
           Fan Speeds (in rpm): cpu: N/A 
Info:      Processes: 219 Uptime: 8:26 Memory: 2611.9/7959.2MB Runlevel: 2 Gcc sys: 4.8.2 Client: Shell inxi: 1.8.4
As you can see this output show a looot more information, you can get a long output also with the option -F (full output).
As last thing, if you are using an Xterm you can choose which color scheme use, and to see which one are available just use the command: inxi -c 94, you’ll get an output similar to this one:
inxi color
Inxi in action:

Tuesday, July 15, 2014

Georgia Tech researchers enlist owners of websites -- and website users -- via Encore project

Georgia Tech researchers are seeking the assistance of website operators to help better understand which sites are being censored and then figure out how to get around such restricted access by examining the data collected.
The open source Encore [Enabling Lightweight Measurements of Censorship with Cross-Origin Requests] tool involves website operators installing a single line of code onto their sites, and that in turn will allow the researchers to determine whether visitors to these sites are blocked from visiting other sites around the world known to be censored. The researchers are hoping to enlist a mix of small and big websites, and currently it is running on about 10 of them.
Georgia Tech Encore tool Georgia Tech
The code works in the background after a page is loaded and Georgia Tech’s team claims the tool won’t slow performance for end users or websites, nor does it track browsing behavior.
+Also on NetworkWorld: 13 of today's Coolest Network Research Projects +
Featured Resource
Presented by Dell Inc.
Improvements in 10GbE technology, lower pricing, and improved performance make 10GbE for the mid-market
Learn More
"Web censorship is a growing problem affecting users in an increasing number of countries," said Sam Burnett, the Georgia Tech Ph.D. candidate who leads the project, in a statement. "Collecting accurate data about what sites and services are censored will help educate users about its effects and shape future Internet policy discussions surrounding Internet regulation and control."
(Burnett’s adviser is Nick Feamster, whose Internet censorship research we’ve written about in the past. I exchanged email with Feamster to gain additional insight into this new research.)
End users won’t even know the baseline data measurement is taking place, which of course when you’re talking about censorship and privacy, can be a sticky subject. Facebook learned that recently when disclosures erupted regarding its controversial secret study of users’ moods. The Georgia Tech researchers in an FAQ say their tool can indicate to users that their browsers are conducting measurements, and that users can opt out.
"Nothing would pop up [in an end user's browser] but a webmaster has an option to make the measurements known/visible," Feamster says.
"They also assure potential Encore users that the list of censored sites compiled by Herdict does not include pornographic ones, so an end user’s browser won’t be directed to such sites in the name of research.
Encore, which is being funded by a National Science Foundation grant on censorship measurement and circumvention as well as via a Google Focused Research Award, has been submitted in hopes of presenting it at the Internet Measurement Conference in November in Vancouver.

How To Enable Storage Pooling And Mirroring Using Btrfs For Linux

If you have multiple hard drives in your Linux system, you don’t have to treat them all as different storage devices. With Btrfs, you can very easily create a storage pool out of those hard drives.
Under certain conditions, you can even enable mirroring so you won’t lose your data due to hard drive failure. With everything set up, you can just throw whatever you want into the pool and make the most use of the storage space you have.
There isn’t a GUI configuration utility that can make all of this easier (yet), but it’s still pretty easy to do with the command line. I’ll walk you through a simple setup for using several hard drives together.

What’s Btrfs?

Btrfs (called B-tree filesystem, “Butter FS”, or “Better FS”) is an upcoming filesystem that incorporates many different features at the filesystem level normally only available as separate software packages. While Btrfs has many noteworthy features (such as filesystem snapshots), the two we’re going to take a look at in this article are storage pooling and mirroring.
If you’re not sure what a filesystem is, take a look at this explanation of a few filesystems for Windows. You can also check out this nice comparison of various filesystems to get a better idea of the differences between existing filesystems.
Btrfs is still considered “not stable” by many, but most features are already stable enough for personal use — it’s only a few select features where you might encounter some unintended results.
While Btrfs aims to be the default filesystem for Linux at some point in the future, it’s still best to use ext4 for single hard drive setups or for setups that don’t need storage pooling and mirroring.

Pooling Your Drives

For this example, we’re going to use a four hard drive setup. There are two hard drives (/dev/sdb and /dev/sdc) with 1TB each, and two other hard drives (/dev/sdd and /dev/sde) with 500GB for a total of four hard drives with a total of 3TB of storage.
You can also assume that you have another hard drive (/dev/sda) of some arbitrary size which contains your bootloader and operating system. We’re not concerning ourselves about /dev/sda and are solely combining the other four hard drives for extra storage purposes.

Creating A Filesystem

btrfs gparted   How To Enable Storage Pooling And Mirroring Using Btrfs For Linux

To create a Btrfs filesystem on one of your hard drives, you can use the command:sudo mkfs.btrfs /dev/sdb
Of course, you can replace /dev/sdb with the actual hard drive you want to use. From here, you can add other hard drives to the Btrfs system to make it one single partition that spans across all hard drives that you add. First, mount the first Btrfs hard drive using the command:
sudo mount /dev/sdb /mnt
Then, run the commands:
sudo mkfs.btrfs /dev/sdc mkfs.btrfs /dev/sdd mkfs.btrfs /dev/sde
Now, you can add them to the first hard drive using the commands:
sudo btrfs device add /dev/sdc /mnt btrfs device add /dev/sdd /mnt btrfs device add /dev/sde /mnt
If you had some data stored on the first hard drive, you’ll want the filesystem to balance it out among all of the newly added hard drives. You can do this with the command:
sudo btrfs filesystem balance /mnt
Alternatively, if you know before you even begin that you want a Btrfs filesystem to span across all hard drives, you can simply run the command:
sudo mkfs.btrfs -d single /dev/sdb /dev/sdc /dev/sdd /dev/sde
Of course this is much easier, but you’ll need to use the method mentioned above if you don’t add them all in one go.
You’ll notice that I used a flag: “-d single”. This is necessary because I wanted a RAID 0 configuration (where the data is split among all the hard drives but no mirroring occurs), but the “single” profile is needed when the hard drives are different sizes. If all hard drives were the same size, I could instead use the flag “-d raid0″. The “-d” flag, by the way, stands for data and allows you to specify the data configuration you want. There’s also an “-m” flag which does the exact same thing for metadata.
Besides this, you can also enable RAID 1 using “-d raid1″ which will duplicate data across all devices, so using this flag during the creation of the Btrfs filesystem that spans all hard drives would mean that you only get 500GB of usable space, as the three other hard drives are used for mirroring.
Lastly, you can enable RAID 10 using “-d raid10″. This will do a mix of both RAID 0 and RAID 1, so it’ll give you 1.5TB of usable space as the two 1TB hard drives are paired in mirroring and the two 500GB hard drives are paired in mirroring.

Converting A Filesystem

btrfs harddiskstack   How To Enable Storage Pooling And Mirroring Using Btrfs For Linux

If you have a Btrfs filesystem that you’d like to convert to a different RAID configuration, that’s easily done. First, mount the filesystem (if it isn’t already) using the command:sudo  mount /dev/sdb1 /mnt
Then, run the command:
sudo btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt
This will change the configuration to RAID 1, but you can replace that with whatever configuration you want (so long as it’s actually allowed — for example, you can’t switch to RAID 10 if you don’t have at least four hard drives). Additionally, the -mconvert flag is optional if you’re just concerned about the data but not the metadata.

If Hard Drive Failure Occurs

If a hard drive fails, you’ll need to remove it from the filesystem so the rest of the pooled drives will work properly. Mount the filesystem with the command:
sudo mount -o degraded /dev/sdb /mnt
Then fix the filesystem with:
sudo btrfs device delete missing /mnt
If you didn’t have RAID 1 or RAID 10 enabled, any data that was on the failed hard drive is now lost.

Removing A Hard Drive From The Filesystem

Finally, if you want to remove a device from a Btrfs filesystem, and the filesystem is mounted to /mnt, you can do so with the command:
sudo btrfs device delete /dev/sdc /mnt
Of course, replace /dev/sdc with the hard drive you want to remove. This command will take some time because it needs to move all of the data off the hard drive being removed, and will likewise fail if there’s not enough room on the other remaining hard drives.

Automatic Mounting

btrfs fstab   How To Enable Storage Pooling And Mirroring Using Btrfs For Linux

If you want the Btrfs filesystem to be mounted automatically, you can place this into your /etc/fstab file:sudo /dev/sdb /mnt btrfs device=/dev/sdb,device=/dev/sdc,device=/dev/sdd,device=/dev/sde 0 0

Mount Options

One more bonus tip! You can optimize Btrfs’s performance in your /etc/fstab file under the mount options for the Btrfs filesystem. For large storage arrays, these options are best: compress-force=zlib,autodefrag,nospace_cache. Specifically, compress=zlib will compress all the data so that you can make the most use of the storage space you have. For the record, SSD users can use these options: noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache. These options go right along with the device specifications, so a complete line in /etc/fstab for SSD users would look like:
sudo /dev/sdb /mnt btrfs device=/dev/sdb,device=/dev/sdc,device=/dev/sdd,device=/dev/sde,
noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache 0 0

How Big Is Your Storage Pool?

Btrfs is a fantastic option for storage pooling and mirroring that is sure to become more popular once it is deemed completely stable. It also wouldn’t hurt for there to be a GUI to make configuration easier (besides in some distribution installers), but the commands you have to use in the terminal are easy to grasp and apply.
What’s the biggest storage pool you could make? Do you think storage pools are worthwhile? Let us know in the comments!

How to set up two-factor authentication for SSH login on Linux

With many high-profile password leaks nowadays, there is a lot of buzz in the industry on "multi-factor" authentication. In a multi-factor authentication system, users are required to go through two distinct authentication procedures: providing something they know (e.g., username/password), and leveraging something they have "physical" access to (e.g., one-time passcode generated by their mobile phone). This scheme is also commonly known as two-factor authentication or two-step verification.

To encourage the wide adoption of two-factor authentication, Google released Google Authenticator, an open-source application that can generate one-time passcode based on open standards (e.g., HMAP/time-based). It is available on multiple platforms including Linux, Android, iOS. Google also offers a pluggable authentication module (PAM) for Google Authenticator, allowing it to be integrated with other PAM-enabled applications such as OpenSSH.

In this tutorial, I will describe how to set up two-factor authentication for an SSH server by integrating Google Authenticator with OpenSSH. I am going to use a Android device to generate one-time passcode. In this tutorial, you will need two things: (1) a Linux host where OpenSSH server is running, and (2) an Android device.

Install Google Authenticator on Linux

The first step is to install Google Authenticator on the Linux host where OpenSSH server is running. Follow this guide to install Google Authenticator and its PAM module on your system.
Once Google Authenticator is ready, you need to go through one-time configuration which involves creating an authentication key from this Linux host, and registering it with an Android device. This will be explained next.

Generate an Authentication Key

To start, simply run Google Authenticator on the Linux server host.
$ google-authenticator
You will see a QR code, as well as a secret key underneath it. The displayed QR code simply represents the numeric secret key. You will need either information to finalize configuration with an Android device.

Google Authenticator will ask you several questions. If you are not sure, you an answer "Yes" to all questions. The emergency scratch codes can be used to regain access to the SSH server in case you lose your Android device, and so cannot generate one-time passcode. So it's better to write them down somewhere.

Run Google Authenticator on Android

As we are going to use an Android device for two-factor authentication, you will need to install Google Authenticator app on Android. Go to Google Play to install it on Android.
When you start Google Authenticator on Android, you will see the following configuration menu.

You can choose either "Scan a barcode" or "Enter provided key" option. The first option allows you to enter the security key, simply by scanning the generated QR code. In this case, you will need to install Barcode Scanner app first. If you choose the second option, you can type the security key using Android keyboard as follows.

Once you register a secret key either way, you will see the following screen on Android.

Enable Google Authenticator on SSH Server

The final step is to integrate Google Authenticator with OpenSSH server. For that, you need to edit two files.
First, edit a PAM configuration file, and append the line below.
$ sudo vi /etc/pam.d/sshd
auth required
Then open an SSH server config file, search for ChallengeResponseAuthentication, and enable it.
$ sudo vi /etc/ssh/sshd_config
ChallengeResponseAuthentication yes
Finally, restart SSH server.
On Ubuntu, Debian or Linux Mint:
$ sudo service ssh restart
On Fedora:
$ sudo systemctl restart sshd
On CentOS or RHEL:
$ sudo service sshd restart

Test Two-factor Authentication

Here is how you use two-factor authentication for SSH logins.
Run Google Authenticator app on Android to obtain one-time verification code. Once generated, a given passcode is valid for 30 seconds. Once it expires, Google Authenticator will automatically generate a new one.

Now log in to the SSH server as you normally do.
$ ssh user@ssh_server
When you are asked to enter "Verification code", type in the verification code generated by Android. After successful verification, then you can type in your SSH login password.

To conclude, two-factor authentication can be an effective means to secure password authentication by adding an extra layer of protection. You can use Google Authenticator to secure other logins such as Google account,,,, etc. Whether you decide to use it or not, it's up to you, but there is a clear industry trend towards the adoption of two-factor authentication.

Open source tools: Five outstanding audio editors

Whether you're producing podcasts or creating highly sophisticated sound recordings, one of these open source apps will suit your needs.
Image: Nivens
A solid audio editor might not seem to belong at the top of your must-have list. It is, however, a tool that can go a long way toward helping you with your business. How? With an audio editor, you can add audio to your business website, create and edit a podcast to help promote your service or product, record and submit audio for radio ads, and more. But what software titles are available from the open source community? Believe it or not, some of the finest audio editors available are open source and offer power and options you might expect only in costly, proprietary software.
Let's take a look at five open source audio editors and see if there's one that will fit your bill.
Note: This article is also available as an image gallery.

1: Audacity

Audacity (Figure A) is the software I've been using for years to record Zombie Radio. It's a powerful multi-track recording app, and it's easy to use. Audacity allows you to record live audio, record from your desktop, convert old tapes/records, edit various formats, cut/copy/splice/mix audio, add effects, change speed/pitch, and much more. At first blush, you might think Audacity is an out-of-date application. But do not let appearances fool you. Audacity is one of the single best recording apps I've ever used. For features and ease of use, you can't beat this recording tool. Audacity is available for Linux, Windows, and Mac.

Figure A

Figure A

2: Ardour

Now we're talking real recording power. Ardour (Figure B) is a digital audio workstation that isn't for the faint of heart. It is to musicians, engineers, soundtrack editors, and composers what Audacity is to podcasters -- the best tool for the job. Not only can you record audio from multiple inputs, you can cut, move, stretch, copy, paste, delete, align, trim, crossfade, rename, snapshot, zoom, transpose, quantize, swing, drag, and drop. The caveat to all of this power is that Ardour comes with a steep learning curve, and It's overkill for podcasters and those wanting to create simple sound recordings.

Figure B

Figure B
Hundreds of plugins are available for this amazing piece of software. The best way to experience Ardour is by downloading and installing Ubuntu Studio or installing on OS X.

3: Traverso

Traverso (Figure C) leans more toward Audacity, but it relies upon the same underlying system that Ardour does: Jack. So although the interface is vastly easier to use than Ardour's, the foundation for connecting to devices (mics, instruments, etc.) is far more complex than Audacity.

Figure C

Figure C
You can use Traverso for a small scale recording session on a netbook or scale up to recording a full-blown orchestra. One outstanding feature that's built into Traverso is the ability to burn your recording straight to CD from within the UI itself. Once you're finished with a project, just burn it and you're done. Traverso is available only for Linux.

4: QTractor

QTractor (Figure D) is another digital audio workstation that requires the Jack Audio Connection Kit. QTractor is a multi-track audio and MIDI sequencing and recording studio. It requires a much better understanding of Jack than Traverso does. But it also delivers a level of power you won't find with lesser applications.

Figure D

Figure D
QTractor lets you drag, move, drop, cut, copy, paste, paste-repeat, delete, split, and merge. It offers unlimited undo/redo, has a built-in patch bay, and much more. QTractor is a great solution for anyone who wants the power of Jack but not the massive complexity (or flexibility and feature set) of Ardour. QTractor is available only for Linux.

5: Linux Multimedia Studio (LMMS)

Linux Multimedia Studio (Figure E) is geared toward songwriters, offering a beat editor and an FX mixer. LMMS includes an incredible array of effects and an impressive number of instruments. With LMMS you can compose entire songs without plugging in a single instrument. Just drag and drop an instrument plug-in to the song editor and you're good to go.

Figure E

LMMS does have a fairly steep learning curve, so be prepared to spend some time getting up to speed with the interface and tools. The name Linux Multimedia Studio a bit misleading, as it is actually available for both Linux and Windows.

Audio tasks?

If you're looking for an audio editor, and you don't want to shell out the money for proprietary software, you don't have to worry about losing features or power. The five editors listed here will get your job done and done right.
How do you make use of audio? Do you use it for training, marketing, PR? Or is audio yet to make its way into your business plan?

Monday, July 14, 2014

How to use systemd for system administration on Debian

Soon enough, hardly any Linux user will be able to escape the ever growing grasp that systemd imposes on Linux, unless they manually opt out. systemd has created more technical, emotional, and social issues than any other piece of software as of late. This predominantly came to show in the heated discussions also dubbed as the 'Init Wars', that occupied parts of the Debian developer body for months. While the Debian Technical Comittee finally decided to include systemd in Debian 8 "Jessie", there were efforts to supersede the decision by a General Resolution, and even threats to the health of developers in favor of systemd.
This goes to show how deep systemd interferes with the way of handling Linux systems that has, in large parts, been passed down to us from the Unix days. Theorems like "one tool for the job" are overthrown by the new kid in town. Besides substituting sysvinit as init system, it digs deep into system administration. For right now a lot of the commands you are used to will keep on working due to the compatibility layer provided by the package systemd-sysv. That might change as soon as systemd 214 is uploaded to Debian, destined to be released in the stable branch with Debian 8 "Jessie". From thereon, users need to utilize the new commands that come with systemd for managing services, processes, switching run levels, and querying the logging system. A workaround is to set up aliases in .bashrc.
So let's have a look at how systemd will change your habits of administrating your computers and the pros and cons involved. Before making the switch to systemd, it is a good security measure to save the old sysvinit to be able to still boot, should systemd fail. This will only work as long as systemd-sysv is not yet installed, and can be easily obtained by running:
# cp -av /sbin/init /sbin/init.sysvinit
Thusly prepared, in case of emergency, just append:
to the kernel boot-time parameters.

Basic Usage of systemctl

systemctl is the command that substitutes the old "/etc/init.d/foo start/stop", but also does a lot more, as you can learn from its man page.
Some basic use-cases are:
  • systemctl - list all loaded units and their state (where unit is the term for a job/service)
  • systemctl list-units - list all units
  • systemctl start [NAME...] - start (activate) one or more units
  • systemctl stop [NAME...] - stop (deactivate) one or more units
  • systemctl disable [NAME...] - disable one or more unit files
  • systemctl list-unit-files - show all installed unit files and their state
  • systemctl --failed - show which units failed during boot
  • systemctl --type=mount - filter for types; types could be: service, mount, device, socket, target
  • systemctl enable debug-shell.service - start a root shell on TTY 9 for debugging
For more convinience in handling units, there is the package systemd-ui, which is started as user with the command systemadm.
Switching runlevels, reboot and shutdown are also handled by systemctl:
  • systemctl isolate - take you to what you know as init 5, where your X-server runs
  • systemctl isolate - take you to what you know as init 3, TTY, no X
  • systemctl reboot - shut down and reboot the system
  • systemctl poweroff - shut down the system
All these commands, other than the ones for switching runlevels, can be executed as normal user.

Basic Usage of journalctl

systemd does not only boot machines faster than the old init system, it also starts logging much earlier, including messages from the kernel initialization phase, the initial RAM disk, the early boot logic, and the main system runtime. So the days where you needed to use a camera to provide the output of a kernel panic or otherwise stalled system for debugging are mostly over.
With systemd, logs are aggregated in the journal which resides in /var/log/. To be able to make full use of the journal, we first need to set it up, as Debian does not do that for you yet:
# addgroup --system systemd-journal
# mkdir -p /var/log/journal
# chown root:systemd-journal /var/log/journal
# gpasswd -a $user systemd-journal
That will set up the journal in a way where you can query it as normal user. Querying the journal with journalctl offers some advantages over the way syslog works:
  • journalctl --all - show the full journal of the system and all its users
  • journalctl -f - show a live view of the journal (equivalent to "tail -f /var/log/messages")
  • journalctl -b - show the log since the last boot
  • journalctl -k -b -1 - show all kernel logs from the boot before last (-b -1)
  • journalctl -b -p err - shows the log of the last boot, limited to the priority "ERROR"
  • journalctl --since=yesterday - since Linux people normally do not often reboot, this limits the size more than -b would
  • journalctl -u cron.service --since='2014-07-06 07:00' --until='2014-07-06 08:23' - show the log for cron for a defined timeframe
  • journalctl -p 2 --since=today - show the log for priority 2, which covers emerg, alert and crit; resembles syslog priorities emerg (0), alert (1), crit (2), err (3), warning (4), notice (5), info (6), debug (7)
  • journalctl > yourlog.log - copy the binary journal as text into your current directory
Journal and syslog can work side-by-side. On the other hand, you can remove any syslog packages like rsyslog or syslog-ng once you are satisfied with the way the journal works.
For very detailed output, append "systemd.log_level=debug" to the kernel boot-time parameter list, and then run:
# journalctl -alb
Log levels can also be edited in /etc/systemd/system.conf.

Analyzing the Boot Process with systemd

systemd allows you to effectively analyze and optimize your boot process:
  • systemd-analyze - show how long the last boot took for kernel and userspace
  • systemd-analyze blame - show details of how long each service took to start
  • systemd-analyze critical-chain - print a tree of the time-critical chain of units
  • systemd-analyze dot | dot -Tsvg > systemd.svg - put a vector graphic of your boot process (requires graphviz package)
  • systemd-analyze plot > bootplot.svg - generate a graphical timechart of the boot process

systemd has pretty good documentation for such a young project under heavy developement. First of all, there is the 0pointer series by Lennart Poettering. The series is highly technical and quite verbose, and holds a wealth of information. Another good source is the distro agnostic Freedesktop info page with the largest collection of links to systemd resources, distro specific pages, bugtrackers and documentation. A quick glance at:
# man systemd.index
will give you an overview of all systemd man pages. The command structure for systemd for various distributions is pretty much the same, differences are found mainly in the packaging.

Monday, July 7, 2014

5 Free Tools for Compliance Management

Most IT pros consider compliance a hassle. Yet the tools of compliance can empower security technologies and simplify risk management. Better yet, some of those tools are free.


Many organizations must comply with regulations such as HIPAA, and the numbers are growing, fueled by constantly evolving legislation that creates new rules, requirements and auditing procedures.
Compliance requirements are often seen as an unnecessary burden that was legislated into existence to protect external entities. However, properly enforced compliance policies can protect organizations from a myriad of problems – ranging from security breaches to lawsuits to corporate espionage.

Compliance's Relationship to Security

Compliance has a symbiotic relationship with the procedures and requirements dictated by computer security. Compliance, like security, is all about managing risk. The risk associated with compliance failures can include financial impact (fines), data loss (intrusions), lost business (customer impacts) or even a suspension of operations.
The Danger Deepens: 2014 Neustar Annual DDoS Attacks and Impact Report

The risks associated with a failure to properly secure IT are similar, if not identical. The only major difference is that most security practices are optional, while compliance practices are required.
While it is easy to see how security and compliance go hand in hand with risk management, the realization does nothing to ease the burdens of compliance and security. It does, however, give some insight into how those burdens can be reduced. Unifying risk management, security management and risk management can lead to an economy of scale, creating efficiencies that lessen the burdens imposed, both in time and budgets.

How Tools Can Help

However, it takes more than an ideology of unification to solve those problems; it takes tangible elements as well – starting with the proper tools. Unified security management tools that offer integration and management modules can often combine risk management, compliance initiatives and security controls into a single managed element, converting compliance to little more than an extension of policy-based security enforcement.
With the proper tool set, compliance management and risk management can become natural extensions of security management, offering managers a clear path to establishing compliance, protecting data and enforcing policy. That holistic approach will reduce costs, while enhancing the benefits of all three.
The market has become all but flooded with compliance tools, yet few of those tools include all of the needed capabilities to combine compliance management with other security capabilities, such as intrusion detection and prevention systems (IDPS),  next generation firewall (NGFW), anti-malware and so on. All of these are rapidly becoming a concern for organizations charged with compliance regulations.
With that in mind, it becomes clear that IT managers may have to build their own solutions and integrate off-the-shelf products with other solutions. Luckily for those choosing a path of self-development, several free tools can become part of an integrated solution. In no particular order, here are five tools that can help IT pros seeking to comply with various regulations:

Wednesday, July 2, 2014

3 open source content management systems compared

Whether you need to set up a blog, a portal for some specific usage, or any other website, which content management system is right for you? is a question you are going to ask yourself early on. The most well-known and widely used open source content management system (CMS) platforms are: Joomla, Wordpress, and Drupal. They are all based on PHP and MySQL and offer a wide range of options to users and developers alike.
To help you choose between these three excellent open source CMS platforms, I've written a comparison based on this criteria: installation complexity, available plugin/themes, ease of use, and more.

Installation time and complexity

Installation is the first thing you would need to do before you start using a CMS, so lets have a look at what it takes to install these tools.
Drupal is considered by many to be the most complex of them all to install and use, but that's simply not true anymore. Drupal has evolved and the process is fairly simple. Download the files from the website, unzip and place the contents in the root folder of your webserver. Then access the root folder from you browser. From there on, you just let the software do it for you. But remember to create a database for your Drupal site and keep the database user name and password on hand before you start the installation process.
Like Drupal, Joomla also needs you to provide the database name during the installation. The installation process in Joomla is similar to Drupal except for a few extra options that Joomla provides during installation. For example, you can choose if your Joomla site should be offline after installation, and you get to see all the configurations before the final installation happens. Also, as a security feature, the installer requires removing the installation code folder after installation.
Most people think that Wordpress is the most easy to use of these three CMS tools. Rightly so. Wordpress requires the same information as the other two, but this is nicely hidden behind two stages of installation. The first part is the creation of config.php file (all of the information about the database, username/password, database host etc. goes in the file). Once this is done, there's just one click for installation of Wordpress. If you have a config.php file ready (from your previous installation or if you manually created it) there is no need to do the first step.  The installer automatically searches for the file and takes you to config.php file creation only if it is not present.
In summary
Installation of all three of these tools is easy and similar with only a few noticeable differences. While Drupal installation looks and feels a bit lengthy, Joomla provides few extra options and a secure feature of installer files deletion. Wordpress has a minimal interface and the quick installation feels nice, but it doesn’t let you configure much during installation. However, all of them need basic information like database name, user ID, and password, among others.

Plugin and theme availability

This is another important aspect of choosing a CMS. You don’t want to get stuck with a CMS that has too few plugins and themes available, because if you don’t find what you want, you may need to get one built as per your requirements and that will directly impact the overall cost of you project! Lets have a look at the total number of plugins and themes available for each of the CMSs in question, though it is possible that you may not find what you want even if the there are more available; but the higher the count the greater the probability that you will find what you are looking for.
At the time of writing this article, Drupal’s official website lists 1223 themes and 14369 modules (plugins are called modules) which are available for free download. This is a pretty good number. If you want to find Drupal themes outside of the theme marketplace though, you will be more hard pressed.
Joomla's official website lists 7437 plugins, and there is no information about themes. But the theme marketplaces have relatively more Joomla themes available than Drupal themes.
If you consider only the numbers, Wordpress wins this round hands down. With 2176 themes and 28593 plugins available on the official website, it quite clearly shows the might of the community behind Wordpress. Even the marketplaces have many Wordpress themes available. This huge number is also attributed to the popularity Wordpress has over other CMS solutions.
In summary
Wordpress' count is not simply an indicator of how good a CMS is, rather it is an indication of how popular it is. Also, there is catch here: as many opine, Wordpress needs more plugins because there are fewer core CMS features supported by Wordpress out of the box. Features such as user access control (syndication, news feed management etc.) have to be implemented using plugins, probably because it evolved (or still evolving) from a blogging tool to a full fledged CMS. But then, community support and the peace of mind that comes with it, is equally important. With a bigger community you can be assured that tomorrow if there is security loophole uncovered that will get fixed quickly.

Ease of use

This is another important aspect of having a CMS. You know that your CMS has many features, but you will need to use them without having the time to read the user manual. So, how easy or difficult it is to figure out things by yourself matters a lot.
Drupal provides some very important features in a very simple and basic user interface (UI). Once you login to the admin account, you have a menu bar on the top, showing all the important aspects of your Drupal site. There is a content link, which shows you a list of all the content and comments on your site and lets you add or manage them. For example, for publish/remove. Other links in the menu are also quite intuitive: Structure, Appearance, People, Modules, Configurations, and Reports. With each name, you can probably guess what’s in there.
When you login to the Joomla admin page for the first time, you will probably feel a little lost. With so many menus on the page, both vertical and horizontal, it is a bit difficult to understand what’s what. But then you will recognize the menu on the left side of the page is just a collection of important links from the main menu on the top. As with Drupal, Joomla lists all the major aspects of the site as different menu items, and below each menu item there is a drop down with more links. Overall the interface of Joomla admin is more polished and refined (compared to Drupal) and also provides more fine-tuned control over the website, but the downside is if you are new to Joomla you will find too many buttons and links all over the place, and it may be difficult to understand their use without looking at the documentation.
Wordpress lives up to being simple and easy to use. The interface is minimal and uses easy to understand language which makes a difference, especially to novices. For example, the button in the admin landing page says "Customize Your Site," encouraging users to go ahead and try it. Compared to the Joomla/Drupal interface that uses more technical language, Wordpress definitely has an edge here.
For websites managed by users with little or no technical background, or small websites with frequent updates required, Wordpress is probably the way to go. The interface is very simple, and you don’t really need to hire someone to do the stuff for you. But if you don’t mind playing around a little and learning things along the way, Joomla is a lot more interesting. It has loads and loads of settings and controls, which let you manage the site to a greater extent. Even Drupal lets you do the same, with a more simple but robust looking interface.

Customization and upgrades

How you can customize and upgrade the CMS is another important aspect you will want to think over before deciding which platform to use. With time, any CMS needs to be upgraded for security or functionality or other reasons, and you may not like to be stuck with a system that is difficult to update or maintain. Also, many times the out of the box solution e.g. themes or plugins are not exactly the way you want them to be, but very close to it. So, you may want to customize things yourself in such cases. Although, customization requires a level of technical expertise, user experience makes the difference. Let’s see how easy or difficult it is to customize or upgrade these CMSs.
After some research I found that, the only way to upgrade a Drupal installation is to do it manually, i.e. backup old files and data, extract the Drupal latest package, and replace all the old files except /sites folder (contains themes and other data) and any other files added. This may sound like a tough task for someone new to the field, there is a certain degree of risk involved as well, and if anything goes wrong you may loose your website altogether. But, if you are an expert, or don’t mind getting expert help, there is no need to worry. Again, to customize your theme, there is no in-application support and you will need to either install a new plugin, which lets you edit themes, or do the customization offline.
Joomla supports upgrading the core from the backend, i.e. you login to the backend, go to Joomla update component (version >= 2.5.4) or Update tab in Joomla Extension Manager (version < 2.5.4), and click install update. That’s it! However, in certain cases, this update method cannot be used. Other methods to update Joomla are Install method, where you select an update file and then tell Joomla to install it and manually update, where you need to manually replace the files. Do remember to always keep a back up before attempting any updates. As far as editing themes is concerned, you need to edit them offline or install the theme editor plugin.
Like Joomla, Wordpress also supports online updates via the admin user interface. Wordpress alerts you whenever there is an update available, if you want to update, just click on update now and Wordpress is updated to latest version! Of course you can take the manual route to update as well. Another interesting feature is the online file editing. It lets you customize your themes or plugins by editing the files in the application itself. Suppose you don’t like an image which is embedded in the theme, and there is no theme setting to change it. Just head over to Administration > Appearance > Editor menu, select the file which you think has that image and edit it. Then you can straightaway review your change as well. Similarly even plugins can be updated; the editor can be found at Administration > Plugins > Editor.
In summary
Wordpress is the winner for customization and upgrades. That means it will be easy if you alone or a small team of people are planning to set up the website. Having said that, Joomla and Drupal can’t be simply written off. Joomla has update features and although Drupal doesn’t offer that right now it has other critical features that make it a leading CMS.