Centralized RSYSLOG Server Monitoring

This tutorial provides step-by-step instructions on how to install RSYSLOG server on Debian 6 (Squeeze) and how to send syslog data to it from multiple client servers. We will also show how to install an Apache webserver and a MySQL database server to gather the syslog data and how to install LogAnalyzer to easily browse the collected data with a web-browser.
I'm using an readymade OpenVZ virtualisation template for this setup but this should also work on most Debian based operating Systems.
I do not issue any guarantee that this will work for you!

Installing RSYSLOG Server

This first part discribes how to build RSYSLOG server that will gather the syslog data from it's clients. In this example our server is called rsyslog.domain.com and it has a fixed IP address of 192.168.0.15.
First we install some dependencies.

apt-get update

apt-get upgrade

apt-get install rsyslog rsyslog-mysql unzip zip binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ mysql-server mysql-client libmysqlclient15-dev apache2 apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libdb4.6-dev libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick php5-imap php5-json php5-mcrypt php5-memcache php5-mhash php5-ming php5-mysql php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl

Create the MySQL server password when the installation asks you to enter it!
My OpenVZ template all ready had this so this command may not be nessecary for you either...

apt-get install linux-kernel-headers

Make sure the services are up and running...

/etc/init.d/rsyslog restart
/etc/init.d/mysql restart
/etc/init.d/apache2 restart

Check that the server is listening on the proper TCP IP ports. (80 & 3306). RSYSLOG does not yet listen on any port at this time.

rsyslog:~# netstat -tapn

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      415/mysqld        
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      581/apache2

Then we can create the rsyslog database:

mysqladmin -u root -p create rsyslog

Next we start the MySQL command shell and create the rsyslog user:

mysql -u root -p

GRANT SELECT, INSERT, UPDATE, DELETE ON rsyslog.* TO 'rsyslog'@'localhost' IDENTIFIED BY 'ENTER-YOUR-NEW-RSYSLOG-PASSWORD-HERE';

FLUSH PRIVILEGES;

quit

Next we configure the rsyslog server to listen on TCP port 514:

vi /etc/rsyslog.conf

Add these lines... (remember to change your password to the one you entered when you created your MySQL server's rsyslog user.)

$ModLoad MySQL
*.*       >127.0.0.1,rsyslog,rsyslog,ENTER-YOUR-NEW-RSYSLOG-PASSWORD-HERE

...and remove the comments in front of the lines that deal with the TCP syslog reception.

# /etc/rsyslog.conf Configuration file for rsyslog v3.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
$ModLoad MySQL
*.* >127.0.0.1,rsyslog,rsyslog,ENTER-YOUR-NEW-RSYSLOG-PASSWORD-HERE
#################
#### MODULES ####
#################
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides --MARK-- message capability
# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
###########################
#### GLOBAL DIRECTIVES ####
###########################

And restart rsyslog service.

/etc/init.d/rsyslog restart

Check that the server is listening on the proper TCP IP ports. (80, 514 & 3306.)

rsyslog:~# netstat -tapn

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:514             0.0.0.0:*               LISTEN      618/rsyslogd   
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      415/mysqld        
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      581/apache2

Then we download the LogAnalyzer and configure the Apache web server to display the logs.

cd /tmp

wget http://download.adiscon.com/loganalyzer/loganalyzer-3.4.1.tar.gz

tar xvzf loganalyzer-3.4.1.tar.gz

mv loganalyzer-3.4.1/ /var/www/

cd /var/www

Then we configure www folders user rights for Apache web server.

chown www-data:www-data * . -Rf

And then we do some 'make nice' work on the LogAnalyzer folders.

mv loganalyzer-3.4.1/ loganalyzer

cd contrib/

cp * ./../src/

cd ./../src/

sh ./configure.sh

And we are ready to enter the final setup of LogAnalyzer using our web-browser. Point your web-browser to the fixed IP address of our rsyslog server that in this demo is http://192.168.0.15/loganalyzer/src/install.php
Go trough the simple setup script (it's pretty much... next -> next.)
Now you should have working rsyslog server with LogAnalyzer up and running.
Next we configure RSYSLOG clients to send their syslog data to the rsyslog server:

Configuring RSYSLOG Clients

Usually all we need to do is configure the rsyslog.conf file and restart the service. (Almost all Debians come with rsyslog preinstalled.)

vi /etc/rsyslog.conf

Add the work spool directory lines that are used in case network connectivity is lost and change your rsyslog server IP ADDRESS to 192.168.0.15. (You may also want to create the /rsyslog/work spool directory with the mkdir command.)

# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
$WorkDirectory /rsyslog/work # default location for work (spool) files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
*.* @@YOUR-RSYSLOG-SERVER-ADDRESS-HERE
###########################
#### GLOBAL DIRECTIVES ####
###########################

And restart rsyslog service.

/etc/init.d/rsyslog restart

Check that the server is connected on the proper TCP IP port (514).

root@ic1:~# netstat -tapn

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 192.168.0.100:49188     192.168.0.15:514         ESTABLISHED 13289/rsyslogd

This is what a successfull setup looks like.