Thursday, March 29, 2012

Using Nagios to Monitor Your Clusters’ Health


The Nagios network monitoring and alerting framework lets you easily keep track of a wide variety of hosts and services, and generate reports and alerts targeted to specific teams or individuals. By using plugins, you can further enhance Nagios’s functionality, giving it capabilities not available in the core product. One such plugin lets you monitor the health of your cluster instead of that of individual hosts.
A cluster is a group of hosts or services that perform a common function in tandem. Some clusters are tied together at the operating system level (Beowulf clusters, for example) while some are defined by the tasks they perform. For instance, say your organization has four machines that work as HTTP servers behind a common front end. These constitute an HTTP service cluster, and you would want Nagios to monitor all of these machines and the HTTP service running on them.
Typically, each node in a such a cluster probably would be configured to pass its work to other nodes should it fail unexpectedly, so it would not be an emergency if one machine went out – though you’d want to know about it immediately. I would worry if two nodes went down, and would declare an emergency if three of them were not in a condition to serve users.
The check_cluster plugin can check the health of two kinds of clusters: host clusters, which comprise all the machines as a whole, and service clusters, which encompass a particular service running on the hosts. To understand the difference, consider a scenario where a particular physical machine is running several HTTP server instances on different ports, running a reverse proxy server on the front end to hide the port. All these HTTP services can be said to be in a service cluster. The Nagios check_cluster plugin can monitor these services and report a state of OK, warning, or critical, depending on the way you define the cluster’s health.

Service Cluster Monitoring

Suppose you want to monitor this HTTP service cluster. You first need to monitor the functionality of each individual server instance using the check_http plugin. Make a note of the service_description of the check, as we will use this in a moment when defining the cluster check. Service description is a kind of check identifier. Once you have set up monitoring individual server instances, you can put them in a cluster check.
First, you need to define the command to execute the check in /etc/nagios/conf.d/. We’ll call it check_cluster_service:
define command{
 command_name check_cluster_service
 command_line path_to_plugin/check_cluster --service -w $ARG1$ -c $ARG2$ -d $ARG3$  }
In the parameters passed to command_line, --service indicates this is a service cluster check, -w defines the number of individual service checks that must fail to get the warning state, -c defines the critical range, and -d contains the list of physical hosts along with the service check description you noted earlier. You can add or remove parameters passed to command_line as you need; see the man page for instructions.
Now you need to decide the warning and critical state thresholds or ranges. Let’s make the failure of two out of four services a warning and three out of four critical:
define service{
  hostgroup_name http-hosts
  service_description Cluster Check - HTTP Service
  check_command check_cluster_service!1:2!2:3!$SERVICESTATEID:host1:HTTP Service Check$,$SERVICESTATEID:host2:HTTP Service Check$,$SERVICESTATEID:host3:HTTP Service Check,$SERVICESTATEID:host4:HTTP Service Check$
  contacts noc, sysad
 }
check_command takes arguments passed according to the definition of command_line in the define command section above. Here we have defined the ranges for warning and critical state range separated by exclamation points. You can read more about defining ranges in the Nagios Plugins development guidelines.
We have also defined the hosts and services to be included using the SERVICESTATEID macro. Macros enable you to use information from various sources in real time. The SERVICESTATEID macro enables you to get the current state of the service from the HTTP check we defined earlier.
Restart Nagios to put these configuration changes into action. You should see the check appearing on the Nagios web interface, hopefully in OK state, which will change to critical or warning depending on number of services that are down. You can define the team or person to alert using Nagios’s contactgroup definition.

Putting in the Host Cluster Check

Setting up host cluster checks is a bit easier than setting up service cluster checks. Suppose you want to monitor whether a cluster of hosts is down instead of tracking the services running on them. As with service clusters checks, you first have to create a define command section:
define command{
 command_name check_cluster_host
 command_line path_to_plugin/check_cluster --host -w $ARG1$ -c $ARG2$ -d $ARG3$
 }
The parameters passed are similar to the ones for service checks, with the exception of --host.
Now you need to decide the alerting thersholds and define the check. Here again let’s keep it similar to the service check, where a warning was the failure of two out of four boxes and critical was the failure of three out of four boxes.
define service{
  hostgroup_name http-hosts
  service_description Cluster Check - HTTP Hosts
  check_command check_cluster_host!1:2!2:3!$HOSTSTATEID:host1$,$HOSTSTATEID:host2$,$HOSTSTATEID:host3$,$HOSTSTATEID:host4$
  contacts noc, sysad
 }
Again, the parameters passed are similar to the service cluster check, with the exception of the macro passed. The HOSTSTATEID macro expands to give the status of the host in real time.
Again, restart Nagios to see the checks appear on your web interface. For medium to large clusters, I usually turn off the alerts for individual hosts and services and only care for the cluster’s health. If I find a problem, I can fix things before users see any downtime.
By using Nagios plugins, you can keep an eye on the health of not just individual nodes and services, but entire clusters. That should give you more time to handle other tasks, and sysadmin time is one resource that’s always at a premium in the data center.

Use NFS to Extend and Integrate Your Filesystems


Someday your server will need more disk space. Rather than purchase new hardware, you may be able to take advantage of disk space you already own by using NFS (Network File System), a distributed filesystem for which support is integrated in the Linux kernel. Any machine can host NFS server software to share its disk space, or use an NFS client to make use of that space.
NAS is not SAN
You may hear NAS and SAN mentioned frequently together as alternatives, but they are quite different. NAS works with data files, while SAN (Storage Area Networking) works with data blocks. NAS is usually simpler to set up, works over TCP/IP, and does not require special hardware. SAN is more complex and does require special hardware.
Very often NFS is offered from a dedicated device as network-attached storage. Let’s see how you can create a simple but powerful NAS server based on CentOS 6.
NFS has been around for more than 20 years, making it very mature technology. CentOS 6 is a good choice for running NFS because it supports the latest version, NFSv4, with full compatibility with older versions and different operating systems.

Server Installation and Configuration

To begin creating a dedicated NAS server, you should have a minimal CentOS 6 installation, which you can install by using the minimal image. For a secure and optimized NAS server you should run only the packages you absolutely need.
Next, consider what filesystem NFS should run on top of. NFS works with well-known Linux filesystems such as ext2, ext3, and the newer ext4. However, for best performance, reliability, and compatibility, use the XFS filesystem for your NFS shares. XFS supports larger files and works well under heavy load and simultaneous access, as you’re likely to have on an NFS server. Just note that XFS is officially supported only for 64-bit architectures and is not recommended for 32-bit operating systems.
To format a partition in XFS, use the mkfs.xfs command-line tool. If there’s already a partition on your hard drive, all the data on that partition will be lost when you format it for XFS.
To run NFS on the server you need installed the packages nfs-utils, rpcbind, and their dependencies. Then, to ensure that all the necessary services are started and stopped when the server boots and shuts down, execute:
chkconfig rpcbind on
chkconfig nfs on
chkconfig nfslock on
Now edit the file /etc/sysconfig/nfs. By default all options are commented out, making them inactive. Configure the server to support only NFSv4 and disable previous versions by uncommenting the following rows:
MOUNTD_NFS_V2="no"
MOUNTD_NFS_V3="no"
RPCNFSDARGS="-N 2 -N 3"
Unlike previous versions, NFSv4 handles UTF-8-encoded files correctly. It’s also faster, with improved caching, and has better crash recovery mechanisms.
Also uncomment:
MOUNTD_PORT=892
STATD_PORT=662
LOCKD_TCPPORT=32803
These lines specify on which ports the NFS services run. The benefit of having the ports explicitly specified is that you can use the port numbers in your firewall rules.
It’s essential to maintain restrictive firewall rules in order to ensure the security of an NFS server. By default, every CentOS 6 installation comes with a firewall enabled and SSH as the only accessible service. You need to add to its rules the ability for NFS clients to access only the ports needed. If, as an example, an NFS client has an IP address of 10.0.0.2, run the following iptables commands in terminal:
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT #NFS core communication
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT #rpcbind/sunrpc TCP port
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT #rpcbind/sunrpc UDP port
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 892 -j ACCEPT #MOUNTD TCP PORT
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m udp -p udp --dport 892 -j ACCEPT #MOUNTD UDP PORT
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 662 -j ACCEPT #STATD port
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 32803 -j ACCEPT #LOCKD TCP port
To save the above rules, run the command service iptables save. Repeat the process for other clients, substituting their IP addresses for 10.0.0.2. Alternatively, you can allow access to whole subnets by specifying their network bits, such as 10.0.0.0/24.
You may notice that the above firewall rules allow mostly TCP connections. TCP is the default communication protocol in NFSv4, though UDP is also supported. TCP’s connection-oriented communication mode provides the required reliability that’s critical for working with files and data.
Next, it’s time to configure the shares. The file /etc/exports contains a path to a shared directory on each row, along with clients’ permissions for it and any custom settings that may apply. A simple configuration example would contain only /nfs 10.0.0.2, which says that the /nfs directory is accessible for a client with the IP address 10.0.0.2. Such short and simple export directives are sufficient to provide secure and optimized access to NFS, thanks to the powerful default NFS settings, which are:
  • Read-only access By default, clients can only read files. To allow write access too, specify immediately after the IP address (rw). If you leave a blank space between the IP address and (rw), you will allow read/write access for any IP address, which would be a security nightmare.
  • Root squash When the remote client identifies itself using the root ID, it receives locally the ID of the user nfsnobody, which prevents remote superusers from gaining absolute powers. It’s highly recommended that you leave this default in place, but you can disable it by adding no_root_squash at the end of the line.
  • Accept connections only from well-known ports The secure directive means that the request should originate on ports lower than 1024, the range of the so-called well-known ports. Only services started by the superuser can initiate such connections, which means a compromised non-root account on the remote server cannot connect to NFS pretending to be another user. You can turn off this option by specifying insecure.
  • Synchronous access This ensures that the NFS server replies to the client after data has been safely stored locally. This pause has a small performance impact but ensures data integrity. With asynchronous behavior, which was the default for NFS versions prior to 4, the server replies to the client as soon as it has processed the request, without taking into consideration whether the data has been successfully stored on the NFS server filesystem. To offer asynchronous access, use the option async.
  • Write delay By default, the NFS server will delay disk writes if it suspects it will get subsequent write requests so that it can process them in batch. This makes the NFS server faster when there are numerous, small write requests at the same time. If you know this is not likely to be the case, you can turn off this option by specifying no_wdelay.

Client Installation and Configuration

A client running Linux and especially CentOS 6 can easily connect to NFS shares. In CentOS you need the package nfs-utils, which is installed by default. It extends the NFS kernel support and provides additional utilities, such as showmount, that facilitate work with NFS.
Before actually mounting NFS shares, you should explore the remote NFS server and discover what it offers, which means looking at its export list. If the NFS server’s IP address is 10.0.0.1, run showmount -e 10.0.0.1. You should see output similar to:
Export list for 10.0.0.1:
/nfs 10.0.0.2
This means that the /nfs share would be accessible to a client with the IP address 10.0.0.2. If that’s you, you can mount it as you would any other media. First create a directory for it, such as /media/nfs. Then run mount 10.0.0.1:/nfs /media/nfs. You should then see your shared files inside /media/nfs.
If you want to make sure that the NFS directory is mounted automatically after reboot, add to the client’s /etc/fstab file a line containing 10.0.0.1:/nfs /media/nfs nfs timeo=10. This instructs the client to mount the /nfs share available at 10.0.0.1. The only custom setting here is the timeout (timeo), which is 10 seconds, in order to prevent the client from hanging for too long waiting for an NFS reply. You can add more custom settings by separating them with commas and no space.
Once you have mounted the NFS share on the client you can try some tests to determine the best settings for your environment. A useful tool for performance tests and benchmarking is nfsiostat. Similar to the popular iostat, nfsiostat gives statistics for read and write operations on NFS shares.
Based on the results of your tests, you can mount NFS shares with different options. For instance, play with the read and write size options, rsize and wsize. They specify the data block size received and sent. On faster networks with no congestion, large sizes may benefit performance.

NFS Security

In the past, NFS was considered insecure by design because it did not offer username and password authorization. With the latest NFSv4 this is not a problem; for authorization you can use Kerberos.
The big question for NFS security is how secure and trustworthy the clients are, because you must rely on them to provide legitimate userids. As long as you receive legitimate userids you can use the native Linux permission and ownership features to refine access and security on the server side.
NFS continues to play a prominent role for distributed file systems in the Linux world thanks to its performance, reliability, robustness, and security. Furthermore, NFS is supported not only in Unix-based operating systems such as Linux and Mac OS, but also under Windows with Windows Services for Unix, making it an excellent option for heterogenous interoperability.

The Magic of Editable PDFs


Stop sending .DOC files as mail attachments! There is a better way!

One of the scourges of e-mail is file attachments, and particularly those from people sending files made by their new word-processor or presentation programme that half the people receiving it can't open. While proprietary software vendors love this errant behaviour (it keeps up the pressure for people to re-purchase software they don't really need so they can read other people's work - AKA "upgrades"), it's really anti-social behaviour.
To start with, most of us don't need the source file for a document or presentation. Usually we just want to read it or review it. Then many of us have mobile devices which may not support the file format you are using. Finally, if we actually do need to collaborate, it would be better to use a collaborative editor than to circulate files and then have the problem of dealing with all the different ways people can comment on them afterwards
For the vast majority of us, a document we can read is sufficient. A PDF, for example - the name itself tells the story, "Portable Document Format". While it was started by Adobe many years ago, today PDF is a standard and there are many ways to read PDF files. Even better, most smartphones and tablets can read PDFs out-of-the-box.
So if you have to send a document attachment, please please please send a PDF so the rest of us can read it. If the file isn't confidential, you might even put it into a "locker" on the internet - say on DropBox - and let people collect the file from there. But make sure that's a PDF too, please, and remember mobile devices may not easily be able to access it!

Portable But Editable

Of course, some people will need to edit the document you are sending. Did you know you can send a final document as a PDF attachment that everyone will be able to open and view, but which people who need to will also be able to open and edit?
The document that can do this magic is called a "Hybrid PDF", and anyone can make one using open source software. LibreOffice (and related packages like OpenOffice.org) have been able to make these editable PDFs for quite some time. A Hybrid PDF is a normal PDF file that any PDF reader can display, but with the added benefit that the original source document is also embedded in the file. Any suitably advanced office suite, such as LibreOffice, is able to retrieve the source and allow you to edit it.
To show how they work, I have prepared a PDF that provides instructions on how to create a hybrid PDF with LibreOffice. Feel free to download it and take a look at the instructions. Naturally, the PDF is itself a hybrid, so try dragging and dropping it onto your office suite and see if it can be opened. If not - well, you need to get a modern, innovative office suite instead of the one you're using!  It also works for presentations - I've prepared a sample presentation about OSI for you to try.
If you like this idea, send the file to your colleagues and friends and tell them all attachments from you in future will be these editable PDFs. Let's start a wave of change!

Tuesday, March 27, 2012

Using ATA Over Ethernet (AoE) On Debian Squeeze (Initiator And Target)


This guide explains how you can set up an AoE target and an AoE initiator (client), both running Debian Squeeze. AoE stands for "ATA over Ethernet" and is a storage area network (SAN) protocol which allows AoE initiators to use storage devices on the (remote) AoE target using normal ethernet cabling. "Remote" in this case means "inside the same LAN" because AoE is not routable outside a LAN (this is a major difference compared to iSCSI). To the AoE initiator, the remote storage looks like a normal, locally-attached hard drive.
I do not issue any guarantee that this will work for you!

1 Preliminary Note

I'm using two Debian Squeeze servers here:
  • server1.example.com (Initiator): IP address 192.168.0.100
  • server2.example.com (Target): IP address 192.168.0.101

2 Loading The aoe Kernel Module On Both Systems

server1/server2:
Before we start, we must make sure that the the kernel supports AoE:
grep ATA_OVER /boot/config-`uname -r`
This should display something like this:
root@server2:~# grep ATA_OVER /boot/config-`uname -r`
CONFIG_ATA_OVER_ETH=m
root@server2:~#
This means that AoE was built as a kernel module. Let's check if the module is already loaded:
lsmod | grep aoe
If you get nothing back, this means it's not loaded. In this case we can load it as follows:
modprobe aoe
Let's check again if the module is loaded:
lsmod | grep aoe
root@server2:~# lsmod | grep aoe
aoe                    21380  0
root@server2:~#
To have the module loaded automatically when the system boots, we add the aoe module to /etc/modules:
vi /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.

aoe
loop
 

3 Setting Up The Target (server2)

server2:
First we set up the target (server2):
apt-get install vblade
We can use unused logical volumes, image files, hard drives (e.g. /dev/sdb), hard drive partitions (e.g. /dev/sdb1) or RAID devices (e.g. /dev/md0) for the storage. In this example I will create a logical volume of 20GB named storage1 in the volume group vg0:
lvcreate -L20G -n storage1 vg0
(If you want to use an image file, you can create it as follows:
mkdir /storage
dd if=/dev/zero of=/storage/storage1.img bs=1024k count=20000
This creates the image file /storage/storage1.img with a size of 20GB.
)
Now we export our storage device as follows:
vbladed 0 1 eth0 /dev/vg0/storage1
The first number (0) is the shelf number (major), the second (1) the slot number (minor), change these numbers to your liking. Each AoE device is identified by a couple major/minor which must be unique (if you are exporting multiple devices), with major between 0-65535 and minor between 0-255. The eth0 part tells vbladed which ethernet device to use (if you ethernet device is eth1, then use eth1 - you can find out about your ethernet devices by running
ifconfig
).
To start the export automatically whenever you boot the target, open /etc/rc.local...
vi /etc/rc.local
... and add the following line to it (before the exit 0 line):
[...]
/usr/sbin/vbladed 0 1 eth0 /dev/vg0/storage1
[...]

4 Setting Up The Initiator (server1)

 
server1:
On server1, we install the initiator:
apt-get install aoetools
Now we check what AoE storage devices are available:
aoe-discover
The command
aoe-stat
should now show the storage devices:
root@server1:~# aoe-stat
      e0.1        21.474GB   eth0 up
root@server1:~#
At this point we have a new block device available on the client box named /dev/etherd/e0.1. If we have a look at the /dev tree a new node appears:
ls -la /dev/etherd/
root@server1:~# ls -la /dev/etherd/
total 0
drwxr-xr-x  2 root root     160 Mar 16 13:34 .
drwxr-xr-x 14 root root    3000 Mar 16 13:33 ..
c-w--w----  1 root disk 152,  3 Mar 16 13:33 discover
brw-rw----  1 root disk 152, 16 Mar 16 13:34 e0.1
cr--r-----  1 root disk 152,  2 Mar 16 13:33 err
c-w--w----  1 root disk 152,  6 Mar 16 13:33 flush
c-w--w----  1 root disk 152,  4 Mar 16 13:33 interfaces
c-w--w----  1 root disk 152,  5 Mar 16 13:33 revalidate
root@server1:~#
In the output of
fdisk -l
you should now also find the new hard drive:
root@server1:~# fdisk -l

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00029d5c

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        3793    30461952   83  Linux
/dev/sda2            3793        3917      992257    5  Extended
/dev/sda5            3793        3917      992256   82  Linux swap / Solaris

Disk /dev/etherd/e0.1: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/etherd/e0.1 doesn't contain a valid partition table
root@server1:~#
To use that device, we must format it:
fdisk /dev/etherd/e0.1
root@server1:~# fdisk /dev/etherd/e0.1
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xa00b110d.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.


The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help):
 <-- n
Command action
   e   extended
   p   primary partition (1-4)

<-- p
Partition number (1-4): <-- 1
First cylinder (1-2610, default 1): <-- ENTER
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-2610, default 2610):
<-- ENTER
Using default value 2610

Command (m for help):
 <-- t
Selected partition 1
Hex code (type L to list codes):
 <-- 83

Command (m for help): <-- w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
root@server1:~#

Afterwards, the output of
fdisk -l
should look as follows:
root@server1:~# fdisk -l

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00029d5c

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        3793    30461952   83  Linux
/dev/sda2            3793        3917      992257    5  Extended
/dev/sda5            3793        3917      992256   82  Linux swap / Solaris

Disk /dev/etherd/e0.1: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0cf58b15

            Device Boot      Start         End      Blocks   Id  System
/dev/etherd/e0.1p1               1        2610    20964793+  83  Linux
root@server1:~#
Now we create a filesystem on /dev/etherd/e0.1p1...
mkfs.ext4 /dev/etherd/e0.1p1
... and mount it for test purposes:
mount /dev/etherd/e0.1p1 /mnt
You should now see the new device in the outputs of...
mount
root@server1:~# mount
/dev/sda1 on / type ext3 (rw,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/etherd/e0.1p1 on /mnt type ext4 (rw)
root@server1:~#
... and
df -h
root@server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              29G  806M   27G   3% /
tmpfs                 249M     0  249M   0% /lib/init/rw
udev                  244M  108K  244M   1% /dev
tmpfs                 249M     0  249M   0% /dev/shm
/dev/etherd/e0.1p1     20G  172M   19G   1% /mnt
root@server1:~#
You can unmount it like this:
umount /mnt
To have the device mounted automatically at boot time, e.g. in the directory /storage, we create that directory...
mkdir /storage
... and add the following line to /etc/fstab:
vi /etc/fstab
[...]
/dev/etherd/e0.1p1       /storage        ext4    defaults,auto,_netdev 0 0
This alone isn't enough to have the device mounted at boot time because the AoE stuff gets loaded after /etc/fstab is read. Therefore we open /etc/rc.local...
vi /etc/rc.local
... and add the following lines to it (before the exit 0 line):
[...]
aoe-discover
sleep 5
mount -a
[...]
For test purposes, you can now reboot the system:
reboot
After the reboot, the device should be mounted:
mount
root@server1:~# mount
/dev/sda1 on / type ext3 (rw,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/etherd/e0.1p1 on /storage type ext4 (rw,_netdev)
root@server1:~#
df -h
root@server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              29G  806M   27G   3% /
tmpfs                 249M     0  249M   0% /lib/init/rw
udev                  244M  108K  244M   1% /dev
tmpfs                 249M     0  249M   0% /dev/shm
/dev/etherd/e0.1p1     20G  172M   19G   1% /storage
root@server1:~#

5 Links

Linux IP Commands


Display Current Config for all NIC’s: ifconfig
Display Current Config for eth0: ifconfig eth0
Assign IP: ifconfig eth0 192.168.1.2
Ping: ping -c 3 192.168.1.1
Assign multiple IP’s: ifconfig eth0:0 192.168.1.2
Assign second IP: ifconfig eth0:1 192.168.1.3
Disable network card: ifconfig eth0 down
Enable network card: ifconfig eth0 up
View current routing table: route “or” route -n
View arp cache: arp “or” arp -n
Assign IP/Subnet: ifconfig eth0 192.168.1.2 netmask 255.255.255.0
Assign Default Gateway: route add default gw 192.168.1.1
Trace Route: traceroute www.howtonetworks.net
Trace Path: tracepath www.howtonetworks.net
DNS Test: host www.howtonetworks.net
Advanced DNS Test: dig www.howtonetworks.net
Reverse Lookup: host 66.11.119.69
Advanced Reverse Lookup: dig -x 66.11.119.69
Linux IP Commands images
*You MUST be at the ROOT user to make/save any changes. Linux users, your distribution will determine the location of your network config file which will need to be updated and saved in order for the changes to remain in effect after rebooting. Network cards are referred to as eth0, eth1, eth2, etc based on their position on the PCI bus.
*Special thanks to Gergely for the Linux commands!

How To Configure PureFTPd To Accept TLS Sessions On CentOS 6.2


FTP is a very insecure protocol because all passwords and all data are transferred in clear text. By using TLS, the whole communication can be encrypted, thus making FTP much more secure. This article explains how to configure PureFTPd to accept TLS sessions on a CentOS 6.2 server.
I do not issue any guarantee that this will work for you!

1 Preliminary Note

You should have a working PureFTPd setup on your CentOS 6.2 server, e.g. as shown in this tutorial: Virtual Hosting With PureFTPd And MySQL (Incl. Quota And Bandwidth Management) On CentOS 6.2.

2 Installing OpenSSL

OpenSSL is needed by TLS; to install OpenSSL, we simply run:
yum install openssl

3 Configuring PureFTPd

Open /etc/pure-ftpd/pure-ftpd.conf...
vi /etc/pure-ftpd/pure-ftpd.conf
If you want to allow FTP and TLS sessions, set TLS to 1:
[...]
# This option can accept three values :
# 0 : disable SSL/TLS encryption layer (default).
# 1 : accept both traditional and encrypted sessions.
# 2 : refuse connections that don't use SSL/TLS security mechanisms,
#     including anonymous sessions.
# Do _not_ uncomment this blindly. Be sure that :
# 1) Your server has been compiled with SSL/TLS support (--with-tls),
# 2) A valid certificate is in place,
# 3) Only compatible clients will log in.

TLS                      1
[...]
If you want to accept TLS sessions only (no FTP), set TLS to 2:
[...]
# This option can accept three values :
# 0 : disable SSL/TLS encryption layer (default).
# 1 : accept both traditional and encrypted sessions.
# 2 : refuse connections that don't use SSL/TLS security mechanisms,
#     including anonymous sessions.
# Do _not_ uncomment this blindly. Be sure that :
# 1) Your server has been compiled with SSL/TLS support (--with-tls),
# 2) A valid certificate is in place,
# 3) Only compatible clients will log in.

TLS                      2
[...]
To not allow TLS at all (only FTP), set TLS to 0:
[...]
# This option can accept three values :
# 0 : disable SSL/TLS encryption layer (default).
# 1 : accept both traditional and encrypted sessions.
# 2 : refuse connections that don't use SSL/TLS security mechanisms,
#     including anonymous sessions.
# Do _not_ uncomment this blindly. Be sure that :
# 1) Your server has been compiled with SSL/TLS support (--with-tls),
# 2) A valid certificate is in place,
# 3) Only compatible clients will log in.

TLS                      0
[...]

4 Creating The SSL Certificate For TLS

In order to use TLS, we must create an SSL certificate. I create it in /etc/ssl/private/, therefore I create that directory first:
mkdir -p /etc/ssl/private/
Afterwards, we can generate the SSL certificate as follows:
openssl req -x509 -nodes -days 7300 -newkey rsa:2048 -keyout /etc/ssl/private/pure-ftpd.pem -out /etc/ssl/private/pure-ftpd.pem
Country Name (2 letter code) [XX]: <-- Enter your Country Name (e.g., "DE").
State or Province Name (full name) []:
<-- Enter your State or Province Name.
Locality Name (eg, city) [Default City]:
<-- Enter your City.
Organization Name (eg, company) [Default Company Ltd]:
<-- Enter your Organization Name (e.g., the name of your company).
Organizational Unit Name (eg, section) []:
<-- Enter your Organizational Unit Name (e.g. "IT Department").
Common Name (eg, your name or your server's hostname) []:
<-- Enter the Fully Qualified Domain Name of the system (e.g. "server1.example.com").
Email Address []:
<-- Enter your Email Address.
Change the permissions of the SSL certificate:
chmod 600 /etc/ssl/private/pure-ftpd.pem
Finally restart PureFTPd:
/etc/init.d/pure-ftpd restart
That's it. You can now try to connect using your FTP client; however, you should configure your FTP client to use TLS - see the next chapter how to do this with FileZilla.

5 Configuring FileZilla For TLS

In order to use FTP with TLS, you need an FTP client that supports TLS, such as FileZilla.
In FileZilla, open the Server Manager:

Select the server that uses PureFTPd with TLS; in the Server Type drop-down menu, select FTPES instead of normal FTP:


Now you can connect to the server. If you do this for the first time, you must accept the server's new SSL certificate:


If everything goes well, you should now be logged in on the server:



6 Links

71 Top Commercial Open Source Projects


A 2011 Gartner survey found that more than half of all enterprises surveyed had adopted some open source software. However, many businesses also need professional support and related services to make open source software viable for their organizations. Commercial open source software vendors fill in the gap for these businesses by providing those additional services while keeping overall costs low.
In addition to low costs, open source software gives businesses the ability to tailor the software to their specific needs. According to Laurie Wurster, research director at Gartner, "Users are beginning to look at OSS differently — if they can customize the code to make it unique to their company, they have created a competitive advantage." She added, "Although a search for reducing costs by adopting OSS continues to be a major driver, with this survey we see more respondents looking at OSS as having much-greater value than simply getting something for free."
With enterprises in mind, we've updated our list of top commercial open source projects. While home users might be interested in some of this software, the commercial versions are all targeted primarily at business users.
Each of these projects offers at least one version of its software under an open source license, while providing additional software, hardware, support and/or services for a fee. This arrangement allows organizations to receive many of the benefits of open source software, along with the other products and services they need to make the software usable. At the same time, the software developers receive the income they need to continue producing quality software.
As always, if you know of another commercial open source project that you think should be included in our list, please feel free to make a note in the comments section below.

Billing

1. jBilling
The self-described "leader in open source billing and rating software," jBilling offers an invoicing platform for telecoms and companies that offer subscription-based services. It comes in either a community or an enterprise version, and the company also offers consulting, support and training services. Operating System: OS Independent.

Business Intelligence

2. Jaspersoft
Jaspersoft claims to be "the world's most widely used business intelligence software." It offers enterprise, professional and express editions in addition to the free community edition. The website above relates primarily to the commercial versions, but you can find out more about the open source tools at JasperForge.org. Operating System: OS Independent.
3. Palo BI Suite/Jedox
This project goes by two different names: Palo BI is the open-source version and Jedox is the commercial version. Both include tools for planning, analytics and reporting, as well as plug-ins that extend the capabilities of Microsoft Excel. Operating System: OS Independent.
4. Pentaho
Although it pays homage to its "open source heritage," the main Pentaho site is all about the commercial version of its business analytics and data integration tools. However, you can still find the open source version at the Pentaho Community Wiki. The full suite includes tools for ETL, OLAP analysis, metadata, data mining, reporting, dashboards and a platform for building your own solutions. Operating System: Windows, Linux, OS X.
5. OpenReports
Web-based OpenReports supports a wide variety of reporting engines, SQL-based reports, multiple parameters and flexible scheduling. The paid professional version adds a reporting dashboard, alerts, conditional report scheduling and report statistics. Operating System: OS Independent.
6. RapidMiner
Rapid-I, the company behind Rapid Miner and sister projects Rapid Analytics and Radoop, claims that 38 percent of all analysts use its open source solutions. Data mining solution RapidMiner has won numerous awards and is used by thousands of organizations in more than 40 countries. It comes in three different supported enterprise versions, as well as the free community version. Operating System: OS Independent.

Business Process Management

7. Bonita Open Solution
Users of this BPM solution include Directv, Trane, the governments of France and the Canary Islands, and Konica Minolta. In addition to the open source version, it comes in teamwork, efficiency and performance subscription packs. Operating System: OS Independent.
8. ProcessMaker
Used by Toyota, Lenovo, the University of Melbourne and many other organizations, ProcessMaker BPM promises to help users "radically reduce paperwork, use resources more efficiently and improve business outcomes." You can deploy the open source and enterprise editions on site or use ProcessMaker in the cloud. Operating System: Windows, Linux.

Cloud Infrastructure

9. Cloud.com CloudStack
Now owned by Citrix, Cloud.com is an open source platform that allows enterprises or service providers to build public or private cloud infrastructure. The company also offers a supported enterprise version. Operating System: Linux.
10. Eucalyptus
Currently powering more than 25,000 private or hybrid clouds, Eucalyptus counts 40 percent of Fortune 500 firms as customers and boasts that it is "the world's most widely deployed cloud computing software platform for on-premise (private) Infrastructure as a Service clouds." Note that the link above will take you to the commercial version of the software; you can find the community version (and a whole lot of information about cloud computing in general) at Open.Eucalyptus.com. Operating System: Linux.
11. Scalr
This open source cloud management software offers auto-scaling, disaster recovery and server management capabilities. It's used by more than 6,000 companies, including Samsung, the Walt Disney Company and Accenture. It works with both public and private clouds, and it's also available as a hosted service. Operating System: Linux.

Content Management

12. Alfresco
According to a Forrester study, Alfresco Enterprise pays for itself within ten months. It includes Web content management, document management, records management, collaboration and a platform for building your own Java-based applications. It's available in both enterprise and community versions or a cloud version. Operating System: Windows, Linux, OS X.
13. DotNetNuke
The "#1 open source Web content platform for business," DotNetNuke currently runs more than 700,000 sites and boasts more than 7 million downloads. You can download the free community version or purchases the professional or enterprise version, and the site also has a large store with add-ons, themes and other tools to help speed the development of your website. Operating System: Windows.
14. Liferay
Named a "leader" in Gartner's Magic Quadrant for Horizontal Portals, LifeRay provides the platform for nearly half a million websites and enterprise portals, including some for Cisco, T-Mobile, Barclays, AutoZone and even Sesame Street. It has both community and enterprise editions, and Liferay also sells support, training and consultation services. Operating System: OS Independent.
15. Magnolia
This open source content management system has recently released version 4.5, which offers instant mobile websites and multi-channel publishing. It's used by thousands of organizations, including the U.S. Navy and Texas State University. Magnolia sells both standard and pro enterprise versions of its software, with multiple levels of support available. Operating System: Windows, Linux.
16. MindTouch
MindTouch creates customer support and help sites, also known as knowledge bases. Its long list of customers features PayPal, Mozilla, HP, Intuit, Fujitsu, Microsoft, The Washington Post and other well-known companies. The "Core" version of the software remains open source, but today the company focuses on the cloud-based version, which is updated every week. Operating System: Windows, Linux.

Customer Relationship Management (CRM)
17. ConcourseSuite
ConcourseSuite combines CRM functionality with Web content management and enterprise collaboration features. Interested organizations can use it on an SaaS basis, or it can be deployed on-premises with maintenance, hosting and support contracts available. Operating System: Windows, Linux, OS X.
18. Orange Leap
While most CRM software is designed to meet the needs of for-profit companies, Orange Leap specifically targets non-profits. Paid support and hosting are available through the company. Operating System: Windows.
19. SourceTap
SourceTap is a sales force automation system that aims to meet the needs of both sales reps and sales managers. Multiple licensing options are available, and the software is also available on an SaaS basis. Operating System: Windows, Linux.
20. SplendidCRM
The community version of Splendid offers accounts, contacts, leads and opportunities modules, while the professional version adds product and order management, and the enterprise version adds more advanced features like workflow, ad-hoc reporting and offline client capabilities. All three versions can be deployed on-premise or accessed in the cloud on an SaaS basis. Operating System: Windows.
21. SugarCRM
Web-based SugarCRM offers a low-cost, adaptable tool to help companies "communicate with prospects, share sales information, close deals and keep customers happy." Thousands of companies, like Avis, Coca-Cola and Chevrolet, have chosen it as their CRM solution. In addition to the open source version, it comes in professional, corporate, enterprise and ultimate versions, all of which can be deployed on premise or in the cloud. Operating System: Windows, Linux, OS X.
22. vTiger
Used by more than 100,000 businesses, vTiger aims to make your sales, marketing and support staff more effective. The latest version also includes mobile support, allowing employees to work from anywhere. You can download the open source version or use the onDemand service which runs on Amazon EC2. Operating System: Windows, Linux, iOS, Android.

Databases

23. MySQL
"The world's most popular open source database," Oracle-owned MySql claims that users can lower their TCO up to 96 percent when compared to using Microsoft SQL Server. The site offers downloads of both the community and enterprise editions of the software, as well as a number of related tools. Operating System: Windows, Linux, OS X.

Data Warehouse (DW)

24. Apatar
Apatar makes it easier for enterprises to integrate data contained in on-premise or cloud-based applications, including many popular CRM tools. Users include Salesforce.com, Hotels.com, University of Maryland, Autodesk, Credit Suisse and others. In addition to the free download, Apatar offers an On-Demand version, as well as support, training, consulting and other paid services. Operating System: Windows, Linux.
25. MailArchiva
MailArchiva stores enterprise e-mail messages, allowing companies to meet compliance requirements, to search old messages quickly, to monitor content and to save on storage costs. The link above will connect you with the enterprise and ISP versions of the software; for the open source version, see SourceForge. Operating System: Windows, Linux.
26. Talend
Named a "leader" in the Forrester Wave for ETL, Talend offers data integration, data quality, master data management and application integration tools used by companies around the world, including The Weather Channel, Xerox, Capgemini, Verizon, Infosys and others. The company offers quite a few different versions of its tools: it markets the open source versions under the name "Talend Open Studio," and it markets the commercial versions under the name "Talend Enterprise." Operating System: Windows, Linux, Unix.

Developer Tools

27. Mono
Now owned by Xamarin, the Mono development framework allows programmers to create cross-platform applications that are compatible with Microsoft's .NET framework. Regular Mono is free and open source; MonoTouch (for iOS development) and Mono for Android require a licensing fee. Operating System: Windows, Linux, OS X, iOS, Android.
28. Sencha Touch
With Sencha Touch, developers can create cross-platform HTML5 Web apps that look and feel like native mobile apps. Both the GPL and commercial license versions of the framework are free; support is available on a paid basis. Operating System: OS Independent.
29. Zend Framework
The Zend Framework helps PHP developers create more secure, reliable, and modern Web 2.0 applications and services. Support and related products can be purchased through Zend's corporate website. Operating System: Windows, Linux, OS X.

Document Management Systems (DMS)

30. LogicalDOC
This user-friendly enterprise document management solution features an intuitive Web interface, integration with Microsoft Office and Outlook, full-text indexing, multiple language support and more. It comes in open source, enterprise and cloud editions. Operating System: OS Independent.
31. OpenKM
Designed to be equally usable by both large and small organizations, OpenKM is a document management system that features a Web 2.0 interface, easy search capabilities, multiple language support, mobile support and a scheduler. In addition to the free open source download, the company offers paid professional support or a subscription-based cloud version. Operating System: OS Independent.

E-Commerce

32. Broadleaf Commerce
Used by companies like PepBoys, the Container Store, Ganz and Waste Management, Broadleaf offers a Java-based e-commerce platform with a sophisticated promotions engine and customization capabilities. The full software is free and open source, but Broadleaf also offers paid support and consulting services. Operating System: Windows, Linux, OS X.
33. Magento
Used by more than 110,000 online merchants, including Office Max, Harbor Freight Tools, K-Swiss and the North Face, eBay-owned Magento has been called an "emerging player to watch" by analysts at Forrester. The company offers enterprise and enterprise premium versions of the software for on-site deployment, a cloud-based version called "Magento Go," paid training and the free open source version. Operating System: Windows, Linux.
34. PrestaShop
Downloaded more than 1.3 million times, award-winning PrestaShop boasts a search-engine-friendly e-commerce platform with more than 275 features. The software is free, but the company sells support, training, add-ons, templates and other services. Operating System: Windows, Linux, OS X.

E-mail/Collaboration/Groupware

35. Collabtive
Very similar to Basecamp, Collabtive is cloud-based groupware for tracking projects, milestones, tasks and time spent. It also imports Basecamp data. You can host it on your own server or use the paid hosting service. Operating System: OS Independent.
36. cyn.in
This collaboration platform helps teams work together with wikis, social networks, blogs, file sharing repositories, micro blogs, discussion boards and other tools. It comes in quite a few different flavors: the free open source version; standard, enterprise or high-availability cloud-based versions; and standard or enterprise on-premise versions. Operating System: Windows, Linux, OS X.
37. Feng Office
More than 350,000 people use Feng Office, including those at NASA, the NBA and the U.S. Department of State. It offers project management, client relationship management, billing and finance modules that are designed with professional services organizations in mind. The company primarily promotes its cloud-based service, but you can also find an on-premise version, other services and the open source version at the site. Operating System: Windows, Linux, OS X.
38. Group-Office
Group-Office includes shared calendar, CRM, e-mail, file sharing, and project management capabilities, all with mobile support. It comes in a community version, professional (on-site) version and a hosted version. Operating System: OS Independent.
39. Scalix
Aimed at hosting providers and ISPs as well as enterprises and small businesses, Scalix offers an alternative to Microsoft Exchange servers for group e-mail and calendaring. It comes in numerous flavors, including the community, enterprise, small business and hosting editions. Operating System: Linux.
40. Zimbra
Another alternative to Microsoft Exchange, Zimbra offers "integrated email, contacts, calendaring, sharing and document management plus mobility and desktop synchronization." In addition to the open source edition, the company also offers a desktop client, a network edition and an appliance edition designed for use with VMware cloud infrastructure. Operating System: Linux, Unix, OS X.

Enterprise Resource Planning (ERP)

41. Compiere ERP + CRM Business Solution
Calling itself the "most modern, adaptable and affordable ERP solution," Compiere offers reporting, dashboards, warehouse management, manufacturing, purchasing, order management, financial management, sales, website, CRM and much more. It comes in open source or enterprise editions, and it's also easy to run the enterprise version in Amazon's EC2 cloud. Operating System: Windows, Linux, OS X.
42. Openbravo ERP
With more than 2 million downloads to its credit, Openbravo claims to be "the leading Web-based professional enterprise resource planning solution. The professional version of the software is available only through Openbravo's channel partners, but you can download the community edition or a free trial of the professional edition from the website. Operating System: Windows, Linux, OS X.
43. Open ERP
OpenERP offers more than 700 modules, so it's easy to tailor to your specific needs. It comes in three flavors: the open source community version, a paid enterprise version, or a cloud-based online version. Operating System: Windows, Linux.
44. opentaps
Used by Toyota, Honeywell and other well-known companies, Opentaps considers itself "the most advanced open source ERP + CRM solution." In addition to the free open source software, the company offers a supported professional version and several cloud-based versions available for deployment on Amazon EC2. Operating System: Windows, Linux.
45. ]project-open[
Downloaded more than 150,000 times, project-open aims to bridge the gap between ERP and project management, combining many of the most important features of both types of solutions. It's particularly well-suited for the needs of advertising and marketing agencies, IT service management companies, translation businesses and consulting firms. The core modules are free, but the company also offers paid support, extension modules, consulting, automatic updates and an SaaS version. Operating System: OS Independent.
46. xTuple
This company claims you can download, install and get up and running with its ERP/CRM solutions in just ten minutes. The PostBooks version of the software is the open source version; xTuple offers many other paid versions (both on-premise and cloud-based) that are tailored for the needs of different types companies. Operating System: Windows, Linux, OS X, mobile OSes.

Firewall

47. Smoothwall
You'll find the open source version of this firewall, Smoothwall Express, at Smoothwall.org. At the main corporate site, Smoothwall offers more complete security and Web filtering products that incorporate the open source firewall. Operating System: Linux, Unix.

Gateway Security/UTM Appliances

48. Endian Firewall
The "leading provider of open source network security solutions," Endian offers a free, open source version of its software that you can use to turn any PC into your own unified threat management appliance. It also sells a variety of commercially supported hardware and software appliances based on the same technology. Operating System: Linux.
49. Untangle Like Endian, the community version of Untangle allows you to create your own UTM appliance. However, the Untangle download includes some commercial features that will deactivate after 14 days unless you purchase a commercial license. The company also sells pre-configured appliances that offer "plug-and-play" network security. Operating System: Linux.

Human Resource Management (HRM)

50. Open Applicant
For companies besieged by job applicants, this software aims to make the process of finding the right employees easy and effective. It comes in open source, professional and hosted versions. Operating System: OS Independent.
51. Orange HRM
Designed for small and medium-sized enterprises (SMEs), Orange includes employee information management, absence/holiday/leave management, recruitment management, employee performance evaluation and many other HR tools. It's available as a free download, with professional support, training and a more advanced hosted version also available. Operating System: Windows, Linux, OS X.

Intrusion Detection and Prevention (IDS/IPS)

52. Snort
The "most widely deployed IDS/IPS technology worldwide," Snort combines signature, protocol, and anomaly-based network inspection into one package. The software is free, but a subscription gets you real-time access to rules updates. (Non-subscribers must wait 30 days.) Operating System: Linux, Unix, BSD, Mac OS X.
53. OSSEC
Downloaded more than 5,000 times every month, this popular host-based intrusion detection system (HIDS) offers features like a powerful correlation and analysis engine, integrating log analysis, file integrity checking, Windows registry monitoring, centralized policy enforcement, rootkit detection, real-time alerting and active response. Companies that need paid support can obtain it through OSSEC-owner Trend Micro. Operating System: Windows, Mac, Linux, Unix, BSD, Solaris.

Middleware

54. JBoss
Used by companies like Priceline.com, GEICO and NYSE Euronext, RedHat's JBoss line of middleware includes an application platform, Web platform, messaging, SOA platform, business rules management system and much more. All of the tools are available in open source and enterprise versions. Operating System: Linux.

Mind Mapper

55. XMind
Ideal for brainstorming and planning sessions, XMind makes it easy to see the connections between ideas—it's sort of like a white board for your computer. In addition to the free version, it comes in a pro version for individuals and an enterprise version for businesses. Operating System: Windows, Linux, OS X.

Network Monitoring/Scanning/Intrusion Detection

56. Nagios
Calling itself the "industry standard in open source monitoring," Nagios counts Unisys, Wells Fargo, AVG, BT, Xerox, Citrix, iRobot, ADP, the Office of the President of the United States and many other organizations among its users. The paid Nagios XI version comes with a support and maintenance contract, but you can also purchase a support subscription for the Core open source version. Operating system: Linux, Unix.
57. Opsview Community
Opsview aims to provide "a single pane of glass for managing and monitoring distributed IT infrastructures," including your cloud infrastructure and mobile devices connected to your network. The community version is free, but the paid enterprise version offers better scalability, reporting and other features. Operating System: Linux.
58. SNARE
Intersect Alliance, the company behind SNARE, offers free, open source downloads of its monitoring agents. For commercial support and a centralized view of event collection, analysis, reporting and archives, you can purchase the proprietary SNARE server. Operating System: Windows, Linux, Unix, Solaris.
59. Wireshark
"The world's foremost network protocol analyzer," Wireshark offers live capture and offline analysis of the traffic running across your network. The software is free, but Riverbed Technologies offers many products and services to enhance its capabilities. Operating System: Windows, Mac, Linux, Unix, BSD, Solaris.
60. Zabbix
This enterprise-class monitoring solution offers both agent and agentless monitoring, availability and SLA reporting, scalability to 100,000 devices, and more. Commercial support, training and other services are available. Operating System: Windows (agent only), Linux, OS X.

Operating System

61.Red Hat
RedHat, which calls itself "the world's open source leader," offers one of the best known distributions of Linux. RedHat Enterprise Linux comes in desktop, server and several other versions. Note that RedHat is a paid product, but a community version is available through Fedora.
62. SUSE Novell counts the London Stock Exchange, Office Depot, Sony and Walgreens among the users of its SUSE Enterprise Linux. It comes in multiple desktop and server versions, including some optimized for virtualized or cloud computing environments. For the open source version, see openSUSE.
63. Ubuntu Used by more than 20 million people, Ubuntu comes in desktop, server, cloud and mobile editions. Professional support, services, training and consulting are available through the Canonical, which owns Ubuntu.

Project Management

64. Onepoint Project
This Web-based application combines project management and project portfolio management into a single package. It comes in open source, group and enterprise versions, and the paid group and enterprise versions can be deployed on your own server or in the cloud. Operating System: Windows, Linux, OS X.

Router Software

65. Vyatta
Vyatta offers both hardware appliances and networking software. Both are based on the open source software available through Vyatta.org. Operating System: Linux.

School Management

66. openSIS
This open source school information system (SIS) claims that it can save schools 75 percent versus commercial solutions. The free community edition is suitable for small or medium-sized schools with their own IT staff, while the SaaS school edition is for small or medium-sized schools who don't want the hassle of running their own servers. Large schools and districts should look at the district edition, which can run in the cloud or on premises. Operating System: OS Independent.

Server Software

67. ClearOS
ClearOS combines a networking server, security gateway, file server, groupware server, Web server and more into a single download. All three versions—enterprise, home and core—are available on a free and open source basis, but paid software subscriptions, technical support, hardware and other services are available through ClearCare. Operating System: Linux.
68.Zentyal
Like ClearOS, Zentyal is a multi-function small business server with unified threat management, officer server, communications and other capabilities. The basic edition is free; small business and enterprise editions, training, and add-ons require a fee. Operating System: Linux.

Storage

69. Openfiler
Downloaded more than 1 million times, Openfiler incorporates both file-based Network Attached Storage (NAS) and block-based Storage Area Networking (SAN) in a single download. Subscriptions, plug-ins and an administrator's guide are available for a fee. Operating System: Linux

Time Tracking

70. TimeTrex
The self-proclaimed "world's most popular open source payroll and time management software," TimeTrex handles scheduling, attendance, job costing and payroll functions. It comes in a free standard edition and paid pro and business editions; the pro and business versions can be purchased on an SaaS basis or deployed on site. Operating System: Windows, Linux, OS X.

User Authentication

71. WiKID
This software-based two-factor authentication system aims to be less expensive and more extensible than competing solutions. The paid enterprise server version adds support and some proprietary functionality not included in the free open source version. Operating System: OS Independent.

Virtual Hosting With PureFTPd And MySQL (Incl. Quota And Bandwidth Management) On CentOS 6.2


This document describes how to install a PureFTPd server that uses virtual users from a MySQL database instead of real system users. This is much more performant and allows to have thousands of ftp users on a single machine. In addition to that I will show the use of quota and upload/download bandwidth limits with this setup. Passwords will be stored encrypted as MD5 strings in the database.
For the administration of the MySQL database you can use web based tools like phpMyAdmin which will also be installed in this howto. phpMyAdmin is a comfortable graphical interface which means you do not have to mess around with the command line.
This tutorial is based on CentOS 6.2. You should already have set up a basic CentOS 6.2 system, for example as described in the first six chapters of this tutorial: The Perfect Server - CentOS 6.2 x86_64 With Apache2 [ISPConfig 3]
This howto is meant as a practical guide; it does not cover the theoretical backgrounds. They are treated in a lot of other documents in the web.
This document comes without warranty of any kind! I want to say that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Preliminary Note

In this tutorial I use the hostname server1.example.com with the IP address 192.168.0.100. These settings might differ for you, so you have to replace them where appropriate.

2 Install MySQL And phpMyAdmin

First we enable the EPEL repository on our CentOS system as some packages that we are going to install in the course of this tutorial are not available in the official CentOS 6.2 repositories:
rpm --import https://fedoraproject.org/static/0608B895.txt
wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm
rpm -ivh epel-release-6-5.noarch.rpm
yum install yum-priorities
Edit /etc/yum.repos.d/epel.repo...
vi /etc/yum.repos.d/epel.repo
... and add the line priority=10 to the [epel] section:
[epel]
name=Extra Packages for Enterprise Linux 6 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
failovermethod=priority
enabled=1
priority=10
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
[...]
Now we can install MySQL and phpMyAdmin as follows:
yum install mysql mysql-server phpMyAdmin httpd
Afterwards, we must edit the file /etc/httpd/conf.d/phpMyAdmin.conf to make phpMyAdmin accessible from all IP addresses. By default, it's accessible only from 127.0.0.1. Comment out this section:
#
#   Order Deny,Allow
#   Deny from All
#   Allow from 127.0.0.1
#   Allow from ::1
#
so that the file looks like this:
vi /etc/httpd/conf.d/phpMyAdmin.conf
# phpMyAdmin - Web based MySQL browser written in php
#
# Allows only localhost by default
#
# But allowing phpMyAdmin to anyone other than localhost should be considered
# dangerous unless properly secured by SSL

Alias /phpMyAdmin /usr/share/phpMyAdmin
Alias /phpmyadmin /usr/share/phpMyAdmin

#
#   Order Deny,Allow
#   Deny from All
#   Allow from 127.0.0.1
#   Allow from ::1
#


   Order Deny,Allow
   Deny from All
   Allow from 127.0.0.1
   Allow from ::1


# These directories do not require access over HTTP - taken from the original
# phpMyAdmin upstream tarball
#

    Order Deny,Allow
    Deny from All
    Allow from None



    Order Deny,Allow
    Deny from All
    Allow from None



    Order Deny,Allow
    Deny from All
    Allow from None


# This configuration prevents mod_security at phpMyAdmin directories from
# filtering SQL etc.  This may break your mod_security implementation.
#
#
#    
#        SecRuleInheritance Off
#    
#
Then we create the system startup links for MySQL and Apache (so that both start automatically whenever the system boots) and start both services:
chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start
chkconfig --levels 235 httpd on
/etc/init.d/httpd start
Create a password for the MySQL user root (replace yourrootsqlpassword with the password you want to use):
mysql_secure_installation
[root@server1 ~]# mysql_secure_installation




NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!


In order to log into MySQL to secure it, we'll need the current
password for the root user.  If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
 <-- ENTER
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

Set root password? [Y/n]
 <-- ENTER
New password: <-- yourrootsqlpassword
Re-enter new password: <-- yourrootsqlpassword
Password updated successfully!
Reloading privilege tables..
 ... Success!


By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n]
 <-- ENTER
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n]
 <-- ENTER
 ... Success!

By default, MySQL comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n]
 <-- ENTER
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n]
 <-- ENTER
 ... Success!

Cleaning up...



All done!  If you've completed all of the above steps, your MySQL
installation should now be secure.

Thanks for using MySQL!


[root@server1 ~]#


3 Install PureFTPd With MySQL Support

The CentOS PureFTPd package supports various backends, such as MySQL, PostgreSQL, LDAP, etc. Therefore, all we have to do is install the normal PureFTPd package:
yum install pure-ftpd
Then we create an ftp group (ftpgroup) and user (ftpuser) that all our virtual users will be mapped to. Replace the group- and userid 2001 with a number that is free on your system:
groupadd -g 2001 ftpgroup
useradd -u 2001 -s /bin/false -d /bin/null -c "pureftpd user" -g ftpgroup ftpuser

4 Create The MySQL Database For PureFTPd

Now we create a database called pureftpd and a MySQL user named pureftpd which the PureFTPd daemon will use later on to connect to the pureftpd database:
mysql -u root -p
CREATE DATABASE pureftpd;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP ON pureftpd.* TO 'pureftpd'@'localhost' IDENTIFIED BY 'ftpdpass';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP ON pureftpd.* TO 'pureftpd'@'localhost.localdomain' IDENTIFIED BY 'ftpdpass';
FLUSH PRIVILEGES;
Replace the string ftpdpass with whatever password you want to use for the MySQL user pureftpd. Still on the MySQL shell, we create the database table we need (yes, there is only one table!):
USE pureftpd;
CREATE TABLE ftpd (
User varchar(16) NOT NULL default '',
status enum('0','1') NOT NULL default '0',
Password varchar(64) NOT NULL default '',
Uid varchar(11) NOT NULL default '-1',
Gid varchar(11) NOT NULL default '-1',
Dir varchar(128) NOT NULL default '',
ULBandwidth smallint(5) NOT NULL default '0',
DLBandwidth smallint(5) NOT NULL default '0',
comment tinytext NOT NULL,
ipaccess varchar(15) NOT NULL default '*',
QuotaSize smallint(5) NOT NULL default '0',
QuotaFiles int(11) NOT NULL default 0,
PRIMARY KEY (User),
UNIQUE KEY User (User)
) ENGINE=MyISAM;
quit;
As you may have noticed, with the quit; command we have left the MySQL shell and are back on the Linux shell.
BTW, (I'm assuming that the hostname of your ftp server system is server1.example.com) you can access phpMyAdmin under http://server1.example.com/phpMyAdmin/ (you can also use the IP address instead of server1.example.com) in a browser and log in as the user pureftpd. Then you can have a look at the database. Later on you can use phpMyAdmin to administrate your PureFTPd server.

5 Configure PureFTPd

 
Edit /etc/pure-ftpd/pure-ftpd.conf and make sure that the ChrootEveryone, MySQLConfigFile, and CreateHomeDir lines are enabled and look like this:
vi /etc/pure-ftpd/pure-ftpd.conf
[...]
ChrootEveryone              yes
[...]
MySQLConfigFile               /etc/pure-ftpd/pureftpd-mysql.conf
[...]
CreateHomeDir               yes
[...]
The ChrootEveryone setting will make PureFTPd chroot every virtual user in his home directory so he will not be able to browse directories and files outside his home directory. The CreateHomeDir line will make PureFTPd create a user's home directory when the user logs in and the home directory does not exist yet.
Then we edit /etc/pure-ftpd/pureftpd-mysql.conf. It should look like this:
cp /etc/pure-ftpd/pureftpd-mysql.conf /etc/pure-ftpd/pureftpd-mysql.conf_orig
cat /dev/null > /etc/pure-ftpd/pureftpd-mysql.conf
vi /etc/pure-ftpd/pureftpd-mysql.conf
MYSQLSocket      /var/lib/mysql/mysql.sock
#MYSQLServer     localhost
#MYSQLPort       3306
MYSQLUser       pureftpd
MYSQLPassword   ftpdpass
MYSQLDatabase   pureftpd
#MYSQLCrypt md5, cleartext, crypt() or password() - md5 is VERY RECOMMENDABLE uppon cleartext
MYSQLCrypt      md5
MYSQLGetPW      SELECT Password FROM ftpd WHERE User="\L" AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MYSQLGetUID     SELECT Uid FROM ftpd WHERE User="\L" AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MYSQLGetGID     SELECT Gid FROM ftpd WHERE User="\L"AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MYSQLGetDir     SELECT Dir FROM ftpd WHERE User="\L"AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MySQLGetBandwidthUL SELECT ULBandwidth FROM ftpd WHERE User="\L"AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MySQLGetBandwidthDL SELECT DLBandwidth FROM ftpd WHERE User="\L"AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MySQLGetQTASZ   SELECT QuotaSize FROM ftpd WHERE User="\L"AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
MySQLGetQTAFS   SELECT QuotaFiles FROM ftpd WHERE User="\L"AND status="1" AND (ipaccess = "*" OR ipaccess LIKE "\R")
Make sure that you replace the string ftpdpass with the real password for the MySQL user pureftpd in the line MYSQLPassword! Please note that we use md5 as MYSQLCrypt method, which means we will store the users' passwords as an MD5 string in the database which is far more secure than using plain text passwords!
Now we create the system startup links for PureFTPd and start it:
chkconfig --levels 235 pure-ftpd on
/etc/init.d/pure-ftpd start

6 Populate The Database And Test

To populate the database you can use the MySQL shell:
mysql -u root -p
USE pureftpd;
Now we create the user exampleuser with the status 1 (which means his ftp account is active), the password secret (which will be stored encrypted using MySQL's MD5 function), the UID and GID 2001 (use the userid and groupid of the user/group you created at the end of step two!), the home directory /home/www.example.com, an upload and download bandwidth of 100 KB/sec. (kilobytes per second), and a quota of 50 MB:
INSERT INTO `ftpd` (`User`, `status`, `Password`, `Uid`, `Gid`, `Dir`, `ULBandwidth`, `DLBandwidth`, `comment`, `ipaccess`, `QuotaSize`, `QuotaFiles`) VALUES ('exampleuser', '1', MD5('secret'), '2001', '2001', '/home/www.example.com', '100', '100', '', '*', '50', '0');
quit;
Now open your FTP client program on your work station (something like WS_FTP or SmartFTP if you are on a Windows system or gFTP on a Linux desktop) and try to connect. As hostname you use server1.example.com (or the IP address of the system), the username is exampleuser, and the password is secret.
If you are able to connect - congratulations! If not, something went wrong.
Now, if you run
ls -l /home
you should see that the directory /home/www.example.com (exampleuser's home directory) has been created automatically, and it is owned by ftpuser and ftpgroup (the user/group we created at the end of step two):
[root@server1 ~]# ls -l /home
total 4
drwxr-xr-x 2 ftpuser ftpgroup 4096 Mar  5 02:13 www.example.com
[root@server1 ~]#

7 Database Administration

For most people it is easier if they have a graphical front-end to MySQL; therefore you can also use phpMyAdmin (in this example under http://server1.example.com/phpMyAdmin/) to administrate the pureftpd database.

Whenever you want to create a new user, you have to create an entry in the table ftpd so I will explain the columns of this table here:

ftpd Table:

  • User: The name of the virtual PureFTPd user (e.g. exampleuser).
  • status: 0 or 1. 0 means the account is disabled, the user cannot login.
  • Password: The password of the virtual user. Make sure you use MySQL's MD5 function to save the password encrypted as an MD5 string:
  • UID: The userid of the ftp user you created at the end of step two (e.g. 2001).
  • GID: The groupid of the ftp group you created at the end of step two (e.g. 2001).
  • Dir: The home directory of the virtual PureFTPd user (e.g. /home/www.example.com). If it does not exist, it will be created when the new user logs in the first time via FTP. The virtual user will be jailed into this home directory, i.e., he cannot access other directories outside his home directory.
  • ULBandwidth: Upload bandwidth of the virtual user in KB/sec. (kilobytes per second). 0 means unlimited.
  • DLBandwidth: Download bandwidth of the virtual user in KB/sec. (kilobytes per second). 0 means unlimited.
  • comment: You can enter any comment here (e.g. for your internal administration) here. Normally you leave this field empty.
  • ipaccess: Enter IP addresses here that are allowed to connect to this FTP account. * means any IP address is allowed to connect.
  • QuotaSize: Storage space in MB (not KB, as in ULBandwidth and DLBandwidth!) the virtual user is allowed to use on the FTP server. 0 means unlimited.
  • QuotaFiles: amount of files the virtual user is allowed to save on the FTP server. 0 means unlimited.

8 Anonymous FTP

If you want to create an anonymous ftp account (an ftp account that everybody can login to without a password), you need a user and a group called ftp. Both have been created automatically when you installed the pure-ftpd package, so you don't need to create them manually. However, ftp's homedir is /var/ftp by default, but I'd like to create the anonymous ftp directory in /home/ftp (the normal users' ftp directories are in /home as well, e.g. /home/www.example.com). But of course, you can use the /var/ftp directory for anonymous ftp, if you prefer it.
If you want to use /home/ftp, open /etc/passwd and change the ftp user's homedir from /var/ftp to /home/ftp (don't do this if you want to use /var/ftp):
vi /etc/passwd
[...]
#ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
ftp:x:14:50:FTP User:/home/ftp:/sbin/nologin
[...]
Then move /var/ftp to /home (don't do this if you want to use /var/ftp):
mv /var/ftp /home
Then we create the directory /home/ftp/incoming which will allow anonymous users to upload files. We will give the /home/ftp/incoming directory permissions of 311 so that users can upload, but not see or download any files in that directory. The /home/ftp directory will have permissions of 555 which allows seeing and downloading of files:
chown ftp:nobody /home/ftp
cd /home/ftp
mkdir incoming
chown ftp:nobody incoming/
chmod 311 incoming/
cd ../
chmod 555 ftp/
(If you want to use /var/ftp instead, replace /home/ftp with /var/ftp in the above commands.)
Anonymous users will be able to log in, and they will be allowed to download files from /home/ftp, but uploads will be limited to /home/ftp/incoming (and once a file is uploaded into /home/ftp/incoming, it cannot be read nor downloaded from there; the server admin has to move it into /home/ftp first to make it available to others).
Now we have to configure PureFTPd for anonymous ftp. Open /etc/pure-ftpd/pure-ftpd.conf and make sure that you have the following settings in it:
vi /etc/pure-ftpd/pure-ftpd.conf
[...]
NoAnonymous                 no
[...]
AntiWarez                   no
[...]
AnonymousBandwidth            8
[...]
AnonymousCantUpload         no
[...]
(The AnonymousBandwidth setting is optional - it allows you to limit upload and download bandwidths for anonymous users. 8 means 8 KB/sec. Use any value you like, or comment out the line if you don't want to limit bandwidths.)
Finally, we restart PureFTPd:
/etc/init.d/pure-ftpd restart

9 Links