Use NFS to Extend and Integrate Your Filesystems

Someday your server will need more disk space. Rather than purchase new hardware, you may be able to take advantage of disk space you already own by using NFS (Network File System), a distributed filesystem for which support is integrated in the Linux kernel. Any machine can host NFS server software to share its disk space, or use an NFS client to make use of that space.

NAS is not SAN You may hear NAS and SAN mentioned frequently together as alternatives, but they are quite different. NAS works with data files, while SAN (Storage Area Networking) works with data blocks. NAS is usually simpler to set up, works over TCP/IP, and does not require special hardware. SAN is more complex and does require special hardware.

Very often NFS is offered from a dedicated device as network-attached storage. Let’s see how you can create a simple but powerful NAS server based on CentOS 6.
NFS has been around for more than 20 years, making it very mature technology. CentOS 6 is a good choice for running NFS because it supports the latest version, NFSv4, with full compatibility with older versions and different operating systems.

Server Installation and Configuration

To begin creating a dedicated NAS server, you should have a minimal CentOS 6 installation, which you can install by using the minimal image. For a secure and optimized NAS server you should run only the packages you absolutely need.
Next, consider what filesystem NFS should run on top of. NFS works with well-known Linux filesystems such as ext2, ext3, and the newer ext4. However, for best performance, reliability, and compatibility, use the XFS filesystem for your NFS shares. XFS supports larger files and works well under heavy load and simultaneous access, as you’re likely to have on an NFS server. Just note that XFS is officially supported only for 64-bit architectures and is not recommended for 32-bit operating systems.
To format a partition in XFS, use the mkfs.xfs command-line tool. If there’s already a partition on your hard drive, all the data on that partition will be lost when you format it for XFS.
To run NFS on the server you need installed the packages nfs-utils, rpcbind, and their dependencies. Then, to ensure that all the necessary services are started and stopped when the server boots and shuts down, execute:

chkconfig rpcbind on
chkconfig nfs on
chkconfig nfslock on

Now edit the file /etc/sysconfig/nfs. By default all options are commented out, making them inactive. Configure the server to support only NFSv4 and disable previous versions by uncommenting the following rows:

MOUNTD_NFS_V2="no"
MOUNTD_NFS_V3="no"
RPCNFSDARGS="-N 2 -N 3"

Unlike previous versions, NFSv4 handles UTF-8-encoded files correctly. It’s also faster, with improved caching, and has better crash recovery mechanisms.
Also uncomment:

MOUNTD_PORT=892
STATD_PORT=662
LOCKD_TCPPORT=32803

These lines specify on which ports the NFS services run. The benefit of having the ports explicitly specified is that you can use the port numbers in your firewall rules.
It’s essential to maintain restrictive firewall rules in order to ensure the security of an NFS server. By default, every CentOS 6 installation comes with a firewall enabled and SSH as the only accessible service. You need to add to its rules the ability for NFS clients to access only the ports needed. If, as an example, an NFS client has an IP address of 10.0.0.2, run the following iptables commands in terminal:

iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT #NFS core communication
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT #rpcbind/sunrpc TCP port
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT #rpcbind/sunrpc UDP port
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 892 -j ACCEPT #MOUNTD TCP PORT
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m udp -p udp --dport 892 -j ACCEPT #MOUNTD UDP PORT
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 662 -j ACCEPT #STATD port
iptables -I INPUT -s 10.0.0.2 -m state --state NEW -m tcp -p tcp --dport 32803 -j ACCEPT #LOCKD TCP port

To save the above rules, run the command service iptables save. Repeat the process for other clients, substituting their IP addresses for 10.0.0.2. Alternatively, you can allow access to whole subnets by specifying their network bits, such as 10.0.0.0/24.
You may notice that the above firewall rules allow mostly TCP connections. TCP is the default communication protocol in NFSv4, though UDP is also supported. TCP’s connection-oriented communication mode provides the required reliability that’s critical for working with files and data.
Next, it’s time to configure the shares. The file /etc/exports contains a path to a shared directory on each row, along with clients’ permissions for it and any custom settings that may apply. A simple configuration example would contain only /nfs 10.0.0.2, which says that the /nfs directory is accessible for a client with the IP address 10.0.0.2. Such short and simple export directives are sufficient to provide secure and optimized access to NFS, thanks to the powerful default NFS settings, which are:

• Read-only access By default, clients can only read files. To allow write access too, specify immediately after the IP address (rw). If you leave a blank space between the IP address and (rw), you will allow read/write access for any IP address, which would be a security nightmare.
• Root squash When the remote client identifies itself using the root ID, it receives locally the ID of the user nfsnobody, which prevents remote superusers from gaining absolute powers. It’s highly recommended that you leave this default in place, but you can disable it by adding no_root_squash at the end of the line.
• Accept connections only from well-known ports The secure directive means that the request should originate on ports lower than 1024, the range of the so-called well-known ports. Only services started by the superuser can initiate such connections, which means a compromised non-root account on the remote server cannot connect to NFS pretending to be another user. You can turn off this option by specifying insecure.
• Synchronous access This ensures that the NFS server replies to the client after data has been safely stored locally. This pause has a small performance impact but ensures data integrity. With asynchronous behavior, which was the default for NFS versions prior to 4, the server replies to the client as soon as it has processed the request, without taking into consideration whether the data has been successfully stored on the NFS server filesystem. To offer asynchronous access, use the option async.
• Write delay By default, the NFS server will delay disk writes if it suspects it will get subsequent write requests so that it can process them in batch. This makes the NFS server faster when there are numerous, small write requests at the same time. If you know this is not likely to be the case, you can turn off this option by specifying no_wdelay.

Client Installation and Configuration

A client running Linux and especially CentOS 6 can easily connect to NFS shares. In CentOS you need the package nfs-utils, which is installed by default. It extends the NFS kernel support and provides additional utilities, such as showmount, that facilitate work with NFS.
Before actually mounting NFS shares, you should explore the remote NFS server and discover what it offers, which means looking at its export list. If the NFS server’s IP address is 10.0.0.1, run showmount -e 10.0.0.1. You should see output similar to:

Export list for 10.0.0.1:
/nfs 10.0.0.2

This means that the /nfs share would be accessible to a client with the IP address 10.0.0.2. If that’s you, you can mount it as you would any other media. First create a directory for it, such as /media/nfs. Then run mount 10.0.0.1:/nfs /media/nfs. You should then see your shared files inside /media/nfs.
If you want to make sure that the NFS directory is mounted automatically after reboot, add to the client’s /etc/fstab file a line containing 10.0.0.1:/nfs /media/nfs nfs timeo=10. This instructs the client to mount the /nfs share available at 10.0.0.1. The only custom setting here is the timeout (timeo), which is 10 seconds, in order to prevent the client from hanging for too long waiting for an NFS reply. You can add more custom settings by separating them with commas and no space.
Once you have mounted the NFS share on the client you can try some tests to determine the best settings for your environment. A useful tool for performance tests and benchmarking is nfsiostat. Similar to the popular iostat, nfsiostat gives statistics for read and write operations on NFS shares.
Based on the results of your tests, you can mount NFS shares with different options. For instance, play with the read and write size options, rsize and wsize. They specify the data block size received and sent. On faster networks with no congestion, large sizes may benefit performance.

NFS Security

In the past, NFS was considered insecure by design because it did not offer username and password authorization. With the latest NFSv4 this is not a problem; for authorization you can use Kerberos.
The big question for NFS security is how secure and trustworthy the clients are, because you must rely on them to provide legitimate userids. As long as you receive legitimate userids you can use the native Linux permission and ownership features to refine access and security on the server side.
NFS continues to play a prominent role for distributed file systems in the Linux world thanks to its performance, reliability, robustness, and security. Furthermore, NFS is supported not only in Unix-based operating systems such as Linux and Mac OS, but also under Windows with Windows Services for Unix, making it an excellent option for heterogenous interoperability.