Saturday, April 17, 2010

Getting to know the hardware of your Linux box

When you buy a new PC, laptop or server and install a Linux you want to know what hardware is actually in the Linux box and more importantly which piece of hardware is supported by the kernel out of the box and which needs special tweaking with modules to get it work.

Here is a list of commands which should help you to troubleshoot your hardware and find some information about it. This is not a ultimate troubleshooting guide but certainly will serve as a good starting point.


NOTE: some commands may not be avaiable for your platform after default installation as well as some commands may be distribution specific only.

1. What hardware is in my Linux box:
$ lspci

or
# dmidecode
2. What hardware is using which module.

NOTE:Root permissions are needed to execute this command:

# lspci -v

or
# lspci -vvv

or
# hardinfo

or
$ hardinfo ( GUI )

or
# lshw

or
# lshw-gtk  ( GUI )

or
# hwinfo
3. BIOS

Retrieve BIOS information:

# biosdecode

Retrieve BIOS vendor information:
# dmidecode -s  bios-vendor
4. Retrieve 
information about your motherboard only:
# dmidecode --type baseboard
5. What USB devices are 
plugged in:
$ lsusb
6. Graphic card information:
$ lspci | grep VGA
7. Check the 
size of the hard drive and partitions

Check the size of the hard drive and what hard drives are available in the system.

This command will also list USB drives and sticks. You need a root permissions to execute this command:
# fdisk -l | grep GB

Check what partitions and file system is in use on my hard drives:
# fdisk -l
8. Locate CD/DVD-ROM device 
file:
$ wodim --devices

or
$ wodim --scanbus
9. Modules

What modules are currently loaded:
$ lsmod

get a information about any particular module:
$ /sbin/modinfo module_name

remove modules:
# modprobe --remove module_name

load a modules to the kernel:
# modprobe module_name

What hardware is using which module.

NOTE:Root permissions are needed to execute this command:
# lspci -v

or
# lspci -vvv
10. Check for PCMCIA cards:
# lspcmcia
11. Processor type:
$ cat /proc/cpuinfo
or
# dmidecode --type 4

Is my processor using 32 or 64 bit instruction set:
$ cat /proc/cpuinfo | grep flags | grep lm

if you get some output you have 64 bit if no output is presented you are using 32 or even 16 bit CPU.

NOTE: grep command looks for lm flag.

12. RAM

How much RAM is installed in my Linux and how much of it is in use ( mega bytes ).

It will also include swap memory:
$ free -m

or
$ top

or
$ cat /proc/meminfo
13. Sound Card

Check sound card settings. This command will reveal whether your sound card is installed and what modules are in use:
$ cat /dev/sndstat
14. Available wireless cards:
$ iwconfig
15. What speed is set to FANs:
$ cat /proc/acpi/ibm/fan
16. Battery

Get a battery information on your laptop:
$ powersave -b
17. List Plug and Play BIOS 
device
# lspnp

5 cool Linux audio editors worth knowing

One good thing about using Linux is the amount of choices one has when it comes to the use of any application.

If you are a music lover, the following 5 music/audio editors are worthy of note

"LMMS is a free cross-platform alternative to commercial programs like FL Studio, which allow you to produce music with your computer.

This includes the creation of melodies and beats, the synthesis and mixing of sounds, and arranging of samples.

You can have fun with your MIDI-keyboard and much more; all in a user-friendly and modern interface."

Talking of features, it boasts of
  • Song-Editor for composing songs
  • A Beat+Bassline-Editor for creating beats and basslines
  • An easy-to-use Piano-Roll for editing patterns and melodies
  • An FX mixer with 64 FX channels and arbitrary number of effects allow  unlimited mixing possibilities
  • Many powerful instrument and effect-plugins out of the box
  • Full user-defined track-based automation and computer-controlled automation sources
  • Compatible with many standards such as SoundFont2, VST(i), LADSPA, GUS Patches, and MIDI
  • Import of MIDI and FLP (Fruityloops Project) files

Available for all three popular platforms, Traverso DAW is "a GPL licensed, cross platform multitrack audio recording and editing suite, with an innovative and easy to master User Interface.

It's suited for both the professional and home user, who needs a robust and solid DAW." 


The developers call it a complete suite because it "is a complete solution from recording to CD Mastering.

By supplying many common tools in one package, you don't have to learn how to use lots of applications with different user interfaces.

This considerably lowers the learning curve, letting you get your audio processing work done faster." Worth Checking out.
Rosegarden, the development of which is sponsored by  Typo 3, "is an easy-to-learn, attractive application that runs on Linux, ideal for composers, musicians, music students, and small studio or home recording environments.


It is a well-rounded audio and  MIDI sequencer, score editor, and general-purpose music" composition and editing environment."

Ardour
Released under the GPL, Ardour is a hard disk recorder and digital audio workstation application that runs on Linux, FreeBSD and Mac OSX.


The application aims to provide a digital audio workstation suitable for professional use with features such as recording, mixing, editing and mastering.

Ardour relies heavily on plugins ot get some features to work.

Hydrogen
Hydrogen is an advanced drum machine for GNU/Linux. It's main goal is to bring professional yet simple and intuitive pattern-based drum programming with features such as

  •     Up to 64 ticks per pattern with individual level per event and variable pattern length.
  •     32 instrument tracks with volume, mute, solo, pan capabilities.
  •     Multi layer support for instruments (up to 16 samples for each instrument).
  •     Ability to import/export song files.
  •     Unique human velocity, human time, pitch and swing functions.
  •     Multiple patterns playing at once.
There are dozens of others out there that easily get the job done no matter its complexity.


If there is any one reason why Linux is such a force to reckon with, it's about how hard it is to choose from lots of solutions to any given problem.


Give any of the above apps a try and tell me what you think of them. If you prefer others not mentioned here, please share them with us in the comments.

Friday, April 16, 2010

tc: Linux HTTP Outgoing Traffic Shaping (Port 80 Traffic Shaping)

I've 10Mbps server port dedicated to our small business server. The server also act as a backup DNS server and I'd like to slow down outbound traffic on port 80.

How do I limit bandwidth allocation to http service 5Mbps (burst to 8Mbps) at peak times so that DNS and other service will not go down due to heavy activity under Linux operating systems?


You need use the tc command which can slow down traffic for given port and services on servers and it is called traffic shaping:
  1. When traffic is shaped, its rate of transmission is under control, in other words you apply some sort of bandwidth allocation for each port or or so called Linux services. Shaping occurs on egress.
  2. You can only apply traffic shaping to outgoing or forwarding traffic i.e. you do not have any control for incoming traffic to server. However, tc can do policing controls for arriving traffic. Policing thus occurs on ingress. This FAQ only deals with traffic shaping.

Token Bucket (TB)

A token bucket is nothing but a common algorithm used to control the amount of data that is injected into a network, allowing for bursts of data to be sent.

It is used for network traffic shaping or rate limiting. With token bucket you can define the maximum rate of traffic allowed on an interface at a given moment in time.

tokens/sec
                                   |   |
                                   |  | Bucket to
                                   |  | to hold b tokens
                             +======+=====+
                                          |
                                          |
        |                                \|/
Packets |      +============+
stream  | ---> | token wait | --->  Remove token  --->  eth0
        |      +============+
  1. The TB filter puts tokens into the bucket at a certain rate.
  2. Each token is permission for the source to send a specific number of bits into the network.
  3. Bucket can hold b tokens as per shaping rules.
  4. Kernel can send packet if you've a token else traffic need to wait.

How Do I Use tc command?

WARNING! These examples requires good understanding of TCP/IP and other networking concepts. All new user should try out examples in test environment.

tc command is by default installed on my Linux distributions. To list existing rules, enter:


# tc -s qdisc ls dev eth0


Sample outputs:
qdisc pfifo_fast 0: root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 2732108 bytes 10732 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0

Your First Traffic Shaping Rule

First, send ping request to cyberciti.biz from your Local Linux workstation and note down ping time, enter:


# ping cyberciti.biz


Sample outputs:
PING cyberciti.biz (74.86.48.99) 56(84) bytes of data.
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=1 ttl=47 time=304 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=2 ttl=47 time=304 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=3 ttl=47 time=304 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=4 ttl=47 time=304 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=5 ttl=47 time=304 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=6 ttl=47 time=304 ms

Type the following tc command to slow down traffic by 200 ms:


# tc qdisc add dev eth0 root netem delay 200ms


Now, send ping requests again:


# ping cyberciti.biz


Sample outputs:
PING cyberciti.biz (74.86.48.99) 56(84) bytes of data.
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=1 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=2 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=3 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=4 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=5 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=6 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=7 ttl=47 time=505 ms
64 bytes from txvip1.simplyguide.org (74.86.48.99): icmp_seq=8 ttl=47 time=505 ms
^C
--- cyberciti.biz ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 7006ms
rtt min/avg/max/mdev = 504.464/505.303/506.308/0.949 ms

To list current rules, enter:


# tc -s qdisc ls dev eth0


Sample outputs:
qdisc netem 8001: root limit 1000 delay 200.0ms
 Sent 175545 bytes 540 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0 

To delete all rules, enter:


# tc qdisc del dev eth0 root
# tc -s qdisc ls dev eth0



TBF Example
To attach a TBF with a sustained maximum rate of 1mbit/s, a peakrate of 2.0mbit/s, a 10kilobyte buffer, with a pre-bucket queue size limit calculated so the TBF causes at most 70ms of latency, with perfect peakrate behavior, enter:


# tc qdisc add dev eth0 root tbf rate 1mbit burst 10kb latency 70ms peakrate 2mbit minburst 1540


HTB - Hierarchy Token Bucket
To control the use of the outbound bandwidth on a given link use HTB:
  1. rate - You can set the allowed bandwidth.
  2. ceil - You can set burst bandwidth allowed when buckets are present.
  3. prio - You can set priority for additional bandwidth. So classes with lower prios are offered the bandwidth first. For example, you can give lower prio for DNS traffic and higher for HTTP downloads.
  4. iptables and tc: You need to use iptables and tc as follows to control outbound HTTP traffic.

Example: HTTP Outbound Traffic Shaping

First , delete existing rules for eth1:


# /sbin/tc qdisc del dev eth1 root


Turn on queuing discipline, enter:


# /sbin/tc qdisc add dev eth1 root handle 1:0 htb default 10


Define a class with limitations i.e. set the allowed bandwidth to 512 Kilobytes and burst bandwidth to 640 Kilobytes for port 80:


# /sbin/tc class add dev eth1 parent 1:0 classid 1:10 htb rate 512kbps ceil 640kbps prio 0


Please note that port 80 is NOT defined anywhere in above class. You will use iptables mangle rule as follows:


# /sbin/iptables -A OUTPUT -t mangle -p tcp --sport 80 -j MARK --set-mark 10


To save your iptables rules, enter (RHEL specific command):


# /sbin/service iptables save


Finally, assign it to appropriate qdisc:


# tc filter add dev eth1 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10


Here is another example for port 80 and 22:


/sbin/tc qdisc add dev eth0 root handle 1: htb
/sbin/tc class add dev eth0 parent 1: classid 1:1 htb rate 1024kbps
/sbin/tc class add dev eth0 parent 1:1 classid 1:5 htb rate 512kbps ceil 640kbps prio 1
/sbin/tc class add dev eth0 parent 1:1 classid 1:6 htb rate 100kbps ceil 160kbps prio 0
/sbin/tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 5 fw flowid 1:5
/sbin/tc filter add dev eth0 parent 1:0 prio 0 protocol ip handle 6 fw flowid 1:6
/sbin/iptables -A OUTPUT -t mangle -p tcp --sport 80 -j MARK --set-mark 5
/sbin/iptables -A OUTPUT -t mangle -p tcp --sport 22 -j MARK --set-mark 6



How Do I Monitor And Test Speed On Sever?
Use the following tools


# /sbin/tc -s -d class show dev eth0
# /sbin/iptables -t mangle -n -v -L
# iptraf
# watch /sbin/tc -s -d class show dev eth0



To test download speed use lftp or wget command line tools.

Tuesday, April 6, 2010

Beginners Guide to File and Directory Permissions ( umask, chmod, read, write, execute )


Unix file and directory permission is in the form of a 3×3 structure. i.e Three permissions (read, write and execute) available for three types of users (owner, groups and others).

In the output of ls -l command, the 9 characters from 2nd to 10th position represents the permissions for the 3 types of users.
-rw-r--r--  1 sathiya sathiya  272 Mar 17 08:22 test.txt
In the above example:
  • User (sathiya) has read and write permission
  • Group has read permission
  • Others have read permission
Three file permissions:
  • read: permitted to read the contents of file.
  • write: permitted to write to the file.
  • execute: permitted to execute the file as a program/script.
Three directory permissions:
  • read: permitted to read the contents of directory ( view files and sub-directories in that directory ).
  • write: permitted to write in to the directory. ( create files and sub-directories in that directory )
  • execute: permitted to enter into that directory.
Numeric values for the read, write and execute permissions:
  • read 4
  • write 2
  • execute 1
To have combination of permissions, add required numbers. For example, for read and write permission, it is 4+2 = 6.

Change File and Directory Permissions Using Chmod Command

You can use either the octal representation or symbolic representation to change the permission of a file or directory.
Octal representation for permissions:
  • First number is for user
  • Second number is for group
  • Third number is for others
For example, give read, write ( 4+2 = 6 ) to user and read ( 4 ) to group and others.
$ chmod 644 filename
For example, give read, execute ( 4 + 1 = 5 ) to user and read (4 ) to group, and nothing ( 0 ) to others.
$ chmod 540 filename
For example, give read, write ( 4 + 2 = 6 ) to user and nothing ( 0 ) to group, and read ( 4 ) to others.
$ chmod 604 filename

Umask 022 is Responsible for the default permission of a file

The default umask value is 0022, which decides the default permission for a new file or directory. Default permission for a directory is 0777, for files the permissions are 0666 from which the default umask value 0022 is deducted to get the newly created files or directory permission.
Final default permission for a file is calculated as shown below:
  • Default file permission: 666
  • Default umask : 022
  • Final default file permission: 644
Final default permission for a directory is calculated as shown below:
  • Default directory permission: 777
  • Default umask: 022
  • Final default directory permission: 755
You can change the umask value to appropriate value of what you need based upon the above calculation.
For example, if you don’t want anybody other than the user (owner) to do anything on the file or directory then you can give umask as 0077.
$ umask 0077
After this, if you create a file or directory, it will have permissions only for the user as shown below:
$ > testfile
$ ls -l testfile
-rw------- 1 sathiya sathiya 0 Mar 17 08:23 testfile

12 More of the Best Free Linux Books

Many computer users have an insatiable appetite to deepen their understanding of computer operating systems and computer software.

Linux users are no different in that respect. At the same time as developing a huge range of open source software, the Linux community fortunately has also written a vast range of documentation in the form of books, guides, tutorials, HOWTOs, man pages, and other help to aid the learning process.

Some of this documentation is intended specifically for a newcomer to Linux, or those that are seeking to move away from a proprietary world and embrace freedom.

There are literally thousands of Linux books which are available to purchase from any good (online) book shop.

However, the focus of this article is to highlight champion Linux books which make an invaluable contribution to learning about Linux, and which are also available to download without charge.

We have tried to select a fairly diverse selection of books in this article so that there should be something of interest here for any type of user whatever their level of computing knowledge.

This article should be read in conjunction with our previous article on free Linux books, entitled 20 of the Best Free Linux Books

1. GNU/Linux Advanced Administration
GNU/Linux Advanced Administration
Websiteftacademy.org
AuthorRemo Suppi Boldrito, Josep Jorba Esteve
FormatPDF
Pages545
We start of this article with a book that is exhaustive in its treatment of system administration. This book examines many different areas involved in administering Linux systems, with each subject being accompanied by a tutorial to act as an aid in the learning process.
Topics covered include:
  • Introduction to Linux
  • Migration and coexistence with non-Linux systems
  • Basic tools for the administrator
  • The kernel
  • Local administration
  • Network administration
  • Server administration
  • Data administration
  • Security administration
  • Configuration, tuning and optimisation
  • Clustering

2. Using Samba
Using Samba
Websiteoreilly.com
AuthorRobert Eckstein, David Collier-Brown, Peter Kelly
FormatPDF, HTML
Pages416
Samba is a suite of tools for sharing resources such as printers and files across a network. Samba uses the Server Message Block (SMB) protocol, which is endorsed by Microsoft and IBM, to communicate low-level data between Windows clients and Unix servers on a TCP/IP network.
It is one of the most important software to bridge the open source and closed source worlds.
The book focuses on two different areas:
  • Installation including configuring Windows clients
  • Configuration and optimization exploring areas such as disk shares, browsing and advanced disk shares, set up users, printer and Windows Internet Naming Service setup with Samba, and troubleshooting tips

3. Slackware Linux Basics
Slackware Linux Basics
Website www.slackbasics.org
Author Daniël de Kok
Format PDF, HTML, Single page HTML
Pages 233
Slackware Linux Basics is a book that aims to provide an introduction to Slackware Linux. It targets people who have little or no GNU/Linux experience. It aims to cover the Slackware Linux installation, basic Linux commands and the configuration of Slackware Linux.
Slackware is one of the earliest Linux distributions with development commencing in 1993.
Covers:
  • Installation including partitioning and custom installation
  • Basic essential information such as the shell, files and directories, text processing, process management, editing and typesetting, and electronic mail
  • System administration covering topics such as user management, printer configuration, X11, package management, building a kernel, system initialization, and security
  • Network administration focusing on network configuration, IPsec, the Internet super server, Apache, and BIND

4. Advanced Bash Scripting Guide
Advanced Bash Scripting Guide
Website www.tldp.org
Author Mendel Cooper
Format PDF, HTML
Pages 945
Advanced Bash-Scripting Guide is an in-depth exploration of the art of scripting. Almost the complete set of commands, utilities, and tools is available for invocation by a shell script.
The book explains:
  • Basics such as special characters, quoting, exit and exit status
  • Beyond the Basics including loops and branches, command substitution, arithmetric expansion, recess time
  • Commands - Internal commands and builtins; External filters, programs and commands; System and Administrative Commands
  • Advanced topics: Regular Expressons, Here Documents, I/O Redirection, Subshells, Restricted Shells, Process Substitution, Functions, Aliases, List Constructs, Arrays, Indirect References, /dev and /proc, Of Zeros and Nulls, Debugging, Options, Gotchas, Scripting with Style
5. Test Driving Linux: From Windows to Linux in 60 Seconds
Test Driving Linux: From Windows to Linux in 60 Seconds
Websitecommons.oreilly.com
AuthorDavid Brickner
FormatHTML
Pages368
Test Driving Linux: From Windows to Linux in 60 Seconds is a detailed step-by-step guide to Linux and selected popular open source programs. It is a gentle introduction for newcomers to Linux showing them how to perform common activities such as surfing the web, using email, instant message with friends, writing letters, creating spreadsheets, and enhancing digital photos.
The book explores a wide range of topics including:
  • Surfing the web
  • File Management
  • Music and Videos
  • Playing Games
  • Email, Organizers and Instant Messaging
  • Office Suite
  • The Command Line
6. Open Sources 2.0
Open Sources 2.0
Website commons.oreilly.com
Author Chris DiBona, Danese Cooper, Mark Stone
Format HTML
Pages 496
Open Sources 2.0 is a collection of insightful and thought-provoking essays from today's technology leaders that continues painting the evolutionary picture that developed in Open Sources: Voices from the Revolution.
Explores:
  • Open Source: Competition and Evolution, covering such topics as:
    • Mozilla, Commoditization of software
    • Dual Licensing - provide a single software product under two different licenses
    • Security
    • Open Source Software in India and China
  • Beyond Open Source: Collaboration and Community
    • The Open Source Paradigm Shift
    • Patterns of Governance in Open Source
7. Linux in a Windows World
Linux in a Windows World
Websitecommons.oreilly.com
AuthorRoderick W. Smith
FormatHTML
Pages496
Linux still faces numerous challenges before it can dominate the computing world, much less the world at large. One of these challenges is the huge installed base of Windows systems. Linux must coexist with these systems. Indeed, the challenge of coexisting with Windows can be viewed as an opportunity: Linux can be integrated into a Windows network, providing a reliable and low-cost platform on which to run vital services for Windows systems, or even serving as a workstation on an otherwise Windows-dominated network. This book is dedicated to describing this opportunity for Linux.
Covers topics such as:
  • Samba - an Open Source/Free Software suite that provides seamless file and print services to SMB/CIFS clients
  • OpenLDAP - an implementation of the Lightweight Directory Access Protocol
  • BIND - Berkeley Internet Name Domain
  • Kerberos Configuration and Use - Kerberos is a network authentication protocol
  • Remote Login Tools such as VNC
  • Configuring Mail Servers
  • Network Backups
  • How to manage a network

8. Linux From Scratch
Linux From Scratch
Website www.linuxfromscratch.org
Author Gerard Beekmans, Matthew Burgess, Bruce Dubbs
Format PDF, HTML, Single page HTML, XML
Pages 290
Linux From Scratch (LFS) is a project that provides you with step-by-step instructions for building your own customized Linux system entirely from source.
Chapters cover:
  • How to create a new Linux native partition and file system
  • Which packages and patches need to be downloaded to build an LFS system and how to store them on the new file system
  • How to Set up an appropriate working environment
  • Installation of packages, and how to build a toolchain
  • Building a full LFS system
9. Beyond Linux from Scratch
Beyond Linux From Scratch
Website www.linuxfromscratch.org/blfs
Author BLFS Development Team
Format PDF, Multi-page HTML, Single-page HTML
Pages 1164
Beyond Linux From Scratch (BLFS) is a project that continues where the Linux From Scratch book finishes. It introduces and guides the reader through additions to the system including networking, graphical interfaces, sound support, and printer and scanner support.
This is a really comprehensive book providing a wealth of information on:
  • Security: OpenSSL, GnuTLS, CrackLib, Linux-PAM etc
  • File systems: ReiserFS, XFS
  • Editors: Vim, Emacs, Bluefish etc
  • Shells: ZSH, ksh, Dash, Tcsh
  • General Libraries and Utilities: PCRE, S-Lang, GLib, and many more
  • Graphics and Font Libraries: libjpeg, libpng, FreeType to name a few covered
  • General Utilities: ImageMagick, ScrollKeeper, Screen, GPerf etc
  • System Utilities: Hdparm, cpio, Apache Ant
  • Programming: GCC, JDK-6, NASM, PHP, Python, Ruby, Tcl ...
  • Networking
  • Mail/News clients
  • Servers
10. Linux 101 Hacks
Linux 101 Hacks
Website www.thegeekstuff.com
Author Ramesh Natarajan
Format PDF, HTML
Pages 140
Linux 101 Hacks is a 12 chapter, 140 page manual which explains easy to follow Linux commands. Ramesh Natarajan describes how to use these commands, with clear and concise examples.
This ebook has chapters on:
  • Powerful Command Hacks
  • Date Manipulation
  • SSH Client Commands
  • Essential Linux Commands
  • Linux Prompts
  • Colorful Shell Prompt Using PS1
  • Archive and Compression
  • Command Line History
  • System Administration Tasks
  • Apachectl and HTTPD Examples
  • Bash Scripting
  • System Monitoring and Performance
  • Bonus hacks
11. The Linux Command Line
The Linux Command Line
Website gd.tuwien.ac.at/linuxcommand.org
Author William D. Shotts, Jnr
Format PDF
Pages 522
This book is a broad overview of 'living' on the Linux command line. Unlike some books that concentrate on just a single program, such as the shell, bash, this book will try to convey how to get along with the command line interface in a larger sense. How does it all work? What can it do? What's the best way to use it?
The book is divided into the following parts:
  • Learning The Shell starts the exploration of the basic language of the command line including such things as the structure of commands, file system navigation, command line editing, and finding help and documentation for commands
  • Configuration And The Environment covers editing configuration files that control the computer's operation from the command line
  • Common Tasks And Essential Tools explores many of the ordinary tasks that are commonly performed from the command line. Unix-like operating systems, such as Linux, contain many 'classic' command line programs that are used to perform powerful operations on data
  • Writing Shell Scripts introduces shell programming, an admittedly rudimentary, but easy to learn, technique for automating many common computing tasks. By learning shell programming, you will become familiar with concepts that can be applied to many other programming languages

    12. Linux Sea
    Linux Sea
    Website swift.siphos.be
    Author Sven Vermeulen
    Format PDF, HTML
    Pages 171
    Linux Sea offers a gentle yet technical (from end-user perspective) introduction to the Linux operating system, using Gentoo Linux as the example Linux distribution.
    Topics covered include:
    • What is Linux / How does Free Software affect Linux
    • The Role of the Community / Running Linux
    • The Linux File System / Working with Processes
    • Configuring a Linux Kernel / Hardware Support
    • Software Management / User Management
    • Network Management / Service Management / Storage Management / System Management
    • Introducing the Graphical Environment
    • Installing Gentoo

    Monday, April 5, 2010

    Top 20 Nginx WebServer Best Security Practices

    Nginx is a lightweight, high performance web server/reverse proxy and e-mail (IMAP/POP3) proxy. It runs on UNIX, GNU/Linux, BSD variants, Mac OS X, Solaris, and Microsoft Windows.

    According to Netcraft, 6% of all domains on the Internet use nginx webserver. Nginx is one of a handful of servers written to address the C10K problem.

    Unlike traditional servers, Nginx doesn't rely on threads to handle requests. Instead it uses a much more scalable event-driven (asynchronous) architecture.

    Nginx powers several high traffic web sites, such as WordPress, Hulu, Github, and SourceForge. This page collects hints how to improve the security of nginx web servers running on Linux or UNIX like operating systems.


    Default Config Files and Nginx Port
    • /usr/local/nginx/conf/ - The nginx server configuration directory and /usr/local/nginx/conf/nginx.conf is main configuration file.
    • /usr/local/nginx/html/ - The default document location.
    • /usr/local/nginx/logs/ - The default log file location.
    • Nginx HTTP default port : TCP 80
    • Nginx HTTPS default port : TCP 443
    You can test nginx configuration changes as follows:


    # /usr/local/nginx/sbin/nginx -t


    Sample outputs:
    the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
    configuration file /usr/local/nginx/conf/nginx.conf test is successful

    To load config changes, type:


    # /usr/local/nginx/sbin/nginx -s reload


    To stop server, type:


    # /usr/local/nginx/sbin/nginx -s stop


    #1: Turn On SELinux
    Security-Enhanced Linux (SELinux) is a Linux kernel feature that provides a mechanism for supporting access control security policies which provides great protection.

    It can stop many attacks before your system rooted. See how to turn on SELinux for CentOS / RHEL based systems.


    Do Boolean Lockdown
    Run the getsebool -a command and lockdown system:
    # getsebool -a | less
    # getsebool -a | grep off
    # getsebool -a | grep o

    To secure the machine, look at settings which are set to 'on' and change to 'off' if they do not apply to your setup with the help of setsebool command.

    Set correct SE Linux booleans to maintain functionality and protection. Please note that SELinux adds 2-8% overheads to typical RHEL or CentOS installation.


    #2: Allow Minimal Privileges Via Mount Options
    Server all your webpages / html / php files via separate partitions. For example, create a partition called /dev/sda5 and mount at the /nginx.

    Make sure /nginx is mounted with noexec, nodev and nosetuid permissions. Here is my /etc/fstab entry for mounting /nginx:
    LABEL=/nginx     /nginx          ext3   defaults,nosuid,noexec,nodev 1 2

    Note you need to create a new partition using fdisk and mkfs.ext3 commands.


    #3: Linux /etc/sysctl.conf Hardening
    You can control and configure Linux kernel and networking settings via /etc/sysctl.conf.
     
    # Avoid a smurf attack
    net.ipv4.icmp_echo_ignore_broadcasts = 1
     
    # Turn on protection for bad icmp error messages
    net.ipv4.icmp_ignore_bogus_error_responses = 1
     
    # Turn on syncookies for SYN flood attack protection
    net.ipv4.tcp_syncookies = 1
     
    # Turn on and log spoofed, source routed, and redirect packets
    net.ipv4.conf.all.log_martians = 1
    net.ipv4.conf.default.log_martians = 1
     
    # No source routed packets here
    net.ipv4.conf.all.accept_source_route = 0
    net.ipv4.conf.default.accept_source_route = 0
     
    # Turn on reverse path filtering
    net.ipv4.conf.all.rp_filter = 1
    net.ipv4.conf.default.rp_filter = 1
     
    # Make sure no one can alter the routing tables
    net.ipv4.conf.all.accept_redirects = 0
    net.ipv4.conf.default.accept_redirects = 0
    net.ipv4.conf.all.secure_redirects = 0
    net.ipv4.conf.default.secure_redirects = 0
     
    # Don't act as a router
    net.ipv4.ip_forward = 0
    net.ipv4.conf.all.send_redirects = 0
    net.ipv4.conf.default.send_redirects = 0
     
    # Turn on execshild
    kernel.exec-shield = 1
    kernel.randomize_va_space = 1
     
    # Tuen IPv6
    net.ipv6.conf.default.router_solicitations = 0
    net.ipv6.conf.default.accept_ra_rtr_pref = 0
    net.ipv6.conf.default.accept_ra_pinfo = 0
    net.ipv6.conf.default.accept_ra_defrtr = 0
    net.ipv6.conf.default.autoconf = 0
    net.ipv6.conf.default.dad_transmits = 0
    net.ipv6.conf.default.max_addresses = 1
     
    # Optimization for port usefor LBs
    # Increase system file descriptor limit
    fs.file-max = 65535
     
    # Allow for more PIDs (to reduce rollover problems); may break some programs 32768
    kernel.pid_max = 65536
     
    # Increase system IP port limits
    net.ipv4.ip_local_port_range = 2000 65000
     
    # Increase TCP max buffer size setable using setsockopt()
    net.ipv4.tcp_rmem = 4096 87380 8388608
    net.ipv4.tcp_wmem = 4096 87380 8388608
     
    # Increase Linux auto tuning TCP buffer limits
    # min, default, and max number of bytes to use
    # set max to at least 4MB, or higher if you use very high BDP paths
    # Tcp Windows etc
    net.core.rmem_max = 8388608
    net.core.wmem_max = 8388608
    net.core.netdev_max_backlog = 5000
    net.ipv4.tcp_window_scaling = 1
     
    See also:

    #4: Remove All Unwanted Nginx Modules

    You need to minimizes the number of modules that are compiled directly into the nginx binary. This minimizes risk by limiting the capabilities allowed by the webserver.

    You can configure and install nginx using only required modules. For example, disable SSI and autoindex module you can type:


    # ./configure --without-http_autoindex_module --without-http_ssi_module
    # make
    # make install



    Type the following command to see which modules can be turn on or off while compiling nginx server:


    # ./configure --help | less


    Disable nginx modules that you don't need.


    (Optional) Change Nginx Version Header
    Edit src/http/ngx_http_header_filter_module.c, enter:


    # vi +48 src/http/ngx_http_header_filter_module.c


    Find line
     
    static char ngx_http_server_string[] = "Server: nginx" CRLF;
    static char ngx_http_server_full_string[] = "Server: " NGINX_VER CRLF;
     
    Change them as follows:
     
    static char ngx_http_server_string[] = "Server: Ninja Web Server" CRLF;
    static char ngx_http_server_full_string[] = "Server: Ninja Web Server" CRLF;
     
    Save and close the file. Now, you can compile the server. Add the following in nginx.conf to turn off nginx version number displayed on all auto generated error pages:

    server_tokens off
    
    
    #5: Use mod_security (only for backend Apache servers)
    mod_security provides an application level firewall for Apache. Install mod_security for all backend Apache web servers. This will stop many injection attacks.


    #6: Install SELinux Policy To Harden The Nginx Webserver
    By default SELinux will not protect the nginx web server. However, you can install and compile protection as follows.

    First, install required SELinux compile time support:


    # yum -y install selinux-policy-targeted selinux-policy-devel


    Download targeted SELinux policies to harden the nginx webserver on Linux servers from the project home page:


    # cd /opt
    # wget 'http://downloads.sourceforge.net/project/selinuxnginx/se-ngix_1_0_10.tar.gz?use_mirror=nchc'



    Untar the same:
    # tar -zxvf se-ngix_1_0_10.tar.gz


    Compile the same
    # cd se-ngix_1_0_10/nginx
    # make



    Sample outputs:
    Compiling targeted nginx module
    /usr/bin/checkmodule:  loading policy configuration from tmp/nginx.tmp
    /usr/bin/checkmodule:  policy configuration loaded
    /usr/bin/checkmodule:  writing binary representation (version 6) to tmp/nginx.mod
    
    Creating targeted nginx.pp policy package
    
    # rm tmp/nginx.mod.fc tmp/nginx.mod

    Install the resulting nginx.pp SELinux module:
    # /usr/sbin/semodule -i nginx.pp


    #7: Restrictive Iptables Based Firewall
    The following firewall script blocks everything and only allows:
    • Incoming HTTP (TCP port 80) requests
    • Incoming ICMP ping requests
    • Outgoing ntp (port 123) requests
    • Outgoing smtp (TCP port 25) requests
    #!/bin/bash
    IPT="/sbin/iptables"
     
    #### IPS ######
    # Get server public ip
    SERVER_IP=$(ifconfig eth0 | grep 'inet addr:' | awk -F'inet addr:' '{ print $2}' | awk '{ print $1}')
    LB1_IP="204.54.1.1"
    LB2_IP="204.54.1.2"
     
    # Do some smart logic so that we can use damm script on LB2 too
    OTHER_LB=""
    SERVER_IP=""
    [[ "$SERVER_IP" == "$LB1_IP" ]] && OTHER_LB="$LB2_IP" || OTHER_LB="$LB1_IP"
    [[ "$OTHER_LB" == "$LB2_IP" ]] && OPP_LB="$LB1_IP" || OPP_LB="$LB2_IP"
     
    ### IPs ###
    PUB_SSH_ONLY="122.xx.yy.zz/29"
     
    #### FILES #####
    BLOCKED_IP_TDB=/root/.fw/blocked.ip.txt
    SPOOFIP="127.0.0.0/8 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8 169.254.0.0/16 0.0.0.0/8 240.0.0.0/4 255.255.255.255/32 168.254.0.0/16 224.0.0.0/4 240.0.0.0/5 248.0.0.0/5 192.0.2.0/24"
    BADIPS=$( [[ -f ${BLOCKED_IP_TDB} ]] && egrep -v "^#|^$" ${BLOCKED_IP_TDB})
     
    ### Interfaces ###
    PUB_IF="eth0"   # public interface
    LO_IF="lo"      # loopback
    VPN_IF="eth1"   # vpn / private net
     
    ### start firewall ###
    echo "Setting LB1 $(hostname) Firewall..."
     
    # DROP and close everything
    $IPT -P INPUT DROP
    $IPT -P OUTPUT DROP
    $IPT -P FORWARD DROP
     
    # Unlimited lo access
    $IPT -A INPUT -i ${LO_IF} -j ACCEPT
    $IPT -A OUTPUT -o ${LO_IF} -j ACCEPT
     
    # Unlimited vpn / pnet access
    $IPT -A INPUT -i ${VPN_IF} -j ACCEPT
    $IPT -A OUTPUT -o ${VPN_IF} -j ACCEPT
     
    # Drop sync
    $IPT -A INPUT -i ${PUB_IF} -p tcp ! --syn -m state --state NEW -j DROP
     
    # Drop Fragments
    $IPT -A INPUT -i ${PUB_IF} -f -j DROP
     
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags ALL FIN,URG,PSH -j DROP
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags ALL ALL -j DROP
     
    # Drop NULL packets
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags ALL NONE -m limit --limit 5/m --limit-burst 7 -j LOG --log-prefix " NULL Packets "
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags ALL NONE -j DROP
     
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
     
    # Drop XMAS
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags SYN,FIN SYN,FIN -m limit --limit 5/m --limit-burst 7 -j LOG --log-prefix " XMAS Packets "
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
     
    # Drop FIN packet scans
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags FIN,ACK FIN -m limit --limit 5/m --limit-burst 7 -j LOG --log-prefix " Fin Packets Scan "
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags FIN,ACK FIN -j DROP
     
    $IPT  -A INPUT -i ${PUB_IF} -p tcp --tcp-flags ALL SYN,RST,ACK,FIN,URG -j DROP
     
    # Log and get rid of broadcast / multicast and invalid
    $IPT  -A INPUT -i ${PUB_IF} -m pkttype --pkt-type broadcast -j LOG --log-prefix " Broadcast "
    $IPT  -A INPUT -i ${PUB_IF} -m pkttype --pkt-type broadcast -j DROP
     
    $IPT  -A INPUT -i ${PUB_IF} -m pkttype --pkt-type multicast -j LOG --log-prefix " Multicast "
    $IPT  -A INPUT -i ${PUB_IF} -m pkttype --pkt-type multicast -j DROP
     
    $IPT  -A INPUT -i ${PUB_IF} -m state --state INVALID -j LOG --log-prefix " Invalid "
    $IPT  -A INPUT -i ${PUB_IF} -m state --state INVALID -j DROP
     
    # Log and block spoofed ips
    $IPT -N spooflist
    for ipblock in $SPOOFIP
    do
             $IPT -A spooflist -i ${PUB_IF} -s $ipblock -j LOG --log-prefix " SPOOF List Block "
             $IPT -A spooflist -i ${PUB_IF} -s $ipblock -j DROP
    done
    $IPT -I INPUT -j spooflist
    $IPT -I OUTPUT -j spooflist
    $IPT -I FORWARD -j spooflist
     
    # Allow ssh only from selected public ips
    for ip in ${PUB_SSH_ONLY}
    do
            $IPT -A INPUT -i ${PUB_IF} -s ${ip} -p tcp -d ${SERVER_IP} --destination-port 22 -j ACCEPT
            $IPT -A OUTPUT -o ${PUB_IF} -d ${ip} -p tcp -s ${SERVER_IP} --sport 22 -j ACCEPT
    done
     
    # allow incoming ICMP ping pong stuff
    $IPT -A INPUT -i ${PUB_IF} -p icmp --icmp-type 8 -s 0/0 -m state --state NEW,ESTABLISHED,RELATED -m limit --limit 30/sec  -j ACCEPT
    $IPT -A OUTPUT -o ${PUB_IF} -p icmp --icmp-type 0 -d 0/0 -m state --state ESTABLISHED,RELATED -j ACCEPT
     
    # allow incoming HTTP port 80
    $IPT -A INPUT -i ${PUB_IF} -p tcp -s 0/0 --sport 1024:65535 --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPT -A OUTPUT -o ${PUB_IF} -p tcp --sport 80 -d 0/0 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT
     
    # allow outgoing ntp
    $IPT -A OUTPUT -o ${PUB_IF} -p udp --dport 123 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPT -A INPUT -i ${PUB_IF} -p udp --sport 123 -m state --state ESTABLISHED -j ACCEPT
     
    # allow outgoing smtp
    $IPT -A OUTPUT -o ${PUB_IF} -p tcp --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
    $IPT -A INPUT -i ${PUB_IF} -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
     
    ### add your other rules here ####
     
    #######################
    # drop and log everything else
    $IPT -A INPUT -m limit --limit 5/m --limit-burst 7 -j LOG --log-prefix " DEFAULT DROP "
    $IPT -A INPUT -j DROP
     
    exit 0
    
    
    #8: Controlling Buffer Overflow Attacks
    Edit nginx.conf and set the buffer size limitations for all clients.


    # vi /usr/local/nginx/conf/nginx.conf


    Edit and set the buffer size limitations for all clients as follows:
     
     ## Start: Size Limits & Buffer Overflows ##
      client_body_buffer_size  1K;
      client_header_buffer_size 1k;
      client_max_body_size 1k;
      large_client_header_buffers 2 1k;
     ## END: Size Limits & Buffer Overflows ##
     
    Where,
    1. client_body_buffer_size 1k - (default is 8k or 16k) The directive specifies the client request body buffer size.
    2. client_header_buffer_size 1k - Directive sets the headerbuffer size for the request header from client. For the overwhelming majority of requests a buffer size of 1K is sufficient. Increase this if you have a custom header or a large cookie sent from the client (e.g., wap client).
    3. client_max_body_size 1k- Directive assigns the maximum accepted body size of client request, indicated by the line Content-Length in the header of request. If size is greater the given one, then the client gets the error "Request Entity Too Large" (413). Increase this when you are getting file uploads via the POST method.
    4. large_client_header_buffers 2 1k - Directive assigns the maximum number and size of buffers for large headers to read from client request. By default the size of one buffer is equal to the size of page, depending on platform this either 4K or 8K, if at the end of working request connection converts to state keep-alive, then these buffers are freed. 2x1k will accept 2kB data URI. This will also help combat bad bots and DoS attacks.
    You also need to control timeouts to improve server performance and cut clients. Edit it as follows:
     
    ## Start: Timeouts ##
      client_body_timeout   10;
      client_header_timeout 10;
      keepalive_timeout     5 5;
      send_timeout          10;
    ## End: Timeouts ##
     
    1. client_body_timeout 10; - Directive sets the read timeout for the request body from client. The timeout is set only if a body is not get in one readstep. If after this time the client send nothing, nginx returns error "Request time out" (408). The default is 60.
    2. client_header_timeout 10; - Directive assigns timeout with reading of the title of the request of client. The timeout is set only if a header is not get in one readstep. If after this time the client send nothing, nginx returns error "Request time out" (408).
    3. keepalive_timeout 5 5; - The first parameter assigns the timeout for keep-alive connections with the client. The server will close connections after this time. The optional second parameter assigns the time value in the header Keep-Alive: timeout=time of the response. This header can convince some browsers to close the connection, so that the server does not have to. Without this parameter, nginx does not send a Keep-Alive header (though this is not what makes a connection "keep-alive").
    4. send_timeout 10; - Directive assigns response timeout to client. Timeout is established not on entire transfer of answer, but only between two operations of reading, if after this time client will take nothing, then nginx is shutting down the connection.

    #9: Control Simultaneous Connections

    You can use NginxHttpLimitZone module to limit the number of simultaneous connections for the assigned session or as a special case, from one IP address. Edit nginx.conf:
     
    ### Directive describes the zone, in which the session states are stored i.e. store in slimits. ###
    ### 1m can handle 32000 sessions with 32 bytes/session, set to 5m x 32000 session ###
           limit_zone slimits $binary_remote_addr 5m;
     
    ### Control maximum number of simultaneous connections for one session i.e. ###
    ### restricts the amount of connections from a single ip address ###
            limit_conn slimits 5;
     
    The above will limits remote clients to no more than 5 concurrently "open" connections per remote ip address.


    #10: Allow Access To Our Domain Only
    If bot is just making random server scan for all domains, just deny it. You must only allow configured virtual domain or reverse proxy requests.

    You don't want to display request using an IP address:
    ## Only requests to our Host are allowed i.e. nixcraft.in, images.nixcraft.in and www.nixcraft.in
          if ($host !~ ^(nixcraft.in|www.nixcraft.in|images.nixcraft.in)$ ) {
             return 444;
          }
    ##
    
    
    #11: Limit Available Methods
    GET and POST are the most common methods on the Internet. Web server methods are defined in RFC 2616.

    If a web server does not require the implementation of all available methods, they should be disabled.

    The following will filter and only allow GET, HEAD and POST methods:
    ## Only allow these request methods ##
         if ($request_method !~ ^(GET|HEAD|POST)$ ) {
             return 444;
         }
    ## Do not accept DELETE, SEARCH and other methods ##
    
    
    More About HTTP Methods
    • The GET method is used to request document such as http://www.cyberciti.biz/index.php.
    • The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
    • The POST method may involve anything, like storing or updating data, or ordering a product, or sending E-mail by submitting the form. This is usually processed using the server side scripting such as PHP, PERL, Python and so on. You must use this if you want to upload files and process forms on server.

    #12: How Do I Deny Certain User-Agents?

    You can easily block user-agents i.e. scanners, bots, and spammers who may be abusing your server.

    ## Block download agents ##

    if ($http_user_agent ~* LWP::Simple|BBBike|wget) {

    return 403;
         }
    ##
    

    Block robots called msnbot and scrapbot:
    ## Block some robots ##
         if ($http_user_agent ~* msnbot|scrapbot) {
                return 403;
         }
    
    
    #12: How Do I Block Referral Spam?
    Referer spam is dengerouns. It can harm your SEO ranking via web-logs (if published) as referer field refer to their spammy site.

    You can block access to referer spammers with these lines.
    ## Deny certain Referers ###
         if ( $http_referer ~* (babes|forsale|girl|jewelry|love|nudit|organic|poker|porn|sex|teen) )
         {
             # return 404;
             return 403;
         }
    ##
    
    
    #13: How Do I Stop Image Hotlinking?
    Image or HTML hotlinking means someone makes a link to your site to one of your images, but displays it on their own site.

    The end result you will end up paying for bandwidth bills and make the content look like part of the hijacker's site.

    This is usually done on forums and blogs. I strongly suggest you block and stop image hotlinking at your server level itself.
    # Stop deep linking or hot linking
    location /images/ {
      valid_referers none blocked www.example.com example.com;
       if ($invalid_referer) {
         return   403;
       }
    }
    
    
    Example: Rewrite And Display Image
    Another example with link to banned image:
    valid_referers blocked www.example.com example.com;
     if ($invalid_referer) {
      rewrite ^/images/uploads.*\.(gif|jpg|jpeg|png)$ http://www.examples.com/banned.jpg last
     }
    

    See also:
    • HowTo: Use nginx map to block image hotlinking. This is useful if you want to block tons of domains.

    #14: Directory Restrictions

    You can set access control for a specified directory. All web directories should be configured on a case-by-case basis, allowing access only where needed.

    Limiting Access By Ip Address

    You can limit access to directory by ip address to /docs/ directory:
    location /docs/ {
      ## block one workstation
      deny    192.168.1.1;
    
      ## allow anyone in 192.168.1.0/24
      allow   192.168.1.0/24;
    
      ## drop rest of the world
      deny    all;
    }
    
    
    Password Protect The Directory
    First create the password file and add a user called vivek:


    # mkdir /usr/local/nginx/conf/.htpasswd/
    # htpasswd -c /usr/local/nginx/conf/.htpasswd/passwd vivek



    Edit nginx.conf and protect the required directories as follows:
    ### Password Protect /personal-images/ and /delta/ directories ###
    location ~ /(personal-images/.*|delta/.*) {
      auth_basic  "Restricted";
      auth_basic_user_file   /usr/local/nginx/conf/.htpasswd/passwd;
    }
    

    Once a password file has been generated, subsequent users can be added with the following command:


    # htpasswd -s /usr/local/nginx/conf/.htpasswd/passwd userName


    #15: Nginx SSL Configuration
    HTTP is a plain text protocol and it is open to passive monitoring. You should use SSL to to encrypt your content for users.


    Create an SSL Certificate
    Type the following commands:


    # cd /usr/local/nginx/conf
    # openssl genrsa -des3 -out server.key 1024
    # openssl req -new -key server.key -out server.csr
    # cp server.key server.key.org
    # openssl rsa -in server.key.org -out server.key
    # openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt



    Edit nginx.conf and update it as follows:
    server {
        server_name example.com;
        listen 443;
        ssl on;
        ssl_certificate /usr/local/nginx/conf/server.crt;
        ssl_certificate_key /usr/local/nginx/conf/server.key;
        access_log /usr/local/nginx/logs/ssl.access.log;
        error_log /usr/local/nginx/logs/ssl.error.log;
    }

    Restart the nginx:


    # /usr/local/nginx/sbin/nginx -s reload


    See also:

    #16: Nginx And PHP Security Tips

    PHP is one of the popular server side scripting language. Edit /etc/php.ini as follows:
     
    # Disallow dangerous functions
    disable_functions = phpinfo, system, mail, exec
     
    ## Try to limit resources  ##
     
    # Maximum execution time of each script, in seconds
    max_execution_time = 30
     
    # Maximum amount of time each script may spend parsing request data
    max_input_time = 60
     
    # Maximum amount of memory a script may consume (8MB)
    memory_limit = 8M
     
    # Maximum size of POST data that PHP will accept.
    post_max_size = 8M
     
    # Whether to allow HTTP file uploads.
    file_uploads = Off
     
    # Maximum allowed size for uploaded files.
    upload_max_filesize = 2M
     
    # Do not expose PHP error messages to external users
    display_errors = Off
     
    # Turn on safe mode
    safe_mode = On
     
    # Only allow access to executables in isolated directory
    safe_mode_exec_dir = php-required-executables-path
     
    # Limit external access to PHP environment
    safe_mode_allowed_env_vars = PHP_
     
    # Restrict PHP information leakage
    expose_php = Off
     
    # Log all errors
    log_errors = On
     
    # Do not register globals for input data
    register_globals = Off
     
    # Minimize allowable PHP post size
    post_max_size = 1K
     
    # Ensure PHP redirects appropriately
    cgi.force_redirect = 0
     
    # Disallow uploading unless necessary
    file_uploads = Off
     
    # Enable SQL safe mode
    sql.safe_mode = On
     
    # Avoid Opening remote files
    allow_url_fopen = Off
     
    See also:

    #17: Run Nginx In A Chroot Jail (Containers) If Possible

    Putting nginx in a chroot jail minimizes the damage done by a potential break-in by isolating the web server to a small section of the filesystem.

    You can't use traditional chroot kind of setup with nginx. However, you can use FreeBSD jails, XEN, or OpenVZ virtualization which uses the concept of containers.


    #18: Limits Connections Per IP At The Firewall Level
    A webserver must keep an eye on connections and limit connections per second. This is serving 101. Both pf and iptables can throttle end users before accessing your nginx server.


    Linux Iptables: Throttle Nginx Connections Per Second
    The following example will drop incoming connections if IP make more than 15 connection attempts to port 80 within 60 seconds:
     
    /sbin/iptables -A INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
    /sbin/iptables -A INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 60  --hitcount 15 -j DROP
    service iptables save
    
    
    BSD PF: Throttle Nginx Connections Per Second
    Edit your /etc/pf.conf and update it as follows. The following will limits the maximum number of connections per source to 100.

    15/5 specifies the number of connections per second or span of seconds i.e. rate limit the number of connections to 15 in a 5 second span.

    If anyone breaks our rules add them to our abusive_ips table and block them for making any further connections.

    Finally, flush keyword kills all states created by the matching rule which originate from the host which exceeds these limits.
    
    
    webserver_ip="202.54.1.1"
    table  persist
    block in quick from 
    pass in on $ext_if proto tcp to $webserver_ip port www flags S/SA keep state (max-src-conn 100, max-src-conn-rate 15/5, overload  flush)

    Please adjust all values as per your requirements and traffic (browsers may open multiple connections to your site). See also:
    1. Sample PF firewall script.
    2. Sample Iptables firewall script.

    #19: Configure Operating System to Protect Web Server

    Turn on SELinux as described above. Set correct permissions on /nginx document root. The nginx runs as a user named nginx.

    However, the files in the DocumentRoot (/nginx or /usr/local/nginx/html) should not be owned or writable by that user. To find files with wrong permissions, use:


    # find /nginx -user nginx
    # find /usr/local/nginx/html -user nginx



    Make sure you change file ownership to root or other user. A typical set of permission /usr/local/nginx/html/


    # ls -l /usr/local/nginx/html/


    Sample outputs:
    -rw-r--r-- 1 root root 925 Jan  3 00:50 error4xx.html
    -rw-r--r-- 1 root root  52 Jan  3 10:00 error5xx.html
    -rw-r--r-- 1 root root 134 Jan  3 00:52 index.html
    

    You must delete unwated backup files created by vi or other text editor:


    # find /nginx -name '.?*' -not -name .ht* -or -name '*~' -or -name '*.bak*' -or -name '*.old*'
    # find /usr/local/nginx/html/ -name '.?*' -not -name .ht* -or -name '*~' -or -name '*.bak*' -or -name '*.old*'

    Pass -delete option to find command and it will get rid of those files too.


    #20: Restrict Outgoing Nginx Connections
    The crackers will download file locally on your server using tools such as wget. Use iptables to block outgoing connections from nginx user.

    The ipt_owner module attempts to match various characteristics of the packet creator, for locally generated packets.

    It is only valid in the OUTPUT chain. In this example, allow vivek user to connect outside using port 80 (useful for RHN access or to grab CentOS updates via repos):

    /sbin/iptables -A OUTPUT -o eth0 -m owner --uid-owner vivek -p tcp --dport 80 -m state --state NEW,ESTABLISHED  -j ACCEPT

    Add above rule to your iptables based shell script. Do not allow nginx web server user to connect outside.

    Bounce Tip: Watching Your Logs & Auditing
    Check the Log files. They will give you some understanding of what attacks is thrown against the server and allow you to check if the necessary level of security is present or not.


    # grep "/login.php??" /usr/local/nginx/logs/access_log
    # grep "...etc/passwd" /usr/local/nginx/logs/access_log
    # egrep -i "denied|error|warn" /usr/local/nginx/logs/error_log



    The auditd service is provided for system auditing. Turn it on to audit service SELinux events, authetication events, file modifications, account modification and so on.

    As usual disable all services and follow our "Linux Server Hardening" security tips.


    Conclusion
    Your nginx server is now properly harden and ready to server webpages. However, you should be consulted further resources for your web applications security needs.

    For example, wordpress or any other third party apps has its own security requirements.

    References: