Monitoring and Dealing With Snort Alerts

Snort, the open source intrusion detection and prevention system, is immensely powerful, but to get the most out of it, you need to configure it correctly for your own setup. My previous Snort article looked at Snort’s own performance monitoring provision and how to use it to tune Snort to get the best throughput at the detection end. Here are some performance tips for dealing well with alerts, looking at alert monitoring, streamlining false positives and genuine but frequent real positives, and logical rule optimization.
The last article looked at monitoring rule performance – that is, how well the Snort detection engine deals with the traffic it sees. The next stage concerns what to do with traffic that Snort flags as suspicious: alerts of various sorts. I won’t go into detail about how to deal with specific alerts, but talk instead about some good Snort management practices.
Snort itself doesn’t actually do anything with its alerts, so it’s important to make sure you have something in place to check for them. You do have a choice of output modules, but (with one exception, alert_unixsock, which I’ll talk more about later) these only output to a file or a database. To monitor your chosen output format, you have several third-party options, with Snort Alert Monitor being one of the more popular ones, as it allows you to configure email alerts among other output types. If you have additional specific requirements, SAM exposes its API and is (at least according to its author!) simple to extend.
You can also find self-rolled script options online. In general, however, it’s best for productivity, usability, and maintainability to avoid rolling your own solution wherever possible. Using a base system that’s in wide use elsewhere increases the likelihood that any inherent bugs will be found and patched by others, so you’re not on your own to fix up your alert system.
Even better, productivity- and efficiency-wise, is to use a Snort plugin for your existing network monitoring system. Munin plugins and a Nagios plugin exist, and it’s possible to set up Snort to output to Zenoss.

Dealing with Alerts

Once you’ve got your alert monitoring set up, you may find that Snort is generating a lot of false positives alongside your genuine positive alerts. Getting rid of these is an important part of fine-tuning Snort. A sea of false positives makes it easy to miss the real problems.
If a rule is generating a lot of false positives, your first option should be to see if you can tune that rule to fit your situation better. For example, you might want to rewrite it to ignore traffic from a particular set of hosts; or there might be a feature common to all the false alerts (perhaps something that gives away to you that it is a false alert) which you can write into the rule. Be careful when rewriting rules not to accidentally cause yourself other problems, or to risk missing genuine alerts. If you find yourself rewriting a lot of rules, a tool like Oinkmaster can help you to manage them.
If you want to suppress a rule only for particular hosts, but still want it in place for others, you can write a suppress statement for the rule by using track by_src or track by_dst in the rule setup. Note that Snort checks for suppress statements after it matches the rule, so you’re still using CPU cycles. If you’re never interested in a particular rule, it’s better to disable it rather than use global suppression.
You can also use a pass rule; these look like alert rules, except that if a packet matches a pass rule, Snort will stop looking at it any further. You could write a pass rule to get rid of false positives, using it only for particular hosts:

pass tcp myserver any - any any

This will ignore any traffic from myserver. However, be warned that you can accidentally affect other rules with similar signatures! For example, this alert rule (a very simplified version of one in the default FTP ruleset; do not use this version of it!) looks for a particular content and regular expression:

alert tcp $EXTERNAL_NET any -> $HOME_NET 21 (msg:"FTP PASS format
attempt"; content:"PASS"; pcre:"^PASS\s[^\n]/smi";)

This pass rule would ignore traffic with the content PASS, thus turning off these sort of alerts:

alert tcp $EXTERNAL_NET any -> $HOME_NET 21 (msg:"FTP PASS format attempt"; content:"PASS";)

However, it would also turn off this totally separate rule:

alert tcp $EXTERNAL_NET any -> $HOME_NET 21 (msg:"FTP adm scan";
content:"PASS ddd@|0A|";)

Now, this example is pretty obvious; for starters, we totally skipped the regular expression in the initial alert rule. But the same class of problem can be much subtler and harder to stop, and thus harder to maintain. You might also carefully check that all your existing rules are unaffected, and then find that a newly introduced one, next time you update your ruleset, has been clobbered. Bottom line: Be very cautious when writing pass rules.
You can use BPF filters (also known as tcpdump filters) to filter traffic from a particular host; check out the snort manpage for more information on using them, or check out the syntax here. Note that if you only block traffic in one direction, you may confuse the stateful parts of Snort as traffic in the other direction is still seen. Depending on your setup, Snort might start blocking this traffic, as it would no longer appear to be part of an existing interaction.
Of course, you can disable a particular rule altogether: find it in the rules file, then either delete it or (better) comment it out by preceding it with #, the Snort comment character. Only do this if you really never care about a particular alert, regardless of which host it comes from.
Note that when you update Snort, your disabled alerts may be re-enabled. A tool like Oinkmaster can help you retain changes when you update the ruleset.

Reacting to Alerts

There are also alerts that you don’t want to disable, but which show up often enough that you start to grow accustomed to them. This is dangerous, as you may end up mentally skipping over new alerts too. Instead, look for other automatic ways of dealing with these alerts, and then adjust either the rulesets or your alert monitoring so you don’t automatically see them.
If you want to set up an automatic reaction to a particular alert, you can use the unixsock output plugin to output to a Unix socket, then use an external script to look for your alert and act accordingly. There are no options to this Snort output setting:

output alert_unixsock

It sends all alerts to the snort_alert socket in the Snort log directory. You can then write a script to unpack the datagram written by Snort and do something with it – for example, add a line to your /etc/hosts.deny file, or a firewall rule. Note that the script will also need to create the socket; Snort won’t do this if it doesn’t already exist. If you do set up unixsock, consider whether you want to also run a script to clean up after yourself; do you really want to ban these hosts forever, or just for a short while? (Most script kiddie attackers will give up and move on after only a few minutes.)
unixsock can be useful if you have very site-specific action requirements for an alert or want to do something unusual, but if there’s an existing solution that fits your alert reaction requirements, it’s always better to use that rather than rolling your own. For instance, the SnortSAM plugin can interface with your firewall to modify its ruleset. SnortSAM supports a variety of firewalls, including iptables and various commercial products. It adds an output plugin to Snort, and installs an intelligent agent to interact with the firewall. Its wide range of capabilities includes whitelist support, a misuse/attack detection engine that aims to minimize the risk of self-inflicted DOS, encryption, logging, and email alerts. In particular, the anti-DOS code is useful, since reactive firewalls make it easier to launch a DOS by spoofing source addresses of particular packets that Snort then blocks, blocking multiple addresses.
Another way to address the requirement for a reactive firewall is to run Snort inline, enabling it to actively intercept and drop traffic. As of Snort 2.9.0, this is done via the DAQ (Data Acquisition Library), and just requires you to run Snort with the -Q switch. You can also set up the DAQ with the config daq_* options.
The AFPACKET DAQ is the easiest way to set up Snort inline. Using Snort this way means that instead of “alert” rules, you use “drop” rules, dropping any packets that match. Snort Inline also interacts with iptables, so you need to make sure that both Snort and iptables are correctly configured.

Making Alerts Easier for Humans to Read

Performance isn’t only about the technical side of things; it’s also about the human who will eventually eyeball at least some of the data. Poorly written rules can make this job much harder. The person who writes the rules isn’t always the person who deals with their output, and what seems obvious when writing a rule isn’t necessarily obvious when reading the resulting alert. Even if the same person is at both ends, they may not remember the details of an alert six months down the line when it finally fires.
To help when reading alerts, Snort has classtype, reference, and priority values that you can add to rules. priority is particularly useful for filtering:

priority: x;

You can then filter on this value in your monitoring program to decide what to do with a particular alert – is it worth bouncing it to someone’s phone or pager, or can it wait with all the routine stuff.
classtype enables you to add an alert classification (such as trojan-activity or policy-violation), and with reference (which can have multiple instances in a single rule) you can add a CVE reference, a URL, or anything else that can help identify the alert further. See the Snort white paper Improving your custom Snort rules for more on this.
If you fine-tune your rules using the advice above, your Snort installation should operate better throughout its cycle, and you should find it more useful than before. Check out the Snort web page for more suggestions on improving and rewriting rules to increase performance, and for other options you may want to investigate.