The IDS Incident Handling 101

We came up with a decent short description of Intrusion Detection System (IDS) incident handling during a recent lunch discussion.

Handling IDS incidents is not very different to making a puzzle. You first gather and identify the individual pieces and then you place them together correctly. While traditional puzzles, once put together, present you with a complete picture, the IDS incidents present you with a possible attack against one of your IP addresses.

The alerts in the current generation of IDS systems come in five pieces. There are always...

the source IP address
the source port
the destination IP address
the destination port

...and the IDS signature, which was triggered by an pattern detected in the network traffic by the IDS sensors.

As with traditional puzzles, you have to examine each given piece thoroughly before attempting to put them together. Some preliminary questions regarding the involved systems are the installed operating systems and the applications. Large part of the IDS signatures detect exploits to specific vulnerabilities in software, so the patching level of the systems and of the destination especially is one of the initial things to check also.

Ideally the incident handler is able to securely connect to the monitored environment and verify all of the above from the systems themselves, but in case this is not possible, ensure the incident response team has a system database of some type available to them. ALL monitored IP addresses really should be listed. I know it may be a taunting task in large environments to identify the various switch port addresses and the virtual addresses used for load balancing and clustering et cetera, but I guarantee you that simply assessing and documenting properly your computing resources is a big step in hardening your environment against misuse.

Among many other things, a proper system database enables your incident response team to react fast and properly to any possibly true positive attacks. The database should include at least the following properties for each monitored IP address:

-DESCRIPTION of the system usage
(as in web server, internet gateway router, LAN server, workstation etc.)
-OPERATING SYSTEM and the patching level
(including the network device firmware etc.)
-APPLICATIONS installed on the systems
(the exact version numbers)
-CONTACT DETAILS of the system administrators
(for running commands on the systems etc.)

Do not overlook the system information carried by the IPv4 addresses. One of the steps to full system identification is identifying the IP address ranges involved. In case the monitored environment includes publicly available servers, pay attention to the source addresses. Any addresses in the private IPv4 address ranges are most likely internal systems and as such possibly indications of more serious compromise.

The next piece to look at are the TCP/UDP ports. Do multiple searches to identify all possible services known to use the involved ports on the given platform. The IANA Port Numbers list is the official resource for figuring out the legitimate use of the Well Known Ports and the Registered Ports (namely the ports between the 0 and the 49151).

The SANS Internet Storm Center (ISC) provides a quality port search on the upper right corner of their diary page often listing the malware known to use the given port also. The DefaultPorts.com is another good database to check and don't forget the frequently updated list of known and detected port usage hosted at nmap.org.

The Trojan List hosted at simovits.com can be sorted by the port numbers also. I am not sure if the list is still maintained actively, but it is worthy information source nevertheless.

Here again access to the monitored environment comes very handy as the incident handler can quickly copy the running processes and the open network connections to the case log for later analysis. You will notice in time, that the mentioned two listings from the destination will be constantly needed.

Take the network design into consideration also when doing preliminary analysis on the ports. It is often an easy way to identify false positives and further implement filters to the sensors. Ensure you know the firewalls and routers examining the monitored traffic AFTER the sensor examines it. Depending on the implementation, the IDS sensor may see the possible attacks before the traffic gets filtered by a firewall or the routers. There may be even high volumes of attacks against the environment, but if the destination port of the attacks is closed by the firewall, no permanent harm can be caused.

After some innovative system identification and port analysis exercises, the incident handler is left with the last piece only. The IDS signature. This is the piece that tells you what actually was detected. Note that the signature does not tell you what the attack was as often used description goes. It tells you what was detected in the traffic between the two end points.

Study the signature that got triggered. Have the relevant IDS signature description library available to you at all times when monitoring. Its THE starting point. I believe all IDS vendors now provide online access to such libraries. Study the description, check all related advisories and references and do some background study if necessary to fully appreciate what the signature detects. This piece really completes the puzzle often with the IDS alerts. Here the next steps most often are revealed. They may be as simple as verifying that a patch is installed or that the software has been updated on the destination, but whatever the next steps may be, they are to be found by analysing the signature as far as possible.

In my opinion the incident handler should have access to the attack packets. At least in order to copy them to the case log for later analysis. The Cisco IDS for example gives you often also the exact character pattern detected by the signature in the same window with the attack packet data.

The attack packet data is priceless when investigating alerts against non-patched vulnerabilities such as the ones in custom web applications, where there is not actually even any specific description of the vulnerability available a part from the generic XSS or SQL Injection papers.

In case you use the much loved Snort IDS, make sure you can read the Snort rule language effortlessly as Snort uses an open signature format which allows you a detailed view into the signature construction. It uses a specific syntax, but the variables are not too many.

Emerging Threats has a good wiki entry titled Snort Signatures 101 to get you over the fundamentals

In my opinion the IDS incident handling process is completed with identifying and disregarding the obvious false positive alerts (and later maybe permanently filtering the reoccurring false positive alerts from the IDS monitoring). After this point the incident handling is over and the incident response begins although they often should overlap each other and occur simultaneously.

I have been the dude introducing various junior agents to the IDS incident handling. Kind of like giving them the first idea of the science here and showing them their first real alerts. I find it very challenging to give them any type of short description or introductory note to start with. The puzzle comparison is the current favourite here by far.

KHOBE Cause Temblor

Matousec did cause some temblor in the infosec community with the KHOBE attack paper published last week. See my earlier post Windows TOCTTOU Attacks with KHOBE for the initial material.

KHOBE has gotten a lot of publicity and has generated active response and commentary from the anti-virus industry already. Temblor reached the internet s0c1ety also due to some "juicy" details published in a GData blog post about their recent communication with Matousec in order to get more information and evaluate the effect on their software correctly.

It is the art of vulnerability disclosure, no? The difficulty to handle all the aspects of disclosing delicate information. Considering that Matousec apparently did disclose privately their full research to their "clients and other software vendors" already in August 2008 and (especially) considering the fact that the KHOBE code is a result of some years of research and development, I personally find it only correct that Matousec now sell the full details and offer audit services for paying customers only. On the other hand, helping out an affected vendor a bit beyond the public paper is not too much to ask either IMO, so lets leave it to 1-1.

The technical talk about the attack has been somewhat limited due to all this. Anti-virus vendors want to see code before assessing the threat further and have concentrated responding only to the facts detailed in the Matousec paper for now. See Paul Ducklin´s blog post from Sophos for a thorough write-up on the issue, which in my opinion does good job in summarizing the initial vendor stance on KHOBE across the field as well.

While I completely agree with the point about layered defense providing security beyond the system call and parameter checks (and find the point about unknown malware bypassing the protection with or without KHOBE to be logical), I think the discussion is far from being over yet. Let's assume we are running a multicore/multiprocessor system for the rest of the post. System where the threads are not competing for the clock cycles of only one processor, but have multiple parallel clock cycles to choose from and are able to actually run parallel in time.

I suspect various security software in this type of Windows system to be highly vulnerable to the attack. I am limited to the information available in the KHOBE paper about Matousec´s findings, but studying the earlier papers published about the TOCTTOU attacks on Windows leaves me feeling like possibly every validation check done on Windows platform is vulnerable.

The problem is not really the SSDT hooking dominating the public discussion at the moment. As far as I can see the root cause of the vulnerability lies somewhere in the way the data is being validated on Windows. In the way it is being referenced in the validation process and especially in the alarming detail that the memory areas being validated can actually be manipulated while they are being validated in some cases.

Seems like the KHOBE code focuses on exploiting the vulnerability in software which uses SSDT hooks in order to intercept the system calls and validate the parameters, but I doubt the exploitation is limited to checks initiated by SSDT hooks. The problem really is the accessibility of the memory areas under validation, not the way the checks are initiated. Any type of validation check requires multiple clock cycles, which possibly allow a parallel thread  running on parallel clock cycles plenty of time to manipulate the values in the memory while they are under examination and possibly cause invalidated malicious parameters actually to be passed to the processor for execution.

The anti-virus vendors rushed a bit in my opinion to declare that any known malware would be detected regardless of KHOBE due to various alternations monitored in the system. While true obviously for big zoos of known malicious code, it does not exactly address the issue sufficiently in the enterprise environments.

Imagine an installer exploiting TOCTTOU vulnerabilities. Used in a staged attack as the initial payload for bypassing the security checks when installing further compromise tools including a malicious communication component again utilizing the technique to bypass the firewall for stealth communications. The race condition exists as long as the user memory objects can be manipulated, while the values are being examined.

It is not the end of the world by any means, but definitely something to keep an eye on. Very possibly maybe a real threat (at least until more details about KHOBE are published), but in any case a serious vulnerability which apparently exists and which probably require some changes both to the Windows kernel and to the security software functionality in order to get solved completely.

One more reason for the enterprises to ensure they have adequate incident response capabilities available in addition to the preventive security mechanisms. All the hope obviously should not be placed on the anti-virus vendor and the end point protection. Preventive security measures will be circumvented repeatedly and intrusions do happen. Just as trusted systems need hardening, they need constant intrusion and integrity monitoring throughout their lifetime.