Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 10 – Wrapping Up

Future work

Although IDSs have matured and gone through tremendous amount of development throughout the years to this day there are however quite a few limitations that still and will continue to exist. This is because currently there isn’t a way to reliably detect unknown attacks and all the existing products seem to focus on attacks that are already known. It is true that anomaly-based detection can be classed as a method of detecting unknown attacks; it is still unreliable and can be easily defeated. It would make a difference if there’s a way to utilise all these methods of detection to make IDSs smarter and make them able to predict new attacks using the same concept professional hackers use to invent new exploits and methods of attacking the network. The following points highlight important facts researchers should focus on:

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 9 – Results Discussion

Discussion

To achieve the objectives of this study the experiments needed to be as realistic as possible, meaning that the same steps an attacker takes to compromise a network would need to be taken. This is to investigate the effectiveness of the IDS in detecting each stage at the same time there’s a need for a way to ensure that any attacks managed to evade detection have done so because of the effectiveness of the technique and not because of misconfiguration or any other issues. For this reason the experiments were divided into two scenarios. These scenarios were further divided into the experiments discussed in the previously (part 1 – part 8).

As explained by (Bejtlich 2004, p.19), in certain phases of a compromise it becomes very difficult to detect the attack; taking this under consideration it was decided to carry out the tests on the phases where if detected would mean that it was very obvious that an attack is being carried out. Table 7.1 taken from (Bejtlich 2004, p.19) shows these phases.

Phases of compromise	Description	Probability of Detection	Attacker’s Advantage	Defender’s Advantage
Reconnaissance	Enumerate hosts, services, and application version.	Medium to high	Attackers perform host and service discovery over a long time frame using normal traffic patterns.	Attackers reveal themselves by differences between their traffic and legitimate user traffic.
Exploitation	Abuse, subvert, or breach services.	Medium	Attackers may attack services offering encryption or obfuscate exploit traffic.	Exploits don’t appear as legitimate traffic, and IDSs will have signatures to detect attacks.
Reinforcement	Retrieve tools to elevate privileges and/or disguise presence.	High	Encryption hides the content of tools.	Outbound activity from servers can be closely monitored and identified.
Consolidation	Communicate via backdoor, typically using covert channel.	Low to medium	With full control over communication endpoints, the attacker’s creativity is limited only by the access and traffic control offered by intervening network devices.	Traffic profiling may reveal unusual patterns corresponding to the attacker’s use of a backdoor.
Pillage	Steal information, damage the asset, for further compromise the organisation.	Low to medium	Once operating from a “trusted host”, the attacker’s activities may be more difficult to notice.	Smart analysts know the sorts of traffic that internal systems should employ and will notice deviations.

Table 7.1: Phases of a compromise (Bejtlich 2004, p.19)

The highlighted top two columns in the above table are the stages chosen to be investigated for this study.

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 8 – Scenario Two Findings – Encryption

SCENARIO 2.3 – ENCRYPTION
EXPERIMENT 8 – SSH TUNNELS

Encrypted data makes it very difficult for IDSs to examine the payload for any attacks because it can’t actually see it, which is one of the limitations that exist in IDSs. For this part of the experiment the author used SSH tunnels to encrypt the traffic between the attacker and the target. Figure 6.45 shows the attacker creating a dynamic SSH port listening on the localhost on port 8888 and an SSH forwarder listening on localhost port 8881 forwarding the traffic to destination target 192.168.59.130 on destination port 3632 (DISTCC).

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 7 – Scenario Two Findings – Evasion

Evasion Techniques

SCENARIO 2.1 – FRAGMENTATION

In this section different evasion techniques have been implemented to evade Snort. The first task of this setup was to run fragtest on the attacker, feeding it the IP address of the target to determine if the target system processes fragmented and overlapped data correctly and not being dropped by a firewall. Then fragroute was executed to fragment packets destined to the target using different fragmentation methods. The tests started with the default fragmentation settings and then the settings were changed accordingly.

Tcpdump was also started to capture the traffic for later analysis. Figure 6.17 illustrates this setup.

Figure 6.17: Evasion Techniques – Fragmentation Test Block Diagram

Figure 6.18: Evasion Techniques – Fragmentation – Testing target’s handling of fragmentation data

Fragtest’s results illustrates the target’s handling of fragmented and overlapped data, such as the target’s distance being one hop away from the attacker, the target’s IP options (sec, ts, esec, rr, satid), the target handles fragmented data correctly, handles overlapped data favouring new data and the target’s timeout setting is 29.99 seconds. All this information is very important to an attacker and would aid the attacker in deciding which fragmentation patterns to use.

The experiments presented in this scenario used the following fragmentation settings:

• Fragroute’s default configuration.
  
• An 8-byte fragments favouring old data.
  
• 4-byte TCP segments favouring new data.
  

EXPERIMENT THREE: DEFAULT SETTINGS

The default settings are as follows:

• tcp_seg 1 new – breaks the data into one byte segments favouring new data.
  
• ip_frag 24 – breaks IP packets up into 24 byte fragments with duplicate IP chaffing (ip_chaff dup) to confuse the IDS.
  
• Order random – orders the data into random order.
  
• Print – prints the output on screen in tcpdump format.
  

As described by fragroute’s manual page ip_chaff is:

“Interleave IP packets in the queue with duplicate IP packets containing different payloads, either scheduled for later delivery, carrying invalid IP options, or bearing short time-to-live values.”

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 6 – Scenario One Findings – No Evasion

Scenario One – No evasion

The overall purpose of this stage was really to confirm that the IDS was configured correctly and to ensure it can detect the attacks that will be used.

When attackers attack networks they first perform some reconnaissance to map out the target network, at this stage they’d want to know the IP range used by the organisation, DNS servers, email addresses, etc. then a network scan is performed to find out which hosts are live, what services and version of services they’ve running and what operating systems they use, this is so they can choose suitable exploits to launch at the systems. There are many tools available out there to perform such steps; however nmap is the most popular tool that’s being used by the majority of professionals as well as attackers and it is the tool of choice for this study. All the attacker needs to do to achieve this is provide nmap with an IP address or a range of IP addresses and the type of scans to use.

To test the IDS it was configured to monitor network 192.168.0.0/16 and two types of scans were performed using nmap. These were as follows:

• Nmap SYN scan.
  
• Nmap fragmented SYN scan using nmap fragmentation option.
  

It is also important to note that for the experiments throughout the study the emphasis is not the returned results of the attack, the aim here is weather snort detects the attack or not.

Experiment one – Normal SYN Scan

RULES ONLY

Command:    nmap –sS –n 192.168.59.130     ß target system

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 5 – Tools and Procedures

Welcome back all, this part of the series we will test some of the methods described previously as outlined on the upcoming sections where we will discuss the tools and the procedures followed to attempt such tests and then in the final sections of this series we will discuss the results.

The tools used in this study are all preinstalled into Backtrack operating system with the exception of snort; although snort is also installed in Backtrack a separate installation was used. The tools discussed in this chapter are used throughout the experiments.

Please note that as previously mentioned, this series contained sections from an entire Distinctions Grade MSc. Dissertation that I have attempted as part of my master’s degree years back and I have tried my best to keep it without any modifications as best as I can.

The dissertation is very long and as was described by my supervisor, was a PHD grade level research, therefore I have tried to keep it as short as possible for this blog series without impacting the benefits I am intending to pass forward to the InfoSec community and I hope you all enjoy it.

Snort IDS

According to Richard Bejtlich (2005, pg 149) snort is a network intrusion detection system that can also be used for packet capture and analysis.

Snort NIDS in standby mode

A more detailed description of snort is given by (Brandon Franklin 2006, p.3):

“Snort is a freeware IDS developed by Martin Roesch and Brian Caswell. It’s a lightweight, network-based IDS that can be set up on a Linux or Windows host. While the core program uses a Command Line Interface (CLI), graphical user interfaces (GUIs) can also be used. Snort operates as a network sniffer and logs activity that matches predefined signatures. Signatures can be designed for a wide range of traffic, including Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Internet Control Message Protocol (ICMP).

Snort consists of two basic parts:

• Header where the rules “actions” are identified.
  
• Options where the rules “alert messages” are identified.“

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 4 – Complex Evasion Techniques

Invalid RST Packets

TCP protocol is known as a connection-oriented protocol that ensures the data sent reaches its destination in a correct manner and if there is an error it requests that data to be sent again. One of the mechanisms it uses to ensure reliable communication is the use of checksum values that are added to every transmitted segment. These values are checked by the receiving end and if this value is different to the one expected by the receiver the packet is dropped.

To end a communication session TCP uses an RST packet that it sends to the other party, which terminates the session. Attackers are able to use these features to confuse the IDS because if the attacker sends an RST packet with invalid checksum the IDS sees this packet and thinks that the communication session has ended and so it stops processing it. However on the other side the receiver examines the checksum sees that it’s invalid and so it drops the RST packet, maintains the session and accepts the packets that follows, while the IDS has stopped processing this session because it thinks the session has ended, which makes other packets attackers send after the RST packet that had the invalid checksum go undetected. A way that may detect this is a signature that looks for an RST packet with invalid checksum followed by a PUSH packet.

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 3 – Basic Evasion

IDS Evasion Techniques

Since the introduction of intrusion detection systems they became very popular amongst network administrators because it provided a way to detect if any attacks took place. At them days (1990s) these systems were very simple devises that monitor the network traffic, match it to a database of signatures and if there is a match an alert is produced that informs the administrator of the event
(Bruneau, 2001, History and Evolution of Intrusion Detection, p. 3). Throughout the years attackers, security professionals, researchers and developers have always played a cat and mouse game. Every time developers, security professionals and researchers find or develop new ways to counteract attacks, attackers seem to develop new ways to attack security devises such as IDSs or even bypass them altogether. This is because of the complexities in capturing, analysing and understanding network traffic (Timm, 2002, IDS Evasion Techniques and Tactics, p. 1). These complexities meant the existence of many techniques that can be used to take advantage of the weaknesses in IDSs. These methods were very simple such as pattern matching, denial of service (DOS) and false positives. However overtime more advanced techniques such as fragmentation, session splicing and polymorphic shell-code techniques began to appear.

As mentioned by (Roberts, 2010)
that stonesoft’s disclosure of a new way to evade IDSs and IPSs (IDPSs) raises some doubts about the effectiveness of many security products already used by many organisations worldwide. However stonesoft’s discovery was not new; anyone has an interest in information security knows that these techniques have been around for a very long time and was first mentioned in 1998 by (Newsham, 1998).

As quoted by (Roberts, 2010):

“Researchers working for Stonesoft have been delving into evasion techniques since 2007 in an effort to improve Stonesoft’s own products, said Matt McKinley, Director of Product Management in the U.S.”

He further quoted:

“”In the process of doing so, we basically discovered that it’s possible to combine multiple evasion techniques together working at different layers (of the IP stack) and they can confound the IPS and become hard to protect,” he said.”

However, I respectfully disagrees with what Mr. Matt McKinley had said; this is because these techniques as will be discussed in the following sections have been around for a very long time and all are based on the techniques mentioned in
(Newsham, 1998)
paper.

In this section the author presents an overview of some of these techniques by dividing them into two categories Basic techniques and Complex techniques.

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 2

A look at IDSs in more details

As described by Karen Scarfone and Peter Mell (2007, page 15):

“Intrusion detection is the process of monitoring the events occurring in a computer system or network and analysing them for signs of possible incidents, which are violations or imminent threats of violation of computer security policies, acceptable use policies, or standard security practices. Incidents have many causes, such as malware (e.g., worms, spyware), attackers gaining unauthorized access to systems from the Internet, and authorized users of systems who misuse their privileges or attempt to gain additional privileges for which they are not authorized.”

They also mentioned that an IDS is a software that automates the process of intrusion detection and an IPS has the same functionality of an IDS but with an extra option that can stop incidents. An IPS can work as an IDS by deactivating the IPS option, which is why the term IDPS is used to mean both (IDS and IPS).

This post provides solid foundations to setup the seen for the understanding the upcoming posts in this series when we delve into different methods used to defeat these systems.

Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 1

Introduction

Computers have advanced to become part of our daily life, everywhere you go there is a computer network that’s been installed to perform a certain number of jobs; and the effectiveness of organisation’s business performance has become greatly dependent on the availability, reliability and security of these networks and the systems connected to it. This has resulted to the unfortunate fact that many systems connected to the internet is a target of a range of attacks especially organisations with hundreds of hosts, services and sensitive data. It is also equally true for small offices and isolated home users with no data to protect. This is because criminals would benefit greatly from the data they steal from big organisations; and in the case of small offices and home users, criminals would benefit from them by using them as a stepping stone so they can launch their attack without being caught.

From the beginning of 2003, the (SANS Internet Storm Centre ISC 2003) started to monitor the average survival time of un-patched machines and found that the time it takes to download patches is greater than the time to install the software. This means that before a system is fully patched attacks have already spread across the network at incredible speed. In many cases the speed of these attacks and the speed they spread across the network exceed the possibility of human intervention. Therefore the development of the components (hardware and software) that detects these attacks becomes extremely important.

As it has been mentioned by (Bruneau 2001); the rule-based method developed by Dorothy Dinning and Peter Neumann between 1984 and 1986 was used by the first IDS system. This work was influenced by a report published by James P Anderson in 1980 titled “How to use accounting audit files to detect unauthorised access”. This model was improved to create what is recognised today as the Next-Generation Intrusion Detection Expert System (Bruneau 2001, p.3).