Investigating the efficiency of modern Intrusion Detection Systems (IDS) in detecting current evasion techniques – Part 4 – Complex Evasion Techniques

Invalid RST Packets

TCP protocol is known as a connection-oriented protocol that ensures the data sent reaches its destination in a correct manner and if there is an error it requests that data to be sent again. One of the mechanisms it uses to ensure reliable communication is the use of checksum values that are added to every transmitted segment. These values are checked by the receiving end and if this value is different to the one expected by the receiver the packet is dropped.

To end a communication session TCP uses an RST packet that it sends to the other party, which terminates the session. Attackers are able to use these features to confuse the IDS because if the attacker sends an RST packet with invalid checksum the IDS sees this packet and thinks that the communication session has ended and so it stops processing it. However on the other side the receiver examines the checksum sees that it’s invalid and so it drops the RST packet, maintains the session and accepts the packets that follows, while the IDS has stopped processing this session because it thinks the session has ended, which makes other packets attackers send after the RST packet that had the invalid checksum go undetected. A way that may detect this is a signature that looks for an RST packet with invalid checksum followed by a PUSH packet.

Fragmentation technique

Using fragmentation the attacker splits or fragments the attacking packet into fragments so that it doesn’t match any of the signatures the IDS has, which is similar to the session splicing method however it’s more powerful and enables the attacker to be more creative. This technique also gives the attacker the choice of two fragmentation methods. One of these methods overwrites a section of the previous fragment (overlapping), while the other one overwrites the whole fragment. Thus it allows the attacker to write a packet full of garbage data and make the attack blend in with the standard protocols. Below is an example by (iDefence Security Team 2006, p.6), which explains this further:

“Attack 1: Overlap Method

Packet 1: GET /cgi-bin/

Packet 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/../phxx

Packet 3: f?

This example of fragmentation can overwrite the ‘xx’ portion of Packet 2 with the data in Packet 3, making the information resemble the following:

GET /cgi-bin/ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/../phf?

Attack 2: Overwrite Method

Packet 1: GET /cgi-bin/

Packet 2: some_normal_filename.cgi

Packet 3: /aaa/../aaa/../aaa/../phxx

Packet 4: f?

This example is similar to the first, but in this example, the ‘xx’ portion is overwritten and the some_normal_filename.cgi packet is completely overwritten with the last two packets. This leaves GET /cgi-bin/phf? as the end result.

It is important to note, however, that IDSs such as Snort have found ways to handle these types of attacks through reassembly.”

Carrying out the experiments discussed in chapter six, the author disagrees to some degree with the above last comment by iDefence when they concluded that “IDSs such as Snort have found ways…“. The experiment’s results indicate that there are variations of this technique that still work especially when they’re combined with other fragmentation methods. Thus these techniques will continue to represent a problem to many IDSs. This is further confirmed by (Timm 2002, p.9).

Time-to-live (TTL) Technique

The TTL method of evading IDSs requires the attacker to have a deep knowledge of the victim’s network topology. The attacker needs to know how many hops or in another words the distance to the target host and whether the IDS is placed in front of the target host or not. Using their knowledge of the topology of the victim’s network; attackers can send a TCP packet with a small TTL flag that the attacker knows that this packet would only reach the IDS. Although the packet is addressed to the end host it will not reach it because of the small TTL flag. This would allow attackers to insert packets into the IDS stream processing. For example taking the malicious packet illustrated earlier (GET /cgi-bin/phf?) in normal circumstances this would generate an alert because the IDS has a signature for it, however if the TTL attack method is used this attack would go undetected. This is further illustrated below:

Would reach

TTL Value

End Host

>= 10




< 5

Table 4.1: TTL evasion technique example TTL values

Packet 1: GET /cgi-bin/p        TTL 10

Packet 2: some_file.cgi?=        TTL 5

Packet 3: kjvskdjbkdjsbkbvk        TTL 4        This would be dropped

Packet 4: hf?                TTL 10

As described in the table above a packet with a TTL of 10 or more would reach the end host, a packet with a TTL of 5 would reach the IDS and a packet with a TTL of less than 5 would be dropped. In the above example there are four packets sent by the attacker the first one would reach the end host as GET /cgi-bin/p, the second one would reach the IDS only, the third packet would be dropped and the fourth packet would also reach the end host as hf?

When the end host puts these packets in order it would result in a packet with a payload of GET /cgi-bin/phf?, which will trigger the attack, while the IDS received a packet with a payload of GET /cgi-bin/psome_file.cgi?=hf and so the attack would go unnoticed because the packet the IDS received doesn’t match any of the signatures in the IDS.

Urgency Flag Method

The urgent flag in the TCP protocol is used to indicate that the data is of an urgent matter. The pointer used marks the beginning of the urgent data in the packet, which is processed first and everything else that is before this pointer is ignored. If the IDS don’t take the TCP urgent feature into consideration it will allow the attacker to evade detection. This is the case of some IDSs, where the attacker is able to insert garbage data before the urgent flag, which makes the IDS read the data without taking into account the urgent flag handling of the end host. This means that there is more data the IDS have than the end host processed. As described by RFC 1122, when the urgent data is joined with normal data the urgency pointer causes one byte of data that is next to the urgent data to be lost. This is shown in the following example:

Packet 1: 12345

Packet 2: 678         Urgent pointer: 3ß this means that packet 3 has the urgent pointer

Packet 3: ABC

End result: 12345678BC

The above example basically shows that we have 3 packets with the third packet having the urgency pointer, when they are combined the urgency pointer causes one byte (corresponds to the letter A in the packet 3) of the third packet to be deleted.

Polymorphic Shellcode

Modern IDSs these days have signatures of known attacks and exploits such as shell code attack. These IDSs are very good at detecting known attacks, however if the attack is still unknown or the attack is changed in a way that it keeps the original attack but modifies its signature, it will be very hard if not impossible for the IDS to detect it. This can be easily performed by encoding the attack and sending it with a stub that will decode it at the target end, when the IDS tries to match it to its signatures there’ll be no match and so the attack would proceed without being noticed. The use of the stub also means that the attack would look completely different every time it is sent, which makes it very difficult to write signatures for the encoded attack.

Polymorphic shell code uses a simple algorithm to encrypt the shell code; this makes shell code signatures useless because the encryption hides the commonly used strings within shell code. Encryption was demonstrated in the last experiment of chapter six (SSH tunnel).

ASCII shelcode

ASCII shell code is very similar to polymorphic shell code; this type of shell code contains only the ASCII characters and hides the string inside the shell code in the same way as the polymorphic shell code. In this way attackers can bypass character restriction mechanisms and manage to evade pattern matching signatures.

An example that explains this further is shown below:

char shellcode[] =











The example shown above executes /bin/sh shell, as it can be seen from the above the “bin” and the “sh” are within the last few bytes of the shell code.

Application layer attacks

Attacks available to the application layer take evasion methods to a new level that provides many different ways to evade IDSs. To increase data transfer speeds many applications that deal with media (images, videos, audios) uses some type of data compression, which compresses the data to a much smaller size than it originally was. If any of these applications have vulnerabilities, the whole attack can happen with the compressed data and so the IDS will not be able to check the compressed data for signatures.

Combining Attacks

There are many techniques used to evade IDSs detection that have been around for a very long time, throughout the years developers have managed to develop solutions to detect them, however these solution are not perfect and on many occasions these techniques can go unnoticed. As described by (Champion Undated, p.8) another problem that could take these techniques to another level and cause modern IDSs and probably future IDSs great difficulties is when two or more of these attacks are combined. These methods could lead to a total detection failure or produce less serious and obscure alerts that would be classed by analysts as false positives. This was exactly what happened on some of the experiments in chapter six.

(Roberts 2010) mentioned in his online paper that Stonesoft developed a tool that uses such methods of combining many other techniques to make new evasion attacks; one of these methods that Stonesoft claimed to have discovered rapidly opens/closes the TCP connection, while manipulating the TCP_TIME_WAIT parameter. This causes the IDS to think that this session had already been checked.

This technique however is not as Stonesoft claims; it existed for quite a long time as described previously in this chapter and also confirmed by Paul Robert in the same article (Roberts 2010).

As described in the recommendations section (chapter 8) of this study the model designed by the author of this study aims at using Artificial intelligence or fuzzy logics so that the IDS itself learns about its weaknesses and strengths and how attackers would use these vulnerabilities to attack it. To do this the system would use such well known techniques to automatically create new methods by combining these techniques and then using packet analysis methods to automatically create new ways of defending against such predicted attacks.

This method according to the author’s research is not used yet; the only studies that looked at using artificial intelligence for IDSs was to reduce false positives and false negatives. The Model also uses a combination of detection methods such as the use of a profile of its self and the devices connected to the network it’s monitoring, that way it knows which OS, services etc. Each system uses and from that it knows how each system can be attacked due to the OS’s implementation of the TCP/IP stack, which is similar to the target-based pre-processor policy used by Snort NIDS.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s