Exscind: Fast pattern matching for intrusion detection using exclusion and inclusion filters

The need for efficient intrusion detection systems increases every day to protect network traffic against emerging attacks. Unfortunately, increasing network speeds and number of signatures makes it harder for the existing signature-based intrusion detection systems to keep up. This makes those systems the weak link and the bottleneck which decreases the overall network performance. Researchers found that 30%-60% of the overall processing time of signature-based intrusion detection systems is spent on pattern matching operations [1]. In this paper, we present a novel and fast software-based pattern matching algorithm to reduce the number of times to perform pattern matching. This new algorithm introduces an exclusion-inclusion filter programmed only with signatures prefixes. It filters out the clean traffic without requiring pattern matching and weeds out suspicious packets to be searched using a specially modified Wu-Manber pattern matching algorithm. The exclusion-inclusion filter is a modified Bloom filter that produces a list of probable matching signatures for each suspect packet. The remaining few suspicious packets are searched only for the probable matches. Compared to the Wu-Manber algorithm used in intrusion detection systems, the experimental results indicate a speed up of 3.4 times on average, 5.5 times for regular traffic, and 1.6 times for worst case traffic. The memory overhead added by the algorithm was limited to 0.11%.