RExACtor: Automatic Regular Expression Signature Generation for Stateless Packet Inspection

Monitoring, security, and network management systems require packet classification solutions to identify particular application-layer protocols. In real-time cases, traffic must sometimes be identified per-packet, and thus cannot rely on traffic flow data or statistical methods for identification. Payload-based signature matching is utilized for clear text data, but the manual creation of regular expression signatures is inefficient and humans can overlook key repeated patterns in unknown traffic. In this paper, we present our system, a Regular Expression Apriori Constructor (RExACtor), which combines sequential pattern data mining using the Apriori algorithm, frequency distribution tables from natural language processing, and pairwise sequence alignment from the Needleman-Wunsch scoring algorithm to automatically generate regular expression signatures for targeted protocols. This solution takes as input traffic PCAPs and uses a modular framework to output signatures or common substrings based on user configurations. Our encoding algorithm creates more specific signatures than previous systems and our results show an 11.8% reduction in heap allocation, making our solution more space efficient than the state of the art. We also provide an implementation of the Hyperscan scanning framework for regular expressions which outperforms packet filtering libraries like Snort used in previous work. RExACtor advances the state of the art as both an automatic signature generator and knowledge discovery toolbox by reducing overall dynamic memory allocation and providing a solution for per-packet, signature-based analysis.