AdaptParse: Adaptive Contextual Aware Attention Network for Log Parsing via Word Classification

Logs are widely used during the development and maintenance of software systems. Logs assist developers and operation & maintenance personnel to understand the state and behavior of systems at runtime. Also, logs can diagnose system failures and conduct abnormal analyses to provide further protection to the security of systems. However, large software systems generate large amounts of semi-structured logging routinely. The first step to support further analysis is how to parse semi-structured records with free-form text log messages into structured templates. Therefore, log parsing is rather challenging. Because logs are generated by static templates (i.e., log statements) in the source code, templates are often not accessible when parsing logs. It is worth noting that most proposed approaches still rely on log-specific heuristics or manual rule extraction. Those existed methods are often specialized for parsing certain log types and often neglect the semantic meaning of log messages, thus limiting performance scores and generalization, hence, in this paper, we propose a new parsing technique - Adaptive Contextual Aware Attention Network for Log Parsing via Word Classification, named AdaptParse. Adapt-Parse transforms the template generation problem into a word classification task, then learns the features of template words and variable words. We evaluate our AdaptParse on 5 realworld log datasets and compare the performance with 7 parsing techniques. Our experimental results show that the proposed approach can effectively understand the semantic meaning of log messages and achieve accurate log parsing results. Overall, AdaptParse achieves state-of-the-art performance on five realworld log datasets, outperforming all the baseline models.