EntityParser: A Log Analytics Parser for Identifying Entities from System Logs

System logs are used to record the operating system and application program runtime status, constituting a form of semi-structured text. Analyzing these logs becomes pivotal in addressing failures or abnormalities within the operating system or application. However, the substantial volume of log data necessitates extensive manual analysis. To overcome this problem, researchers propose an automated log parsing method that reduces the volume of logs by parsing them into templates. However, a profound understanding of these templates still requires domain-specific knowledge. Therefore, this paper proposes EntityParser, which aims at enhancing log semantics. EntityParser identifies entities within log templates, mapping abstract log messages to specific, meaningful entities, thereby enhancing the readability of the logs. Additionally, EntityParser maintains two high-quality knowledge bases and can enrich them through self-iteration.Finally, in order to validate the efficacy of EntityParser, we conduct experiments on the LogHub dataset and the Linux system log dataset collected from real production environments. The results demonstrate the effectiveness of the proposed method in improving log semantics, discovering new entities, and proving its generalization. Furthermore, it enables the discovery of all potential entities within the current log through self-iteration.