Extracting Log Patterns from System Logs in LARGE

This paper introduces algorithms of extracting log patterns from system logs in LARGE system. LARGE is a log analyzing framework deployed in the Scientific Computing Grid Environment in Chinese Academy of Sciences. In the process of monitoring systems logs, there is the need of obtaining patterns from log records, and set corresponding rules to deal with logs in certain patterns. Two algorithms named identical words rate and tree-matching to fulfill this requirement are illustrated, and their performances are compared to show the capability and efficiency of extracting log patterns. This work can be extended to various distributed and cloud environments for log monitoring.