Logs record the software and hardware running information. By viewing system logs, you can find system faults and causes to ensure system stability and normal running. The purpose of log parsing is to parse semi-structured original logs into readable log templates. The existing parsing methods often only focus on parsing the original logs, but ignore the later template processing. As a result, the accuracy of the results cannot be further improved. Since then, a new log parsing method FMLogs (Logs parsing based on Frequency and Minhash algorithm) is proposed. This method designs regular expressions and adjusts threshold parameters to obtain the best performance. Meanwhile, character-level frequency statistics and MinHash methods are used to merge log templates with the same length and different log templates. FMLogs conducted extensive experiments on seven real datasets, achieving an average analytic accuracy of 0.924 and an F1-Score of 0.983. The experimental results show that FMLogs is an effective log parsing method with high accuracy, high efficiency, and stable performance.

[2024-03-07] Accepted Paper /doc/2023.12.0612/v1