LAnoBERT: System log anomaly detection based on BERT masked language model

Yukyung Lee, Jina Kim, Pilsung Kang

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

The system log generated in a computer system refers to large-scale data that are collected simultaneously and used as the basic data for determining errors, intrusion and abnormal behaviors. The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention, which is a critical problem in the industry. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template using a parser. Particularly, a template corresponding to a specific event should be defined in advance for all the log data using which the information within the log key may get lost. In this study, we propose LAnoBERT, a parser free system log anomaly detection method that uses the BERT model, exhibiting excellent natural language processing performance. The proposed method, LAnoBERT, learns the model through masked language modeling, which is a BERT-based pre-training method, and proceeds with unsupervised learning-based anomaly detection using the masked language modeling loss function per log key during the test process. In addition, we also propose an efficient inference process to establish a practically applicable pipeline to the actual system. Experiments on three well-known log datasets, i.e., HDFS, BGL, and Thunderbird, show that not only did LAnoBERT yield a higher anomaly detection performance compared to unsupervised learning-based benchmark models, but also it resulted in a comparable performance with supervised learning-based benchmark models.

Original languageEnglish
Article number110689
JournalApplied Soft Computing
Volume146
DOIs
Publication statusPublished - 2023 Oct

Bibliographical note

Funding Information:
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) ( No. 2021-0-00034, Clustering technologies of fragmented data for time-based data analysis) .

Publisher Copyright:
© 2023 Elsevier B.V.

Keywords

  • Anomaly detection
  • BERT
  • Log data analysis
  • Transformer

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'LAnoBERT: System log anomaly detection based on BERT masked language model'. Together they form a unique fingerprint.

Cite this