LogSD: Detecting Anomalies from System Logs through Self-supervised Learning and Frequency-based Masking
arxiv(2024)
摘要
Log analysis is one of the main techniques that engineers use for
troubleshooting large-scale software systems. Over the years, many supervised,
semi-supervised, and unsupervised log analysis methods have been proposed to
detect system anomalies by analyzing system logs. Among these, semi-supervised
methods have garnered increasing attention as they strike a balance between
relaxed labeled data requirements and optimal detection performance,
contrasting with their supervised and unsupervised counterparts. However,
existing semi-supervised methods overlook the potential bias introduced by
highly frequent log messages on the learned normal patterns, which leads to
their less than satisfactory performance. In this study, we propose LogSD, a
novel semi-supervised self-supervised learning approach. LogSD employs a
dual-network architecture and incorporates a frequency-based masking scheme, a
global-to-local reconstruction paradigm and three self-supervised learning
tasks. These features enable LogSD to focus more on relatively infrequent log
messages, thereby effectively learning less biased and more discriminative
patterns from historical normal data. This emphasis ultimately leads to
improved anomaly detection performance. Extensive experiments have been
conducted on three commonly-used datasets and the results show that LogSD
significantly outperforms eight state-of-the-art benchmark methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要