Unsupervised Learning for Log Data Analysis Based on Behavior and Attribute Features
Proceedings of the 2019 International Conference on Artificial Intelligence and Computer Science(2019)
摘要
In some special application environments, network fault can lead to loss of important information or even mission failures, resulting in unpredictable losses. Therefore, it has certain research significance and practical value to evaluate the network status and predict the possible faults before performing the key tasks. Based on the logs collected by the router board in the real network, this paper analyses the behavior type, attribute information and the corresponding status value, and detects the hidden fault or network attack, so as to provide early warning information for operators. We propose a deep neural network model utilizing Long Short-Term Memory (LSTM) to predict the current number of level-1 logs. By comparing the predicted number of level-1 logs, it can detect abnormal behavior such as a surge in the number of logs. What's more, we perform semantic analysis on attribute information to construct attribute syntax forest, which assists maintenance staff to monitor the system through key fingerprint information in the log. In addition, we adopt attribute information and status value to train the unsupervised learning algorithm models such as Isolation Forest, OneClassSVM and LocalOutlierFactor. What's more, this paper analyses the results to find out the causes of log surge, and to assist operators in subsequent maintenance of the system.
更多查看译文
关键词
LSTM, Log Analysis, Network Fault, Unsupervised Machine Learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络