In this paper, we propose a novel Intrusion Detection System (IDS) architecture utilizing both anomaly and misuse detection approaches. This hybrid Intrusion Detection System architecture consists of an anomaly detection module, a misuse detection module and a decision support system combining the results of these two detection modules. The proposed anomaly detection module uses a Self-Organizing Map (SOM) structure to model normal behavior. Deviation from the normal behavior is classified as an attack. The proposed misuse detection module uses J.48 decision tree algorithm to classify various types of attacks. The principle interest of this work is to benchmark the performance of the proposed hybrid IDS architecture by using KDD Cup 99 Data Set, the benchmark dataset used by IDS researchers. A rule-based Decision Support System (DSS) is also developed for interpreting the results of both anomaly and misuse detection modules. Simulation results of both anomaly and misuse detection modules based on the KDD 99 Data Set are given. It is observed that the proposed hybrid approach gives better performance over individual approaches.

Network-based anomaly intrusion detection systems using artificial neural networks are investigated. From knowledge of only normal traffic data, a mathematical model describing normal traffic is constructed and a test is conducted based on the deviations from the mathematical model. A self-organizing map (SOM) structure is used for constructing the mathematical model describing normal traffic and anomaly detection. The SOM structure preserves topological mappings between representations. A feature which is desired when classifying normal or intrusive behavior for network data, our hypothesis is that normal traffic representing normal behavior would be clustered around one or more cluster centers and any irregular traffic representing abnormal, and possibly suspicious, behavior would be clustered outside of the normal clustering or inside with high quantization error. The SOM is trained with normal traffic data and by considering the best matching unit or clustering region and the quantization error, the type of traffic is determined.

We propose a host-based intrusion detection system for Microsoft Windows. The proposed system detects attacks on a host machine by monitoring anomalous accesses to the Windows registry. First, a model of normal registry behavior is trained for a host and then this model is used to detect abnormal registry accesses. The system trains a normal model using data that contains no attacks and then checks each access to the registry to determine whether or not the behavior is abnormal and corresponds to an attack. A new approach to register anomaly detection (RAD) is proposed in the meaning of model generator and anomaly detector. A self organizing map (SOM), a type of artificial neural network model, is used as an anomaly detection algorithm. The system is trained on a set of normal registry accesses using SOM algorithm and then it is used to detect the behavior of malicious software. The results of this study show that the proposed system is effective in detecting the behavior of malicious software and has a low rate of false alarms compared to other host-based intrusion detection systems.