Comparative Analysis on Imbalanced Multi-class Classification for Malware Samples Using CNN

Malware is considered as one of the main actors in cyber attacks. The number of unique malware samples is constantly on the rise; however, the ratio of benign software still greatly outnumbers malware samples. In machine learning, such datasets are known as imbalanced, where the majority class label greatly dominates over others. In this paper, we present a comparative analysis and evaluation of some of the proposed techniques in the literature in order to address the problem of classifying imbalanced multi-class malware datasets. More specifically, we use Convolutional Neural Network (CNN) as a classification algorithm to study the effect of imbalanced datasets on deep learning approaches. These experiments are conducted on three publicly available imbalanced datasets. Our performance analysis demonstrates that methods such as cost sensitive learning, oversampling and cross validation have positive effects on the model classification performance, albeit in varying degrees. Meanwhile others like using pre-trained models require more special parameter settings. However, best practices may change in accordance with the problem domain.