An Automatic Classification Framework Using Multi-Modal Deep Learning for Lung Diseases.

Disease identification across multiple modalities remains a challenging task. Recent research has explored early, automatic disease prediction using single modality inputs and computer vision with deep learning techniques. In this study, we introduce the Robust Multi-modal Deep Learning-based (RMDL) framework for automatic lung disease classification using both computed tomography (CT) and X-ray scans. The RMDL framework comprises segmentation, feature extraction, and classification. Input lung images undergo pre-processing, segmentation, and region of interest (ROI) extraction. The core contribution of RMDL lies in its deep learning-based automatic feature extraction and classification. We modify the Convolutional Neural Network (CNN) model to address overfitting and exploding gradients, enabling efficient feature learning with minimal computing cost. The modified CNN model employs L1 normalization and novel convolutional layers for automatic low-dimensional feature extraction. We further utilize the Long Short-Term Memory (LSTM) classifier to mitigate the exploding gradient problem, enabling accurate and early lung disease prediction. RMDL's performance was assessed using CXRTD and CCSC datasets. CCS CONCEPTS • Keywords: Classification, CNN, automatic disease prediction, deep learning, multi-modal.