A Two-Step Feature Selection Using Permutation Based Feature Importance and Wrapper-Based Bio-Inspired Algorithms for Classifying Clinical Dataset

Abstract The proposed framework aims to address the challenge of high-dimensional, noisy, irrelevant, and redundant features in clinical datasets by using a two-step feature selection approach. The first step involves a wrapper-based method for calculating the Permutation Based Feature Importance (PBFI) score using the Support Vector Machine (SVM) classifier. The second step employs three wrapper-based bio-inspired optimization algorithms, namely Harris Hawk Optimization (HHO), Binary Grasshopper Optimization (BGOA), and Whale Optimization (WOA), with the weighted F1-Score, measured by the SVM classifier, as the fitness function. The selected features from both steps are then combined using a union operation and used to train four classifiers: SVM, K-Nearest Neighbor (K-NN), Linear Discriminant Analysis (LDA), and Naive Bayes (NB). The proposed framework is also evaluated using a non-parametric hypothesis testing method, the Kruskal-Wallis Test, on eight medical datasets from the Machine Learning Repository (MLR) maintained by the University of California, Irvine (UCI). The proposed approach is compared with feature selection using the Particle Swarm Optimization algorithm (PSOA) and Lion's Algorithm, and is shown to perform better.