Region Purity-Based Local Feature Selection: A Multiobjective Perspective

In contrast to the traditional feature selection (FS), local FS (LFS) partitions the whole sample space and obtains the feature subset for each local region. However, most existing LFS algorithms lack a problem-specific objective function and instead simply apply the distance-like objective function, which limits their classification performance. In addition, obtaining a good LFS model is essentially a multiobjective optimization problem. Therefore, in this article, we propose a region purity (RP)-based LFS (RP-LFS) where, besides the proportion of the selected features and region-based distance metric, we design a novel objective function, RP, from the perspective of combining local features with classifiers. To solve the RP-LFS, an improved nondominated sorting genetic algorithm III is proposed. Specifically, a network-inspired crossover operator and a quick bit mutation are applied, which can improve the ability to search for better solutions. A regional feature sharing strategy between different local models is developed, which can preserve more effective features. Experimental studies on 11 UCI datasets and nine high-dimensional datasets validate the effectiveness of our proposed RP. In comparison with various state-of-the-art FS and LFS algorithms, RP-LFS can achieve very competitive classification accuracy while obtaining a reduced feature subset size.