Heterogeneous Feature Selection Based on Neighborhood Combination Entropy

Feature selection aims to remove irrelevant or redundant features and thereby remain relevant or informative features so that it is often preferred for alleviating the dimensionality curse, enhancing learning performance, providing better readability and interpretability, and so on. Data that contain numerical and categorical representations are called heterogeneous data, and they exist widely in many real-world applications. Neighborhood rough set (NRS) can effectively deal with heterogeneous data by using neighborhood binary relation, which has been successfully applied to heterogeneous feature selection. In this article, the NRS model as a unified framework is used to design a feature selection method to handle categorical, numerical, and heterogeneous data. First, the concept of neighborhood combination entropy (NCE) is presented. It can reflect the probability of pairs of the neighborhood granules that are probably distinguishable from each other. Then, the conditional neighborhood combination entropy (cNCE) based on NCE is proposed under the condition of considering decision attributes. Moreover, some properties and relationships between cNCE and NCE are derived. Finally, the functions of inner and outer significances are constructed to design a feature selection algorithm based on cNCE (FScNCE). The experimental results show the effectiveness and superiority of the proposed algorithm.