Information-theoretic feature selection based on the Weight of the New Classification Information

Many objective functions of feature selection methods are positively correlated with the amount of the new classification information and negatively correlated with the amount of redundant information. However, a critical issue in these methods is they ignore the magnitude of the negative correlation of redundant information. So, we proposed a new method from another perspective to measure redundancy between features, and employ the Weight of the New Classification Information (WNCI) to ensure features are related with class label when selected feature is given. Finally, we tested the classification accuracy of the proposed method and compared with five other methods on eighteen data sets. Then the experiment results show that the new proposed method performs better than these other methods in terms of classification accuracy.