Bi-level ensemble method for unsupervised feature selection

Unsupervised feature selection is an important machine learning task and thus attracts increasingly more attention. However, due to the absence of labels, unsupervised feature selection often suffers from stability and robustness problems. To tackle these problems, some works try to ensemble multiple feature selection results to obtain a consensus result. Most of the existing methods do the ensemble on the feature level, i.e., they directly ensemble feature selection results by feature ranking or voting aggregation, without paying any attention to the following downstream tasks. In this paper, we take clustering as the downstream task and wish to ensemble the base results to select features which are appropriate for clustering. To this end, we propose a novel bi-level feature selection ensemble method, which ensembles on two levels: the feature level and the clustering level. Together with feature level ensemble, we also learn a consensus clustering result from base feature selection results with self-paced learning. Then, we apply the consensus clustering result to guide the feature selection in turn. Extensive experiments are conducted to demonstrate that the proposed method outperforms other state-of-the-art feature selection and feature selection ensemble methods in the clustering task. The codes of this paper are released in https://doctor-nobody.github.io/codes/BLFSE.zip.