Recovering PCA and Sparse PCA via Hybrid-(l1, l2) Sparse Sampling of Data Elements.

This paper addresses how well we can recover a data matrix when only given a few of its elements. We present a randomized algorithm that element-wise sparsifies the data, retaining only a few of its entries. Our new algorithm independently samples the data using probabilities that depend on both squares (l(2) sampling) and absolute values (l(1) sampling) of the entries. We prove that this hybrid algorithm (i) achieves a near-PCA reconstruction of the data, and (ii) recovers sparse principal components of the data, from a sketch formed by a sublinear sample size. Hybrid-(l(1);l(2)) inherits the l(2)-ability to sample the important elements, as well as the regularization properties of l(1) sampling, and maintains strictly better quality than either l(1) or l(2) on their own. Extensive experimental results on synthetic, image, text, biological, and financial data show that not only are we able to recover PCA and sparse PCA from incomplete data, but we can speed up such computations significantly using our sparse sketch