Matrix Entry-wise Sampling: Simple is Best [Extended Abstract]

Sparsfying matrices is a ubiquitous operation in large scale machine learning, data mining and signal processing. More formally, given a large matrix A, we aim to find another matrix B, such that }A B} ¤ ε with B being significantly sparser than A. Using B as a surrogate for A is more efficient and often provides provably good approximations for many tasks. In this paper, we suggest an element-wise sampling scheme for producing B. We prove it is superior to previously suggested schemes using a relatively new matrix-valued version of the Bernstein inequality, which is known to be tight up to logarithmic factors. Moreover, the sampling scheme can be executed in the streaming model where single matrix non-zeros are presented to the algorithm in an arbitrary order. We support our theoretical findings with experimental results that corroborate our claims.