Sketching for M-estimators: a unified approach to robust regression
SODA(2015)
摘要
We give algorithms for the M-estimators minx||Ax − b||G, where A ∈ RnXd and b ∈ Rn, and ||y||G for y ∈ Rn is specified by a cost function G: R → R≥0, with ||y||G ≡ ΣiG(yi). The M-estimators generalize lp regression, for which G(x) = |x|p. We first show that the Huber measure can be computed up to relative error ε in O(nnz(A) log n + poly(d(log n)/ε)) time, where nnz(A) denotes the number of non-zero entries of the matrix A. Huber is arguably the most widely used M-estimator, enjoying the robustness properties of l1 as well as the smoothness properties of l2. We next develop algorithms for general M-estimators. We analyze the M-sketch, which is a variation of a sketch introduced by Verbin and Zhang in the context of estimating the earthmover distance. We show that the M-sketch can be used much more generally for sketching any M- estimator provided G has growth that is at least linear and at most quadratic. Using the M-sketch we solve the M-estimation problem in O(nnz(A) + poly(d log n)) time for any such G that is convex, making a single pass over the matrix and finding a solution whose residual error is within a constant factor of optimal, with high probability.
更多查看译文
关键词
regression,unified approach,m-estimators
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络