The dual randomized kaczmarz algorithm

Consider finding the minimum norm solution to a consistent system of linear equations Ax = b, where A is an element of R-mxn is a real matrix with row vectors a(i)(inverted perpendicular) , i = 1, 2, ... , m, and b is an element of R-m is a known real vector. It is well known that the sequence {x(k))(k=1)(infinity) generated by the randomized Kaczmarz (RK) algorithm with an arbitrary initial guess x(0) is an element of span{a(1), a(2), ... , a(m)} converges to the minimum norm solution of Ax = b. Based on the fact that x(k) is a linear combination of the vectors a(i), i = 1, 2, ... , m, for each k >= 0, the dual randomized Kaczmarz (DRK) algorithm is proposed in this article for finding the minimum norm solution of Ax = b. Except for the final iteration, each iteration of the DRK algorithm only updates the combination coefficients of x(k) with respect to a(i), i = 1, 2, ... , m, instead of x(k) itself. We prove that, for the same number of iterations K, the ratio of the average computation work required by the DRK algorithm to the RK algorithm is: kappa := 1/4 (2m+1/n + 2m-1/K). This means that the DRK algorithm is better than the RK algorithm provided kappa < 1. Especially, the DRK algorithm has more obvious advantages than the RK algorithm for the underdetermined systems of linear equations. Numerical results show the superiority of the DRK algorithm for the underdetermined systems of linear equations and the CT problem.