The Geometry of LISTA [Perspectives]

The celebrated iterative soft thresholding algorithm (ISTA) and its accelerated variant, fast ISTA (FISTA), are classical signal processing methods used to solve the LASSO problem [1]. This problem spans various applications, including medical imaging, direction-of-arrival estimation, astronomy, and sparse coding. Within the broader trend in signal processing of transitioning from classical model-based approaches to deep learning-based methods, the Learned ISTA (LISTA) algorithm was proposed as a way to solve the LASSO problem using deep learning while preserving the original ISTA structure [2]. LISTA learns a fast approximation to the LASSO problem by casting the weight matrices of the ISTA algorithm as learnable parameters and making them unique for each iteration, a technique now known as deep unfolding [3]. LISTA achieves superior reconstruction results in fewer iterations compared to ISTA. This is due to two main factors. First, by learning its weights, LISTA addresses potential modeling mismatches, such as imperfect knowledge of noise behavior or the forward model. Second, even with an accurately known model, the sparsifying basis might be too complex to implement efficiently using classical methods. Here, we consider a geometric interpretation to gain insight into why LISTA performs well with significantly fewer iterations (or folds) compared to ISTA. To that end, we use the fact that both models are continuous piecewise linear (CPWL) functions. Our main contributions are as follows: • We demonstrate that existing bounds on the complexity of the geometry for ISTA and LISTA are insufficient to explain their differences; their geometries must be assessed experimentally. • We introduce the concepts of expected knot density and decision density as practical metrics to evaluate the geometry of these algorithms. • We establish a lower bound on the MAP optimal decision density for sparse linear inverse problems. • We demonstrate the effect of the loss function on the geometry of LISTA. Training LISTA with an L1 norm produces fewer larger regions and a lower knot density, compared to an L2 loss, which creates many small regions and a higher knot density. • We highlight that LISTA converges faster than ISTA and, when trained with L1, produces a simpler geometry with lower knot and decision densities closer to the optimal.