Distributed Composite Regularization Via Dual-Layer Consensus Optimization

Consensus algorithms are widely used in distributed optimization for large-scale problems such as sparse estimation, ridge regression, and optimal control. In these settings, regularization is employed to enforce task-specific structural constraints on the decision variables, with prior work focusing primarily on promoting sparsity or low-energy solutions. However, less attention has been devoted to achieving consensus on the regularization policy adopted across the network. This paper introduces a novel consensus-based algorithm with $\ell ^{1}$ and $\ell ^{2}$ regularization. The proposed mechanism adaptively controls the trade-off between sparsity and energy of the solution, by reaching a consensus on the weight-mixing parameter. We prove convergence of the dual-layer consensus process and link the distributed regularization dynamics to opinion formation in social learning. The effectiveness of the proposed Distributed Composite Regularization (DiCoR) algorithm is demonstrated on the Low Rank Matrix Completion problem (LRMC) using real image datasets under both centralized and decentralized architectures. The DiCoR approach is also compared against frameworks that employ distributed $\ell ^{1}$, $\ell ^{2}$ dynamic regularization and ElasticNet. We demonstrate that our approach achieves superior image-reconstruction quality measured in terms of Peak-Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM), across multiple scenarios.