Variable projection algorithms with sparse constraint for separable nonlinear models

Separable nonlinear models are widely used in various fields such as time series analysis, system modeling, and machine learning, due to their flexible structures and ability to capture nonlinear behavior of data. However, identifying the parameters of these models is challenging, especially when sparse models with better interpretability are desired by practitioners. Previous theoretical and practical studies have shown that variable projection (VP) is an efficient method for identifying separable nonlinear models, but these are based on L_2 penalty of model parameters, which cannot be directly extended to deal with sparse constraint. Based on the exploration of the structural characteristics of separable models, this paper proposes gradient-based and trust-region-based variable projection algorithms, which mainly solve two key problems: how to eliminate linear parameters under sparse constraint; and how to deal with the coupling relationship between linear and nonlinear parameters in the model. Finally, numerical experiments on synthetic data and real time series data are conducted to verify the effectiveness of the proposed algorithms.