Generalized Adaptive Bridge Regression: a Unified Framework for High-Dimensional Architecture Discovery

In high-dimensional regression, the choice of regularization penalty typically forces a rigid assumption upon the underlying signal structure, dichotomizing data into strictly sparse (Lasso, q=1 ) or entirely dense (Ridge, q=2 ) regimes. However, real-world data generating mechanisms frequently exist on a continuum between these extremes, requiring flexible geometries to handle varying degrees of sparsity and considerable multicollinearity. In this work, we propose a data-driven framework to learn the optimal regularization norm by elevating the L_q exponent ( q ∈ (0, 2] ) from a discrete choice to a strictly continuous, learnable hyper-parameter. To overcome the computational bottleneck of evaluating non-convex and non-smooth penalty landscapes, we develop a universal proximal coordinate descent solver that utilizes a safeguarded jumping threshold operator and a novel empirical Karush-Kuhn-Tucker (KKT) verification strategy. This solver is coupled with a stochastic Tree-structured Parzen Estimator (TPE) utilizing randomized internal validation splits, enabling the rapid discovery of optimal penalty geometries without over-fitting. We evaluate the framework on simulated architectures, demonstrating its dynamic adaptivity to structural sparsity, collinearity, and varying signal-to-noise ratios. Applied to four high-dimensional genomic datasets (scaling up to P ≈ 50,000 features), our generalized adaptive bridge regression (GABR) framework successfully identifies optimal, off-grid grouping architectures ( q ≈ 1.63 to 1.80), outperforming purely sparse and purely dense alternatives. These results demonstrate that the exact regression geometry can be efficiently learned from the data, enabling a unified approach to high-dimensional inference without the computational restrictions of exhaustive discrete grid searches.