Sparse PCA: Phase Transitions in the Critical Regime

This work studies sparse principal component analysis (PCA) in high dimensions. Given n independent p-dimensional Gaussian samples with covariance Σ := (λ − 1)vv⊤ + Ip, our goal is to estimate v under the assumption of sparsity. On the one hand, if the sparsity level m := ∥v∥0 satisfies m ≲ √ n, algorithms such as covariance thresholding (Krauthgamer et al., 2015) consistently outperform PCA. On the other hand, if m ≫ √ n, it is conjectured that no polynomial-time algorithm can recover v below the detection threshold of PCA. We investigate the “critical” high-dimensional regime, where n, p,m → ∞ with m/ √ n → β and p/n → γ, and study estimators based on kernel PCA, generalizing covariance thresholding. Within this framework, we achieve a fine-grained understanding of signal detection and recovery. Our main result establishes a detection phase transition, analogous to the Baik–Ben Arous–Péché (BBP) transition for PCA: above a signal strength threshold—depending on the kernel function, γ, and β—kernel PCA is informative. Conversely, below the threshold, kernel principal components are asymptotically orthogonal to the signal. Notably, (1) above this threshold, consistent support recovery is possible with high probability, (2) for all β ∈ (0,∞), kernel PCA strictly outperforms PCA, and (3) as β → ∞, kernel PCA and PCA coincide. We identify optimal kernel functions for detection and support recovery, and numerical calculations suggest that soft thresholding is nearly optimal. Our key technical contribution is approximation guarantees for deterministic equivalents of kernel random matrices, which enable sharp estimates of coordinate fluctuations of kernel principal components.