Sparse neural network methods typically assume that the target sparsity (or density) is fixed in advance, even though the relationship between network capacity and performance is generally unknown and task-dependent. Existing approaches (including iterative pruning, dynamic sparse training, and pruning at initialization) either rely on dense pretraining, incur substantial retraining cost, or require a preset sparsity budget. We propose Gradual Capacity Growth (GCG), a constructive sparse-to-dense training framework that allocates network capacity progressively during training. Starting from a sparse seed network, GCG grows new connections in stages using PathGrow, a probabilistic path-based growth rule that biases additions toward high-signal input–output pathways while preserving structural diversity. Growth is interleaved with limited training, and a lightweight performance–density extrapolation rule is used to estimate the smallest density beyond which further capacity increases yield diminishing accuracy gains. Experiments on CIFAR, TinyImageNet, and ImageNet show that GCG efficiently identifies sparse networks that achieve near-dense performance at moderate densities, without requiring dense pretraining or exhaustive retraining at each sparsity level. While growth-only methods do not reach the extreme sparsity achievable by pruning or dynamic reallocation techniques, GCG substantially reduces total training cost compared to iterative magnitude pruning and provides a practical mechanism for exploring the accuracy–density tradeoff under limited optimization budgets.
Temporal action localization (TAL) aims to recognize actions and localize their temporal boundaries. Existing TAL methods mainly rely on RGB frames, sometimes with optical flow. However, relying solely on RGB struggles in cluttered backgrounds and for fine-grained motion, while optical flow is computationally expensive and sensitive to illumination. Human skeleton information provides a background-robust representation which explicitly describes human motion. In this work, firstly, we extract 2D poses from a widely used TAL dataset to construct a skeleton-modal TAL benchmark, and conduct a study of how skeleton information can benefit TAL. Then we compare existing methods on skeleton-only data, and explore different RGB–skeleton fusion strategies, revealing when skeleton information is most helpful. Since the skeleton keypoints which often exhibit large offset magnitudes around action onset and transition moments, we finally propose Keypoint-Movement-Aware Former (KeyMA), an RGB–skeleton fusion framework that models keypoint offsets as temporal movement features to highlight potential boundary frames. Extensive experiments are conducted to analyze the influence of skeleton information for TAL and show that KeyMA achieves state-of-the-art performance compared to existing methods, demonstrating the potential for enhancing TAL with skeleton information.
In parameter-efficient class-incremental learning for Vision Transformers, accuracy often trades off against inference cost. Methods that expand task-specific modules can reduce parameter interference across tasks, but their test-time cost grows linearly with the number of tasks. A single shared adapter keeps inference cost fixed, yet repeated updates can shift the representation space toward new tasks, making old-class features less discriminative over time. We show that a shared-adapter design can substantially narrow this accuracy gap without giving up its cost advantage. We propose HPS-Replay, a continual tuning framework built on one frozen ViT backbone and one shared adapter throughout learning. To prevent the classifier from drifting, the method replays a compact prototype memory and applies post-task alignment. To prevent the adapter from drifting, it replays synthetic intermediate states sampled from stored class-conditional means and variances. A momentum-based anchor stabilizes both signals. Across five benchmarks, HPS-Replay leads on three of five benchmarks in average incremental accuracy while maintaining constant inference cost, substantially reducing persistent replay memory and using far fewer inference FLOPs than expanding-module alternatives on long task sequences.
Multivariate time series forecasting, which aims to predict the future evolution of multiple interdependent temporal variables, remains a challenging task due to nonlinear trends, complex seasonal patterns, and intricate inter-variable interactions, particularly in real-world systems characterized by abrupt changes and noisy observations. Despite recent advances, existing methods still face several key limitations. First, decomposition-based approaches often rely on linear assumptions, which restrict their ability to capture nonlinear dynamics. Second, current temporal modeling techniques primarily focus on time-domain dependencies, while overlooking informative patterns embedded in the frequency domain. Third, channel-independent strategies typically model each variable in isolation, ignoring dynamic inter-variable interactions and thereby reducing robustness in the presence of confounding or irrelevant factors. To address these limitations, we propose TFDF-Net, a time-frequency feature fusion framework. TFDF-Net introduces an adaptive decomposition module with learnable Gaussian kernel weights to accurately separate trend and seasonal components. Multi-scale wavelet decomposition combined with a frequency-domain enhancement module enables effective extraction of seasonal time-frequency features, while a stochastic-pooling-based global representation aggregation-redistribution mechanism models nonlinear inter-variable dependencies. Extensive experiments demonstrate that TFDF-Net consistently improves forecast accuracy and robustness in complex multivariate time series.
Many optimization algorithms used in Machine Learning arise as instances of De Giorgi’s minimizing movement scheme, where each step minimizes the objective plus a distance-based penalty. The dynamics is fully determined by the pair (objective, distance): different distances recover gradient descent, natural gradient, and mirror descent as special cases. In this paper we propose learning the distance from data. The standard well-posedness and energy-decrease guarantees carry over unchanged to the learned-distance setting. We give a variational formulation of the classical fact that the Hessian is the optimal quadratic metric, and extend the framework to non-quadratic penalties. We identify power-law penalties as an interesting parametric family for which the proximal update takes the explicit form with learnable exponents. With per-layer exponents, the resulting optimizer smoothly interpolates between gradient descent and sign-like methods, producing a family of update rules beyond existing mirror descent and learned-preconditioner approaches.
