Few-shot action recognition (FSAR) has recently made notable progress through set matching and efficient adaptation of large-scale pre-trained models. However, two key limitations persist. First, existing set matching metrics typically rely on cosine similarity to measure inter-frame linear dependencies and then perform matching with only instance-level information, thus failing to capture more complex patterns such as nonlinear relationships and overlooking task-specific cues. Second, for efficient adaptation of CLIP to FSAR, recent work performing fine-tuning via skip-fusion layers (which we refer to as side layers) has significantly reduced memory cost. However, the newly introduced side layers are often difficult to optimize under limited data conditions. To address these limitations, we propose TS-FSAR, a framework comprising three components: (1) a visual Ladder Side Network (LSN) for efficient CLIP fine-tuning; (2) a metric called Task-Specific Distance Correlation Matching (TS-DCM), which uses alpha-distance correlation to model both linear and nonlinear inter-frame dependencies and leverages a task prototype to enable task-specific matching; and (3) a Guiding LSN with Adapted CLIP (GLAC) module, which regularizes LSN using the adapted frozen CLIP to improve training for better α-distance correlation estimation under limited supervision. Extensive experiments on five widely-used benchmarks demonstrate that our TS-FSAR yields superior performance compared to prior state-of-the-arts.

Bridging contrastive language-image pre-training (CLIP) to video action recognition has attracted growing interest. Human actions are inherently rich in spatial and temporal contexts, involving dynamic interactions among people, objects, and the environment. Accurately recognizing actions requires effectively capturing these fine-grained elements and modeling their relationships with language. However, most existing methods rely on cosine similarity--practically equivalent to the Pearson correlation coefficient--between global tokens for video-language alignment. As a result, they have limited capacity to model complex dependencies and tend to overlook local tokens that encode critical spatio-temporal cues. To overcome these limitations, we propose BDC-CLIP, a novel framework that leverages Brownian Distance Covariance (BDC) to align visual and textual representations. Our method can capture complex relationships--both linear and nonlinear--between all visual and textual tokens, enabling fine-grained modeling in space, time, and language. BDC-CLIP achieves state-of-the-art performance across zero-shot, few-shot, base-to-novel, and fully supervised action recognition settings, demonstrating its effectiveness and broad applicability.

Adapting CLIP models for few-shot recognition has recently attracted significant attention. Despite considerable progress, these adaptations remain hindered by the pervasive challenge of data scarcity. Text-to-image models, capable of generating abundant photorealistic labeled images, offer a promising solution. However, existing approaches simply treat synthetic images as complements to real images, rather than as standalone knowledge repositories stemming from distinct foundation models. To overcome this limitation, we frame synthetic images as an imagined base set (iBase), i.e., an independent, large-scale synthetic dataset encompassing diverse concepts. Building on this perspective, we introduce ImagineFSL, a novel CLIP adaptation methodology that pretrains on iBase and then fine-tunes for downstream few-shot tasks. We find that, compared to no pretraining, both supervised and self-supervised pretraining are beneficial, with the latter providing better performance. Based on on this finding, we propose an improved self-supervised method tailored for few-shot scenarios, enhancing the transferability of representations from synthetic to real image domains. Additionally, we present a systematic and scalable pipeline that employs chain-of-thought and in-context learning techniques, harnessing foundation models to automatically generate diverse, realistic images. Validated across eleven datasets, our methods consistently outperform state-of-the-art approaches by substantial margins.

Since pioneering work of Hinton et al., knowledge distillation based on Kullback-Leibler Divergence (KL-Div) has been predominant, and recently its variants have achieved compelling performance. However, KL-Div only compares probabilities of the corresponding category between the teacher and student while lacking a mechanism for cross-category comparison. Besides, KL-Div is problematic when applied to intermediate layers, as it cannot handle non-overlapping distributions and is unaware of geometry of the underlying manifold. To address these downsides, we propose a methodology of Wasserstein Distance (WD) based knowledge distillation. Specifically, we propose a logit distillation method called WKD-L based on discrete WD, which performs cross-category comparison of probabilities and thus can explicitly leverage rich interrelations among categories. Moreover, we introduce a feature distillation method called WKD-F, which uses a parametric method for modeling feature distributions and adopts continuous WD for transferring knowledge from intermediate layers. Comprehensive evaluations on image classification and object detection have shown (1) for logit distillation WKD-L outperforms very strong KL-Div variants; (2) for feature distillation WKD-F is superior to the KL-Div counterparts and state-of-the-art competitors.

Recent works in few-shot learning (FSL) for visual recognition have indicated that dense features benefit representation learning across novel categories. One of particularly interesting methods is DeepEMD that is formalized as optimal matching of dense features via an effective statistical distance, i.e., Earth Mover's Distance. Despite its competitive performance, DeepEMD is computationally very expensive due to inherent linear programming. Towards addressing this problem, we propose a metric-based Gaussian EMD (GEMD-M) for FSL. We adopt Gaussians for modeling distributions and closed form EMD between Gaussians as a dis-similarity measure. We illuminate that this metric amounts to feature matching, in which the optimal matching flows follow a joint Gaussian and can be expressed analytically. As the distance in GEMD-M is entangled and not that GPU-friendly, we further present a transfer learning-based Gaussian EMD (GEMD-T). The key idea is to learn a parametric EMD for a more discriminative metric based on square-roots of covariance matrices (via learnable orthogonal matrices) and mean vectors. The learnable metric in GEMD-T is decoupled and thus can be implemented by a fully-connected layer followed by a softmax classifier, very suitable for GPU. We conduct extensive experiments on large-scale Meta-Dataset and three small-scale benchmarks. The results show our GEMD is superior to DeepEMD and achieves compelling performance compared to state-of-the-art methods.

tieredImageNet The tieredImageNet [S-19] is also a fewshot benchmark for general object recognition. It is constructed from ImageNet [S-6] as well, which, different from miniImageNet, considers the hierarchical structure of ImageNet. This dataset contains 608 classes from 34 superclasses and a total of 779,165 images. Among these classes, 20 super-classes (351 classes) are used for meta-training, 6 super-classes (97 classes) for meta-validation and the remaining 8 super-classes (160 classes) for meta-testing.

Few-shot classification is a challenging problem as only very few training examples are given for each new task. One of the effective research lines to address this challenge focuses on learning deep representations driven by a similarity measure between a query image and few support images of some class. Statistically, this amounts to measure the dependency of image features, viewed as random vectors in a high-dimensional embedding space. Previous methods either only use marginal distributions without considering joint distributions, suffering from limited representation capability, or are computationally expensive though harnessing joint distributions. In this paper, we propose a deep Brownian Distance Covariance (DeepBDC) method for few-shot classification. The central idea of DeepBDC is to learn image representations by measuring the discrepancy between joint characteristic functions of embedded features and product of the marginals. As the BDC metric is decoupled, we formulate it as a highly modular and efficient layer. Furthermore, we instantiate DeepBDC in two different few-shot classification frameworks. We make experiments on six standard few-shot image benchmarks, covering general object recognition, fine-grained categorization and cross-domain classification. Extensive evaluations show our DeepBDC significantly outperforms the counterparts, while establishing new state-of-the-art results. The source code is available at http://www.peihuali.org/DeepBDC.