With the rapid development of artificial intelligence, deep neural networks have achieved great performance in many tasks. However, traditional deep learning methods require a large amount of training data, which may not be available in certain practical scenarios. In contrast, few-shot learning aims to learn a model that can be readily adapted to new unseen classes from only one or a few labeled examples. Despite this success, most existing methods rely on pre-trained feature extractor networks trained with global features, ignoring the discrimination of local features, and weak generalization capabilities limit their performance. To address the problem, according to the human's coarse-to-fine cognition paradigm, we propose an Inverted Pyramid Network with Spatial-adapted and Task-oriented Tuning (TIPN) for few-shot learning. Specifically, the proposed framework represents local features for categories that are difficult to distinguish by global features and recognizes objects from both global and local perspectives. Moreover, to ensure the calibration validity of the proposed model at the local stage, we introduce the Spatial-adapted Layer to preserve the discriminative global representation ability of the pre-trained backbone network. Meanwhile, as the representations extracted from the past categories are not applicable to the current new tasks, we further propose the Task-oriented Tuning strategy to adjust the parameters of the Batch Normalization layer in the pre-trained feature extractor network, to explicitly transfer knowledge from base classes to novel classes according to the support samples of each task. Extensive experiments conducted on multiple benchmark datasets demonstrate that our method can significantly outperform many state-of-the-art few-shot learning methods.

Open Vocabulary Object Detection (OVD) aims to detect objects from novel classes described by text inputs based on the generalization ability of trained classes. Existing methods mainly focus on transferring knowledge from large Vision and Language models (VLM) to detectors through knowledge distillation. However, these approaches show weak ability in adapting to diverse classes and aligning be-tween the image-level pre-training and region-level detection, thereby impeding effective knowledge transfer. Moti-vated by the prompt tuning, we propose scene-adaptive and region-aware multi-modal prompts to address these issues by effectively adapting class-aware knowledge from VLM to the detector at the region level. Specifically, to enhance the adaptability to diverse classes, we design a scene-adaptive prompt generator from a scene perspective to consider both the commonality and diversity of the class distributions, and formulate a novel selection mechanism to facilitate the ac-quisition of common knowledge across all classes and spe-cific insights relevant to each scene. Meanwhile, to bridge the gap between the pre-trained model and the detector, we present a region-aware multi-modal alignment module, which employs the region prompt to incorporate the po-sitional information for feature distillation and integrates textual prompts to align visual and linguistic representations. Extensive experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art models on the OV-COCO and OV-LVIS datasets, sur-passing the current method by 3.0% mAP and 4.6% APr.

Predicting future frames using historical spatiotemporal data sequences is challenging and critical, and it is receiving a lot of attention these days from academic and industrial scholars. Most spatiotemporal predictive algorithms ignore the valuable backward reasoning ability and the disparate learning complexities among different layers and hence, cannot build good long-term dependencies and spatial correlations,resulting in suboptimal solutions. To address the aforementioned issues, we propose a two-stage coarse-to-fine spatiotemporal predictive model with bidirectional distillation and level-specific meta-adaptation(See More)in this paper, which includes a bidirectional distillation network(BDN) and a level-specific meta-adapter(LMA), to gain bidirectional multilevel reasoning. In the first stage, BDN concentrates on bidirectional dynamics modeling and coarsely constructs spatial correlations of different layers, while LMA is introduced in the second fine-tuning stage to refine the multilevel spatial correlations from a meta-learning perspective.In particular, BDN mimics the forward and backward reasoning abilities of humans in a distillation manner,which aids in the development of long-term dependencies. The LMA views learning of different layers as disparate but related tasks and guides the transfer of learning experiences among these tasks through learning complexities. Thus, each layer could be closer to its solutions and could extract more informative spatial correlations. By capturing the enhanced short-term spatial correlations and long-term temporal dependencies,the proposed model could extract adequate knowledge from sequential historical observations and accurately predict future frames whose backtracking preconditions are consistent with the historical sequence. Our work is general and robust enough to be integrated into most spatiotemporal predictive models without requiring additional computation or memory cost during inference. Extensive experiments on four widely used predictive learning benchmarks validated the proposed model's effectiveness in comparison to state-of-the-art approaches(e.g., 10.6% improvement of Mean Squared Error on the Moving MNIST dataset).

人工智能技术的快速发展引领了第四科学研究范式—“智能科学”的发展。在该人工智能驱动科学研究的新范式下,计算机能够利用海量数据,通过机器学习、深度学习等算法,达到自主学习和推理的类人智能水平（如ChatGPT、AlphaFold等人工智能技术）,为科学研究中数据、模型和实验等多个方面提供了显著助益。在新范式下,诞生了一大批新的科学模型,取代了传统的经典科学模型表示方法,通过构建基于数据-知识耦合驱动的人工智能分析机制,形成了一系列深度学习增强的复合式科学模型,有望带来更多的科学突破和创新。结合“智能科学”科学研究新范式,科学研究中实验数据、计算模型和科学实验等方面的效率能够大幅提升,科学家能够高效完成研究命题设想、研究方案设计和实验仿真验证,以获得科学结论并推动科学发展。

Transfer learning could improve the robustness and generalization of the model, reducing potential privacy and security risks. It operates by fine-tuning a pre-trained model on downstream datasets. This process not only enhances the model’s capacity to acquire generalizable features but also ensures an effective alignment between upstream and downstream knowledge domains. Transfer learning can effectively speed up the model convergence when adapting to novel tasks, thereby leading to the efficient conservation of both data and computational resources. However, existing methods often neglect the discrepant downstream–upstream connections. Instead, they rigidly preserve the upstream information without an adequate regularization of the downstream semantic discrepancy. Consequently, this results in weak generalization, issues with collapsed classification, and an overall inferior performance. The main reason lies in the collapsed downstream–upstream connection due to the mismatched semantic granularity. Therefore, we propose a discrepant semantic diffusion method for transfer learning, which could adjust the mismatched semantic granularity and alleviate the collapsed classification problem to improve the transfer learning performance. Specifically, the proposed framework consists of a Prior-Guided Diffusion for pre-training and a discrepant diffusion for fine-tuning. Firstly, the Prior-Guided Diffusion aims to empower the pre-trained model with the semantic-diffusion ability. This is achieved through a semantic prior, which consequently provides a more robust pre-trained model for downstream classification. Secondly, the discrepant diffusion focuses on encouraging semantic diffusion. Its design intends to avoid the unwanted semantic centralization, which often causes the collapsed classification. Furthermore, it is constrained by the semantic discrepancy, serving to elevate the downstream discrimination capabilities. Extensive experiments on eight prevalent downstream classification datasets confirm that our method can outperform a number of state-of-the-art approaches, especially for fine-grained datasets or datasets dissimilar to upstream data (e.g., 3.75% improvement for Cars dataset and 1.79% improvement for SUN dataset under the few-shot setting with 15% data). Furthermore, the experiments of data sparsity caused by privacy protection successfully validate our proposed method’s effectiveness in the field of artificial intelligence security.