Robust 6D pose estimation under severe occlusion remains challenging due to fragmented feature representations in traditional paradigms. We propose the Moment-guided Progressive Geometric Reasoning (MPGR) framework, which reformulates pose estimation as a distribution-aware inference problem. By treating incomplete observations as feature distributions, MPGR restores occlusion-corrupted geometric structures through high-order moment modeling. Key components include a Global Second-order Statistical Recalibrator (GSSR) to enhance backbone semantic consistency and a Multi-moment Feature Recalibration Module (MFRM) to repair multi-scale features by jointly modeling first-order responses and second-order cross-channel covariances. Extensive evaluations on BOP benchmarks demonstrate that MPGR consistently outperforms state-of-the-art baselines. Results and ablation studies confirm that integrating high-order statistical modeling significantly improves feature propagation and estimation robustness, offering a powerful solution for robotic perception and augmented reality in complex industrial environments.
Contrastive language-image pretraining has established a robust foundation for video action recognition. Existing methods mainly enhance video modeling through long-range temporal aggregation or prompt design, but most still rely on coarse-grained aggregated representations and sparse action labels, and therefore are insufficient for characterizing the local dynamic differences that determine category boundaries. In fact, the key discriminativecues in action recognition lie not only in whether videos share the same global semantics, but also in the distributional differences exhibited by local spatiotemporal patterns during temporal evolution. To this end, this paper proposes MHoS-Adapter, a parameter-efficient multimodal adaptation framework that couples distribution-sensitive visual modeling with semantically densified textual supervision to alleviate the insufficient characterization of local dynamic differences and the limited expression of language supervision in CLIP-based video transfer. First, we design a higher-order statistical visual adapter(HoS-Adapter) that explicitly models higher-order statistical information of feature distributions on top of lightweight spatiotemporal enhancement, thereby improving the sensitivity of frozen pretrained representations to fine-grained action differences. Second, we introduce a parallel text adapter(Para-Adapter) in the text branch and expand category semantics with caption-style action descriptions, thereby upgrading language supervision from short category names to richer action-semantic supervision. Furthermore, we jointly optimize the two branches with a multimodal objective consistent across multiple granularities, simultaneously constraining global video-text alignment, category-level discriminative boundaries, and token-level semantic correspondence. Extensive experiments on MiniKinetics-200, Kinetics-400, UCF101, and HMDB51 show that the proposed method achieves competitive performance while maintaining high parameter efficiency. In particular, on Kinetics-400, it reaches 83.2% Top-1 accuracy with only 19M trainable parameters and remains highly competitive in zero-shot and few-shot transfer scenarios. Code is released at https://github.com/xuxue152/MHoS_Adapter.
Video action recognition, as a core task in computer vision, plays an important role in multimedia content understanding. Although vision-language models (e.g., CLIP) have demonstrated strong generalization capabilities in multimodal tasks, directly transferring them to the video domain still faces challenges such as insufficient temporal modeling and high cost of parameter fine-tuning. Existing Parameter-Efficient Fine-Tuning (PEFT) methods generally exhibit limitations in spatial modeling when adapting to videos, making it difficult to capture complex high-order spatiotemporal dependencies between frames. This paper proposes a multimodal and multitask video action recognition framework based on high-order Adapters, named HM-CLIP. We design a Global Temporal Difference Adapter (GT Adapter), which integrates a frame-difference mechanism with a global second-order covariance matrix (GSOP) to effectively model high-order spatiotemporal statistical dependencies in videos. Furthermore, we propose a Video Caption Prompting mechanism that utilizes pre-trained vision-language models to generate rich video-level textual descriptions, thereby enhancing the semantic representation capability of the text branch. Additionally, we introduce a lightweight Temporal Feature Aggregation (TFA) block, which adaptively learns frame-level importance through an attention-weighted mechanism, significantly improving temporal modeling performance. Extensive experiments on standard benchmarks such as Kinetics-400, UCF101, and HMDB51 show that HM-CLIP achieves a Top-1 accuracy of 83.5
Recently, Contrastive Language-Image Pre-Training(CLIP) and Parameter-Efficient Fine-Tuning(PEFT) have attracted significant attention in the field of video action recognition. Nevertheless, effectively capturing both temporal dynamics and spatial local variations remains essential. Previous methods either concentrated on modeling long-term dependencies in video-level features while ignoring intra-frame spatial details, which frequently contain crucial discriminative information, or captured lower-order features in spatial details. In this paper, a multimodal higher-order statistical adapter (MHoS-Adapter) approach for CLIP is proposed, introducing a visual-branch Higher-Order Statistical Adapter (HS-Adapter) and a textual-branch Parallel Text Adapter (Para-Adapter). To drastically cut down on computational expense and time overhead, both employ adapter tuning, which involves freezing the initial two-branch structure and adjusting just a few parameters. Higher-order statistics of feature maps, including long-term and higher-order data, are modeled by HS-Adapter to improve local detail discrimination and capture long-term dependencies with minimal temporal complexity. Para-Adapter improves text-visual feature alignment by using Video Caption to produce video-level text descriptions for labels. Comprehensive tests on standard benchmarks like Kinetics-400, Mini Kinetics-200, UCF101, and HMDB51 validate the method’s effectiveness and show that MHoS-Adapter outperforms most full fine-tuning approaches, achieving 83.2
High pharmacological doses of zinc oxide (ZnO) are widely used to control post-weaning diarrhea, but environmental pollution and potential adverse effects necessitate the search for effective low-dose alternatives. This study aimed to investigate the effects of dietary level of mechanically activated zinc oxide on growth performance and diarrhea in weaned piglets. A total of 1152 healthy weaned piglets (6.72 ± 0.63 kg) at 21 d of age were randomly assigned to six treatment groups with six replicates of 32 pigs per pen. Piglets received either a basal diet (BD), the BD supplemented with 100, 200, 400, or 600 mg Zn/kg mechanically activated zinc oxide (100 Zn, 200 Zn, 400 Zn, and 600 Zn), or the BD supplemented with 1600 mg Zn/kg conventional ZnO (1600 Zn) over a 28-d feeding period. Compared with the BD group, dietary 400 Zn supplementation significantly increased average daily gain during d 1 to 14, and decreased the feed/gain ratio during d 1 to 14 and d 1 to 28 (P < 0.05). Additionally, dietary 400 Zn supplementation consistently decreased the diarrhea rate regardless of the experimental period (P = 0.001). Notably, these effects are comparable to those observed with 1600 Zn treatment. Moreover, compared to 1600 Zn group, 400 or 600 Zn supplementation significantly reduced interleukin-6 (IL-6) content and diamine oxidase (DAO) activity, as well as significantly increased superoxide dismutase (SOD) activity (P < 0.05) in serum at d 28. Conversely, no significant differences were observed in serum malondialdehyde (MDA) and interleukin-1β (IL-1β) concentrations between the 400 or 600 Zn groups and the 1600 Zn group throughout the experimental period (P > 0.05). Further gut microbiome and serum metabolomic analysis found that the abundances of Lactobacillus, Ligilactobacillus, and Roseburia increased and tryptophan metabolism pathway was enriched by 400 Zn supplementation. Furthermore, the differential metabolites involved in tryptophan metabolism significantly correlated with most of differential genera. In conclusion, dietary supplementation with mechanically activated zinc oxide at 400 mg Zn/kg could exerted a certain positive effect on the growth performance of weaned piglets, which was comparable to or even superior to that of 1600 mg Zn/kg ZnO, indicating that mechanically activated zinc oxide could serve as an effective alternative to high-dose ZnO used in weaned piglets.
Large-scale contrastive vision-language pretraining models have demonstrated impressive zero-shot recognition capabilities by aligning images and text within a shared embedding space. However, transferring such image-text models directly to video action recognition remains challenging due to the inherent differences between static images and dynamic video data. In this study, we propose M2-CLIP++, a multimodal and parameter-efficient video action recognition framework that integrates structured visual modeling with semantically enriched textual prompting. M2-CLIP++ addresses two key limitations of existing adaptation methods: (1) insufficient fine-grained spatiotemporal structural modeling in video encoders, and (2) semantic sparsity caused by using only short category labels as textual supervision. To mitigate these issues, we introduce a global second-order pooling (GSoP) module into the visual branch to capture high-order spatial structures and channel-wise dependencies. Additionally, we design a Vision-Driven Semantic Prompting strategy that utilizes a vision-language model to generate rich descriptive sentences from class labels, which are further refined by a large language model. By jointly enhancing visual and textual representations, M2-CLIP++ strengthens cross-modal semantic alignment and disentangles action-specific features from unrelated contextual information in the latent space. Without updating the CLIP backbone, our framework achieves competitive performance on several video action benchmarks including Kinetics-400, UCF101, and HMDB51, under supervised learning settings, while maintaining low additional parameter overhead and improved interpretability.
Referring Image Segmentation (RIS) requires precise alignment between free-form natural language expressions and fine-grained visual regions in complex scenes. While recent methods improve token-level cross-modal alignment, they often overlook distributional structure shared across modalities, limiting the capture of nuanced semantics and context for disambiguation. To this end, we introduce QHSP-Net, which realizes higher-order cross-modal alignment by injecting two complementary modules, Higher-order Statistical Pooling (HSP) and Query-aware Higher-order Statistical Pooling (QHSP), into frozen vision and text encoders. On the text side, HSP summarizes token distributions via higher-order moments (e.g., variance, skewness, kurtosis), producing distribution-aware query embeddings. On the vision side, QHSP first aligns visual tokens with the expression through lightweight query-aware cross-modal conditioning and then pools higher-order moments over the aligned features to recalibrate spatial tokens. Together, HSP and QHSP couple the two modalities at the distribution level, enabling query-sensitive higher-order reasoning and alignment. The modules can be inserted at multiple layers of frozen encoders, avoiding full model finetuning. Extensive experiments on RefCOCO, RefCOCO+, and G-Ref show consistent gains in oIoU and mIoU over strong baselines, demonstrating superior segmentation accuracy.
Multimodal vision-language contrastive models play a crucial role in video action recognition, thanks to their dual-encoder architecture, which effectively processes both visual and language information. However, these models are often limited to processing a single modality, which restricts the development of multimodal frameworks. Although visual-language guidance frameworks have been proposed, most fail to implement deep cross-modal feature interaction within the backbone network, making it difficult to effectively align information from different modalities in the latent feature space. To address this limitation, we propose a novel Multimodal Backbone Network Guidance Framework (MBGF) that introduces interaction between visual and language modalities within the backbone network, which not only enhances feature representation but also significantly improves the depth of interaction between visual and language modalities. MBGF modifies the traditional dual-encoder structure by connecting the parallel encoding layers of the video and language encoders, forming a unified encoding framework, which is further divided into guided and separation layers. Furthermore, the GE block is introduced in the guided layer to facilitate bidirectional guidance between the video and language modalities through the cross-attention mechanism, effectively enhancing the representation ability of visual and textual features. Through extensive experiments on multiple benchmark datasets, including Kinetics-400, mini Kinetics-200, HMDB51, and UCF101, in both fully supervised and few-shot settings, we demonstrate the effectiveness and practicality of the MBGF framework.
In the task of few-shot action recognition, data scarcity further amplifies the limitations of multimodal temporal feature modeling. Additionally, noise interference in the multimodal fusion process is difficult to effectively filter out, leading to more ambiguous video features under limited sample conditions, thereby affecting recognition accuracy. To address this issue, this paper proposes a visual-language prototype hierarchical temporal enhancement network architecture. This architecture innovatively introduces a shift-and-semantic temporal aggregation block to enhance video-level temporal modeling and alleviate ambiguities in multimodal temporal feature matching. Furthermore, a visual-semantic interaction module is designed to leverage textual semantic information to constrain visual features, guiding the construction of visual prototypes and achieving spatiotemporal semantic collaborative modeling. Extensive experimental evaluations validate the superior performance of the proposed method, fully demonstrating its application value and potential in the field of few-shot action recognition.
The video recognition community is undergoing a significant change in backbone shifting from CNNs to transformers. However, due to the temporal information existing in the video, vision transformers, which have been shown to be effective in image tasks, cannot model spatio-temporal structure in video recognition. In addition, current transformer-based video models exclusively utilize either class tokens or visual tokens for classification, failing to combine the merits of these two types of tokens. To solve the above problems, we propose a novel video second-order transformer network (ViSoT). As there are complex motion and appearance information in video, visual tokens of the last ViSoT layer are aggregated by cross-covariance pooling to model spatio-temporal information, which is combined with class tokens for classification. Meanwhile, a fast singular value power normalization is further introduced to achieve effective aggregation of visual tokens. For temporal modeling, before ViSoT block temporal convolutions are performed continuously in the early stage after the convolution stem via a short-cut connection. In ViSoT, token shift module and space–time attention are proposed for modeling temporal relations and spatio-temporal interaction across adjacent frames respectively. Validating ViSoT on four video benchmarks demonstrates its effectiveness and efficiency, comparable to or better than state-of-the-art methods with fewer parameters and GFLOPs.
Multi-image Super-Resolution (MISR) reconstructs high-resolution images from multiple satellite-acquired low-resolution images, emerging as a key technique in remote sensing. However, image sequences collected by satellites usually have diverse views and extended time spans, making the integration of multiple low-resolution images into a single high-resolution image with intricate details a challenging problem. In this paper, we propose AttMISR, an attention-based multi-image super-resolution network for remote sensing. AttMISR is composed of three key modules: a feature extraction module utilizing residual dynamic convolution blocks, a hybrid non-local feature fusion module, and a multi-attention-based image reconstruction module. The feature extraction module integrates the residual structure with dynamic convolution to efficiently capture complex features and textures in remote sensing images, while the hybrid non-local feature fusion module optimizes feature aggregation across multiple remote sensing images by calculating both cross-correlation and non-cross-correlation features. Moreover, a coordinate-window attention mechanism is proposed to construct the multi-attention-based image reconstruction module, enabling more precise reconstruction. Comprehensive experiments conducted on PROBA-V Kelvin dataset demonstrate the superiority of the proposed method.
Few-shot learning offers a promising solution for computational pathology by alleviating the reliance on large an-notated datasets, but faces challenges from the high redundancy in whole slide images and underutilized cross-modal knowledge. Existing methods typically use foundation models only for pre-liminary feature extraction while employing fixed or single-level prompts that lack multi-scale pathological representation. To address these limitations, we propose a Hierarchical Vision-Text Prompt (H- VTP) framework that enables multi-level cross-modal interaction through GPT-4 generated Local Instance Prompts for patch-level morphological details and Global Semantic Prompts for slide-level diagnostic context. A dual-branch decoding mech-anism with Text-Guided-Patch Decoder and Patch-Augmented-Text Decoder facilitates closed-loop vision-text fusion, while a parameter-efficient adaptation strategy trains only lightweight prompts and adapters. Extensive experiments on three cancer subtype datasets demonstrate the superiority of H-VTP in few-shot WSI classification, confirming its effectiveness for clinical applications.
Thanks to capability to alleviate the cost of large-scale annotation, few-shot action recognition (FSAR) has attracted increased attention of researchers in recent years. Existing FSAR approaches typically neglect the role of individual motion pattern in comparison, and under-explore the feature statistics for video dynamics. Thereby, they struggle to handle the challenging temporal misalignment in video dynamics, particularly by using 2D backbones. To overcome these limitations, this work proposes an adaptively aligned multi-scale second-order moment network, namely A^2M^2-Net, to describe the latent video dynamics with a collection of powerful representation candidates and adaptively align them in an instance-guided manner. To this end, our A^2M^2-Net involves two core components, namely, adaptive alignment (A^2 module) for matching, and multi-scale second-order moment (M^2 block) for strong representation. Specifically, M^2 block develops a collection of semantic second-order descriptors at multiple spatio-temporal scales. Furthermore, A^2 module aims to adaptively select informative candidate descriptors while considering the individual motion pattern. By such means, our A^2M^2-Net is able to handle the challenging temporal misalignment problem by establishing an adaptive alignment protocol for strong representation. Notably, our proposed method generalizes well to various few-shot settings and diverse metrics. The experiments are conducted on five widely used FSAR benchmarks, and the results show our A^2M^2-Net achieves very competitive performance compared to state-of-the-arts, demonstrating its effectiveness and generalization.
BACKGROUND AND OBJECTIVE:Few-shot learning has emerged as a key technological solution to address challenges such as limited data and the difficulty of acquiring annotations in medical image classification. However, relying solely on a single image modality is insufficient to capture conceptual categories. Therefore, medical image classification requires a comprehensive approach to capture conceptual category information that aids in the interpretation of image content. METHODS:This study proposes a novel medical image classification paradigm based on a multi-modal foundation model, called PM2. In addition to the image modality, PM2 introduces supplementary text input (prompt) to further describe images or conceptual categories and facilitate cross-modal few-shot learning. We empirically studied five different prompting schemes under this new paradigm. Furthermore, linear probing in multi-modal models only takes class token as input, ignoring the rich statistical data contained in high-level visual tokens. Therefore, we alternately perform linear classification on the feature distributions of visual tokens and class token. To effectively extract statistical information, we use global covariance pool with efficient matrix power normalization to aggregate the visual tokens. We then combine two classification heads: one for handling image class token and prompt representations encoded by the text encoder, and the other for classifying the feature distributions of visual tokens. RESULTS:Experimental results on three datasets: breast cancer, brain tumor, and diabetic retinopathy demonstrate that PM2 effectively improves the performance of medical image classification. Compared to existing multi-modal models, PM2 achieves state-of-the-art performance. CONCLUSIONS:Integrating text prompts as supplementary samples effectively enhances the model's performance. Additionally, by leveraging second-order features of visual tokens to enrich the category feature space and combining them with class token, the model's representational capacity is significantly strengthened.
Identified and validated BnaA03.XTH4 as a negative regulator of plant height through QTL mapping and RNA-seq, with promoter structural variation influencing expression, providing a foundation for dwarf breeding in B. napus. Rapeseed (Brassica napus L.) is an important oilseed crop that faces yield limitations due to lodging susceptibility. Plant height (PH) significantly affects seed yield and lodging resistance; however, the genetic mechanisms remain largely elusive. In this study, we developed multiple genetic populations derived from a dwarf line DZ and a tall line GW. Through simple sequence repeat mapping and specific-locus amplified fragment sequencing, we identified twenty-three preliminary quantitative trait loci, explaining 0.82
With the objective of investigating the basis of phosphorus (P) utilization efficiency (PUE), physiological and morphological traits, two P-efficient and two P-inefficient rapeseed (Brassica napus L.) cultivars were compared at the seedling stage. P-efficient cultivars showed root morphological adaptation, high P uptake activity, and greater phospholipid degradation under low P stress. Improving root morphological adaptation and reducing lipid-P allocation could allow increasing PUE in rapeseed seedlings.
High-quality genome information is essential for efficiently deciphering and improving crop traits. Here, we report a highly contiguous and accurate hexaploid genome assembly for the key wheat breeding parent Zhou8425B, an elite 1BL/1RS translocation line with durable adult plant resistance (APR) against yellow rust (YR) disease. By integrating HiFi and Hi-C sequencing reads, we have generated a 14.75-Gb genome assembly for Zhou8425B with a contig N50 of 70.94 and a scaffold N50 of 735.11 Mb. Comparisons with previously sequenced common wheat cultivars shed light on structural changes in the 1RS chromosome arm, which has been extensively used in wheat improvement. Interestingly, Zhou8425B 1RS carries more genes encoding AP2/ERF-ERF or B3 transcription factors than its counterparts in four previously sequenced wheat and rye genotypes. The Zhou8425B genome assembly aided in the fine mapping of a new APR locus (YrZH3BS) that confers resistance to YR disease and promotes grain yield under field conditions. Notably, pyramiding YrZH3BS with two previously characterized APR loci (YrZH22 and YrZH84) can further reduce YR severity and enhance grain yield, with the triple combination (YrZH3B + YrZH22 + YrZH84) having the greatest effect. Finally, the founder genotype effects of Zhou8425B were explored using publicly available genome resequencing data, which reveals the presence of important Zhou8425B genomic blocks in its derivative cultivars. Our data demonstrate the value of the Zhou8425B genome assembly for further study of the structural and functional characteristics of 1RS, the genetic basis of durable YR resistance, and founder genotype effects in wheat breeding. Our resources will facilitate the development of elite wheat cultivars through genomics-assisted breeding.
Sound source localization in complex acoustic environments is a challenging task. The spectrogram contains rich sound features, including the distribution of frequency components, intensity variations, and harmonic structures, which are crucial for accurate sound source localization. In this paper, we design a multi-scale feature aggregation network (MFAnet) to effectively utilize the sound features in the spectrogram for accurate sound source localization. Specifically, the multi-scale features are extracted by combining the Res2Net architecture with dynamic convolution to capture the rich contextual dependencies across multiple layers. Moreover, this network incorporates a dual attention mechanism, including both spatial and channel attention modules, in the feature extraction stage to help the network focus on the most discriminative parts of the features. Finally, we use the multi-output regression to estimate the 3D coordinates of sound sources. Experimental results indicate that the proposed MFAnet performs well in complex acoustic environments, especially in the case of overlapping sound sources conditions, and is significantly outperform the existing methods. Additionally, we conduct ablation experiments to verify the effectiveness of the proposed method.