While multimodal fusion with injective embedding has shown significant effectiveness in multimodal sentiment analysis, it is constrained by the requirement of a single-point embedding and the intrinsic ambiguity within multimodal signals, limiting its practical applications. In this paper, we propose Prompt-based Distribution Learning (PDL), a novel divide-and-conquer framework designed for multimodal sentiment analysis. PDL aims to enhance the representation of multimodal information by jointly performing prompt tuning and sentiment distribution learning. Specifically, PDL utilizes modality-aware prompts as probes to decompose multimodal features and mine the latent sentiment distributions for each modality, thereby capturing a wide range of sentiment information. We conduct extensive experiments on three benchmark datasets: CMU-MOSI, CMU-MOSEI, and CH-SIMS, to evaluate PDL. The experimental results validate the efficacy of PDL, showing that PDL outperforms existing approaches in capturing and representing the complex sentiment information present in multimodal data. Code is available at https://github.com/Dongri404/PDL.

Multimodal recommender systems enhance recommendation accuracy by leveraging multimodal item information together with users' interaction histories. Existing methods mainly use item-item semantic graphs and user-item interaction graphs to model item and user preferences. However, in item modeling, existing methods often overlook collaborative relationships between items, which makes it difficult to capture comprehensive item features. On the interaction side, they treat all interactions as equally reliable, overlooking differences across behavior types and their varying importance to users. To address these issues, we propose Noise-Robust Item Modeling and Dynamic Multiview Contrastive Learning (NRDMC), a novel method for multimodal recommendation. First, we extract latent collaborative signals between items from user-item interactions to enhance comprehensive modeling of item features. Second, we design intra-modal and cross-modal attention mechanisms. The intra-modal attention suppresses noise within each modality, while the cross-modal attention exploits complementary information across modalities. Finally, NRDMC adopts dynamic multi-view contrastive learning to capture latent behavior prototypes and their importance in user-item interactions. This enables the model to construct higher-quality contrastive views and improve recommendation performance. To ensure fair comparison, we evaluate NRDMC on three public Amazon Reviews benchmark datasets (Baby, Sports, and Clothing) under standard settings. Experimental results show that NRDMC consistently outperforms strong baselines, achieving average performance gains of 3.49%, 7.54%, and 1.64% on Baby, Sports, and Clothing, respectively. Our code is available at https://github.com/50hehe/NRDMC.

Knowledge-Editing-based (KE-based) detoxification has emerged as a promising approach for mitigating harmful behaviours in Large Language Models. Existing evaluations, however, largely rely on automatic toxicity classifiers, implicitly assuming that reduced toxicity scores reflect genuine behavioural suppression. In this work, we propose a robustness-oriented evaluation framework for KE-based detoxification that examines its reliability beyond standard classifier-based metrics along three dimensions: optimisation robustness, compositional robustness, and cross-lingual robustness. We identify pseudo-detoxification as a common failure mode, where apparent toxicity reductions arise from degenerate generation behaviours rather than meaningful suppression of unsafe content. We further show that detoxification effectiveness degrades when multiple unsafe behaviours are edited jointly, and that both monolingual and cross-lingual detoxification remain effective only under specific model-method combinations. Overall, our results indicate that KE-based detoxification is robust only for certain models, limited numbers of detoxification objectives, and a subset of languages.

The proliferation of misinformation on social networks poses an acute threat to public trust and safety. Although large language models (LLMs) have shown promise in detecting false claims, they falter when confronted with rapidly emerging information because they cannot reliably gauge the boundaries of their own knowledge, i.e., when to trust their parametric memory and when to seek external evidence. We present SWIFT, a novel lightweight framework that equips LLMs with a self-reflective knowledge switch. SWIFT introduces sparse, trainable activations on knowledge-critical layers that learn to emit a calibrated confidence score; when confidence falls below a learned threshold, the model automatically queries a retrieval module for up-to-date evidence. Extensive experiments on five challenging, time-varying benchmarks show that SWIFT improves the F1 score by 40 points over the baseline while cutting the inference-time cost by 90%.

Multimodal aspect-based sentiment classification (MABSC) aims to identify the sentiment polarity toward specific aspects in multimodal data. It has gained significant attention with the increasing use of social media platforms. Existing approaches primarily focus on analyzing the content of posts to predict sentiment. However, they often struggle with limited contextual information inherent in social media posts, hindering accurate sentiment detection. To overcome this issue, we propose a novel multimodal dual cause analysis (MDCA) method to track the underlying causes behind expressed sentiments. MDCA can provide additional reasoning cause (RC) and direct cause (DC) to explain why users express certain emotions, thus helping improve the accuracy of sentiment prediction. To develop a model with MDCA, we construct MABSC datasets with RC and DC by utilizing large language models (LLMs) and visual-language models. Subsequently, we devise a multitask learning framework that leverages the datasets with cause data to train a small generative model, which can generate RC and DC, and predict the sentiment assisted by these causes. Experimental results on MABSC benchmark datasets demonstrate that our MDCA model achieves the state-of-the-art performance, and the small fine-tuned model exhibits superior adaptability to MABSC compared to large models like ChatGPT and BLIP-2.

Influenza-like illness (ILI) forecasting is crucial for effective public health intervention, but existing models often fail to capture the complex temporal and semantic patterns inherent in epidemic data. Traditional statistical techniques and even advanced deep learning methods predominantly leverage numerical time series data, thereby overlooking contextual medical and epidemiological insights that could enhance the performance. Recent progress in large language models (LLMs) has illustrated their exceptional effectiveness in integrating semantic understanding into natural language processing tasks within the medical context. Motivated by these developments, we propose SMILE (Semantic Multi-scale Integration and LLM-Enhanced network), a novel multi-modal forecasting framework designed to integrate LLM-derived semantic features with multi-scale temporal analysis. Built upon TimeMixer architecture, SMILE introduces an automatic semantic feature extraction system using LLMs, adaptive fusion mechanisms for integrating textual and temporal data, and demonstrates robust performance improvements. Extensive experiments on ILI and benchmark datasets confirm that SMILE significantly outperforms state-of-the-art forecasting methods, highlighting the value of incorporating semantic context into time series disease prediction.

The rapid growth in the parameter size of Large Language Models (LLMs) has led to the development of Parameter-Efficient Fine-Tuning (PEFT) methods to alleviate the computational costs of fine-tuning. Among these, Fisher Induced Sparse uncHanging (FISH) Mask is a selection-based PEFT technique that identifies a subset of pre-trained parameters for fine-tuning based on approximate Fisher information. However, the integration of FISH Mask with other PEFT methods, such as LoRA and Adapters, remains underexplored. In this paper, we propose FISH-Tuning, a novel approach that incorporates FISH Mask into addition-based and reparameterization-based PEFT methods, including LoRA, Adapters, and their variants. By leveraging Fisher information to select critical parameters within these methods, FISH-Tuning achieves superior performance without additional memory overhead or inference latency. Experimental results across various datasets and pre-trained models demonstrate that FISH-Tuning consistently outperforms the vanilla PEFT methods with the same proportion of trainable parameters.

Aspect-Based Sentiment Analysis (ABSA) aims to analyze sentiment toward specific aspect terms, requiring fine-grained understanding of both aspect semantics and sentiment polarity. However, existing datasets are often limited in size, expression diversity, and interpretability, causing models to rely on superficial input-label mappings and struggle with nuanced sentiment. To address these challenges, we propose Multi-faceted Data Augmentation (MDA), a framework that leverages large language models (LLMs) to enrich ABSA training data. MDA integrates paraphrasing-based data expansion, dual confidence filtering, aspect clarification, and sentiment reasoning generation into a collaborative strategy, enhancing data diversity, semantic depth, and logical consistency. Furthermore, by reformulating ABSA as a generation-based task, MDA enables models to jointly predict sentiment polarity and generate reasoning statements, fostering a deeper understanding of the underlying causes behind sentiment expressions. Extensive experiments on benchmark datasets demonstrate that MDA outperforms data augmentation-based baselines, and improves model performance, robustness.

With the exponential growth of video content, aiming at localizing relevant video moments based on natural language queries, video moment retrieval (VMR) has gained significant attention. Existing weakly supervised VMR methods focus on designing various feature modeling and modal interaction modules to alleviate the reliance on precise temporal annotations. However, these methods have poor generalization capabilities on compositional queries with novel syntactic structures or vocabulary in real-world scenarios. To this end, we propose a new task: weakly supervised compositional moment retrieval (WSCMR). This task trains models using only video-query pairs without precise temporal annotations, while enabling generalization to complex compositional queries. Furthermore, a proposal-centric network (PC-Net) is proposed to tackle this challenging task. First, video and query features are extracted through frozen feature extractors, followed by modality interaction to obtain multimodal features. Second, to handle compositional queries with explicit temporal associations, a dual-granularity proposal generator decodes multimodal global and frame-level features to obtain query-relevant proposal boundaries with fine-grained temporal perception. Third, to improve the discrimination of proposal features, a proposal feature aggregator is constructed to conduct semantic alignment of frames and queries, and employ a learnable peak-aware Gaussian distributor to fit the frame weights within the proposals to derive proposal features from the video frame features. Finally, the proposal quality is assessed based on the results of reconstructing the masked query using the obtained proposal features. To further enhance the model's ability to capture semantic associations between proposals and queries, a quality margin regularizer is constructed to dynamically stratify proposals into high and low query-relevance subsets and enhance the association between queries and common elements within proposals, and suppress spurious correlations via inter-subset contrastive learning. Notably, PC-Net achieves superior performance with 54\% fewer parameters than prior works by parameter-efficient design. Experiments on Charades-CG and ActivityNet-CG demonstrate PC-Net’s ability to generalize across diverse compositional queries. Code is available at https://github.com/mingyao1120/PC-Net.

Few-shot multi-intent spoken language understanding (SLU) aims to identify users’ multiple intents and key slots using a tiny amount of annotated data. Recent advances in large language models (LLMs) have utilized instruction learning frameworks to model intent-slot interdependencies, typically requiring abundant data for effective training. However, in few-shot scenarios, these frameworks face challenges such as mismatches between the number of generated slots and input lengths, relational confusion in multi-intent scenarios and neglect of task-specific variations in intent counts across utterances. To overcome the challenges, we propose PICD-Instruct, a novel generative framework based on Basic Instructions (BI), Pairwise Interaction Instructions (PII) and Contrastive Distinct Instructions (CDI). Specifically, BI directs LLMs to generate entities along with associated words, thereby mitigating mismatches in quantitative correspondences. PII explicitly captures dual-task interdependencies by guiding LLMs to pair each intent with its related entities. CDI enhances understanding of utterances by guiding LLMs to determine whether two utterances share the same intent count. Experimental results on public datasets indicate that PICD-Instruct achieves state-of-the-art performance.

This paper investigates a new task, Weakly- supervised Group Activity Recognition in Still-images (WGARS), which aims to extend the applicability of Group Activity Recognition (GAR) to broader scenarios, such as low-latency domains. To tackle this challenge, we propose a Spatial Dual Context Transformer (SDCT), comprising a Dual Context Encoder (DCE) and a Dual Context Decoder (DCD). The DCE module individually encodes holistic context with integral relations of overall actors, and encodes partial context with individual features in still images. Subsequently, the DCD module explores the complementarity between holistic and partial contexts, and alternatively updates these encoded contexts to enhance the interaction of actors. Additionally, auxiliary supervised contrastive learning is incorporated to mitigate activity confusion. The proposed SDCT attains state-of-the-art performance on Volleyball and NBA datasets in WGARS. Notably, SDCT even outperforms recent methods when extended to the weakly-supervised GAR in videos task on Volleyball dataset.

Chinese Spell Checking (CSC) is a widely used technology, which plays a vital role in speech to text (STT) and optical character recognition (OCR). Most of the existing CSC approaches relying on BERT architecture achieve excellent performance. However, limited by the scale of the foundation model, BERT-based method does not work well in few-shot scenarios, showing certain limitations in practical applications. In this paper, we explore using an in-context learning method named RS-LLM (Rich Semantic based LLMs) to introduce large language models (LLMs) as the foundation model. Besides, we study the impact of introducing various Chinese rich semantic information in our framework. We found that by introducing a small number of specific Chinese rich semantic structures, LLMs achieve better performance than the BERT-based model on few-shot CSC task. Furthermore, we conduct experiments on multiple datasets, and the experimental results verified the superiority of our proposed framework.

Fine-tuning all parameters of Large Language Models (LLMs) is computationally expensive. Parameter-Efficient Fine-Tuning (PEFT) methods address this by selectively fine-tuning specific parameters. Most of the parameter efficient fine-tuning (PEFT) methods center on selecting or introducing a set of parameters to be fine-tuned. However, there are few methods that consider the impact of data samples on parameter selecting. Representative data driven methods include FISH Mask based method, which randomly selects a portion of data samples as a basis when selecting parameters. However, this random data sample selection method cannot select optimal parameters for unstable data distribution. In this work, we introduce a data-centric approach and propose the Iterative Range Decreasing (IRD) algorithm to optimize the sample-parameter pair selection in FISH Mask. IRD iteratively refines the selection by identifying subsets of samples and parameters exhibiting higher Fisher information. We demonstrate the effectiveness and rationality of proposed strategy by conducting experiments on GLUE benchmark. Experimental results show our strategy optimizes the parameter selection and achieves preferable performance over some typical baseline methods.

Recently, weakly supervised temporal sentence grounding in videos (TSGV) has attracted extensive attention because it does not require precise start-end time annotations during training, and it can quickly retrieve interesting segments according to user needs. In weakly supervised TSGV, query reconstruction (QR)-based methods are the current mainstream, and the quality of proposals determines their performance. QR-based methods have two problems in proposal quality. First, a multi-modal global token is usually mapped to proposals with limited duration diversity, making it difficult to capture relevant segments at varying durations in real scenarios. Additionally, Gaussian functions are typically used to generate relatively fixed weights for frames within proposals, which weigh the original video features to generate proposal-specific features. This results in query-irrelevant frames affecting the discrimination of the proposal features. In this study, we propose a query-aware multi-scale proposal network (QMN). Initially, pre-trained encoders are used to extract video and query features. Subsequently, a multi-scale proposal generation module is designed to refine video features guided by queries and diversify the duration of the proposal. This module performs multi-modal interaction and multi-scale modeling to obtain proposals of different durations. Furthermore, to extract discriminative proposal features and enhance the modeling of proposal frame correlation, a query-aware weight generator is constructed to learn frame weights to suppress query-irrelevant frame representations through contrastive learning. Finally, the masked query is reconstructed using the proposal features to select the best proposal. The effectiveness of the proposed QMN is verified through experiments on the Charades-STA and ActivityNet-Captions datasets.

Commonsense question answering (CQA) requires understanding and reasoning over QA context and related commonsense knowledge, such as a structured Knowledge Graph (KG). Existing studies combine language models and graph neural networks to model inference. However, traditional knowledge graph are mostly concept-based, ignoring direct path evidence necessary for accurate reasoning. In this paper, we propose MRGNN (Meta-path Reasoning Graph Neural Network), a novel model that comprehensively captures sequential semantic information from concepts and paths. In MRGNN, meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously. We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets, showing the effectiveness of MRGNN. Also, we conduct further ablation experiments and explain the reasoning behavior through the case study.

Ancient Chinese poetry (ACP) is a vital component of Chinese traditional culture. Enhancing the performance of related downstream tasks demands the development of high-quality pre-trained language models (PLMs) dedicated to ACP. Notably, the semantics of ACP significantly differ from modern Chinese. Existing PLMs have limited knowledge of ACP and are inadequately aligned with the semantic space of modern Chinese, which constrains the utility for tasks related to ACP. In this paper, we propose a fine-tuning strategy to establish a precise alignment between ACP and modern Chinese semantics on sentence level. This strategy involves the inclusion of corresponding modern Chinese translations alongside original ancient poems, creating a hybrid corpus. This corpus facilitates a more effective transfer of knowledge from existing PLMs to the domain of ACP. Furthermore, we employ a training strategy based on a glyph-based foundational PLM, enabling meticulous fine-tuning. Consequently, we develop a specialized PLM named CP-ChineseBERT. To evaluate the effectiveness of our proposed strategies, we conducted experiments on two real-world datasets, focusing on tasks related to ACP sentiment classification and ACP title prediction. The experimental results demonstrate the significant improvements in performance achieved through our innovative approaches.

Spoken language understanding (SLU) is an essential part of a task-oriented dialogue system, which mainly includes intent detection and slot filling. Some existing approaches obtain enhanced semantic representation by establishing the correlation between two tasks. However, those methods show little improvement when applied to BERT, since BERT has learned rich semantic features. In this paper, we propose a BERT-based model with the probability-aware gate mechanism, called PAGM ( P robability A ware G ated M odel). PAGM aims to learn the correlation between intent and slot from the perspective of probability distribution, which explicitly utilizes intent information to guide slot filling. Besides, in order to efficiently incorporate BERT with the probability-aware gate, we design the stacked fine-tuning strategy. This approach introduces a mid-stage before target model training, which enables BERT to get better initialization for final training. Experiments show that PAGM achieves significant improvement on two benchmark datasets, and outperforms the previous state-of-the-art results.

Identifying rumor sources in social networks is one of the key tasks for defeating rumors automatically. Many efforts have been devoted to locating rumor sources with an assumption that the infected status of each node is known in advance, while other efforts focus on identifying sources with partial infection knowledge, such as wavefront, sparse observers, and snapshots. Wavefront is a set of nodes that are infected at the latest propagation in social networks, which is originally defined for analyzing the SARS epidemic, and shows considerable importance in information source locating task. However, only a few studies are proposed to solve the multiple rumor source detection (MRSD) problem by using wavefront. In this paper, we propose a sequence-to-sequence model, called Graph Constraint based Sequential Source Identification (GCSSI), which takes wavefront as input to solve the MRSD problem. By adopting encoder-decoder structure and graph constraint based multi-task learning, GCSSI estimates the reverse rumor dissemination at each time step and predicts sources in an end-to-end way. We conduct experiments on several real datasets and the experimental results show the superiority of our model compared with existing work.

Due to the distributed and decentralized nature of social media, respective content that contains misinformation is usually propagated without any type of moderation, which may mislead the public and have a profound real-world impact. In addition, it is quite challenging to distinguish misinformation with high precision, since the content is often short and lacks of semantics. A promising solution is to utilize the crowdsourcing wisdom that pushes the suspected misinformation to relevant users based on the expertise and collects the assessments to judge the credibility. Even though a lot of expert finding models have been employed, however, these methods cannot effectively deal with the misinformation-oriented expert matching tasks since the data collected from social network is different form traditional text collection. To this end, we focus on how to obtain an appropriate matching between the suspect misinformation and corresponding experts, and propose a multi-topic expert finding method, called LTM (List based Topic Model), to sufficiently utilize crowdsourcing wisdom. Moreover, we optimize the query results with the help of supervised information that extracted from Twitter Lists . Finally, we demonstrate the effectiveness of our work with experiments on real-world data and verify the superiority of our proposed model in accuracy.