Traditional speaker diarization systems have primarily focused on constrained scenarios such as meetings and interviews, where the number of speakers is limited and acoustic conditions are relatively clean. To explore open-world speaker diarization, we extend this task to the visual media domain, encompassing complex audiovisual programs such as films and TV series. This new setting introduces several challenges, including long-form video understanding, a large number of speakers, cross-modal asynchrony between audio and visual cues, and uncontrolled in-the-wild variability. To address these challenges, we propose Cinematic Speaker Registration & Diarization (CineSRD), a unified multimodal framework that leverages visual, acoustic, and linguistic cues from video, speech, and subtitles for speaker annotation. CineSRD first performs visual anchor clustering to register initial speakers and then integrates an audio language model for speaker turn detection, refining annotations and supplementing unregistered off-screen speakers. Furthermore, we construct and release a dedicated speaker diarization benchmark for visual media that includes Chinese and English programs. Experimental results demonstrate that CineSRD achieves superior performance on the proposed benchmark and competitive results on conventional datasets, validating its robustness and generalizability in open-world visual media settings.

With the continuous advancement of reasoning abilities in Large Language Models (LLMs), their application to scientific reasoning tasks has gained significant research attention. Current research primarily emphasizes boosting LLMs' performances on scientific QA benchmarks by training on larger, more comprehensive datasets with extended reasoning chains. However, these approaches neglect the essence of scientific reasoning process -- logicality, which is the rational foundation to ensure the validity of reasoning steps leading to reliable conclusions. In this work, we make the first systematic investigation into the internal logicality underlying LLM scientific reasoning, and develop a scientific logicality enriched methodology, including a set of assessment criteria and data sampling methods for logicality-guided training, to improve the logical faithfulness as well as task performance. Further, we take physics, characterized by its diverse logical structures and formalisms, as an exemplar discipline to practise the above methodology. For data construction, we extract scientific problems from academic literature and sample a high-quality dataset exhibiting strong logicality. Experiments based on three different backbone LLMs reveal that: 1) the training data we constructed can effectively improve the scientific logicality in LLM reasoning; and 2) the enriched scientific logicality plays a critical role in solving scientific problems.

Reinforcement learning (RL) with verifiable rewards has recently catalyzed a wave of “MLLM-r1” approaches that bring RL to vision language models. Most representative paradigms begin with a cold start, typically employing supervised fine-tuning (SFT), to initialize the policy before RL. However, SFT-based cold start adopts the reasoning paradigm intertwined with task solution and output format, which may induce instruction-style overfitting, weakens out-of-distribution generalization, and ultimately affects downstream RL. We revisit the cold start along two views, its training method and data construction, and introduce the Generalization Factor (GF) coefficient to quantify the generalization capability under different methods. Our empirical study finds that preference–based training methods (e.g. DPO) generalizes better than SFT-based methods in cold start. Motivated by this, we propose $\textbf{SPECS}$—a $\textbf{S}$elf-distilled, $\textbf{P}$r$\textbf{e}$ference-based $\textbf{C}$old $\textbf{S}$tart framework that decouples multimodal learning: (1) generates introspective preference data pairs via self-distillation, avoiding reliance on larger teachers or manual annotation; (2) performs preference–based training to learn, focusing on shallow, transferable surface-form criteria (format, structure, style) rather than memorizing content; and (3) hands off to RL with verifiable rewards for deep reasoning results. Experimental results across multiple multimodal benchmarks show that our decoupling learning framework yields consistent performance gains over strong baselines, improving MEGA-Bench by 4.1\% and MathVista by 12.2\%. Additional experiments indicate that SPECS contributes to reducing in-distribution “stuckness,” improving exploration, stabilizing training, and raising the performance ceiling.

Retrieval-augmented generation (RAG) connects large language models (LLMs) to external knowledge, but single-round retrieval is often insufficient for complex multi-hop questions. To enhance search capabilities for complex tasks, most existing works integrate multi-round iterative retrieval with reasoning processes via end-to-end training. While these approaches improve problem-solving performance, they still face challenges in task reasoning and model training, especially ambiguous retrieval execution paths and sparse rewards in end-to-end reinforcement learning (RL), which can lead to inaccurate retrieval results and lower performance. We attribute these failures to hierarchical credit entanglement: a single final reward updates planning and execution together, so the model cannot clearly separate plan errors from retrieval errors. We propose APEX-Searcher, which uses a Refining Credit Assignment paradigm: planning is optimized by RL with a plan-level reward, while execution is learned by SFT. Extensive experiments show consistent gains in both multi-hop RAG and task planning across benchmarks.

Deep hashing has emerged as an effective and widely adopted framework for similarity retrieval, owing to its computational efficiency and the powerful feature extraction capabilities of deep learning (DL). Despite their strong performance in multi-modal and high-dimensional data retrieval tasks, recent studies have exposed the susceptibility of DL models to adversarial attacks, including in retrieval scenarios. This underscores the critical need for enhancing robustness in DL models to ensure reliable inference. However, most adversarial robustness studies focus on classification tasks, where explicit labels facilitate supervised adversarial training. In contrast, retrieval tasks typically rely on similarity matrices rather than explicit labels, making direct adversarial optimization challenging and limiting its application in large-scale retrieval settings. To address this gap, we propose Deep Supervised Adversarial Robust Hashing (DSARH), an end-to-end framework that leverages similarity matrices and learnable hash codes to construct gradient-based worst-case perturbations, enabling efficient adversarial training and robust feature learning for retrieval. Extensive experiments on cross-modal and image retrieval tasks demonstrate that existing deep hashing models are highly vulnerable to adversarial perturbations, whereas DSARH achieves superior robust generalization across a wide range of adversarial scenarios. Moreover, the robust visual features learned by DSARH help mitigate modality heterogeneity, resulting in consistent improvements in both standard and adversarial performance across multiple image-text retrieval benchmarks compared to state-of-the-art baselines. These results highlight the critical role of adversarial robustness in developing reliable and effective multi-modal retrieval systems.

While Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective for closed-ended tasks, extending it to open-ended social language games via self-play reveals a critical issue: evolution impasse. Due to the vast strategy space, language agents frequently converge to homogenized behaviors, leading to deterministic match outcomes that eliminate the gradient signals necessary for policy evolution. To tackle this issue, we propose Dual-scale Evolutionary Policy Training (DEPT) for social language games. DEPT introduces a time-scaled evolutionary perception mechanism that detects impasse by quantifying dual-scale value baseline divergence alongside match entropy. Upon perceiving the collapse, it then activates asymmetric advantage reshaping to dynamically modulate the optimization landscape for intervention. Thus, our method effectively restores gradient signals and enforces sustained strategic exploration. Extensive experiments on multiple social language games demonstrate that DEPT outperforms strong baselines, avoiding policy degeneration and driving the continuous evolution of social language agents.

Interlingual subtitling, which translates subtitles of visual media into a target language, is essential for entertainment localization but has not yet been explored in machine translation. Although Large Language Models (LLMs) have significantly advanced the general capabilities of machine translation, the distinctive characteristics of subtitle texts pose persistent challenges in interlingual subtitling, particularly regarding semantic coherence, pronoun and terminology translation, and translation expressiveness. To address these issues, we present Hermes, an LLM-based automated subtitling framework. Hermes integrates three modules: Speaker Diarization, Terminology Identification, and Expressiveness Enhancement, which effectively tackle the above challenges. Experiments demonstrate that Hermes achieves state-of-the-art diarization performance and generates expressive, contextually coherent translations, thereby advancing research in interlingual subtitling.

Effective social intelligence simulation requires language agents to dynamically adjust reasoning depth, a capability notably absent in current studies. Existing methods either lack explicit reasoning or employ lengthy Chain-of-Thought reasoning uniformly across all scenarios, resulting in excessive token usage and inflexible social behaviors in tasks such as negotiation or collaboration. To address this, we propose an $\textbf{A}$daptive $\textbf{S}$ocial $\textbf{L}$earning ($\textbf{ASL}$) framework in this paper, aiming to improve the adaptive reasoning ability of language agents in dynamic social interactions. To this end, we first identify the hierarchical reasoning modes under such context, ranging from intuitive response to deep deliberation based on the cognitive control theory. We then develop the $\textbf{A}$daptive $\textbf{M}$ode $\textbf{P}$olicy $\textbf{O}$ptimization ($\textbf{AMPO}$) algorithm to learn the context-aware mode adaptation and reasoning. Our framework advances existing research in three key aspects: (1) Multi-granular reasoning mode design, (2) Context-aware mode switching in rich social interaction, and (3) Token-efficient reasoning with depth adaptation. Extensive experiments on the benchmark social intelligence environment verify that ASL achieves 15.6\% higher task performance than GPT-4o. Notably, our AMPO outperforms GRPO by 7.0\% with 32.8\% shorter thinking chains, demonstrating the advantages of our AMPO and the learned adaptive reasoning ability over GRPO's solution.

Mobile GUI agents have shown strong potential in real-world automation and practical applications. However, most existing agents remain reactive, making decisions mainly from current screen, which limits their performance on long-horizon tasks. Building a world model from repeated interactions enables forecasting action outcomes and supports better decision making for mobile GUI agents. This is challenging because the model must predict post-action states with spatial awareness while remaining efficient enough for practical deployment. In this paper, we propose MobileDreamer, an efficient world-model-based lookahead framework to equip the GUI agents based on the future imagination provided by the world model. It consists of textual sketch world model and rollout imagination for GUI agent. Textual sketch world model forecasts post-action states through a learning process to transform digital images into key task-related sketches, and designs a novel order-invariant learning strategy to preserve the spatial information of GUI elements. The rollout imagination strategy for GUI agent optimizes the action-selection process by leveraging the prediction capability of world model. Experiments on Android World show that MobileDreamer achieves state-of-the-art performance and improves task success by 5.25

Neurosymbolic frameworks have become mainstream paradigms in intelligent system design. For intelligent decision making, reinforcement learning (RL) and automated planning are the representative neural and symbolic techniques, respectively, which can facilitate each other. Despite the rapid development and wide applications of deep RL (DRL), its drawbacks on sample efficiency and convergence in sparse-reward environments have become the major obstacles hindering its advancement. To address these issues, in this article, we propose a neurosymbolic framework that integrates DRL with hierarchical plans. Specifically, we develop a selective Monte Carlo tree search algorithm, in which hierarchical plans are dynamically constructed during the learning process. The constructed plans in turn provide high-level guidance for RL to constrain the subtasks, leading to goal attainment, thus reducing useless/redundant exploration in RL. Experiments on five challenging scenarios show that our framework achieves better sample efficiency and faster convergence compared to the state-of-the-art approaches.

Video Moment Retrieval (VMR) aims to localize a video segment that corresponds to a text query, which is an important task in multi-modal information retrieval. Recently, to better address the practical considerations in real-world scenarios, this task has been extended to the weakly supervised setting, which utilizes simple videotext pairs rather than specific temporal boundary annotations. The main challenge of this task lies in the temporal and semantic discrepancies between different modalities. Existing models are typically built on video and text content, often neglecting audio as an important intermediate modality to enhance cross-modal alignment. To address these limitations, we incorporate the audio information for the first time in weakly supervised VMR and propose a novel audio-augmented fusion model, aiming to enhance temporal consistency and semantic association. To achieve this, we design an audio-mediated feature interactor to facilitate tri-modality feature interactions, and a multi-source proposal generator to select highly-relevant video clips. Additionally, we adopt a diffusion model-based reconstructor to improve video-text alignment. We evaluate our model on the Charades-STA and ActivityNet Captions benchmark datasets. The experimental results show that our model achieves the state-of-the-art performance, indicating the importance of the audio modality in the weakly supervised VMR task.

Although fine-tuning pre-trained language models (PLMs) on downstream tasks has become standard practice, this paradigm is oversensitive to specific noises, making it brittle under adversarial attacks. To alleviate this adversarial vulnerability problem, in this paper, we propose a novel in-context interaction learning framework to enhance the utilization of PLMs by capturing the interaction between instances. Our method first converts text-based downstream tasks to the pretraining task, thus leveraging the pre-trained task layer of PLMs to aggregate task information. It then develops the in-context interaction module to make predictions by learning the interaction relations based on the current instance and the context constructed with other instances. Compared to the typical incontext learning approach, our framework can effectively learn the interaction relations between instances to better leverage the in-context ability of PLMs and enhance model robustness. We conduct extensive experiments on multiple downstream tasks in the adversarial robustness benchmark. Experimental results demonstrate that our framework consistently outperforms both adversarial training and robust fine-tuning methods, verifying the effectiveness of our proposed framework.

Argument generation is a core research task in social media computing and can facilitate multiple securityrelated applications. It aims to automatically generate persuasive and coherent arguments that express supportive or opposite stance towards a given claim. Previous research has mainly emphasized the issues such as aspect control and factual accuracy, largely overlooking the diversity issue in argument generation. Generating diverse arguments is essential to helping users articulate a wide spectrum of personal opinions. With the rapid development of Large Language Models (LLMs), computational argumentation has emerged as an important means to promoting positive online interactions and fostering active community communications among participants. However, it also brings the risk of producing harmful contents in the meanwhile. Thus, generating diverse arguments under ethical considerations is another important issue. In this paper, we propose a Diverse Argument Generation (DAG) framework to address these issues. Our DAG integrates safety-aware supervised fine-tuning and diversity-driven data construction with theme-level preference optimization, which encourages claim-stance relevant and ethics-aware argument generation with thematic and semantic diversity. Specifically, DAG first fine-tunes a model for argument generation and uses the fine-tuned model to generate arguments, both aligning with ethical constraints. It then constructs paired preference data guided with thematic relevance and semantic diversity measures via theme modeling, which are finally utilized by themelevel preference optimization for enhancing diverse argument generation. We conduct experiments to verify the effectiveness of our framework and the results show that DAG achieves better performances in generating diverse and controllable arguments compared to baselines, while maintaining comparable performance in text quality.

Video Moment Retrieval (VMR) aims to temporally localize a query-specified moment in untrimmed videos. Surveillance videos are crucial and indispensable for public security, and an automatic understanding of surveillance video content is crucial to enhance the existing investigative measures. Although existing VMR methods work reasonably well for conventional activity videos, they face particular challenges for videos in security domains, which are often of low quality and involve specific event-driven human behaviors. Specifically, existing methods are unable to capture precise foreground relevant information in noisy frames, and the over-reliance on word-level textual query features fails to model hierarchical semantics, hindering alignment with complex multi-event videos. To address these challenges, we propose an Event-driven Localization with Foreground-enhanced Representation (ELFR) framework, which consists of two key components: 1) in order to suppress background noise and enhance the saliency of foreground elements, the Foreground-Enhanced Representation (FER) module refines cross-modal alignment using spatio-temporal cross-modal attention; 2) the Event-Driven Localization (EDL) module extracts event-level semantic units and integrates these event features with enhanced visual representations to generate boundary predictions. Experiments conducted on surveillance-focused UCA datasets demonstrate that our proposed method has achieved state-of-the-art performances.

Large language models (LLMs) have made dialogue one of the central modes of human-machine interaction, leading to the accumulation of vast amounts of conversation logs and increasing demand for dialogue generation. A conversational life-cycle spans from the Prelude through the Interlocution to the Epilogue, encompassing various elements. Despite the existence of numerous dialogue-related studies, there is a lack of benchmarks that encompass comprehensive dialogue elements, hindering precise modeling and systematic evaluation. To bridge this gap, we introduce an innovative research task Dialogue Element MOdeling, including Element Awareness and Dialogue Agent Interaction, and propose a novel benchmark, DEMO, designed for a comprehensive dialogue modeling and assessment. Inspired by imitation learning, we further build the agent which possesses the adept ability to model dialogue elements based on the DEMO benchmark. Extensive experiments indicate that existing LLMs still exhibit considerable potential for enhancement, and our DEMO agent has superior performance in both in-domain and out-of-domain tasks.

Graph representation learning is a fundamental research theme and can be generalized to benefit multiple downstream tasks from the node and link levels to the higher graph level. In practice, it is desirable to develop task-agnostic graph representation learning methods that are typically trained in an unsupervised manner. However, existing unsupervised graph models, represented by the variational graph auto-encoders (VGAEs), can only address node- and link-level tasks while manifesting poor generalizability on the more difficult graph-level tasks because they can only keep low-order isomorphic consistency within the subgraphs of one-hop neighborhoods. To overcome the limitations of existing methods, in this paper, we propose the Isomorphic-Consistent VGAE (IsoC-VGAE) for multi-level task-agnostic graph representation learning. We first devise an unsupervised decoding scheme to provide a theoretical guarantee of keeping the high-order isomorphic consistency within the VGAE framework. We then propose the Inverse Graph Neural Network (Inv-GNN) decoder as its intuitive realization, which trains the model via reconstructing the node embeddings and neighborhood distributions learned by the GNN encoder. Extensive experiments on multi-level graph learning tasks verify that our model achieves superior or comparable performance compared to both the state-of-the-art unsupervised methods and representative supervised methods with distinct advantages on the graph-level tasks.

Emotion cause analysis is a critical topic in natural language processing. Key tasks include emotion cause extraction (ECE), emotion-cause pair extraction (ECPE), social emotion cause identification (SECI) as well as social emotion mining and its cause identification (SEMCI). While current emotion cause analysis methods often focus on task-specific model design, they tend to overlook the underlying common ground across these tasks rooted in cognitive emotion theories, in particular, the cognitive structure of emotions. Drawing inspiration from this theory, in this paper, we propose a unified model capable of tackling diverse emotion cause analysis tasks, which constructs the emotion cognitive structure through LLM-based in-context learning. To mitigate the hallucination inherent in LLMs, we introduce a self-promote mechanism built on iterative refinement. It dynamically assesses the reliability of substructures based on their cognitive consistency and leverages the more reliable substructures to promote the inconsistent ones. Experimental results on multiple emotion cause analysis tasks ECE, ECPE, SECI, and SEMCI demonstrate the superiority of our unified model over existing SOTA methods and LLM-based baselines.

Medical artificial intelligence (AI) is a cross-disciplinary field focused on developing advanced computing and AI technologies to benefit medicine and healthcare. Globally, medical AI has tremendous potential to support the United Nations’ sustainable development goals pertaining to health and well-being. In particular, large language models (LLMs) afford opportunities for positively disrupting medical AI-related research and practice. We present a research framework for LLMs in medical AI. Our framework considers the interplay between health and well-being goals, disease lifecycle stages, and the important emerging role of LLMs in medical AI processes related to various lifecycle stages. As part of our framework, we describe the LLM multiplex - important multimodal, multi-model, multicultural, and multi-responsibility considerations for LLMs in medical AI. We discuss how the five articles in the special issue relate to this framework and are helping us learn about the opportunities and challenges for LLMs in medical AI.

In subjective natural language generation tasks, generating diverse perspectives is essential for fostering balanced discourse and mitigating bias. Argument generation with diverse perspectives plays a vital role in advancing the understanding of controversial claims. Despite the strong generative capabilities of large language models (LLMs), the diversity of perspectives remains insufficiently explored within argument generation task. Moreover, there remains a significant research gap in developing methods that explicitly generate multi-perspective arguments under the quality control of claim-stance alignment constraints. In this paper, we propose POEM, a P erspective-aware Preference O ptimization with E ntropy M aximization framework for diverse argument generation. It enhances perspective diversity through preference optimization based on the constructed preference dataset via perspective mining and diversity measuring. It further introduces entropy maximization to promote perspective diversity by encouraging dispersed semantic representations among the generated arguments. Experimental results on claim-stance argument generation benchmarks show that POEM is capable of generating diverse arguments while maintaining comparable performances in claim and stance controllability as well as text quality compared to the state-of-the-art baselines and human evaluation.