We examine the Evidence Lower Bound (ELBO) within diffusion models (DMs) applied to speech enhancement (SE) and dereverberation (SD). We focus in particular on the interplay between the ELBO and Gaussian noise schedule (GNS), and the choice of practical loss functions. We hypothesize that the suboptimal performance of DM-based SE and SD can arise from the absence of a well-calibrated GNS. We therefore refine the noise schedule design by controlling the minimum and maximum noise variances. Additionally, we introduce the Importance of Condition as a novel metric that quantitatively assesses the influence of noise variance on the model behavior during reverse diffusion processes. Our analysis reveals that changing the GNS configuration substantially affects the reliance of the model on the input condition, thereby impacting the overall performance. Furthermore, we demonstrate that conventional loss functions used in DMs inherently impose a performance ceiling that prevents convergence to the theoretical optimum of the ELBO, resulting in suboptimal SE and SD outcomes. We propose a two-stage training framework to alleviate this limit. First, a score-based DM uses an optimized GNS to perform initial enhancement. Second, a dedicated refinement model is trained to further improve the ELBO and enhance speech quality. Our comprehensive experimental validation demonstrates the effectiveness of the proposed framework on both SE and SD tasks.

Handwritten Chinese Character Error Correction (HCCEC), which aims to detect and correct glyph-level misspellings in handwritten Chinese characters has recently gained attention from both academia and industry. A common solution is to decompose each character into an Ideographic Description Sequence (IDS). A key challenge lies in handling the linguistic information embedded in IDS, which encodes the grammatical rules and correlations among radicals. While such information benefits the decomposition of correctly-spelled characters, it can mislead the misspelled ones. Based on the observation that although a misspelled character may contain misused, missing, or additional radicals, its overall IDS structure still follows the grammatical rules of Chinese character composition, we propose the Two-Stage Decomposition Network (TDNet). TDNet separates the decomposition process into two stages. In the first stage, TDNet analyzes the structural layout and treats all radicals as placeholder masks to preserve structural priors. In the second stage, it identifies the actual radicals from visual features to suppress the linguistic bias. Moreover, adjacent IDS symbols may correspond to spatially distant image regions, causing abrupt attention jumps. This problem is further exacerbated in the first stage of TDNet since radical categories are masked. To alleviate this, we redesign the traversal order from preorder to inorder, producing more spatially coherent decoding, while a root-sibling prediction branch resolves the resulting one-to-many ambiguity. Finally, TDNet can be integrated into both GRU-based and Transformer-based architecture, and experiments demonstrate consistent and substantial performance gains over existing methods.

Audio-visual Sound Event Localization and Detection with Source Distance Estimation (AV 3D SELD) is designed to use synchronized audio-visual streams to simultaneously identify event categories, 3D spatial positions, and temporal intervals. In real-world scenarios, the visual modality provides rich spatiotemporal semantic cues regarding sound sources. Although recent AV 3D SELD studies have attempted to introduce visual depth cues, they still struggle to effectively exploit depth-aware visual geometry for distance estimation and cross-modal alignment. Furthermore, the absence of effective mechanisms to distill sound-related visual regions often introduces redundant background information, leading to performance degradation. To address these issues, we propose a novel framework called Spatial Semantic-Guided Network (SSGNet). Specifically, we introduce a Spatial Semantic Guidance Loss (SSGL), which guides the model to focus on sound-source-related spaces, effectively suppressing background interference. Additionally, we incorporate a spatial geometric prior to improve the association between sound sources and their visual locations, thereby boosting cross-modal fusion performance. Extensive experimental results on both DCASE 2023 and 2024 Challenge SELD tasks demonstrate that our method significantly outperforms existing SELD approaches.

Wake-up word spotting (WWS) for dysarthric speech remains challenging because it results in highly variable articulation, unstable timing, and irregular prosodic rhythm. Moreover, existing systems remain largely speaker-dependent, lacking adequate generalization to speaker-independent settings. Motivated by neurophysiological findings that intentional utterances induce preparatory motor activity that facilitates regular rhythmic patterns, we have developed a Rhythm-Aware Wake-up Word Spotting (RAWS) framework that explicitly leverages these cues for dysarthric speech. RAWS comprises three components: (1) a Temporal Prosody Structure Encoder (TPSE) that models speaking-rate and pause-durations via feature extraction, positional encoding, and transformer-based temporal processing; (2) a Large Language Model-Guided Auxiliary Learning (LLM-GAL) mechanism that provides perceptual rhythm-naturalness scores as auxiliary supervision; and (3) a Progressive Adapter-based Domain Alignment (PADA) strategy that enables effective non-dysarthric-to-dysarthric speech transfer while reducing cross-speaker variability. To the best of our knowledge, this study represents the first investigation of speaker-independent dysarthric WWS, with experimental validations on the Mandarin dysarthric speech corpus (MDSC) and its extended version (MDSC v2) demonstrating that RAWS outperforms strong competitive systems, achieving state-of-the-art results.

Recent advances in large language models (LLMs) and multimodal LLMs (MLLMs) have created new opportunities for wearable speech interfaces, with smart glasses providing an egocentric platform for continuous audio sensing and assistance. However, speech recognition and understanding in this setting remain challenging because of dynamic acoustic conditions, speaker overlap, and the spatial ambiguity introduced by wearer-centered recording geometry. To support systematic evaluation in this setting, we introduce the IEEE SLT 2026 SmartGlasses Challenge for egocentric multi-speaker speech processing. The challenge consists of two tracks, Dyadic Dialogue Understanding and Multi-party Meeting Understanding, and jointly evaluates Time-Stamped Speaker-Attributed Automatic Speech Recognition (TSA-ASR) and Spoken Language Understanding (SLU). It is built on a 106-hour four-channel egocentric speech dataset containing 714 sessions collected in real-world scenarios. This paper describes challenge tasks, dataset construction, submissions, and summarizes the main findings from the shared evaluation. The results show that heavy speaker overlap remains a major factor affecting TSA-ASR performance, while paralinguistic acoustic understanding continues to be difficult for current audio-language models in complex SLU settings. Further details can be found on the official challenge website.

3D Gaussian Splatting (3D-GS) has emerged as an efficient 3D representation and a promising foundation for semantic tasks like segmentation. However, existing 3D-GS-based segmentation methods typically rely on high-dimensional category features, which introduce substantial memory overhead. Moreover, fine-grained segmentation remains challenging due to label space congestion and the lack of stable multi-granularity control mechanisms. To address these limitations, we propose a coarse-to-fine binary encoding scheme for per-Gaussian category representation, which compresses each feature into a single integer via the binary-to-decimal mapping, drastically reducing memory usage. We further design a progressive training strategy that decomposes panoptic segmentation into a series of independent sub-tasks, reducing inter-class conflicts and thereby enhancing fine-grained segmentation capability. Additionally, we fine-tune opacity during segmentation training to address the incompatibility between photometric rendering and semantic segmentation, which often leads to foreground-background confusion. Extensive experiments on multiple benchmarks demonstrate that our method achieves state-of-the-art segmentation performance while significantly reducing memory consumption and accelerating inference.

Reinforcement learning has emerged as an effective paradigm for enhancing the mathematical reasoning capabilities of large language models. Among existing policy optimization methods, Proximal Policy Optimization (PPO) remains particularly appealing because its learned critic can, in principle, provide token-level credit assignment. However, in mathematical reasoning tasks characterized by long reasoning horizons and sparse outcome rewards, reliable token-level credit assignment remains challenging. The standard critic often fails to accurately evaluate intermediate reasoning states, resulting in noisy advantage estimates and suboptimal policy updates. In this paper, we propose ReDiPPO, a Reference-guided and Discrepancy-aware PPO framework for mathematical reasoning. ReDiPPO introduces a reference-guided critic that uses reference answers as training-time privileged signals to provide more accurate value estimation. Meanwhile, it retains a standard critic and quantifies the token-level reference-standard discrepancy between the standard value estimate and the reference-guided value estimate. This discrepancy serves as an indicator of difficult reasoning states and is used to reweight the corresponding token-level advantages during PPO optimization. Extensive experiments on diverse mathematical reasoning benchmarks demonstrate that ReDiPPO improves value-estimation accuracy and consistently outperforms strong policy optimization baselines, including PPO, DAPO, and GSPO, in final reasoning performance. Our code is available on https://github.com/cii030/ReDiPPO.

Traditional audio-visual speaker diarization (AVSD) approaches exhibit limited robustness to cross-modal heterogeneity and complex inter-speaker interactions, particularly under dynamic, unconstrained real-world conditions. To overcome these limitations, we propose a novel neural architecture, termed the hierarchical modality-speaker adaptive network (HMSA-Net), which integrates two sequentially structured modules and an ad-hoc multi-stage training strategy. The first consistency-gated inter-modal attention (CGIMA) module dynamically estimates cross-modal synchrony between audio and visual embeddings and adaptively regulates their mutual influence during feature fusion, thereby mitigating modality mismatch. The second dense inter-speaker attention (DISA) module explicitly captures complex inter-speaker relationships by applying multi-head attention from a target speaker representation to a densely aggregated bank of non-target speaker embeddings, enabling fine-grained discrimination in overlapping speech conditions without enforcing a fixed upper bound on the number of speakers. To further enhance optimization stability, a multi-stage optimization (MSO) scheme is introduced, which consistently achieves lower convergence loss than end-to-end training. Extensive evaluations on standard AVSD benchmarks demonstrate that CGIMA effectively suppresses modality-specific noise while amplifying complementary cross-modal cues, resulting in more robust fused representations. Meanwhile, DISA improves frame-level speaker discrimination by modeling dense cross-speaker dependencies. As a result, HMSA-Net trained with MSO achieves state-of-the-art performance on the AMI, MISP2022, and AVA-AVD.

This work presents HDA-SELD, a unified hierarchical distillation and augmentation framework for audio-visual (AV) sound event localization and detection (SELD) designed to address the challenge of data scarcity. The proposed framework integrates hierarchical cross-modal distillation (HCMD) to transfer knowledge from a robust audio-only SELD teacher to an AV student through both output responses and intermediate hidden representations. To enhance learning, we introduce a multi-level data augmentation strategy that mixes features randomly selected from multiple network layers and associated loss functions tailored to the SELD task. By employing loss interpolation instead of direct label manipulation, the strategy ensures spatial consistency during the augmentation process. Extensive experiments on the DCASE 2023 and 2024 Challenge SELD datasets show that the proposed method significantly improves AV SELD performance, yielding relative gains of 21%-38% in the overall metric over the baselines. Notably, our proposed HDA-SELD achieves results comparable to or better than teacher models trained on much larger datasets, surpassing state-of-the-art methods on both DCASE 2023 and 2024 Challenge SELD tasks.

Large Language Models (LLMs) have shown promise as educational tutors, yet effective tutoring requires more than solving problems: it must provide progressive Socratic guidance and balance multiple pedagogical objectives across multi-turn interactions. However, training such tutors remains challenging due to limited-fidelity and weakly controllable student simulation, under-specified pedagogical reward modeling, and unstable multi-objective optimization. To overcome these limitations, we propose PEARL, a pedagogically aligned reinforcement learning framework for training Socratic tutoring agents, consisting of three key components. First, we introduce a controllable student simulator that decouples latent cognitive states from response generation to model diverse abilities and misconceptions. Second, we develop a generative reward model that jointly evaluates pedagogical quality and objective correctness for policy optimization. Finally, we propose a stable multi-objective RL scheme that discretizes rewards within each dimension and aggregates normalized advantages across dimensions, preventing high-variance objectives from dominating updates. Experiments on multiple benchmarks show that PEARL achieves the best performance among open-source models and remains competitive with leading proprietary LLMs, despite using only a 30B policy model.

Self-supervised models such as WavLM have demonstrated strong performance for neural speaker diarization. However, these models are typically pre-trained on single-channel recordings, limiting their effectiveness in multi-channel scenarios. Existing diarization systems built on these models often rely on DOVER-Lap to combine outputs from individual channels. Although effective, this approach incurs substantial computational overhead and fails to fully exploit spatial information. In this work, building on DiariZen, a pipeline that combines WavLM-based local endto-end neural diarization with speaker embedding clustering, we introduce a lightweight approach to make pre-trained WavLM spatially aware by inserting channel communication modules into the early layers. Our method is agnostic to both the number of microphone channels and array topologies, ensuring broad applicability. We further propose to fuse multi-channel speaker embeddings by leveraging spatial attention weights. Evaluations on five public datasets show consistent improvements over single-channel baselines and demonstrate superior performance and efficiency compared with DOVER-Lap. Our source code is publicly available at https://github.com/BUTSpeechFIT/DiariZen.

The talking head generation task synthesizes videos from a single portrait image and audio input, animating the portrait to deliver the speech content. Non-autoregressive (NAR) approaches for talking head generation have demonstrated impressive quality and generation speeds by producing video frames in parallel, thereby overcoming the error accumulation problems inherent in frame-wise autoregressive (AR) methods. However, NAR methods have limited practical applicability due to prohibitive VRAM requirements, especially when generating long sequences (≥ 1000 frames) at high resolution (512 × 512). This paper proposes a novel framework that enables high-quality, non-autoregressive talking head generation while significantly reducing computational resource demands for both training and inference. We enhance efficiency through our Multi-stage Dynamics Compression Framework (MDCF), a two-stage pipeline that achieves a 768× compression ratio for pixel-level dynamics representations. Additionally, we demonstrate that this two-stage architecture cannot be effectively optimized via standard end-to-end training. We therefore introduce a Two-Phase Cascaded (TPC) training strategy to stably optimize the MDCF while effectively alleviating error accumulation during multi-stage compression. Experimental results demonstrate that our framework can non-autoregressively generate talking head videos with over 1600 frames at up to 512 × 512 resolution on a 16GB GPU, with state-of-the-art quality and inference speed. Our approach represents a significant advancement toward practical, resource-efficient talking head synthesis for real-world applications. The source code in the supplementary material will be publicly available.

Current multi-object tracking (MOT) algorithms typically overlook issues inherent in low-quality videos, leading to significant degradation in tracking performance when confronted with image degradation conditions. Therefore, advancing the application of MOT algorithms in low-quality video conditions represents a critical and meaningful endeavor. To address the challenges posed by low-quality scenarios, inspired by vision-language models, this paper proposes a Visual Semantic Enhancement-guided Multi-Object Tracking framework (VSE-MOT). Specifically, we first design a dual-branch architecture that leverages a vision-language model to extract global visual semantic information from images and fuse it with query vectors. Subsequently, to further enhance the utilization of visual semantic information, we introduce the Multi-Object Tracking Adapter (MOT-Adapter) and the Visual Semantic Fusion Module (VSFM). The MOT-Adapter adapts the extracted global visual semantic information to suit multi-object tracking tasks, while the VSFM improves the efficacy of feature fusion. Analytical experiments show the proposed method’s performance improvement mainly comes from robust, degradation-resistant pre-trained visual representations of the frozen CLIP image encoder, which are from multimodal pre-training rather than explicitly text-aligned semantics. Extensive experiments verify that our feature-prior-driven design works effectively in low-quality videos and remains robust in conventional scenarios.

Audio-driven talking head generation aims to create vivid and realistic videos from a static portrait and speech. Existing AR-based methods rely on intermediate facial representations, which limit their expressiveness and realism. Meanwhile, diffusion-based methods generate clip-by-clip, lacking fine-grained control and causing inherent latency due to overall denoising across the window. To address these limitations, we propose EARTalking, a novel end-to-end, GPT-style autoregressive model for interactive audio-driven talking head generation. Our method introduces a novel frame-by-frame, in-context, audio-driven streaming generation paradigm. For inherently supporting variable-length video generation with identity consistency, we propose the Sink Frame Window Attention (SFA) mechanism. Furthermore, to avoid the complex, separate networks that prior works required for diverse control signals, we propose a streaming Frame Condition In-Context (FCIC) scheme. This scheme efficiently injects diverse control signals in a streaming, in-context manner, enabling interactive control at every frame and at arbitrary moments. Experiments demonstrate that EARTalking outperforms existing autoregressive methods and achieves performance comparable to diffusion-based methods. Our work demonstrates the feasibility of in-context streaming autoregressive control, unlocking a scalable direction for flexible, efficient generation. The code will be released for reproducibility.

Diffusion models have significantly advanced the field of talking head generation (THG). However, slow inference speeds and prevalent non-autoregressive paradigms severely constrain the application of diffusion-based THG models. In this study, we propose REST, a pioneering diffusion-based, real-time, end-to-end streaming audio-driven talking head generation framework. To support real-time end-to-end generation, a compact video latent space is first learned through a spatiotemporal variational autoencoder with a high compression ratio. Additionally, to enable semi-autoregressive streaming within the compact video latent space, we introduce an ID-Context Cache mechanism, which integrates ID-Sink and Context-Cache principles into key-value caching for maintaining identity consistency and temporal coherence during long-term streaming generation. Furthermore, an Asynchronous Streaming Distillation (ASD) strategy is proposed to mitigate error accumulation and enhance temporal consistency in streaming generation, leveraging a non-streaming teacher with an asynchronous noise schedule to supervise the streaming student. REST bridges the gap between autoregressive and diffusion-based approaches, achieving a breakthrough in efficiency for applications requiring real-time THG. Experimental results demonstrate that REST outperforms state-of-the-art methods in both generation speed and overall performance.

The introduction of diffusion models has brought significant advances to the field of audio-driven talking head generation. However, the extremely slow inference speed severely limits the practical implementation of diffusion-based talking head generation models. In this study, we propose READ, a real-time diffusion-transformer-based talking head generation framework. Our approach first learns a spatiotemporal highly compressed video latent space via a temporal VAE, significantly reducing the token count to accelerate generation. To achieve better audio-visual alignment within this compressed latent space, a pre-trained Speech Autoencoder (SpeechAE) is proposed to generate temporally compressed speech latent codes corresponding to the video latent space. These latent representations are then modeled by a carefully designed Audio-to-Video Diffusion Transformer (A2V-DiT) backbone for efficient talking head synthesis. Furthermore, to ensure temporal consistency and accelerated inference in extended generation, we propose a novel asynchronous noise scheduler (ANS) for both the training and inference processes of our framework. The ANS leverages asynchronous add-noise and asynchronous motion-guided generation in the latent space, ensuring consistency in generated video clips. Experimental results demonstrate that READ outperforms state-of-the-art methods by generating competitive talking head videos with significantly reduced runtime, achieving an optimal balance between quality and speed while maintaining robust metric stability in long-time generation.

Reinforcement Learning with Verifiable Rewards (RLVR) elicits long chain-of-thought reasoning in large language models (LLMs), but outcome-based rewards lead to coarse-grained advantage estimation. While existing approaches improve RLVR via token-level entropy or sequence-level length control, they lack a semantically grounded, step-level measure of reasoning progress. As a result, LLMs fail to distinguish necessary deduction from redundant verification: they may continue checking after reaching a correct solution and, in extreme cases, overturn a correct trajectory into an incorrect final answer. To remedy the lack of process supervision, we introduce a training-free probing mechanism that extracts intermediate confidence and correctness and combines them into a Step Potential signal that explicitly estimates the reasoning state at each step. Building on this signal, we propose Step Potential Advantage Estimation (SPAE), a fine-grained credit assignment method that amplifies potential gains, penalizes potential drops, and applies penalty after potential saturates to encourage timely termination. Experiments across multiple benchmarks show SPAE consistently improves accuracy while substantially reducing response length, outperforming strong RL baselines and recent efficient reasoning and token-level advantage estimation methods. The code is available at https://github.com/cii030/SPAE-RL.

Current learning objectives for blind source separation (BSS) mainly focus on inter-speaker differences. However, in real-world scenarios where mixed speakers exhibit high timbre similarity, state-of-the-art (SOTA) models degrade severely. To evaluate and address this challenge, we first construct the WSJ0-same-mix benchmark, which mixes different utterances from the same speaker to simulate extreme similarity conditions. We further propose a Hierarchical Contrastive Learning with Speech Language Model (HCL-SLM) framework, which introduces dual contrastive losses to exploit layer-specific representations from Qwen2-Audio, pulling together segments from the same speaker while pushing apart those from different speakers. Extensive experiments demonstrate that several representative SOTA BSS models suffer significant degradation on the WSJ0-same-mix benchmark, whereas HCL-SLM consistently improves their performance. Code and dataset resources are available at https://github.com/proudpie/HCL-SLM.

Khmer Stone Inscription Analysis (KSIA) is an important research direction in historical document analysis and digital cultural heritage preservation, motivated by the need to document epigraphic records across ancient temples in Cambodia and broader Southeast Asia. Unlike planar manuscripts, stone inscriptions present distinctive challenges, including relief-induced shading, environmental erosion, biological growth, complex stone textures, and linguistic variation. To address these challenges, we introduce Angkorian-KSI, the first multi-task benchmark for automated digital analysis of Khmer stone inscriptions. Curated from in situ captures across multiple sites within a UNESCO World Heritage archaeological region, Angkorian-KSI contains 230 full inscription images, 760 annotated text regions, 2,733 text lines, 3,493 binarization masks, and 534 script-period labels. The benchmark supports three tasks: KSI-LA for structural detection, KSI-B for binary text-mask extraction, and KSI-C for historical period recognition across Pre-Angkorian, Angkorian, and Post-Angkorian eras. Baseline results reveal a clear domain gap caused by carved-stone degradation, highlighting the need for dedicated evaluation protocols for Khmer stone inscription analysis.

This paper introduces EGGCodec, a robust neural Encodec framework engineered for electroglottography (EGG) signal reconstruction and F0 extraction. We propose a multi-scale frequency-domain loss function to capture the nuanced relationship between original and reconstructed EGG signals, complemented by a time-domain correlation loss to improve generalization and accuracy. Unlike conventional Encodec models that extract F0 directly from features, EGGCodec leverages reconstructed EGG signals, which more closely correspond to F0. By removing the conventional GAN discriminator, we streamline EGGCodec’s training process without compromising efficiency, incurring only negligible performance degradation. Trained on a widely used EGG-inclusive dataset, extensive evaluations demonstrate that EGGCodec outperforms state-of-the-art F0 extraction schemes, reducing mean absolute error (MAE) from 15.9 Hz to 13.7 Hz, and improving voicing decision error (VDE) by 42.1%. Moreover, extensive ablation experiments validate the contribution of each component of EGGCodec.