Image editing has advanced significantly with the development of diffusion models using both inversion-based and instruction-based methods. However, current inversion-based approaches struggle with big modifications (e.g., adding or removing objects) due to the structured nature of inversion noise, which hinders substantial changes. Meanwhile, instruction-based methods often constrain users to black-box operations, limiting direct interaction for specifying editing regions and intensity. To address these limitations, we propose BrushEdit, a novel inpainting-based instruction-guided image editing paradigm, which leverages multimodal large language models (MLLMs) and image inpainting models to enable autonomous, user-friendly, and interactive free-form instruction editing. Specifically, we devise a system enabling free-form instruction editing by integrating MLLMs and a dual-branch image inpainting model in an agent-cooperative framework to perform editing category classification, main object identification, mask acquisition, and editing area inpainting. Extensive experiments show that our framework effectively combines MLLMs and inpainting models, achieving superior performance across seven metrics including mask region preservation and editing effect coherence.

Recent work on test-time scaling for large language model (LLM) reasoning typically assumes that allocating more inference-time computation uniformly improves correctness. However, prior studies show that reasoning uncertainty is highly localized: a small subset of low-confidence tokens disproportionately contributes to reasoning errors and unnecessary output expansion. Motivated by this observation, we propose Thinking by Subtraction, a confidence-driven contrastive decoding approach that improves reasoning reliability through targeted token-level intervention. Our method, Confidence-Driven Contrastive Decoding, detects low-confidence tokens during decoding and intervenes selectively at these positions. It constructs a contrastive reference by replacing high-confidence tokens with minimal placeholders, and refines predictions by subtracting this reference distribution at low-confidence locations. Experiments show that CCD significantly improves accuracy across mathematical reasoning benchmarks while substantially reducing output length, with minimal KV-cache overhead. As a training-free method, CCD enhances reasoning reliability through targeted low-confidence intervention without computational redundancy. Our code will be made available at: https://github.com/bolo-web/CCD.

This paper presents PhoenixCodec, a comprehensive neural speech coding and decoding framework designed for extremely low-resource conditions. The proposed system integrates an optimized asymmetric frequency–time architecture, a Cyclical Calibration and Refinement (CCR) training strategy, and a noise-invariant fine-tuning procedure. Under stringent constraints—computation below 700 MFLOPs, latency less than 30 ms, and dual-rate support at 1 kbps and 6 kbps—existing methods face a trade-off between efficiency and quality. PhoenixCodec addresses these challenges by alleviating the resource-scattering of conventional decoders, employing CCR to enhance optimization stability, and enhancing robustness through noisy-sample fine-tuning. In the LRAC 2025 Challenge Track 1, the proposed system ranked third overall and demonstrated the best performance at 1 kbps in both real-world noise and reverberation and intelligibility in clean tests, confirming its effectiveness.

Large vision-language models (LVLMs) have demonstrated impressive capabilities across diverse multimodal tasks, yet they remain highly susceptible to visual hallucinations (VH), often producing confident but inaccurate descriptions of visual content. Building on the insight that not all tokens and attention heads contribute equally to VH mitigation, we introduce VisFlow, a lightweight and training-free framework that alleviates hallucinations by directly modulating attention patterns during inference. To address two primary challenges of VH, namely insufficient visual attention and the dominance of language priors, we identify three problematic attention behaviors in LVLMs: (1) disproportionate allocation of attention to uninformative or trailing visual tokens, (2) over-dependence on the previously generated token, and (3) excessive fixation on system prompts that hinders multimodal integration. To overcome these issues, VisFlow introduces a dual-level Attention Intervention, consisting of Token-level Attention Intervention (TAI), which reinforces attention to salient visual regions, and Head-level Attention Intervention (HAI), which suppresses undue focus on system prompts and adjacent text tokens. Together, these interventions strengthen visual alignment while reducing linguistic bias. Extensive experiments across diverse models and benchmarks demonstrate that VisFlow effectively mitigates hallucinations with minimal computational overhead.

Speaker diarization is a fundamental task in speech processing aims to determine 'who speaks when'. When combined with ASR, it enables speaker-labeled transcription with broad practical value. Most existing methods rely on frame-level classification, but the high cost of annotating mixed-speaker audio limits the availability of large-scale, accurately labeled datasets. As a result, even state-of-the-art models struggle with imprecise speaker boundary detection and semantic segmentation errors, which degrade timestamp accuracy and downstream ASR performance. To address these challenges, we propose WhisperDiari, a unified framework for speaker diarization and ASR. We first construct LibriDiari, a dataset derived from LibriSpeech, containing 2–4 speaker mixed audio annotated with transcripts and speaker labels. WhisperDiari builds on the Whisper model, incorporating speaker adapters and Speaker Similarity Matrix Supervision to enhance speaker representation. In addition, a dedicated speaker decoder fuses speaker embeddings with contextual semantics from Whisper's decoder, enabling token-level diarization. This design effectively resolves segmentation ambiguity, aligns diarization with semantic units, and jointly models 'who speaks what and when', producing accurate, timestamped transcripts. We train the model on LibriDiari and evaluate it on both LibriDiari and the real-world AMI corpus. Experimental results demonstrate that WhisperDiari consistently outperforms state-of-the-art open-source baselines.

Vision-Language Models (VLMs) such as CLIP demonstrate remarkable zero-shot recognition, yet their adaptation to unseen domains without labels remains challenging. We propose PromptSID, a self-iterative distillation framework for unsupervised adaptation of VLMs. PromptSID operates in two stages: (1) generating high-confidence pseudo-labels from zero-shot CLIP to train a prompt-enhanced student model; (2) iteratively replacing the teacher with the best student to refine both pseudo-labels and prompts in a loop. This iterative mechanism enables stable adaptation by leveraging pseudo-supervision and dynamic prompt optimization. Experiments on 13 image classification benchmarks show that PromptSID significantly surpasses zero-shot CLIP, demonstrating consistent gains across diverse domains. Our framework highlights the potential of prompt-based self-iterative distillation to unlock hidden capacities of VLMs under fully unlabeled conditions, providing a simple yet effective approach for real-world deployment where domain-specific annotations are unavailable.

Recent audio generation models typically rely on Variational Autoencoders (VAEs) and perform generation within the VAE latent space. Although VAEs excel at compression and reconstruction, their latents inherently encode low-level acoustic details rather than semantically discriminative information, leading to entangled event semantics and complicating the training of generative models. To address these issues, we discard VAE acoustic latents and introduce semantic encoder latents, thereby proposing SemanticVocoder, a generative vocoder that directly synthesizes waveforms from semantic latents. Equipped with SemanticVocoder, our text-to-audio generation model achieves a Frechet Distance of 12.823 and a Frechet Audio Distance of 1.709 on the AudioCaps test set, as the introduced semantic latents exhibit superior discriminability compared to acoustic VAE latents. Beyond improved generation performance, it also serves as a promising attempt towards unifying audio understanding and generation within a shared semantic space. Generated samples are available at https://zeyuxie29.github.io/SemanticVocoder/.

Multimodal Large Language Models (MLLMs) show promise for medical applications, yet progress in dermatology lags due to limited training data, narrow task coverage, and lack of clinically-grounded supervision that mirrors expert diagnostic workflows. We present a comprehensive framework to address these gaps. First, we introduce DermoInstruct, a large-scale morphology-anchored instruction corpus comprising 211,243 images and 772,675 trajectories across five task formats, capturing the complete diagnostic pipeline from morphological observation and clinical reasoning to final diagnosis. Second, we establish DermoBench, a rigorous benchmark evaluating 11 tasks across four clinical axes: Morphology, Diagnosis, Reasoning, and Fairness, including a challenging subset of 3,600 expert-verified open-ended instances and human performance baselines. Third, we develop DermoGPT, a dermatology reasoning MLLM trained via supervised fine-tuning followed by our Morphologically-Anchored Visual-Inference-Consistent (MAVIC) reinforcement learning objective, which enforces consistency between visual observations and diagnostic conclusions. At inference, we deploy Confidence-Consistency Test-time adaptation (CCT) for robust predictions. Experiments show DermoGPT significantly outperforms 16 representative baselines across all axes, achieving state-of-the-art performance while substantially narrowing the human-AI gap. DermoInstruct, DermoBench and DermoGPT will be made publicly available at https://github.com/mendicant04/DermoGPT upon acceptance.

Achieving fine-grained and structurally sound controllability is a cornerstone of advanced visual generation. Existing part-based frameworks treat user-provided parts as an unordered set and therefore ignore their intrinsic spatial and semantic relationships, which often results in compositions that lack structural integrity. To bridge this gap, we propose Graph-PiT, a framework that explicitly models the structural dependencies of visual components using a graph prior. Specifically, we represent visual parts as nodes and their spatial-semantic relationships as edges. At the heart of our method is a Hierarchical Graph Neural Network (HGNN) module that performs bidirectional message passing between coarse-grained part-level super-nodes and fine-grained IP+ token sub-nodes, refining part embeddings before they enter the generative pipeline. We also introduce a graph Laplacian smoothness loss and an edge-reconstruction loss so that adjacent parts acquire compatible, relation-aware embeddings. Quantitative experiments on controlled synthetic domains (character, product, indoor layout, and jigsaw), together with qualitative transfer to real web images, show that Graph-PiT improves structural coherence over vanilla PiT while remaining compatible with the original IP-Prior pipeline. Ablation experiments confirm that explicit relational reasoning is crucial for enforcing user-specified adjacency constraints. Our approach not only enhances the plausibility of generated concepts but also offers a scalable and interpretable mechanism for complex, multi-part image synthesis. The code is available at https://github.com/wolf-bailang/Graph-PiT.

We present a family of open-source Music Foundation Models designed to advance large-scale music understanding and generation across diverse tasks and modalities. Our framework consists of four major components: (1) HeartCLAP, an audio-text alignment model; (2) HeartTranscriptor, a robust lyric recognition model optimized for real-world music scenarios; and (3) HeartCodec, a low-frame-rate (12.5 Hz) yet high-fidelity music codec tokenizer that captures long-range musical structure while preserving fine-grained acoustic details and enabling efficient autoregressive modeling; (4) HeartMuLa, an LLM-based song generation model capable of synthesizing high-fidelity music under rich, user-controllable conditions (e.g., textual style descriptions, lyrics, and reference audio). In addition, it provides two specialized modes: (i) fine-grained musical attribute control, which allows users to specify the style of different song sections (e.g., intro, verse, chorus) using natural language prompts; and (ii) short, engaging music generation, which is suitable as background music for short videos. Lastly, HeartMuLa improves significantly when scaled to 7B parameters. For the first time, we show that a Suno-level, commercial-grade system can be reproduced using academic-scale data and GPU resources. We expect these foundation models to serve as strong baselines for future research and to facilitate practical applications in multimodal content production.

With the rapid development of online conferences and live streaming, personalized real-time communication (PRTC) has emerged as a critical capability for next-generation communication systems, placing demands on latency, complexity, and security. This paper presents LiteSwitchCodec, a lightweight neural speech codec specifically designed for PRTC services on operator-managed platforms. It aims to achieve high-quality speech compression while facilitating PRTC by integrating a voice adaptation (VA) module, thereby avoiding potential risks to voice copyright and security. For speech compression, we first design LiteSpeechCodec, which employs fully causal convolutional layers as the encoder and decoder and reduces the complexity of the decoder through a mirrored structure. We introduce scalar quantization (SQ) as an alternative to residual vector quantization (RVQ), reducing model complexity while maintaining high speech quality. This approach facilitates the learning of a high-quality compression domain, which in turn simplifies the generation of the quantized tokens. A lightweight causal U-Net model is introduced in the token space to extract global information for personalized VA, supporting dynamic switching of target speakers. Specifically, we propose a two-stage training strategy. First, we train the LiteSpeechCodec on public datasets for speech compression. We then construct a voice conversion (VC) dataset to train the token-level causal U-Net VA network. Experiments demonstrate that LiteSwitchCodec maintains RTC quality while reducing model parameters by 38 ×  compared to the state-of-the-art codec, achieving an objective quality of ViSQOL: 4.32 at 7.2 kbps. Moreover, LiteSwitchCodec achieves real-time VA with a low latency of 40 ms. Compared with VC models in RTC transmission, our method shows superior performance in both subjective and objective metrics, achieving an objective Resemblyzer similarity of 92.61% and a subjective speaker-similarity score (S-MOS: 4.67 vs. 3.59), highlighting its effectiveness in PRTC. Critically, LiteSwitchCodec inherently safeguards voice copyrights and prevents unauthorized impersonation, satisfying the security demands of commercial RTC deployments.

We present VChangeCodec, an ultra low-complexity neural speech codec with built-in timbre adaptation capabilities designed for customized real-time communication (RTC). Unlike cascaded pipelines that combine neural speech codecs (NSCs) with separate voice conversion (VC) systems, which have high latency, VChangeCodec integrates VC into the NSC itself and can seamlessly switch between original and voice change modes. Specifically, we design the NSC by proposing a fully causal convolutional network with scalar quantization to generate compact tokens. In addition, we introduce a lightweight causal projection network that adapts the tokens with target speaker embeddings for decoding into the converted voice, enabling seamless timbre conversion without the need for an additional VC system. Experiments show that, compared to the state-of-the-art speech codec, VChangeCodec reduces parameters by 96.3%, while maintaining competitive speech quality. More importantly, while the cascaded pipeline suffers from more than 100 ms latency, VChangeCodec achieves integrated timbre customization with only 40 ms, making it well-suited for RTC scenarios.

Task-oriented dialogue (ToD) systems facilitate goal-oriented interactions by understanding user intents, accessing external APIs, and generating appropriate responses. Recent large language model (LLM)-based approaches have improved end-to-end dialogue capabilities, but they still face two major limitations: static prompts lack adaptability in dynamic multi-turn conversations, and full-model fine-tuning is computationally expensive and prone to overfitting. We propose D2-AutoTOD, a parameter-efficient framework that integrates a disentangled context encoder with a Feature-wise Linear Modulation (FiLM) based dynamic prompting mechanism. The model explicitly encodes four complementary signals, namely the user utterance, dialogue history, previous system action, and execution feedback, and generates turn-specific modulation vectors through a lightweight Dialogue Meta-Network to dynamically condition a frozen LLM. Experiments on MultiWOZ 2.2 and SGD demonstrate that D2-AutoTOD significantly improves task success, dialogue quality, and robustness under noisy conditions while achieving strong cross-domain generalization. These results highlight the potential of dynamic and feedback-aware prompting for building scalable and adaptive task-oriented dialogue systems. Code: https://github.com/GongFuXiong/AutoTOD.

Image customization, a crucial technique for industrial media production, aims to generate content that is consistent with reference images. However, current approaches conventionally separate image customization into position-aware and position-free customization paradigms and lack a universal framework for diverse customization, limiting their applications across various scenarios. To overcome these limitations, we propose IC-Custom, a unified framework that seamlessly integrates position-aware and position-free image customization through in-context learning. IC-Custom concatenates reference images with target images to a polyptych, leveraging DiT's multi-modal attention mechanism for fine-grained token-level interactions. We propose the In-context Multi-Modal Attention (ICMA) mechanism, which employs learnable task-oriented register tokens and boundary-aware positional embeddings to enable the model to effectively handle diverse tasks and distinguish between inputs in polyptych configurations. To address the data gap, we curated a 12K identity-consistent dataset with 8K real-world and 4K high-quality synthetic samples, avoiding the overly glossy, oversaturated look typical of synthetic data. IC-Custom supports various industrial applications, including try-on, image insertion, and creative IP customization. Extensive evaluations on our proposed ProductBench and the publicly available DreamBench demonstrate that IC-Custom significantly outperforms community workflows, closed-source models, and state-of-the-art open-source approaches. IC-Custom achieves about 73\% higher human preference across identity consistency, harmony, and text alignment metrics, while training only 0.4\% of the original model parameters.

Active Speaker Detection (ASD) aims to identify speakers in videos using audio-visual cues. Existing methods typically fuse audio and visual features but often lack fine-grained alignment between speech and speaker activity, and seldom explicitly model speech or speaker transition boundaries. To address these limitations, we propose an audio-guided multi-modal method. First, we enrich the AVA dataset with voice activity labels derived from Silero VAD and visual labels, providing more comprehensive training signals. Second, we introduce a semantic alignment strategy that guides visual features to align with pretrained voice activity features. Third, we construct a boundary modeling network that combines pretrained voice activity and speaker features with visual cues to capture transition boundaries, and is optimized with a dedicated loss to capture transition dynamics and enhance boundary precision. Experiments on target benchmarks demonstrate state-of-the-art performance, validating the effectiveness of our method.

Although Chain-of-Thought (CoT) reasoning has achieved success in task-oriented dialogue, the use of ineffective demonstrations has little impact on its reasoning performance. Inspired by the human ability to learn from both positive and negative examples, we propose Contrastive Memory-of-Thought (CMoT) to enhance the reasoning capabilities of language models and improve their accuracy, efficiency, and robustness through contrastive supervision. Compared with conventional chain-of-thought reasoning, we design a memory module to store a set of high-level reasoning informative, termed thought templates, distilled from the problem-solving processes across diverse tasks. In addition, to bridge the gap between unstructured text and structured knowledge bases (KBs), we further introduce a structured knowledge-aware pretraining task. This task guides the model to effectively capture implicit relations within complex subgraphs and to learn more expressive representations. Extensive experiments and analyses on public task-oriented dialogue datasets, SMD and CamRest, demonstrate that Contrastive Memory-of-Thought can be applied as a general-purpose augmentation of chain-of-thought prompting, significantly improving the reasoning ability and accuracy of language models. Code: https://github.com/GongFuXiong/CMoT.

This work presents STAR, the first end-to-end speech-to-audio generation framework, designed to enhance efficiency and address error propagation inherent in cascaded systems. Unlike prior approaches relying on text or vision, STAR leverages speech as it constitutes a natural modality for interaction. As an initial step to validate the feasibility of the system, we demonstrate through representation learning experiments that spoken sound event semantics can be effectively extracted from raw speech, capturing both auditory events and scene cues. Leveraging the semantic representations, STAR incorporates a bridge network for representation mapping and a two-stage training strategy to achieve end-to-end synthesis. With a 76.9

Element-level visual manipulation is essential in digital content creation, but current diffusion-based methods lack the precision and flexibility of traditional tools. In this work, we introduce BlobCtrl, a framework that unifies element-level generation and editing using a probabilistic blob-based representation. By employing blobs as visual primitives, our approach effectively decouples and represents spatial location, semantic content, and identity information, enabling precise element-level manipulation. Our key contributions include: 1) a dual-branch diffusion architecture with hierarchical feature fusion for seamless foreground-background integration; 2) a self-supervised training paradigm with tailored data augmentation and score functions; and 3) controllable dropout strategies to balance fidelity and diversity. To support further research, we introduce BlobData for large-scale training and BlobBench for systematic evaluation. Experiments show that BlobCtrl excels in various element-level manipulation tasks while maintaining computational efficiency, offering a practical solution for precise and flexible visual content creation. Project page: https://liyaowei-stu.github.io/project/BlobCtrl/

The rapid development of generative audio raises ethical and security concerns stemming from forged data, making deepfake sound detection an important safeguard against the malicious use of such technologies. Although prior studies have explored this task, existing methods largely focus on binary classification and fall short in explaining how manipulations occur, tracing where the sources originated, or generalizing to unseen sources-thereby limiting the explainability and reliability of detection. To address these limitations, we present FakeSound2, a benchmark designed to advance deepfake sound detection beyond binary accuracy. FakeSound2 evaluates models across three dimensions: localization, traceability, and generalization, covering 6 manipulation types and 12 diverse sources. Experimental results show that although current systems achieve high classification accuracy, they struggle to recognize forged pattern distributions and provide reliable explanations. By highlighting these gaps, FakeSound2 establishes a comprehensive benchmark that reveals key challenges and aims to foster robust, explainable, and generalizable approaches for trustworthy audio authentication.

Recent advancements in non-autoregressive single-task speech synthesis have garnered significant attention. However,traditional single-task speech synthesis methods focus primarily on mapping semantic tokens to acoustic tokens, which overlooking the internal relationships within acoustic features. Addressing this gap, we propose SpeechSEC, a unified multi-task framework designed for Speech Synthesis, Editing, and Continuation tasks by dynamically adjusting input conditions. SpeechSEC not only surpasses previous state-of-the-art method in audio quality (4.20 vs 4.00), and voice preservation (0.72 vs 0.58) for synthesis task by acquiring shared knowledge, but also efficiently executes editing and continuation tasks with good performance via non-autoregressive techniques. Additionally, SpeechSEC exhibits a strong adaptability to current speech discretization methods, like Hubert, Descript-Audio-Codec and SpeechTokenizer, which showcases robustness of our approach. Audio samples are available.(1)