旷视科技

Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distribution, while learning-based noise modeling impractically depends on paired real data. In this paper, we propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency. Based on this strategy, we introduce an efficient physics-informed noise neural proxy (PNNP) to approximate the real-world sensor noise model. Specifically, we integrate physical priors into neural proxies and introduce three efficient techniques: physics-guided noise decoupling (PND), physics-aware proxy model (PPM), and differentiable distribution loss (DDL). PND decouples the dark frame into different components and handles different levels of noise flexibly, which reduces the complexity of noise modeling. PPM incorporates physical priors to constrain the synthetic noise, which promotes the accuracy of noise modeling. DDL provides explicit and reliable supervision for noise distribution, which promotes the precision of noise modeling. PNNP exhibits powerful potential in characterizing the real noise distribution. Extensive experiments on public datasets demonstrate superior performance in practical low-light raw image denoising. The source code will be publicly available at the https://fenghansen.github.io/publication/PNNP.

Text-Based Person Search (TBPS) aims to retrieve target person images from a large-scale database using natural language descriptions, serving as a critical task in multimodal perception and visual pattern recognition. Bridging the semantic gap between heterogeneous modalities while capturing fine-grained correspondences remains a fundamental challenge, especially when discriminating visually similar individuals based on complex textual semantics. To address these challenges, we propose Sew Calibration and Masked Modeling (SCMM), a unified framework that calibrates cross-modal representations for effective multimodal visual-textual pattern matching. Concretely, SCMM introduces two principal components: a sew calibration loss that dynamically aligns image-text features via a quality-guided adaptive margin governed by textual information density, and a masked caption modeling loss that establishes fine-grained semantic correspondences through transformer-based masked prediction. The sew calibration mechanism imposes bidirectional constraints to compactly cluster same-identity features in a shared embedding space. Simultaneously, the masked modeling component acts as a cross-modal decoder that learns word-level representations, effectively discriminating subtle attribute differences. Importantly, our dual-encoder architecture strikes an optimal balance between representation expressiveness and computational efficiency by adopting a training-only decoder design. Extensive experiments on CUHK-PEDES, ICFG-PEDES, and RSTPReID datasets demonstrate that SCMM achieves state-of-the-art performance with Rank-1 accuracies of 73.81%, 64.25%, and 57.35%, respectively. Thorough ablation studies confirm the efficacy of each proposed mechanism in establishing robust cross-modal patterns for multimodal perception and recognition.

Vision-language models can perform new tasks without parameter updates through in-context learning (ICL), whose core mechanism is utilizing the support set for task induction. In standard ICL setting, once the task is induced, its decision boundary, i.e., the criterion, remains fixed. However, in real-world applications, many tasks exhibit a stable high-level intent, while their decision criteria shift according to specific requirements. Thus we introduce a new test setting, denoted as Criterion-Conditional In-Context Learning (CC-ICL), where models must infer the latent criterion from context under a fixed task semantics. To evaluate this capability, we propose two complementary metrics, Criterion-Sensitivity and Criterion-Invariance, capturing model's robustness and adaptability under criterion shifts. We further construct CC-Bench, a multi-domain benchmark that supports evaluation under the CC-ICL setting through hierarchical annotation, enabling legitimate ground-truth variation under fixed tasks. Experiments on CC-Bench reveal that most models exhibit a ''rigid boundary'' bias, struggling to align their decisions with the latent criterion. We also find that even a simple multi-criteria training strategy can significantly reduce this bias, improving Criterion-Sensitivity and enabling 7B-scale models to surpass proprietary models without degrading general multimodal performance.

Recent unified open-vocabulary detection (OVD) supports heterogeneous prompts, including text queries, visual exemplars, and their combinations, but often rely on increasingly complex designs such as heavy cross-modal fusion, staged training, and iterative annotation pipelines. We revisit whether such complexity is necessary in the era of stronger foundation models. Our finding is that unified OVD can be made substantially simpler with semantic-rich visual representations and scalable grounding supervision. We present OPUS (Open-vocabulary, Prompt-Unified, Simple), a unified detector supporting text, interactive visual, generic visual, and mixed prompting within one framework. OPUS adopts a simple three-part design. Its model architecture combines a semantic-rich visual encoder, built on a DINOv3-ConvNeXt-B backbone with efficient hybrid encoding, with a prompt-aware decoder that avoids prompt-specific branches for unified prompt reasoning. OPUS is trained with a one-stage text-visual training strategy with Instance-level Contrastive Alignment (ICA), and is supported by a SAM3-based single-pass data engine for heterogeneous grounding supervision. Experiments on COCO, LVIS-minival, and ODinW35 show that OPUS achieves state-of-the-art Visual-I performance, reaching 68.1/69.2/54.7 AP, while maintaining balanced Text and Visual-G accuracy. OPUS also turns mixed prompting from interference into complementarity, improving over text or visual prompt alone. These results show that simplicity and strong unified prompting capability can be achieved together.

High-fidelity 3D generation predominantly relies on scaling model capacity and data, which incurs prohibitive computational costs. This paradigm typically requires learning geometry from scratch and overlooks the rich semantic and structural priors already encapsulated in discriminative 3D foundation models. We contend that leveraging the profound understanding of the 3D world possessed by these discriminative models can significantly reduce generative cost. To this end, we propose ROAD, a framework that reduces the training cost of 3D generation by transferring these rich discriminative priors into diffusion transformers. To address the inherent semantic-structural heterogeneity between generative and discriminative latents, we introduce a reciprocal-objective alignment strategy. This method synergizes Holistic Semantic Condensing to enforce global semantic coherence and Structural Optimal Alignment, which is formulated as a bipartite matching problem to rigorously align microscopic geometric details between disparate latent spaces. The 3D foundation model is only used for training-time supervision of alignment and is not used at inference, incurring no additional inference cost. Compared with the industrial baseline Step1X-3D, the proposed ROAD achieves highly competitive generation performance with only 1.5