Text-prompted image segmentation enables fine-grained visual understanding and is critical for applications such as human-computer interaction and robotics. However, existing supervised fine-tuning methods typically ignore explicit chain-of-thought (CoT) reasoning at test time, which limits their ability to generalize to unseen prompts and domains. To address this issue, we introduce LENS, a scalable reinforcement-learning framework that jointly optimizes the reasoning process and segmentation in an end-to-end manner. We propose unified reinforcement-learning rewards that span sentence-, box-, and segment-level cues, encouraging the model to generate informative CoT rationales while refining mask quality. Using a publicly available 3-billion-parameter vision–language model, i.e., Qwen2.5-VL-3B-Instruct, LENS achieves an average cIoU of 81.2% on the RefCOCO, RefCOCO+, and RefCOCOg benchmarks, outperforming the strong fine-tuned method, i.e., GLaMM, by up to 5.6%. These results demonstrate that RL-driven CoT reasoning significantly enhances text-prompted segmentation and offers a practical path toward more generalizable Segment Anything models (SAM).

While 10B-level industrial foundation models have pushed the boundaries of image inpainting, their prohibitive computational costs severely hinder practical deployment. Constructing a highly optimized task-specific specialist offers a promising solution; however, extreme structural compression inevitably triggers a severe representation bottleneck. To conquer this, we propose Moebius, a highly efficient lightweight inpainting framework. We systematically reconstruct the diffusion backbone by introducing the Local-λ Mix Interaction (LλMI) block. Comprising Local-λ and Interactive-λ modules, it elegantly summarizes spatial contexts and global semantic priors into fixed-size linear matrices, preserving complex latent interactions while drastically shedding parameters. Furthermore, to unlock the full representational capacity of this highly compact architecture, we synergistically pair it with an adaptive multi-granularity distillation strategy. Operating strictly within the latent space to avoid expensive pixel-space decoding, this strategy dynamically balances multiple gradient-based losses to achieve high-fidelity alignment. Extensive experiments across natural and portrait benchmarks demonstrate that this optimal synergy enables Moebius to rival or even surpass the generation quality of the 10B-level industrial generalist FLUX.1-Fill-Dev. Remarkably, Moebius achieves this using less than 2% of the parameters (0.22B vs. 11.9B) while delivering a >15× acceleration in total inference time, setting a new efficiency standard for high-fidelity inpainting. Project page at https://hustvl.github.io/Moebius.

Segment Anything Model (SAM) has attracted widespread attention for its superior interactive segmentation capabilities. Despite this, the potential of text prompts for SAM remains largely unexplored. In this paper, we investigate what text prompt encoders are beneficial for adapting SAM for Referring Expression Segmentation (RES) and observe that (1) multimodal prompts and (2) the early-fusion mechanism incorporating text-to-image attention are crucial factors. We introduce the Early Vision-Language Fusion (EVF) framework and further formulate the EVF-SAM, which is simple but demonstrates superior effectiveness and efficiency compared to mainstream methods based on Large Language Models (LLM). We also successfully integrate segmentation data from diverse tasks into an unified hybrid dataset to conduct joint training. The well-designed data strategies resolve issues such as semantic conflict and ambiguity. The proposed EVF-SAM based on BEIT-3 obtains stateof-the-art performance on RES tasks (79.0 in average on RefCOCO/+/g), and extends RES capability to various granularities, e.g., semantic-level RES and part-level RES. EVF-SAM maintains the model size with only 1.32B parameters, considerably fewer than the mainstream LLM-based methods of over 7B parameters. Code and models are released at https://github.com/hustvl/EVF-SAM.

Learning a human-like driving policy from large-scale driving demonstrations is promising, but the uncertainty and non-deterministic nature of planning make it challenging. Existing learning-based planning methods follow a deterministic paradigm to directly regress the action, failing to cope with the uncertainty problem. In this work, we propose a probabilistic planning model for end-to-end autonomous driving, termed VADv2. We resort to a probabilistic field function to model the mapping from the action space to the probabilistic distribution. Since the planning action space is a high-dimensional continuous spatiotemporal space and hard to tackle, we first discretize the planning action space to a large planning vocabulary and then tokenize the planning vocabulary into planning tokens. Planning tokens interact with scene tokens and output the probabilistic distribution of action. Mass driving demonstrations are leveraged to supervise the distribution. VADv2 achieves state-of-the-art closed-loop performance on the CARLA Town05 benchmark, significantly outperforming all existing methods. We also provide comprehensive evaluations on the NAVSIM dataset and a large-scale 3DGS-based benchmark, demonstrating its effectiveness in real-world applications. Code will be released to facilitate future research.

Anterior segment diseases are a major global cause of preventable blindness, especially in regions with limited access to specialized ophthalmic care. Diagnosis typically requires slit-lamp biomicroscopy, creating a significant access bottleneck in primary care and rural settings. Existing AI solutions often lack the efficiency and generalizability necessary for widespread mobile deployment and primarily focus on individual conditions, failing to meet the demand for scalable, multi-disease screening platforms. We developed the Intelligent Detection System (IDS), a smartphone-compatible AI platform for real-time, automated, multi-disease screening of anterior segment diseases. The core model, Eye-YOLO, is a novel, lightweight deep learning model optimized for standard smartphone images. IDS integrates image quality assessment and urgency classification. The model was rigorously trained on a large, heterogeneous multi-center dataset of 24,671 images (comprising 17,853 slit-lamp and 6,818 smartphone images) collected from three tertiary hospitals in China. IDS was prospectively deployed and validated via a widely accessible WeChat Mini Program for community screening. Eye-YOLO achieved a mean average precision of 0.816 with an exceptionally compact architecture (2.77 million parameters and 7.3G FLOPs), enabling real-time inference at 131 FPS on mobile devices. In controlled testing, IDS demonstrated high diagnostic performance (93.50

End-to-end driving models achieve strong planning performance with large-scale data but remain constrained by training-data bias and often struggle in complex and challenging scenarios. In contrast, Large Vision-Language Models (LVLMs) exhibit promising scene understanding and reasoning abilities. The path forward lies in merging the strengths of both approaches. Prior attempts using LVLMs to predict trajectories or control signals yield suboptimal results, as LVLMs are not well-suited for precise numerical outputs. We introduce Senna, an autonomous driving framework combining an LVLM (Senna-VLM) with an end-to-end model (Senna-E2E). Senna decouples high-level planning from low-level prediction. Senna-VLM produces natural language planning decisions, and Senna-E2E converts them into precise trajectories. Senna-VLM utilizes a multi-image encoding mechanism and multi-view prompts for efficient scene reasoning. Besides, we introduce planning-oriented QAs and a three-stage training strategy, which together enhance Senna-VLM’s scene understanding and planning capabilities. Comprehensive open-loop and closed-loop experiments show that Senna achieves state-of-the-art planning performance. Notably, with pre-training on a large-scale dataset DriveX and fine-tuning on nuScenes, Senna significantly reduces planning error by 35.59 https://github.com/hustvl/Senna to facilitate future research.

Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.

Existing Image-based virtual try-on (VTON) methods primarily focus on single-layer or multi-garment VTON, neglecting multi-layer VTON (ML-VTON), which involves dressing multiple layers of garments onto the human body with realistic deformation and layering to generate visually plausible outcomes. The main challenge lies in accurately modeling occlusion relationships between inner and outer garments to reduce interference from redundant inner garment features. To address this, we propose GO-MLVTON, the first multi-layer VTON method, introducing the Garment Occlusion Learning module to learn occlusion relationships and the StableDiffusion-based Garment Morphing Fitting module to deform and fit garments onto the human body, producing high-quality multi-layer try-on results. Additionally, we present the MLG dataset for this task and propose a new metric named Layered Appearance Coherence Difference (LACD) for evaluation. Extensive experiments demonstrate the state-of-the-art performance of GO-MLVTON. Project page: https://upyuyang.github.io/go-mlvton/.

Transformer has been very successful in various computer vision tasks and understanding the working mechanism of transformer is important. As touchstones, weakly-supervised semantic segmentation (WSSS) and class activation map (CAM) are useful tasks for analyzing vision transformers (ViT). Based on the plain ViT pre-trained with ImageNet classification, we find that multi-layer, multi-head self-attention maps can provide rich and diverse information for weakly-supervised semantic segmentation and CAM generation, e.g., different attention heads of ViT focus on different image areas and object categories. Thus we propose a novel method to end-to-end estimate the importance of attention heads, where the self-attention maps are adaptively fused for high-quality CAM results that tend to have more complete objects. Besides, we propose a ViT-based gradient clipping decoder for online retraining with the CAM results efficiently and effectively. Furthermore, the gradient clipping decoder can make good use of the knowledge in large-scale pre-trained ViT and has a scalable ability. The proposed plain Transformer-basedWeakly-supervised learning method (WeakTr) obtains the superior WSSS performance on standard benchmarks, i.e., 78.5% mIoU on the val set of PASCAL VOC 2012 and 51.1% mIoU on the val set of COCO 2014. Source code and checkpoints are available at https://github.com/hustvl/WeakTr.

There is a growing demand for deploying large generative AI models on mobile devices. For recent popular video generative models, however, the Variational AutoEncoder (VAE) represents one of the major computational bottlenecks. Both large parameter sizes and mismatched kernels cause out-of-memory errors or extremely slow inference on mobile devices. To address this, we propose a low-cost solution that efficiently transfers widely used video VAEs to mobile devices. (1) We analyze redundancy in existing VAE architectures and get empirical design insights. By integrating 3D depthwise separable convolutions into our model, we significantly reduce the number of parameters. (2) We observe that the upsampling techniques in mainstream video VAEs are poorly suited to mobile hardware and form the main bottleneck. In response, we propose a decoupled 3D pixel shuffle scheme that slashes end-to-end delay. Building upon these, we develop a universal mobile-oriented VAE decoder, Turbo-VAED. (3) We propose an efficient VAE decoder training method. Since only the decoder is used during deployment, we distill it to Turbo-VAED instead of retraining the full VAE, enabling fast mobile adaptation with minimal performance loss. To our knowledge, our method enables real-time 720p video VAE decoding on mobile devices for the first time. This approach is widely applicable to most video VAEs. When integrated into four representative models, with training cost as low as $95, it accelerates original VAEs by up to 84.5× at 720p resolution on GPUs, uses as low as 17.5% of original parameter count, and retains 96.9% of the original reconstruction quality. Compared to mobile-optimized VAEs, Turbo-VAED achieves a 2.9× speedup in FPS and better reconstruction quality on the iPhone 16 Pro.

World Action Models (WAMs) learn action-relevant representations by predicting how the observed world will evolve. Most existing WAMs define this future in RGB space, where task-relevant state transitions are entangled with nuisance variations in texture, illumination, background, and viewpoint. We argue that WAMs should explicitly predict action-relevant future state rather than relying on RGB prediction alone. We introduce DreamWAM, which reformulates future prediction as structured world modeling beyond RGB, representing future states through complementary views of appearance, motion, geometry, and semantics. During training, DreamWAM combines joint latent denoising of RGB and motion with lightweight gated residual branches for geometry and semantics. Shared attention between VideoDiT and ActionDiT allows the action branch to learn from these future-state predictions, while all beyond-RGB supervision branches are disabled at inference and deployment remains RGB-only. Across both no-rollout and joint video-action inference, DreamWAM consistently improves the matched RGB-only baselines on LIBERO, from 97.30% to 98.40% and from 98.00% to 98.90%, respectively. The gains become larger under unseen LIBERO-Plus perturbations, from 51.36% to 63.44% and from 69.16% to 75.47%. The same robustness extends to real-world manipulation, where DreamWAM attains an average success rate of 74.4% across unseen changes in lighting, background, and object layout, compared with 55.6% for Fast-WAM-Joint. These results show that robust world-action learning depends not only on predicting the future, but on representing it in a form that matters for action. The code and models are publicly released at https://github.com/hustvl/DreamWAM.

Transformers are revolutionizing the landscape of artificial intelligence, unifying the architecture for natural language processing, computer vision, and more. In this paper, we explore how far a Transformer-based architecture can go for object detection - a fundamental task in computer vision and applicable across a range of engineering applications. We found that introducing an early detector can improve the performance of detection transformers, allowing them to know where to focus. To this end, we propose a novel attention map to feature map auxiliary loss and a novel local bipartite matching strategy to cost-freely get a BEtter early detector for high-performance detection TRansformer (BETR). On the COCO dataset, BETR adds no more than 6 million parameters to the Swin Transformer backbone, achieving the highest AP and latency among existing fully Transformer-based detectors across different model scales. As a Transformer detector, BETR also demonstrates accuracy, speed, and parameters on par with previous state-of-the-art CNN-based GFLV2 framework for the first time.

Generating visual text in natural scene images is a challenging task with many unsolved problems. Different from generating text on artificially designed images (such as posters, covers, and cartoons), existing methods for natural scene visual text generation still have significant deficiencies: methods based on rendering engines rely on manually crafted rules, which struggle to adapt to diverse backgrounds and leave obvious artificial traces, while their text layouts may be placed in unreasonable areas (e.g., sky or ground) and text content is semantically disconnected from the scene; diffusion model-based methods, on the other hand, face difficulties in generating small characters, depend on manually designed prompts to ensure reasonable layout and content, fail to generate text at precise locations, and cannot effectively control text attributes (e.g., font and color). In this paper, we propose a two-stage method named SceneVTG++ to address these issues. SceneVTG++ comprises two core components: a Text Layout and Content Generator (TLCG) and a Controllable Local Text Diffusion (CLTD). The former leverages the world knowledge and visual reasoning capabilities of multimodal large language models to identify reasonable text areas and recommend scene-relevant text content based on natural scene background images; the latter generates controllable multilingual text using a diffusion model, ensuring alignment with the outputs of TLCG. Through extensive experiments, we verified the effectiveness of both TLCG and CLTD, and demonstrated that SceneVTG++ achieves state-of-the-art performance in natural scene visual text generation. Additionally, the images generated by SceneVTG++ exhibit superior utility for training natural scene optical character recognition (OCR) tasks, including text detection and text recognition. Codes and datasets will be made publicly available.

Recent advancements in unified multimodal understanding and visual generation (or multimodal generation) models have been hindered by their quadratic computational complexity and dependence on large-scale training data. We present OmniMamba, the first linear-architecture-based multimodal generation model that generates both text and images through a unified next-token prediction paradigm. The model fully leverages Mamba-2's high computational and memory efficiency, extending its capabilities from text generation to multimodal generation. To address the data inefficiency of existing unified models, we propose two key innovations: (1) decoupled vocabularies to guide modality-specific generation, and (2) task-specific LoRA for parameter-efficient adaptation. Furthermore, we introduce a decoupled two-stage training strategy to mitigate data imbalance between two tasks. Equipped with these techniques, OmniMamba achieves competitive performance with JanusFlow while surpassing Show-o across benchmarks, despite being trained on merely 2M image-text pairs, which is 1,000 times fewer than Show-o. Notably, OmniMamba stands out with outstanding inference efficiency, achieving up to a 119.2 times speedup and 63 Transformer-based counterparts. Code and models are released at https://github.com/hustvl/OmniMamba

World Action Models (WAMs) unify future environment prediction with action generation for autonomous driving, yet existing approaches optimize only the final outputs, leaving intermediate representations as incidental byproducts. We present ReWorld, the first representation learning framework specifically designed for autonomous-driving WAMs. ReWorld explicitly optimizes the latent world-to-action pathway through three complementary mechanisms. First, it imposes future-predictive supervision on intermediate Video DiT states to encode temporal scene dynamics, enabling self-guided sampling and a roughly twofold convergence speedup. Second, it aligns Action DiT states with their attended video readouts so that the retrieved world information is retained in the representations used for planning. Third, it shapes the action space using geometrically close yet low-scoring hard negatives to separate the expert trajectory from nearby unsafe alternatives. ReWorld constructs supervision entirely from the WAM's own generation targets and attended features, requiring no external encoders or teacher models and introducing only 0.3

World models have become crucial for autonomous driving, as they learn how scenarios evolve over time to address the long-tail challenges of the real world. However, current approaches relegate world models to limited roles: they operate within ostensibly unified architectures that still keep world prediction and motion planning as decoupled processes. To bridge this gap, we propose DriveLaW, a novel paradigm that unifies video generation and motion planning. By directly injecting the latent representation from its video generator into the planner, DriveLaW ensures inherent consistency between high-fidelity future generation and reliable trajectory planning. Specifically, DriveLaW consists of two core components: DriveLaW-Video, our powerful world model that generates high-fidelity forecasting with expressive latent representations, and DriveLaW-Act, a diffusion planner that generates consistent and reliable trajectories from the latent of DriveLaW-Video, with both components optimized by a three-stage progressive training strategy. The power of our unified paradigm is demonstrated by new state-of-the-art results across both tasks. DriveLaW not only advances video prediction significantly, surpassing best-performing work by 33.3% in FID and 1.8% in FVD, but also achieves a new record on the NAVSIM planning benchmark.

3D point clouds constitute the quintessential digital representation of the physical world and serve as the founda-tional data format for the emerging era of spatial computing and digital transformation,propelled by advances in high-precision sensing,high-performance computing,and next generation communication networks.These data are indispens-able across a wide range of frontier applications,from the precise environmental mapping required for autonomous driving and the preservation of cultural heritage to the immersive interactive experiences of virtual reality and the complex simula-tions found in digital twins and deep space exploration.A point cloud is a massive collection of disordered data points,with each individual point encoding precise geometric coordinates along with a rich set of attributes,such as color,reflectance,and surface normals,which allow for a high-fidelity reconstruction of 3D scenes that far exceeds the capabilities of tradi-tional 2D imagery.However,this comprehensive spatial representation comes at the cost of an enormous data volume.For instance,a single frame of a large-scale scene can contain millions of points with high-precision geometry and attribute information,generating a staggering amount of data that poses severe challenges to existing storage capabilities and trans-mission bandwidth.Consequently,high-efficiency 3D point cloud coding and communication systems have become a criti-cal research focus for industry and academia,enabling the scalable deployment of 3D visual services.This study provides a comprehensive review of the state-of-the-art developments in the field,systematically analyzing advancements from indus-trial standardization efforts to cutting-edge academic research.First,from an industrial perspective,this study elucidates the technical evolution and architectural differences between international and domestic standards.The Moving Picture Experts Group has led the international standardization through the development of two distinct frameworks known as video-based point cloud compression(V-PCC)and geometry-based point cloud compression(G-PCC).V-PCC is designed to leverage the mature and highly optimized hardware ecosystem of existing video codecs by projecting 3D data onto 2D planes,generating occupancy,geometry,and attribute maps that can be compressed using standards,such as high effi-ciency video coding,making it an ideal solution for dense and dynamic contents,such as telepresence and volumetric video.Meanwhile,G-PCC addresses the unique characteristics of sparse data typical of LiDAR sensors by utilizing native 3D coding structures,including octrees and prediction trees,to model geometry directly in the spatial domain without pro-jection while utilizing sophisticated techniques,such as the region adaptive hierarchical transform,to compress attributes.Alongside these international developments,the audio video coding standard workgroup of China(AVS)has made signifi-cant strides in establishing domestic standards that compete on the global stage by introducing novel optimizations specifi-cally tailored for LiDAR scenarios,such as spherical coordinate prediction trees that align with the scanning mechanisms of sensors and semantic aware coding strategies that prioritize objects of interest.Second,this study comprehensively surveys academic research progress across four dimensions of the communication system:compression coding,sampling and enhancement,quality assessment,and transmission control.In the realm of compression coding,this study highlights the paradigm shift from traditional signal processing to deep learning architectures.In geometry compression,this study evolves from simple octree-based entropy coding to sophisticated methods utilizing voxel-based autoencoders,sparse convo-lutions,and Transformers,which effectively capture long-range spatial dependencies and local geometric details to achieve superior rate-distortion performance.In attribute compression,which often consumes the majority of the bitstream,this study examines the evolution from region-adaptive hierarchical transforms and lifting schemes to learnable transforms and joint geometry-attribute coding frameworks that exploit the correlation between spatial structures and color information to minimize redundancy.Regarding sampling and enhancement,this study analyzes strategies for addressing the sparse,unstructured,and non-uniform nature of point clouds.This study reviews up-sampling algorithms that reconstruct dense surfaces from sparse inputs and quality enhancement techniques-such as graph signal processing and neural network-based filtering-that mitigate compression artifacts and restore visual fidelity.In the domain of quality assessment(QA),this study traces the development of metrics from simple geometric measures,such as point-to-point(p2p)and point-to-plane(p2pl)distances,to complex perceptual evaluations.This study emphasizes the recent surge in projection-based and multi-modal fusion methods and highlights the critical advancement of no-reference(NR)quality assessment using deep learning,an area where domestic researchers have shown significant leadership by developing models that accurately pre-dict human visual perception without requiring pristine reference data.In transmission control,this study discusses mecha-nisms to ensure reliable delivery over bandwidth-constrained and error-prone networks,including fine-grained rate control algorithms that dynamically adjust quantization parameters based on content analysis,and joint source-channel coding(JSCC)schemes that optimize the trade-off between compression efficiency and error resilience.Furthermore,this study explores emerging research on semantic communication,which proposes a shift from signal-level fidelity to semantic-level understanding,optimizing transmission for machine vision tasks rather than mere reconstruction.In addition,this study critically analyzes the disparity between domestic and international research landscapes.Although the international commu-nity,led by MPEG,maintains a mature standardization ecosystem with comprehensive software libraries and datasets,Chi-nese researchers have established a significant innovative advantage in AI-driven compression algorithms,particularly for dynamic and LiDAR point clouds,as well as in blind quality assessment methodologies.Finally,this study offers forward-looking predictions on future development trends,highlighting several key directions:improving attribute compression effi-ciency to match geometry coding gains;minimizing the computational complexity of deep learning models for real-time edge deployment;developing unified frameworks that integrate sampling,compression,and enhancement;and evolving toward semantic-aware communication architectures that support the next generation of intelligent systems.This study aims to provide a comprehensive and in-depth technical reference for researchers and engineers,fostering further innova-tion in the rapidly evolving field of 3D point cloud coding and communication.The above-mentioned methods are linked at:.

Streaming video generation holds strong potential for world modeling, where future frames must be inferred online sequentially to form a continuous video stream. However, streaming video diffusion models introduce a fundamental train-inference mismatch: inference follows a specialized denoising order, whereas advanced training strategies typically require diverse noise-level configurations. To address this trade-off between train-inference consistency and training coverage, we reformulate the video diffusion sampling as a frame-indexed stochastic process over noise levels. Within this stochastic process space, we construct a continuous training trajectory along which the sampling schedule progressively evolves from independent sampling to inference-consistent sampling. We further introduce a joint calibration algorithm and a temporal correlative sampling algorithm to ensure trajectory smoothness and cross-frame correlation. Building on these designs, we propose Stream Forcing, a unified training framework for streaming video generation that balances training sufficiency and inference efficiency. Extensive experiments demonstrate that Stream Forcing significantly improves generation quality with a 36.6