Text-conditioned image editing has greatly benefitted from the advancements in Image Diffusion Models. However, extending these techniques to facial video editing introduces challenges in preserving facial identity throughout the source video and ensuring consistency of the edited subject across frames. In this paper, we introduce DiffMagicFace, a unique video editing framework that integrates two fine-tuned models for text and image control. These models operate concurrently during inference to produce video frames that maintain identity features while seamlessly aligning with the editing semantics. To ensure the consistency of the edited videos, we develop a dataset comprising images showcasing various facial perspectives for each edited subject. The creation of a data set is achieved through rendering techniques and the subsequent application of optimization algorithms. Remarkably, our approach does not depend on video datasets but still delivers high-quality results in both consistency and content. The excellent effect holds even for complex tasks like talking head videos and distinguishing closely related categories. The videos edited using our framework exhibit parity with videos that are made using traditional rendering software. Through comparative analysis with current state-of-the-art methods, our framework demonstrates superior performance in both visual appeal and quantitative metrics.
Many analyses in geotechnical engineering can be conceptualized as the interaction of a continua with rigid bodies or intricate boundaries. The coupling MPM-DEM method is recognized as an effective numerical framework for resolving such problems. Nevertheless, most existing study suffer from intrinsic performance constraints imposed by their program-structure design, and rarely can be extended to simulations involving million-scale particles on the desktop-class graphics processing unit (GPU). In this study, a fully GPU-resident MPM-DEM framework based on CUDA is proposed. A linked-cell neighbour search accelerated with two parallel prefix-sum scans, together with a tunable Verlet buffer, reduces the cost of broad-phase collision detection, whereas racefree particle-to-grid transfers are enforced with CUDA atomic operations. Verification against (i) a half cylinder-plane contact, (ii) projectile penetration, and (iii) granular-flow impact on stacked blocks shows excellent agreement with analytical solutions and laboratory measurements. The Baige landslide in China was simulated with 4,398,192 material points and 641,280 triangular facets. This large-scale case verifies the method's applicability to real-world geohazards and achieves a computational speed approximately 30 times faster than the runtime reported in a conventional GPU-MPM analysis. These results confirm that the proposed framework delivers strong scalability and substantial computational speed-ups for coupled MPM-DEM analyses of complex, large-scale geotechnical processes.
Creating visually appealing textured 3D scenes is pivotal to the successful production of video games and movies. With the advancement of text-to-image models, numerous studies have achieved impressive results for single-object texturing, while generating style-consistent textures at the scene level remains a challenge due to multi-view inconsistencies, texture discontinuities, and computational inefficiency. This paper introduces ScenePainter, a novel object-scene-object framework for scene-level texture generation based on diffusion models. ScenePainter addresses the fundamental limitations of existing approaches by strategically decomposing the generation process into three stages: first establishing semantic structures through object-level texturing, then enforcing global style consistency through scene-level refinement, and finally preserving fine-grained details through object completion. Experiments demonstrate that ScenePainter produces geometrically accurate, semantically coherent, and stylistically unified textures, achieving superior aesthetic scores and significantly outperforming baseline methods in user studies.
Diffusion models demonstrate outstanding performance in image generation, but their multi-step inference mechanism requires immense computational cost. Previous works accelerate inference by leveraging layer or token cache techniques to reduce computational cost. However, these methods fail to achieve superior acceleration performance in few-step diffusion transformer models due to inefficient feature caching strategies, manually designed sparsity allocation, and the practice of retaining complete forward computations in several steps in these token cache methods. To tackle these challenges, we propose a differentiable layer-wise sparsity optimization framework for diffusion transformer models, leveraging token caching to reduce token computation costs and enhance acceleration. Our method optimizes layer-wise sparsity allocation in an end-to-end manner through a learnable network combined with a dynamic programming solver. Additionally, our proposed two-stage training strategy eliminates the need for full-step processing in existing methods, further improving efficiency. We conducted extensive experiments on a range of diffusion-transformer models, including DiT-XL/2, PixArt-$\alpha$, FLUX, and Wan2.1. Across these architectures, our method consistently improves efficiency without degrading sample quality. For example, on PixArt-$\alpha$ with 20 sampling steps, we reduce computational cost by 54% while achieving generation metrics that surpass those of the original model, substantially outperforming prior approaches. These results demonstrate that our method delivers large efficiency gains while often improving generation quality. .
Unoriented surface reconstruction is an important task in computer graphics and has extensive applications. Based on the compact support of wavelet and orthogonality properties, classic wavelet surface reconstruction achieves good and fast reconstruction. However, this method can only handle oriented points. Despite some improved attempts for unoriented points, such as iWSR, these methods perform poorly on sparse point clouds. To address these shortcomings, we propose a wavelet-based method to represent the mollified indicator function and complete both the orientation and surface reconstruction tasks. We use the modifying kernel function to smoothen out discontinuities on the surface, aligning with the continuity of the wavelet basis function. During the calculation of coefficient, we fully utilize the properties of the convolutional kernel function to shift the modifying computation onto wavelet basis to accelerate. In addition, we propose a novel method for constructing the divergence-free function field and using them to construct the additional homogeneous constraints to improve the effectiveness and stability. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both orientation and reconstruction for sparse models. We align the matrix construction with the compact support property of wavelet basis functions to further accelerate our method, resulting in efficient performance on CPU. Our source codes will be released on GitHub.
Multi-view video reconstruction plays a vital role in computer vision, enabling applications in film production, virtual reality, and motion analysis. While recent advances such as 4D Gaussian Splatting (4DGS) have demonstrated impressive capabilities in dynamic scene reconstruction, they typically rely on the assumption that input video streams are temporally synchronized. However, in real-world scenarios, this assumption often fails due to factors like camera trigger delays or independent recording setups, leading to temporal misalignment across views and reduced reconstruction quality. To address this challenge, a novel temporal alignment strategy is proposed for high-quality 4DGS reconstruction from unsynchronized multi-view videos. Our method features a coarse-to-fine alignment module that estimates and compensates for each camera's time shift. The method first determines a coarse, frame-level offset and then refines it to achieve sub-frame accuracy. This strategy can be integrated as a readily integrable module into existing 4DGS frameworks, enhancing their robustness when handling asynchronous data. Experiments show that our approach effectively processes temporally misaligned videos and significantly enhances baseline methods.
Reconstructing dynamic 3D scenes with photorealistic detail and temporal coherence remains a significant challenge. Existing Gaussian splatting approaches modeling scenes rely on per-frame optimization, causing them to overfit to instantaneous states rather than learning true motion dynamics. To address this, we present Mango-GS, a multi-frame, node-guided framework for high-fidelity 4D reconstruction. Our approach leverages a temporal Transformer to learn complex motion dependencies across a window of frames, ensuring the generation of plausible trajectories. For efficiency, this temporal modeling is confined to a sparse set of control nodes. These nodes are uniquely designed with decoupled position and latent codes, which provide a stable semantic anchor for motion influence and prevents correspondence errors for large movements. Our framework is trained end-to-end, enhanced by a input masking strategy and two multi-frame loss to ensure robustness. Extensive experiments demonstrate that Mango-GS achieves state-of-the-art quality and fast rendering speed, enabling high-fidelity reconstruction and real-time rendering of dynamic scenes.
3D Gaussian Splatting (3DGS) enables real-time novel view synthesis for static scenes. Extending it to dynamic scenes via deformation fields has recently attracted significant attention, particularly for dynamic scene reconstructionband distractor-free. However, existing deformation networks lack explicit motion awareness: they neither capture long-term motion intensity nor exploit short-term temporal coherence, leading to inaccurate foreground deformation and pseudo-static residuals in the background. We present MVFusion-GS, a method that enhances deformation networks with two complementary motion-aware mechanisms. The Motion-Variance Guided Refinement aggregates per-Gaussian deformation statistics across time to estimate motion variance and uses it to guide dynamic-static separation during deformation prediction. The MotionFormer Temporal Attention module applies Transformer self-attention over neighboring timesteps to model local motion dependencies and improve temporal consistency. Extensive experiments on both dynamic scene reconstruction and distractor-free reconstruction benchmarks demonstrate state-of-the-art performance, showing that explicit motion awareness improves both foreground motion modeling and static background reconstruction.
Recent visual generation models have made major progress in photorealism, typography, instruction following, and interactive editing, yet they still struggle with spatial reasoning, persistent state, long-horizon consistency, and causal understanding. We argue that the field should move beyond appearance synthesis toward intelligent visual generation: plausible visuals grounded in structure, dynamics, domain knowledge, and causal relations. To frame this shift, we introduce a five-level taxonomy: Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation, progressing from passive renderers to interactive, agentic, world-aware generators. We analyze key technical drivers, including flow matching, unified understanding-and-generation models, improved visual representations, post-training, reward modeling, data curation, synthetic data distillation, and sampling acceleration. We further show that current evaluations often overestimate progress by emphasizing perceptual quality while missing structural, temporal, and causal failures. By combining benchmark review, in-the-wild stress tests, and expert-constrained case studies, this roadmap offers a capability-centered lens for understanding, evaluating, and advancing the next generation of intelligent visual generation systems.
Diffusion models have achieved remarkable success in image and video generation. However, their inherently multiple step inference process imposes substantial computational overhead, hindering real-world deployment. Accelerating diffusion models is therefore essential, yet determining how to combine multiple model acceleration techniques remains a significant challenge. To address this issue, we introduce a framework driven by large language models (LLMs) for automated acceleration code generation and evaluation. First, we present DiffBench, a comprehensive benchmark that implements a three stage automated evaluation pipeline across diverse diffusion architectures, optimization combinations and deployment scenarios. Second, we propose DiffAgent, an agent that generates optimal acceleration strategies and codes for arbitrary diffusion models. DiffAgent employs a closed-loop workflow in which a planning component and a debugging component iteratively refine the output of a code generation component, while a genetic algorithm extracts performance feedback from the execution environment to guide subsequent code refinements. We provide a detailed explanation of the DiffBench construction and the design principles underlying DiffAgent. Extensive experiments show that DiffBench offers a thorough evaluation of generated codes and that DiffAgent significantly outperforms existing LLMs in producing effective diffusion acceleration strategies.
The rapid rise of Deepfake technology poses serious threats to digital forensics, including risks of identity theft, misinformation, and online abuse. Although recent deep learning-based detectors achieve promising results on benchmarks, their generalization to real-world scenarios remains limited because detection relies on fragile low-level cues sensitive to noise, compression, and device-specific artifacts. We propose Self-Reference Guided Adaptive Classification (SRGAC), a framework that improves robustness by dynamically generating image-specific classification parameters through self-referential reference features. To further address spurious correlations in Vision Transformer (ViT) architectures, we introduce the Soft Invariant Constraint (SIC), an extension of invariant risk minimization that suppresses background biases while preserving critical feature diversity. Extensive experiments on multiple Deepfake benchmarks show that SRGAC with SIC achieves state-of-the-art results, delivering stronger cross-dataset generalization and robustness than existing approaches. These findings highlight the potential of our method for practical and reliable Deepfake detection in real-world forensic applications.
Look-Up Table based methods have emerged as a promising direction for efficient image restoration tasks. Recent LUT-based methods focus on improving their performance by expanding the receptive field. However, they inevitably introduce extra computational and storage overhead, which hinders their deployment in edge devices. To address this issue, we propose ShiftLUT, a novel framework that attains the largest receptive field among all LUT-based methods while maintaining high efficiency. Our key insight lies in three complementary components. First, Learnable Spatial Shift module (LSS) is introduced to expand the receptive field by applying learnable, channel-wise spatial offsets on feature maps. Second, we propose an asymmetric dual-branch architecture that allocates more computation to the information-dense branch, substantially reducing inference latency without compromising restoration quality. Finally, we incorporate a feature-level LUT compression strategy called Error-bounded Adaptive Sampling (EAS) to minimize the storage overhead. Compared to the previous state-of-the-art method TinyLUT, ShiftLUT achieves a 3.8$\times$ larger receptive field and improves an average PSNR by over 0.21 dB across multiple standard benchmarks, while maintaining a small storage size and inference time.
Diffusion models achieve strong generative performance but remain slow at inference due to the need for repeated full-model denoising passes. We present Token-Adaptive Predictor (TAP), a training-free, probe-driven framework that adaptively selects a predictor for each token at every sampling step. TAP uses a single full evaluation of the model's first layer as a low-cost probe to compute proxy losses for a compact family of candidate predictors (instantiated primarily with Taylor expansions of varying order and horizon), then assigns each token the predictor with the smallest proxy error. This per-token “probe-then-select’’ strategy exploits heterogeneous temporal dynamics, requires no additional training, and is compatible with various predictor designs. TAP incurs negligible overhead while enabling large speedups with little or no perceptual quality loss. Extensive experiments across multiple diffusion architectures and generation tasks show that TAP substantially improves the accuracy–efficiency frontier compared to fixed global predictors and caching-only baselines.
In many high-precision industrial applications, robots are deployed to perform precision peg-in-hole assembly on large batches of manufactured pegs and holes. When the nominal design adopts a transition fit, machining errors may cause each peg-hole pair to exhibit either a clearance fit or a slight interference fit, with an unknown and continuously varying fit amount. This paper addresses robotic batch precision assembly tasks under uncertain fit types and fit amounts, and proposes a novel multi-stage framework for systematically constructing a robust assembly control strategy. The overall batch precision assembly task is first decomposed into multiple deterministic subtasks characterized by different but fixed fit amounts. For these subtasks, a force-vision fusion controller-driven reinforcement learning method combined with a multi-task reinforcement learning training framework (FVFC-MTRL) is developed to jointly learn multiple compliance control strategies. Subsequently, a multi-teacher policy distillation scheme is designed to integrate multiple trained strategies into a single student strategy, yielding a robust control strategy. Real-world experiments demonstrate that the proposed method successfully constructs a robust control strategy for high-precision assembly tasks with varying fit types and fit amounts. Moreover, the MTRL framework significantly improves training efficiency by 50%, and the final control strategy achieves superior force compliance and a higher success rate compared with several existing methods.
In the field of image editing, inpainting tasks that aim to integrate customized elements into a given image context pose significant challenges. Existing methods often fail to adequately incorporate contextual features from the source image and exhibit limitations in maintaining subject consistency. This paper presents a novel approach to subject-driven image inpainting, leveraging a context-aware framework to achieve superior semantic integration of subjects. Our method introduces two key innovations. First, we employ an online augmented dataset constructed through image stitching, enabling the diffusion inpainting model to learn contextual embeddings of subject images. Second, we enhance the model’s semantic understanding by integrating subject image encoding into the diffusion UNet via cross-attention mechanisms and incorporating a prompt identifier. Through extensive experimentation, we demonstrate state-of-the-art performance across overall quality score metrics, surpassing recent methods in the field.
In some high-precision industrial applications, robots are deployed to perform precision assembly tasks on mass batches of manufactured pegs and holes. If the peg and hole are designed with transition fit, machining errors may lead to either a clearance or an interference fit for a specific pair of components, with uncertain fit amounts. This paper focuses on the robotic batch precision assembly task involving components with uncertain fit types and fit amounts, and proposes an efficient methodology to construct the robust and compliant assembly control strategy. Specifically, the batch precision assembly task is decomposed into multiple deterministic subtasks, and a force-vision fusion controller-driven reinforcement learning method and a multi-task reinforcement learning training method (FVFC-MTRL) are proposed to jointly learn multiple compliance control strategies for these subtasks. Subsequently, the multi-teacher policy distillation approach is designed to integrate multiple trained strategies into a unified student network, thereby establishing a robust control strategy. Real-world experiments demonstrate that the proposed method successfully constructs the robust control strategy for high-precision assembly task with different fit types and fit amounts. Moreover, the MTRL framework significantly improves training efficiency, and the final developed control strategy achieves superior force compliance and higher success rate compared with many existing methods.
Despite significant progress in 3D content generation, producing high-resolution textures for 3D meshes remains challenging. In this work, we present HRDreamer, a novel high-resolution texture generation framework that employs multi-scale hierarchical diffusion guidance for precise texture alignment with geometric details. Central to our approach is a multi-scale interval score matching strategy, which seamlessly integrates global context and local details across multiple resolution scales to ensure accurate texture alignment. In addition, we introduce a hierarchical semantic guidance module, leveraging multimodal large language models to achieve fine-grained semantic control. This resolves the semantic pollution issue commonly seen in patch-based high-resolution generation. Our method generates high-quality textures with rich details while maintaining semantic consistency. Extensive experiments on diverse 3D meshes confirm the effectiveness of our approach.
As the scale of vision models continues to grow, Visual Prompt Timing (VPT) has emerged as a parameter-efficient transfer learning technique, noted for its superior performance compared to full fine-tuning. However, indiscriminately applying prompts to every layer without considering their inherent correlations, can cause significant disturbances, leading to suboptimal transferability. Additionally, VPT disrupts the original self-attention structure, affecting the aggregation of visual features, and lacks a mechanism for explicitly mining discriminative visual features, which are crucial for classification. To address these issues, we propose a Semantic Hierarchical Prompt (SHIP) fine-tuning strategy. We adaptively construct semantic hierarchies and use semantic-independent and semantic-shared prompts to learn hierarchical representations. We also integrate attribute prompts and a prompt matching loss to enhance feature discrimination and employ decoupled attention for robustness and reduced inference costs. SHIP significantly improves performance, achieving a 4.9% gain in accuracy over VPT with a ViT-B/16 backbone on VTAB-1k tasks. Our code is available at https://github.com/haoweiz23/SHIP.
Diffusion models have gained attention for their success in modeling complex distributions, achieving impressive perceptual quality in SR tasks. However, existing diffusion-based SR methods often suffer from high computational costs, requiring numerous iterative steps for training and inference. Existing acceleration techniques, such as distillation and solver optimization, are generally task-agnostic and do not fully leverage the specific characteristics of low-level tasks like super-resolution (SR). In this study, we analyze the frequency- and spatial-domain properties of diffusion-based SR methods, revealing key insights into the temporal and spatial dependencies of high-frequency signal recovery. Specifically, high-frequency details benefit from concentrated optimization during early and late diffusion iterations, while spatially textured regions demand adaptive denoising strategies. Building on these observations, we propose the Time-Spatial-aware Sampling strategy (TSS) for the acceleration of Diffusion SR without any extra training cost. TSS combines Time Dynamic Sampling (TDS), which allocates more iterations to refining textures, and Spatial Dynamic Sampling (SDS), which dynamically adjusts strategies based on image content. Extensive evaluations across multiple benchmarks demonstrate that TSS achieves state-of-the-art (SOTA) performance with significantly fewer iterations, improving MUSIQ scores by 0.2 - 3.0 and outperforming the current acceleration methods with only half the number of steps.
This article focuses on improving the compliance, efficiency, and robustness of batch precision assembly of small-scale pegs flexibly absorbed by a suction cup. The main contribution is that a force-vision fusion fuzzy control method (FVFFC) is proposed to achieve precision assembly with unknown clearance or interference fit. Both visual and force features are designed to describe the state of the peg and hole with the deformation of the suction cup. Then, a force-vision fusion control framework is proposed, where the visual features dynamically modify the reference position of admittance control and guide compliant adjustment of the peg angles. Furthermore, based on theoretical analysis, two fuzzy logic inference modules are developed to estimate the contact state as well as either the clearance or interference amount between the peg and hole in order to adaptively tune the control parameters. Finally, sufficient experiments are conducted to demonstrate the superiority and robustness of the FVFFC method.
Jean-Claude Paul合作论文数Institute of Computer Aided Design12