Many biological materials exhibit time-dependent reflectance changes, such as fruit oxidation and leaf senescence. We present a novel, low-cost, and portable multispectral imaging solution, aided by machine learning, for real-time monitoring of dynamic reflectance. Built around a miniaturized 16-channel spectral camera (415-809 nm), the system captures wide-field, time-resolved spectral reflectance data without bulky hardware. Experiments on leaf senescence and fruit oxidation demonstrate the ability to capture wavelength-dependent and spatially heterogeneous reflectance variations over long-duration or short-duration. The results highlight the potential of the compact multispectral imaging for non-destructive monitoring application in agriculture, food science, and primary industries.

Unsupervised image stitching aims to align multi-view images with overlaps by learning from unlabeled datasets. Although great progress has been made to improve the robustness and generalization in recent years, most of them focus on the natural warping, large parallax, etc., and neglect the boundary regularity, which may undermine the wide-angle effects. To address the limitations above, we propose BRecStitch, which further incorporates the boundary rectification, achieving a good balance between content alignment and boundary regularity. Considering that both stitching and boundary rectification are non-trivial tasks, we propose a two-step unsupervised learning strategy. First, we design a novel unsupervised network that integrates a global bidirectional homography decomposition strategy to encourage balanced warping across views, and a local residual mesh motion regression to ensure precise alignment and boundary regularity. With the output mesh warpings in the first step, we design a fine-tuning approach to further optimize the stitching results by iterative alignment and rectangular boundary convergence. Extensive experiments and evaluations demonstrate the effectiveness of our method and the advantages over state-of-the-art methods.

Generating human motion that satisfies customized zero-shot goal functions, enabling applications such as controllable character animation and behavior synthesis for virtual agents, is a critical capability. While current approaches handle many unseen constraints, they fail on tasks with very challenging spatiotemporal restrictions, such as severe spatial obstacles or specified numbers of walking steps. To equip motion generators for these highly constrained tasks, we present a retrieval-guided method built on the training-free diffusion noise optimization framework. The key idea is to search within large motion datasets for guidance that can potentially satisfy difficult constraints. We introduce relational task parsing to group target constraints and identify the difficult ones to be handled by retrieved reference. A better initialization for diffusion noise is then obtained via a reward-guided mask that combines random noise with retrieved noise. By optimizing diffusion noise from this improved initialization, we successfully solve highly constrained generation tasks. By leveraging LLM for relational task parsing, the whole framework is further enabled to automatically reason for what to retrieve, improving the intelligence of moving agents under a training-free optimization scheme. Code will be released upon publication.

We present Mover360, a controllable object manipulation framework for 360° images. Unlike perspective images, 360° images in equirectangular projection (ERP) exhibit horizontal wrap-around, latitude-dependent distortion, and global scene continuity, which makes object-level edits difficult for existing perspective editors to produce and for users to specify. To address this, Mover360 centers on object Translation (relocating a specified object within an existing panorama) while supporting reference-guided Insert and Remove as auxiliary tasks. Its interface unifies point-, bbox-, and mask-guided control by encoding each task into a fixed prompt and a compact, ERP-aligned instruction map. In the default point mode, a single click relocates an object, allowing the model to infer a plausible size, support, and illumination using panoramic context and an auxiliary depth condition. Structurally, Mover360 is a lightweight adaptation of a pretrained diffusion transformer. To generate paired supervision, we construct a UE5 data-generation pipeline with surface-aware object placement and randomized illumination, yielding large-scale paired data and a dual-domain benchmark of synthetic and real panoramas with ground truth for all three tasks. Across both test domains and two evaluation protocols, Mover360 outperforms strong baselines for perspective editing, insertion, and inpainting in reconstruction fidelity, semantic consistency, and distributional quality. Code and our benchmark dataset are available at https://zhonghaoyi.github.io/Mover360/.

Reference-based line art video colorization colorizes the target line art according to reference images, which is an essential stage for the cartoon production workflow. However, the manual colorization process is time-consuming and repetitive, making automatic video colorization highly desirable. Existing cartoon colorization methods struggle with domain misalignment between the reference and line art images and the loss of details caused by compression into a low-dimensional space in the existing video diffusion models, reducing colorization quality. In this paper, we propose an Offset Prior-based Diffusion Model (OPDM) for cartoon video colorization, which utilizes the powerful generation capability of the diffusion model and cross-domain matching priors to generate high-quality colorization results. Specifically, we design a simple and effective Offset-Adapter that leverages the idea of sampling offsets in deformable convolution to estimate the cross-domain spatial offset features between the target line arts and reference images. We further introduce a new training strategy that combines forward diffusion and reverse denoising in the training stage to ensure content consistency. Experiments on a public cartoon dataset and our newly constructed long cartoon video dataset demonstrate that our proposed method outperforms the existing state-of-the-art line art coloring methods. Code is available at https://github.com/xzh976/OPDM.

While instruction-based image editing is emerging, extending it to 360° panorama introduces additional challenges. Existing methods often produce implausible results in both equirectangular projections (ERP) and perspective views. To address these limitations, we propose SE360, a novel framework for multi-condition guided object editing in 360° panoramas. At its core is a novel coarse-to-fine autonomous data generation pipeline without manual intervention. This pipeline leverages a Vision-Language Model (VLM) and adaptive projection adjustment for hierarchical analysis, ensuring the holistic segmentation of objects and their physical context. The resulting data pairs are both semantically meaningful and geometrically consistent, even when sourced from unlabeled panoramas. Furthermore, we introduce a cost-effective, two-stage data refinement strategy to improve data realism and mitigate model overfitting to erasing artifacts. Based on the constructed dataset, we train a Transformer-based diffusion model to allow flexible object editing guided by text, mask, or reference image in 360° panoramas. Our experiments demonstrate that our method outperforms existing methods in both visual quality and semantic accuracy.

Image retargeting aims to resize images while preserving salient content free from distortion and stretching. Recent learning-based solutions reduce reliance on traditional hand-crafted features and optimization procedures, leading to more efficient and generalizable performance. Nevertheless, existing approaches still struggle to preserve global structure and content fidelity. To address these limitations, we propose AttenIR, an unsupervised image retargeting framework requiring no manual labels, based on content-aware mesh warping, which effectively couples dense visual attention with sparse mesh constraints. We formulate the mesh deformation as a vertex displacement regression task. Guided by saliency priors, our method jointly optimizes scale, structure, and geometry preservation losses. By combining dense visual attention with semantic features, it achieves robust, unsupervised retargeting without manual annotations. Our primary contributions include (1) a lightweight unsupervised method for image retargeting that ensures content fidelity and preserves global structure, and (2) a coarse-to-fine projection strategy to align dense visual attention priors with sparse mesh constraints, enabling content-aware deformation at the pixel level. Experiments demonstrate that our method achieves competitive and superior performance compared with both traditional and state-of-the-art learning-based methods.

360° depth estimation is a challenging research problem due to the difficulty of finding a representation that both preserves global continuity and avoids distortion in spherical images. Existing methods attempt to leverage complementary information from multiple projections, but struggle with balancing global and local consistency. Their local patch features have limited global perception, and the combined global representation does not address discrepancies in feature extraction at the boundaries between patches. To address these issues, we propose Cross360, a novel cross-attention-based architecture integrating local and global information using less-distorted tangent patches along with equirectangular features. Our Cross Projection Feature Alignment module employs cross-attention to align local tangent projection features with the equirectangular projection's 360° field of view, ensuring each tangent projection patch is aware of the global context. Additionally, our Progressive Feature Aggregation with Attention module refines multi-scaled features progressively, enhancing depth estimation accuracy. Cross360 significantly outperforms existing methods across most benchmark datasets, especially those in which the entire 360° image is available, demonstrating its effectiveness in accurate and globally consistent depth estimation. The code and model are available at https://github.com/huangkun101230/Cross360.

Blind 360° image quality assessment (IQA) aims to predict perceptual quality for panoramic images without a pristine reference. Unlike conventional planar images, 360° content in immersive environments restricts viewers to a limited viewport at any moment, making viewing behaviors critical to quality perception. Although existing scanpath-based approaches have attempted to model viewing behaviors by approximating the human view‑then‑rate paradigm, they treat scanpath generation and quality assessment as separate steps, preventing end-to-end optimization and task-aligned exploration. To address this limitation, we propose RL‑ScanIQA, a reinforcement‑learned framework for blind 360° IQA. RL-ScanIQA optimize a PPO-trained scanpath policy and a quality assessor, where the policy receives quality-driven feedback to learn task-relevant viewing strategies. To improve training stability and prevent mode collapse, we design multi-level rewards, including scanpath diversity and equator-biased priors. We further boost cross‑dataset robustness using distortion‑space augmentation together with rank‑consistent losses that preserve intra‑image and inter‑image quality orderings. Extensive experiments on three benchmarks show that RL‑ScanIQA achieves superior in‑dataset performance and cross‑dataset generalization. Code will be released upon publication.

The growing demand for diverse and realistic character animations in video games and films has driven the development of natural language-controlled motion generation systems. While recent advances in text-driven 3D human motion synthesis have made significant progress, generating realistic multi-person interactions remains a major challenge. Existing methods, such as denoising diffusion models and autoregressive frameworks, have explored interaction dynamics using attention mechanisms and causal modeling. However, they consistently overlook a critical physical constraint: the explicit spatial distance between interacting body parts, which is essential for producing semantically accurate and physically plausible interactions. To address this limitation, we propose InterDist, a novel masked generative Transformer model operating in a discrete state space. Our key idea is to decompose two-person motion into three components: two independent, interaction-agnostic single-person motion sequences and a separate interaction distance sequence. This formulation enables direct learning of both individual motion and dynamic spatial relationships from text prompts. We implement this via a VQ-VAE that jointly encodes independent motions and relative distances into discrete codebooks, followed by a bidirectional masked generative Transformer that models their joint distribution conditioned on text. To better align motion and language, we also introduce a cross-modal interaction module to enhance text-motion association. Our approach ensures the generated motions exhibit both semantic alignment with textual descriptions and preserving plausible inter-character distances, setting a new benchmark for text-driven multi-person interaction generation.

Reconstructing a 4D spatio-temporal representation of a dynamic scene from monocular video is a fundamental yet highly challenging problem in computer vision and computer graphics. Recent advances in 3D Gaussian Splatting (3DGS) for static scenes have significantly improved rendering efficiency and visual fidelity. However, extending 3DGS to dynamic scenes from single-view input remains difficult, as the lack of dynamic point cloud supervision often hinders the accurate modelling of moving objects, leading to suboptimal performance. In this paper, we introduce SD-4DGS, a novel 4D Gaussian Splatting (4DGS) method with spatial densification, specifically designed for dynamic scene reconstruction from monocular video. SD-4DGS features a fast spatial densification strategy that converts sparse point clouds into dense representations to better capture high-frequency geometry and textures. In addition, a sliding window motion regularizer, together with a stage-wise training schedule, progressively refines appearance and motion. Experiments show that our method significantly outperforms existing methods in terms of modelling quality across various datasets.

We present SSD-GS, a physically-based relighting framework built upon 3D Gaussian Splatting (3DGS) that achieves high-quality reconstruction and photorealistic relighting under novel lighting conditions. In physically-based relighting, accurately modeling light-material interactions is essential for faithful appearance reproduction. However, existing 3DGS-based relighting methods adopt coarse shading decompositions, either modeling only diffuse and specular reflections or relying on neural networks to approximate shadows and scattering. This leads to limited fidelity and poor physical interpretability, particularly for anisotropic metals and translucent materials. To address these limitations, SSD-GS decomposes reflectance into four components: diffuse, specular, shadow, and subsurface scattering. We introduce a learnable dipole-based scattering module for subsurface transport, an occlusion-aware shadow formulation that integrates visibility estimates with a refinement network, and an enhanced specular component with an anisotropic Fresnel-based model. Through progressive integration of all components during training, SSD-GS effectively disentangles lighting and material properties, even for unseen illumination conditions, as demonstrated on the challenging OLAT dataset. Experiments demonstrate superior quantitative and perceptual relighting quality compared to prior methods and pave the way for downstream tasks including controllable light source editing and interactive scene relighting.

With the development of deep neural networks and differentiable rendering techniques, neural rendering methods, represented by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have made significant progress. NeRF represents a 3D scene by encoding the appearance and geometry of the scene through neural networks, which are conditioned on both position and viewpoint. In contrast, 3DGS models the scene with a set of Gaussian ellipsoids, allowing for efficient rendering through the rasterization of these ellipsoids into images. However, both two methods are limited to representing static scenes. The rendering and reconstruction of dynamic scenes are critical in virtual reality and computer graphics. As such, extending neural rendering methods from static to dynamic scenes has become an important area of research. This survey organizes dynamic scene rendering methods based on NeRF and 3DGS and categorizes them according to different motion representations. Furthermore, it highlights the relevant applications of dynamic scene rendering, such as autonomous driving, digital humans, and 4D generation. Finally, we summarize the development of dynamic scene rendering and discuss the remaining limitations and open challenges.

We propose a method to reconstruct dynamic fire in 3D from a limited set of camera views with a Gaussian-based spatiotemporal representation. Capturing and reconstructing fire and its dynamics is highly challenging due to its volatile nature, transparent quality, and multitude of high-frequency features. Despite these challenges, we aim to reconstruct fire from only three views, which consequently requires solving for under-constrained geometry. We solve this by separating the static background from the dynamic fire region by combining dense multi-view stereo images with monocular depth priors. The fire is initialized as a 3D flow field, obtained by fusing per-view dense optical flow projections. To capture the high frequency features of fire, each 3D Gaussian encodes a lifetime and linear velocity to match the dense optical flow. To ensure sub-frame temporal alignment across cameras we employ a custom hardware synchronization pattern – allowing us to reconstruct fire with affordable commodity hardware. Our quantitative and qualitative validations across numerous reconstruction experiments demonstrate robust performance for diverse and challenging real fire scenarios.

Recent advances in 3D generative models have rapidly improved image-to-3D synthesis quality, enabling higher-resolution geometry and more realistic appearance. Yet fidelity, which measures pixel-level faithfulness of the generated 3D asset to the input image, still remains a central bottleneck. We argue this stems from an implicit 2D-3D correspondence issue: most 3D-native generators synthesize shape in canonical space and inject image cues via attention, leaving pixel-to-3D associations ambiguous. To tackle this issue, we draw inspiration from 3D reconstruction and propose Pixal3D, a pixel-aligned 3D generation paradigm for high-fidelity 3D asset creation from images. Instead of generating in a canonical pose, Pixal3D directly generates 3D in a pixel-aligned way, consistent with the input view. To enable this, we introduce a pixel back-projection conditioning scheme that explicitly lifts multi-scale image features into a 3D feature volume, establishing direct pixel-to-3D correspondence without ambiguity. We show that Pixal3D is not only scalable and capable of producing high-quality 3D assets, but also substantially improves fidelity, approaching the fidelity level of reconstruction. Furthermore, Pixal3D naturally extends to multi-view generation by aggregating back-projected feature volumes across views. Finally, we show pixel-aligned generation benefits scene synthesis, and present a modular pipeline that produces high-fidelity, object-separated 3D scenes from images. Pixal3D for the first time demonstrates 3D-native pixel-aligned generation at scale, and provides a new inspiring way towards high-fidelity 3D generation of object or scene from single or multi-view images. Project page: https://ldyang694.github.io/projects/pixal3d/

Recent advancements in deep learning-based detection methods have led to substantial improvements in the performance of infrared small target detection (IRSTD). However, most methods rely solely on the original image and overlook the valuable prior knowledge it contains. As a result, these methods exhibit limited capability in extracting and enhancing target features. To tackle this problem, a fusion framework named the Infrared Small Target Detection Fusion Network (ISTDFN) has been developed, consisting mainly of a decomposition network and a detection network. Initially, to guide the network’s focus toward target features, we use the decomposition network to factorize images into sparse and low-rank components. The target information will primarily be contained in the sparse components. Furthermore, to improve the efficiency of feature extraction, the detection network incorporates a feature amplifier based on the Rayleigh distribution, which is inspired by the thermal diffusion model of small targets in physics, enabling dynamic enhancement of target features. Finally, after extracting and enhancing features from the two decomposed components, a singular value feature fusion (SVFF) module is applied in the detection network to facilitate effective information interaction and fusion. Comprehensive experiments demonstrate that the proposed approach can accurately and efficiently detect targets, even under challenging background conditions.

Image stitching allows wide field-of-view images to be created. However, handheld shooting and alignment of overlapping regions in image stitching intrinsically result in irregular boundaries, compromising the wide-angle effect. To address this problem, we propose an unsupervised warping-based method for rectangling stitched images. We formulate irregular mesh prediction as a mesh motion regression task, constrained by three complementary objectives: shape-preserving, boundary-fitting, and content-preserving losses. This approach leverages geometric and semantic features of images to achieve rectangling without requiring labeled training data. Our primary contributions include (1) a label-free learning framework that improves rectification performance and generalization capability, and (2) a novel boundary-fitting scheme that reconstructs well-aligned meshes, producing visually natural rectangling results across diverse scenarios. Experiments demonstrate that our method achieves competitive or superior performance compared with state-of-the-art supervised methods.

3D Gaussian representations have emerged as a powerful paradigm for digital head modeling, achieving photorealistic quality with real-time rendering. However, intuitive and interactive creation or editing of 3D Gaussian head models remains challenging. Although 2D sketches provide an ideal interaction modality for fast, intuitive conceptual design, they are sparse, depth-ambiguous, and lack high-frequency appearance cues, making it difficult to infer dense, geometrically consistent 3D Gaussian structures from strokes—especially under real-time constraints. To address these challenges, we propose SketchFaceGS, the first sketch-driven framework for real-time generation and editing of photorealistic 3D Gaussian head models from 2D sketches. Our method uses a feed-forward, coarse-to-fine architecture. A Transformer-based UV feature-prediction module first reconstructs a coarse but geometrically consistent UV feature map from the input sketch, and a 3D UV feature enhancement module refines it with high-frequency, photorealistic detail to produce a high-fidelity 3D head. For editing, we introduce a UV Mask Fusion technique combined with a layer-by-layer feature-fusion strategy, enabling precise, real-time, free-viewpoint modifications. Extensive experiments show that SketchFaceGS outperforms existing methods in both generation fidelity and editing flexibility, producing high-quality, editable 3D heads from sketches in a single forward pass.

We present a novel 360° panoramic video conferencing system that dynamically synchronizes virtual backgrounds with real-time camera motion, addressing the limitations of static backgrounds in conventional systems. By integrating robust human segmentation, monocular visual odometry (VO), and virtual environment rendering, our method achieves seamless alignment between foreground participants and immersive 3D virtual scenes. Unlike prior approaches that suffer from foreground-background desynchronization during camera rotations or user movements, our framework estimates camera rotation in 3-DoF using a hybrid pipeline combining feature-based patch tracking and pose smoothing, while ignoring translation artifacts to maintain stability. This work bridges the gap between computational efficiency and MR-driven telepresence, offering a practical solution for next-generation virtual collaboration.

Omnidirectional capture of dynamic scenes facilitates the creation of immersive virtual reality assets and holistic scene understanding. Outward-facing multi-fisheye camera rigs offer an efficient solution for full-scene coverage, using fewer lenses than conventional pinhole arrays while enabling all-directional observation of complex, time-varying environments. By continuously recording scene evolution from every angle, these systems naturally enable a richer characterization of dynamic interactions. Despite these advantages, dynamic scene modeling in this setting remains underexplored. Existing methods, typically designed for fixed pinhole configurations or monocular setups, rely heavily on photometric cues and often neglect the strong geometric and semantic priors inherent in multi-fisheye omnidirectional data. To address this gap, we present OmniPrior, a Gaussian Splatting-based framework for outward-facing, multi-fisheye omnidirectional capture. Our approach incorporates metric-geometry-aware initialization with multi-prior guidance, introducing a dynamicness-aware Gaussian representation that encodes both object motion and subtle temporal variations. The resulting representations are physically consistent and temporally stable. Extensive experiments validate the effectiveness of our method in novel view synthesis across new viewpoints and timestamps. We demonstrate its utility in two representative applications derived from our learned representations: 6DoF rendering with flexible FoV and motion-freeze rendering.