Generating high-fidelity, expressive, and realistic 3D head avatars remains a fundamental challenge for immersive applications such as virtual reality, gaming, and telepresence. This task requires not only precise modeling of non-rigid facial deformations but also semantically controllable expression synthesis under diverse viewpoints and motion contexts. We present DynAvatar, a novel framework that integrates expression-guided deformation into the 3D Gaussian splatting pipeline to produce photorealistic and emotionally resonant head avatars. Our method introduces two key innovations: (1) an expression-guided Gaussian deformation module that tightly couples geometric displacement with high-level semantic cues, enabling fine-grained and anatomically meaningful facial animation; and (2) a spatial context embedding mechanism that encodes the canonical position of each Gaussian to preserve semantic coherence and spatial consistency during expression generation. Extensive experiments on both controlled and in-the-wild datasets demonstrate that DynAvatar significantly outperforms state-of-the-art methods in terms of visual realism, expression fidelity, and rendering quality.

Although speech-driven 3D facial animation has been widely used for VR applications such as virtual announcers and made significant progress in recent years, the primary challenge lies in combining speech-related lip movements with facial expressions. Existing methods mainly focus on accurate lip synchronization while ignoring facial emotion expressions, resulting in limited emotional facial realism and a lack of flexibility in the generation process. In this paper, we propose a novel emotional speech-driven 3D facial animation framework, EmoDiffuser, based on the diffusion mechanism. Our model integrates audio, emotion, and facial motion features to generate rich expressions. By introducing an innovative facial denoising decoder, Multi-Conditional Hierarchical Cross-attention (MCHC), which hierarchically injects audio and emotion, we facilitate emotion-related subtle facial expression generation by learning disentangled representations of content and emotion from facial motion. In addition, our model can achieve flexible and continuous emotion transitions by controlling emotion and audio, where the multi-emotion smoothing algorithm makes the emotion transition process more realistic. Comprehensive qualitative and quantitative evaluations indicate that EmoDiffuser surpasses the state-of-the-art (SOTA) performance techniques, with generated facial movements exhibiting superior lip synchronization and more refined facial expressions.

Video saliency detection models have been achieving steady, significant improvements thanks to rapid advances in deep learning and the wide availability of large-scale training sets. However, deep learning-based visual-audio fixation prediction is still in its infancy. At present, only a few visual-audio sequences have been furnished, with real fixations being recorded in real visual-audio environments. Hence, it would neither be efficient nor necessary to recollect real fixations under the same visual-audio circumstances. To address this problem, this paper promotes a novel weakly supervised approach that alleviates the demand for large-scale training sets for visual-audio model training. By using only the video category tags, we propose the selective class activation mapping (SCAM) and its upgrade (SCAM+). In the spatial-temporal-audio circumstance, the former follows a coarse-to-fine strategy to select the most discriminative regions, which are usually capable of exhibiting high consistency with real human-eye fixations. The latter equips the SCAM with an additional multigranularity perception mechanism, making the whole process more consistent with that of the real human visual-audio system. Moreover, we distill knowledge from these regions to obtain completely new spatial-temporal-audio (STA) fixation prediction (FP) networks, enabling broad applications when video tags are unavailable. Without resorting to any real human-eye fixation, the performances of these STAFP networks are comparable to those of fully supervised networks. The code and results are publicly available at https://github.com/guotaowang/STANet .

Realistic and coherent human animations are essential for virtual reality applications to effectively deliver semantic content. Nevertheless, synthesizing such animations poses significant challenges due to the complexity inherent in integrating detailed geometric characteristics with natural, dynamic movements. To address these challenges, we introduce the Contrastive Learning-based Cross-Modal Variational Auto-Encoder (CLCM-VAE), a novel data-driven framework that utilizes semantic information to enhance animation fidelity and generate human-like behaviors. Our proposed CLCM-VAE bridges the gap between mesh-level animations and semantic guidance by applying contrastive learning across modalities, embedding context-aware features from adjacent frames into a unified latent representation to maintain spatial and temporal coherence. To further refine geometric detail and semantic alignment, we employ meta animations as geometric priors, effectively guiding the integration of semantic inputs with mesh animations. Additionally, we propose a novel Spatio-Temporal Alignment Mechanism (STAM) designed to retain discriminative cross-modal features while enhancing intra-class compactness and inter-class separation within the generated animations. Comprehensive evaluations confirm that our method consistently produces vivid, flexible, and realistic animations, even for categories unseen during training, thereby establishing a new benchmark for coherent and semantically enriched human animation synthesis. The source code will be publicly available.

Cardiac electrophysiological modeling is an effective approach for investigating the mechanisms of arrhythmogenesis. So far, the mainstream of cardiac myocyte modeling depicts the electrophysiological properties of cardiac cells by using biophysically detailed differential equations to describe the activation and inactivation processes of ion channel dynamics. However, this approach is cumbersome and time-consuming to construct the mathematical representations for specific cases, impeding physiological adaptive modeling. Here, we introduce an innovative method called physiology-augmented multivariate temporal simulation (PAMTS), to rapidly and adaptively describe the cell-specific physiological behavior of cardiomyocytes. Based on the excitability of biological systems, PAMTS decouples cellular electric activity into external environmental influences and internal multivariate dynamic mechanisms. Then, the physiological information embedded in the mechanistic models is characterized as physiological templates that guide the complex dependencies across multi-physiological attributes, enabling stable and accurate prediction of future cellular physiological behavior. After systematically evaluating PAMTS in multiple electrophysiological modeling tasks, we demonstrate that PAMTS can be accurately and reliably applied to conduct adaptive modeling of the physiological behavior of various cell types. The study suggests that PAMTS has a promising potential to propel the development of personalized cardiac physiological models, which provide novel insights for rapid and adaptive modeling of complex physiological systems.

3D multi-human reconstruction from single images holds significant potential for advancing AR/VR applications. While remarkable progress has been made in single-human reconstruction, existing methods face challenges when reconstructing multiple humans. These challenges include: (1) severe inter-occlusion that disrupts individual body structures, and (2) the absence of physically plausible relative positioning among subjects. We present DECON, a novel DEcouple-and-reCONstruct framework that systematically addresses these limitations through two technical innovations: (1) a decouple-and-reconstruct framework with multi-view synthesis. It separates individuals and reconstructs detailed 3D bodies from a single image. (2) a Perspective-Aware Position Optimization (PAPO) approach. It ensures realistic positioning by fixing overlaps and gaps between subjects. Extensive experiments demonstrate our method's capability to reconstruct fully separated, anatomically complete 3D humans with clothed-geometric details and plausible interactions. Quantitative evaluations show a 54% reduction in Chamfer Distance and 35% in Point-to-Surface Distance compared to state-of-the-art methods.

Medical image registration is essential for aligning heterogeneous imaging data, yet simultaneously capturing fine-grained local details while maintaining global spatial coherence remains a significant challenge. To address this challenge, we propose MGGA, an anatomy-guided, annotation-free framework that integrates the generalization ability of the MedSAM foundation model with a geometry-aware fusion strategy. Specifically, we leverage a fine-tuned MedSAM encoder to extract 2D slice-level structural priors, and associate them with anatomical semantics using a CLIP-based text encoder, thereby reducing sensitivity to intensity variations without requiring task-specific annotations. To bridge the dimensionality gap between these 2D representations and the 3D global context extracted by a parallel spatial encoder, we introduce a Geometry-Aware 2D-3D Feature Fusion Module (GA-FFM). This module utilizes implicit neural representations to project 2D features into volumetric space, guided by a dual-metric mechanism based on Gaussian and Cosine similarities to adaptively weight features according to geometric consistency. Furthermore, a dual-layer regularization strategy is employed to reinforce anatomical plausibility. Comprehensive evaluations on diverse benchmarks, covering both unimodal and multimodal tasks across brain MRI and abdominal CT datasets, demonstrate that MGGA consistently outperforms state-of-the-art methods. The observed robustness to pathological anatomy and anisotropic spacing, together with improved topological validity, suggests that the framework may be applicable to challenging clinical scenarios. The code is available at https://github.com/goghfan/MGGA.

Conventional EEG-based supervised emotion recognition typically requires large manually annotated multi-channel EEG data. The susceptibility of EEG to noise can lead to corruption of certain channels, making it challenging for emotion recognition from few available channels (denoted as few-channel in this paper). Masked Autoencoder (MAE) can learn highly generalizable representations by reconstructing the original signals from the visible subset of channels. However, most studies do not consider temporal masking, and random spatial masking overlooks the brain's inherent structure, impeding reconstruction efficacy. In this paper, we propose a novel pre-training framework, STMAE-Few, based on a spatial-temporal masked autoencoder, which introduces spatial-temporal masking strategies at both the EEG channel and frame levels to extract robust feature representations and enhance the performance of few-channel emotion recognition in real-world scenarios. Inspired by neuroscience, our approach integrates region and region-selection spatial masking strategies and introduces true position encoding during pre-training to capture richer spatial information. Extensive experimental results on SEED, SEED-IV, and SEED-V datasets demonstrate that our pre-training framework can effectively capture the temporal dynamics and spatial correlations within and between different brain regions.

Generative content is increasingly applied in VR to provide immersive experiences, yet maintaining high generation quality remains challenging for audiovisual effects. Particularly in dynamic fluid phenomena, achieving realism and presence requires adherence to physical laws. To accomplish this objective, this work proposes an audiovisual synergy fluid animation generation framework, which enhances immersion by improving motion texture fidelity and audiovisual consistency. It comprises Detail-Enhanced Texture generator (DET) and Physics-Guided Audio generator (PGA). DET integrates Global-Local Physics guidance (GLP) and Temporal Texture Modeling (TTM) to produce video textures, explicitly optimizing dynamic details by leveraging local motion cues and assigned cumulative differences. PGA incorporates Visual Semantic Augmenter (VSA) and Rhythm Semantic Adapter (RSA) to synchronize audio by fusing static visual semantics with dynamic motion semantics to improve temporal coherence. By integrating DET and PGA, this framework strengthens audiovisual immersion in VR natural dynamic scenes from both visual and auditory perspectives. Quantitative and qualitative evaluations demonstrate that our approach surpasses most existing methods in terms of texture realism and audiovisual synchronization, offering new insights for advancing immersive experiences in dynamic VR phenomena.

Immersive VR applications demand high resolutions and refresh rates, posing significant challenges for real-time rendering. Foveated rendering mitigates this cost by exploiting properties of the Human Visual System (HVS), but conventional approaches often rely on oversimplified heuristic models that neglect high-level attentional cues, resulting in artifacts in peripheral regions. To this end, we present a neural saliency-driven foveated ray tracing framework that overcomes these limitations. Our method introduces a motion-aware foveation model to capture temporal dynamics and employs a lightweight convolutional neural network to predict saliency maps that reflect complex attentional patterns derived from eye-gaze data. The combination of these guides adaptive path tracing and filtering, enabling perceptually optimized rendering with minimal artifacts. Experimental results show that our approach improves perceptual quality over prior methods while sustaining real-time performance.

Generating human motion in complex 3D scenes from text is a challenging task with broad applications. However, existing methods often overlook realistic physical contact, resulting in visually plausible but physically unrealistic motion, e.g., penetration. To alleviate this, we propose IntentMotion, a novel framework that generates human motion in 3D scenes from natural language instructions by explicitly modeling intent. We first introduce the Intention-Guided Contact Field (IGCF). This differentiable voxel-based contact region representation explicitly aligns parsed language roles with spatial contact regions through a hierarchical attention mechanism. IGCF is jointly trained with a diffusion-based motion generator, allowing contact predictions to adapt dynamically through gradient feedback. To improve the controllability and physics-aware motion, we further propose an Intention-Aware Diffusion Model (IADM), which decouples the high-level semantic planning from the low-level contact refinement in a coarse-to-fine process. The optimized contact cues are utilized to guide the synthesis of a coarse trajectory, followed by refining detailed pose sequences under IGCF supervision. Experiments on the HUMANISE and LINGO datasets demonstrate that our IntentMotion outperforms recent baselines in contact accuracy, semantic alignment, and generalization to unseen scenes.

Accurately reconstructing high-fidelity human models from single images is critical for virtual reality applications. Existing methods often rely on 3D features from the estimated parametric human model to provide geometric priors. This approach addresses challenges such as missing limbs or deformations, which often arise due to viewpoint limitations and self-occlusion. However, accurately predicting 3D features from monocular images remains a significant challenge. This limitation poses difficulties for the fusion of 2D and 3D information. In this paper, we introduce HFHuman, a novel approach for high-fidelity human reconstruction from a single image using multi-modality fusion. HFHuman effectively fuses multiple modalities, including geometric and depth, directly from images. Our method introduces three key innovations: (1) a depth and geometric parallel reconstruction framework that simultaneously handles whole-body geometry and detailed depth reconstruction, refining a parameterized 3D human model under progressive depth guidance; (2) a pixel-voxel feature fusion strategy that combines pixel-aligned features with voxel-aligned features using a multi-modality adaptor; and (3) a depth-refined technique that integrates RGB imagery with surface normals and depth mapping. By addressing the challenge of blending 2D and 3D modalities, HFHuman results in more accurate and realistic human reconstructions. Experimental results demonstrate that HFHuman outperforms state-of-the-art methods, setting a new standard for realistic 3D human body reconstruction.

High-fidelity 3D MRI synthesis requires both globally coherent anatomy and fine-grained voxel-level detail. Although latent diffusion makes volumetric generation tractable, its image autoencoder introduces a reconstruction bottleneck that can limit the fine detail recoverable in the final volume. We present VoxStruct3D, a voxel-space flow-matching framework that directly models full-resolution MRI volumes using a clean-data prediction objective. Its Volumetric Voxel Generator (VVG) combines factorized 3D patch embedding with overlapping upsampling, time-modulated residual refinement, and skip fusion, enabling neighboring tokens to jointly reconstruct shared voxel regions and suppress patch-boundary artifacts. To complement direct voxel-space modeling with an explicit anatomical prior, we further introduce a Structure-First, Image-Follows (SFIF) strategy. A frozen pretrained 3D medical encoder and a StructVAE extract compact structure tokens that preserve dominant anatomy, while a structure-leading schedule keeps their trajectory ahead of the image trajectory. Patch-Aligned RoPE spatially aligns the unequal token grids, and asymmetric attention enforces one-way guidance from structure to image. Experiments on pathological and healthy T1-weighted brain MRI datasets show that VoxStruct3D achieves the strongest overall performance across feature-distribution alignment, sample diversity, and perceptual quality, producing anatomically coherent and visually realistic volumes.

Generating continuous and expressive human motion from textual descriptions is a critical challenge in applications such as gaming and filmmaking. Existing methods often struggle to maintain global coherence, realistic frame continuity, and smooth transitions. To address these limitations, we propose FCMD, a novel diffusion-based model for generating cohesive motion sequences from fine-grained textual descriptions. FCMD introduces three key innovations: (1) Fine-grained Text Fusion, which integrates detailed textual cues with transitional narratives to enhance semantic consistency; (2) History Motion Guidance, ensuring motion accuracy and consistency across consecutive frames; and (3) Smooth Stitching Sampling, which leverages preceding and current motion information to achieve seamless transitions. Additionally, FCMD employs a large language model (LLM) to refine motion datasets by extracting fine-grained textual descriptions. Extensive experiments demonstrate that FCMD outperforms state-of-the-art methods in generating coherent, natural, and highly controllable motion sequences.

Depth can resolve appearance ambiguity in RGB-D salient object detection (SOD), yet sensor depth is not uniformly reliable. Missing regions, blurred boundaries, and structural artifacts can propagate through multimodal fusion and make an RGB-D detector less accurate than its RGB-only counterpart. Existing quality-aware approaches regulate observed depth but remain dependent on the same potentially defective modality. We propose , a reliability-aware geometry distillation framework developed for RGB-D SOD benchmarks without using dataset-provided depth during training or inference. A frozen Depth Anything V2 model serves only as a training-time teacher, transferring dense relative geometry, hierarchical spatial attention, and boundary structure to a compact edge-aware geometry branch. Pooled bidirectional interaction aligns geometry with appearance, and a pixel-wise reliability estimator selectively injects geometry that is compatible with the current RGB representation. The teacher is removed after training, leaving an RGB-only inference network. Trained on 2,985 RGB-mask pairs, achieves the best or tied-best result in 26 of 36 metric-dataset comparisons against ten recent RGB-D SOD methods, including a 13.4% relative MAE reduction on ReDWeb-S. When retrained on DUTS-TR, it also improves the strongest prior F-measure by 4.2% on PASCAL-S, showing that the distilled geometry transfers beyond a particular sensor or dataset domain. Code will be released upon publication.

Uncalibrated volumetric video streaming for human reconstruction is essential for holographic communication and AR/VR, yet remains challenging due to the need for temporal consistency and computational efficiency from sparse-view inputs. Existing methods rely on per-scene optimization or calibrated cameras, while recent feed-forward models are limited to low-resolution (0.5K) single-frame synthesis. We present HiReFF, a feed-forward method for 2K-resolution 360° human video reconstruction from uncalibrated sparse-view videos. Our framework decomposes the problem into two key tasks: foreground 3D Gaussian reconstruction from sparse-view videos (four views separated by 90°) and computationally efficient high-resolution synthesis. To enable the former, we propose Scale-synchronized Camera Calibration to resolve scale ambiguity for multi-view supervision, and Gaussian-wise Foreground Masking to reconstruct clean foregrounds by modulating Gaussian parameters. For efficient high-resolution synthesis, our High-resolution Side-tuning achieves 2K rendering by augmenting the Gaussian head with supplementary features while keeping the backbone at 0.5K, drastically reducing computational overhead. Experiments demonstrate that HiReFF significantly outperforms existing methods in high-resolution streaming volumetric video reconstruction. https://iridescentjiang.github.io/HiReFF

Haptic-based surgical simulation is widely utilized for training surgical skills. However, simulating the interaction between rigid surgical instruments and soft tissues presents significant technical challenges. In this paper, we propose an energy-based haptic rendering method to achieve both large deformations and rigid-soft haptic interaction. Different from existing methods, both the rigid tools and soft tissues are modeled by an energy-based virtual coupling system. The constraints of soft deformation, tool-object interaction and haptic rendering are defined by potential energy. Benefit from energy-based constraints, we can realize complex surgical operations, such as inserting tools into soft tissue. The virtual coupling of soft tissue enables the separation of haptic interaction into two components: soft deformation with high computational complexity, and high-frequency haptic rendering. The soft deformation with shape constraints is accelerated GPU at a relatively low frequency(60Hz similar to 100Hz), while the haptic rendering runs in another thread at a high frequency (>= 1000Hz). We have implemented haptic simulation for two commonly used surgical operations, pressing and pulling. The experimental results show that our method can achieve stable feedback force and non-penetration between the tool and soft tissue under the condition of large soft deformation.

Understanding complex surgical scenes requires recognizing multiple interdependent entities, such as instruments, actions, and targets, while maintaining their relational consistency across time. Existing surgical triplet recognition methods struggle to jointly model intra-frame label dependencies and inter-frame temporal semantics in a unified manner. To address these limitations, we propose a unified framework that integrates spatial, relational, and temporal cues for robust surgical triplet recognition. Specifically, class-specific spatial priors are first extracted through a multi-scale encoder. These priors are then refined by a Label Correlation Modeling module with multi-scale class activation map-guided relational extraction (MS-CAMRE), enabling the model to capture both static co-occurrence patterns and dynamic contextual dependencies among triplet components. Furthermore, a Bidirectional Temporal-Relational Fusion Attention (BTRFA) module harmonizes temporal and relational representations to achieve coherent temporal reasoning. We also introduce a new evaluation metric, the Triplet Consistency Error Rate (TCER), which quantitatively measures the model's ability to preserve causal and semantic consistency across triplets. Extensive experiments on the CholecT45 and ProstaTD datasets show that our method achieves state-of-the-art performance, improving AP_IVT by 5.1 percent and 7.8 percent, respectively. Moreover, according to TCER, our approach achieves relative reductions of more than 36 percent and 25 percent on the two datasets, respectively, demonstrating the effectiveness of our framework in temporal-relational co-reasoning.

Diffusion-based motion generation has advanced rapidly, but current methods still struggle with long-horizon consistency, style control, and multi-condition guidance. A major reason is the fused-conditioning design, where semantic, stylistic, and temporal signals share a single pathway, causing interference and limiting controllability.We propose MoCoDiff, a controlable autoregressive diffusion framework that introduces Injection Modulation Controllers (IMC). IMC is a lightweight, modality-specific linear modulation modules that inject text, style, and history signals through separate conditioning paths. IMC preserves the simplicity of a frozen backbone while avoiding the entanglement inherent to fused conditioning, enabling more stable and interpretable multi-condition control.To further enhance long-range synthesis, we develop a controllable autoregressive diffusion model equipped with Temporal IMC (TIMC), which applies history as a timestep-dependent corrective signal. This controllable formulation actively suppresses drift, enforces smooth transitions across motion segments, and significantly improves temporal coherence over extended sequences.Experiments show that MoCoDiff achieves state-of-the-art style fidelity, transition quality, and efficiency, while supporting flexible and interpretable multi-condition motion synthesis without retraining.

While feed-forward 3D Gaussian Splatting (3DGS) enables efficient 3D reconstruction, achieving high-fidelity rendering remains challenging. Existing pixel-aligned approaches suffer from spatial inflexibility and massive structural redundancy, whereas query-based methods lack 3D priors and entangle geometry with appearance, yielding blurry, pose-dependent results. To overcome these deficiencies, we propose QuerySplat, a feed-forward 3DGS framework driven by geometric priors and explicit appearance decoupling. Specifically, we design a dual-branch query-based decoder: the geometry branch leverages a pretrained Vision Geometric Model for spatial understanding, which intrinsically endows QuerySplat with pose-free modeling capabilities, while the appearance branch recovers high-frequency details through a dedicated pathway separated from geometric attribute regression. Extensive experiments demonstrate that QuerySplat mitigates the blurry rendering issues of early query-based models and consistently outperforms pixel-aligned approaches in rendering fidelity. On the challenging DL3DV benchmark, it achieves state-of-the-art novel view synthesis performance, with average PSNR gains of 2.30 dB and 1.04 dB over the best pose-free and pose-required baselines, respectively. Project Page: https://inspatio.github.io/querysplat.