Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.
Abstract Recent advances in generative motion synthesis have enabled realtime autoregressive generation of diverse and realistic character animations conditioned on user inputs, as demonstrated by models such as the Conditional Autoregressive Motion Diffusion Model (CAMDM). However, real‐world applications (e.g., computer games) often demand faster‐than‐realtime performance for large numbers of characters. We introduce F3AMD (Fast FiLM‐conditioned Fourier Autoregressive Motion Diffusion), a framework that achieves an order of magnitude speedup over state‐of‐the‐art systems for multi‐character animation on both GPUs and CPUs while maintaining high motion quality. Our key insight is that autoregressive motion diffusion is primarily bottlenecked by architectural and sampling inefficiencies. To address this, F3AMD employs Fourier Neural Operators (FNOs) as encoder‐decoder modules, substitutes Transformer backbones with FNO blocks, replaces condition concatenation with lightweight Feature‐wise Linear Modulation (FiLM), and adopts a variance‐exploding noise schedule with a deterministic sampler. This design enables a substantially lower‐dimensional latent space, facilitates learning in both the spectral and temporal domains, and significantly improves sample efficiency. We conduct systematic ablations of key design factors, including latent dimension, backbone type, diffusion window length, and number of denoising steps. Our recommended configuration, F3AMD‐FNO‐96, achieves 20x speedup over the baseline CAMDM model, while maintaining comparable motion quality.
Abstract Physics‐based simulation can augment facial rigs with high‐quality deformations, but at steep computational costs. While neural surrogates can substantially reduce computation time, existing methods do not generalize across identity variations and instead require expensive per‐case retraining. In this work, we present LORAMI—a method for learning physics‐based face rigs over a continuous space of identities using low‐rank adaptation. LORAMI addresses the challenge of identity variation through a novel architecture that combines a shared neural surrogate with low‐rank weight adaptations. Instead of training a dense model across identity space, we modulate a shared base network using low‐rank factors. These factors are scaled by diagonal matrices predicted from identity parameters. This design enables efficient modeling of identity‐dependent variations while preserving the generic deformation behavior of the underlying physics‐based rig. Our experiments show that LORAMI achieves deformation accuracy on par with single‐identity models and outperforms fully dense identity‐conditioned networks. As a result, our method enables real‐time physics‐based facial animation with continuous identity control.
Abstract We introduce stylized phase manifolds—a compact, interpretable latent representation that disentangles motion content (e.g. “jumping”, “walking”), the temporal structure (e.g. motion cycle frequency, gait timing), and style (i.e. how the motion is performed). Learned in an unsupervised manner and inherently low‐dimensional, the manifold offers intuitive and flexible editing. Building on this representation, we develop a diffusion‐based motion generator that enables fine‐grained control over semantic, temporal, and stylistic aspects of motion. To connect high‐level intent with low‐level motion, we treat the stylized manifold as an intermediate representation—a structured bridge between natural language and motion. By first mapping text into this manifold, our two‐stage pipeline improves the control over for text‐based motion generation, while producing high‐quality, diverse motion outputs.
Abstract This paper proposes t‐Regina, a novel iterative scheme that automates the manipulation of grid handles in free‐form deformation (FFD) for deformable registrations of cartoon character sprites. First, we build a data‐driven FFD (dFFD) that enables users to handle locations of FFD handles from lower‐dimensional parameters. To prepare training data, we implement an FFD‐based puppet tool and recorded the user‐designed locations of grid handles. Second, we iteratively optimize the parameters of dFFD by using a locally optimal block matching algorithm with almost‐uniform sampling strategies. t‐Regina is effective and easy to integrate into existing drawing systems. This paper shows some examples of deformation results to demonstrate the robustness of t‐Regina.
Abstract In recent years, the Smoothed Particle Hydrodynamics (SPH) approach has been increasingly used for multiphase simulations involving interactions between diverse materials. A critical component of an SPH simulator is the pressure solver, which not only facilitates the simulation of compressible or incompressible fluids but also handles contact by preventing penetration between different materials. Currently, most SPH simulations in computer graphics employ implicit dual pressure solvers such as PBF, IISPH, or DFSPH. However, these solvers often exhibit instability when simulating high density ratios. Furthermore, they are difficult to strongly couple with many existing methods for non‐pressure forces, which typically utilize primal formulations. Consequently, pressure and non‐pressure solvers are often only weakly coupled, which can lead to stability issues. We present a novel implicit primal SPH pressure solver designed for multiphase simulations. Our method enables stable simulation of multiple interacting materials with large density ratios. We show that our solver robustly handles ratios of up to 1:1000 (e.g., air‐water interactions) which was not possible with previous implicit SPH pressure solvers. Moreover, we demonstrate how our solver allows for strong coupling with existing implicit simulation methods for viscosity, elasticity, and surface tension. Overall, our strong coupling significantly improves stability in complex multiphase simulations involving fluids, highly viscous materials, and deformable solids.
Creating photorealistic 3D human avatars with realistic upper-body motion remains challenging. Existing approaches either focus on the head and overlook hand gestures, or reconstruct the full body but fail to preserve fine-grained facial fidelity and hand pose accuracy. As a result, current methods struggle to capture the subtle dynamics of facial expressions and hand gestures that are crucial for natural human communication. While methods based on full-body parametric models enable avatar reconstruction from monocular or multi-view inputs, they often lack accurate facial animation and detailed hand articulation. To address these limitations, we propose MVFGA, a novel multi-view-consistent pipeline for generating realistic upper-body avatars. Our approach models the face and hands separately and fuses them with a parametric upper-body mesh model, enabling the capture of fine-grained facial expressions and hand poses for accurate upper-body avatar reconstruction. We then splat 3D Gaussians onto the obtained mesh, enabling high-quality rendering of dynamic avatars from novel viewpoints. Furthermore, we introduce MVFGA-MoCap, a multi-view upper-body motion capture dataset featuring controlled facial expression sequences, diverse hand gestures, and free-form communication. Experiments show that MVFGA generates visually realistic avatars with high-fidelity facial expressions and hand motions, outperforming baselines for upper-body avatar animation. Project page: https://dfki-av.github.io/MVFGA/
Motion editing is central to VFX and game development, where it is used extensively to modify and augment existing movements to conform to new environments or changes in artistic direction. While traditional motion editing can do small modifications, it cannot accommodate larger structural edits, resulting in visual warping artifacts that require authoring new motion. Conversely, recent advances in large-scale generative modeling have unlocked newfound capabilities for authoring entire movements by directly manipulating sparse spatial constraints. While impressive at creating new movements, these methods lack the capability to preserve and edit existing motion interactively. In this work, we introduce scheduled inpainting, a method that enables interactive generative motion editing, a novel paradigm unifying motion synthesis and editing by leveraging generative models. Scheduled inpainting is a simple yet powerful inference-based technique that enables fine-grained spatiotemporal control over the balance between preserving the original motion and generating new content. By building atop generative models that support direct manipulation, our system allows artists to interactively refine existing animations while ensuring results remain natural and consistent with the learned motion distribution. Scheduled inpainting is versatile and supports many editing applications, such as extending, stitching, and compositing different clips. Finally, we extensively validate our approach by comparing with four baselines, conducting ablations of our design, and reporting user feedback.
Realistic animal motion for virtual production is typically obtained either through motion capture of highly trained performers who accurately mimic animal behavior, or by retargeting ordinary human motion using complex control setups. Both approaches are challenging and often fail to fully reproduce the nuances of natural animal motion, motivating data-driven alternatives. We present an automatic human-to-quadruped puppeteering framework that produces plausible and controllable quadruped motions from ordinary human motion data. Our approach employs a two-stage generative diffusion model trained purely on quadruped motion data. By introducing a structured conditioning and inpainting strategy, our method supports a wide range of actions, including walking, running, jumping, sitting, and lying. Furthermore, we enable fine-grained intuitive control of the quadruped motion such as head movement control and individual limb puppeteering. Experimental results demonstrate improved motion realism and controllability compared to existing retargeting approaches, highlighting the effectiveness of our framework as a tool for animation and virtual production applications.
Existing approaches to frictional contact dynamics typically either modify the Coulomb law to improve numerical robustness or solve the exact law in a fully coupled monolithic form. However, in its reduced form, exact Coulomb friction can be written as a cone complementarity problem with an augmented velocity, which reveals a natural split between a cone-constrained linear response and a scalar non-associated coupling induced by tangential velocity. We exploit this structure in the solver design. Our method uses an outer iteration to update the non-associated coupling explicitly, and an inner solve for a strongly convex cone-constrained quadratic program. This separation also makes the inner solver modular, so different numerical schemes can be used without changing the outer iteration. We evaluate the method on rigid-body benchmarks with stick-slip transitions and frictional stacking, and show that it reproduces exact Coulomb complementarity without smoothing or relaxing the friction law.
Simulating inviscid, incompressible fluids on non-simply-connected curved surfaces requires careful treatment of the flow's local and global behavior. While recent theoretical advancements have established the critical dynamics of the harmonic component in such flows, practical applications remain computationally restricted by a lack of spatial and temporal adaptivity. Furthermore, simulations on poor-quality meshes often lead to numerical instability and a failure to preserve the flow's underlying harmonic component when using naive interpolation methods. In this paper, we introduce Adaptive Fluid Cohomology, a framework that integrates dynamic spatial and temporal refinement into the simulation of the Euler equations. We leverage a posteriori error estimation to adjust spatial resolution on the fly, alongside a standard Dormand-Prince 5(4) time-stepping scheme for temporal accuracy. To ensure stability during mesh mutations, we develop a novel method that robustly transfers the harmonic basis during remeshing. While our experimental evaluation focuses on 2D surface flows, the underlying theoretical formulation is presented to capture the 3D setting as well. Our evaluation demonstrates that this adaptive approach accurately recreates the dynamics of high-resolution simulations while reducing the memory footprint by up to 86
Creating realistic 3D human-human interactions in virtual environments is challenging due to the high degrees of freedom in the human body and the need for physically accurate poses that do not collide with each other. Traditional methods for human-human interaction are based on motion tracking or 3D body reconstruction, but lack generative capabilities. Recent generative methods enable the synthesis of individual or interacting motions via text or image input, but generally fall short in modeling close interactions. This paper introduces a novel generative model for close 3D human-human interactions using a conditional variational autoencoder (cVAE), which generates poses for one human conditioned on the pose of another, allowing for controlled and diverse interaction synthesis. To train our model, we address two underlying long-standing challenges in the field of human-human interaction: data scarcity, for which we propose an automated supervised data augmentation strategy that generates synthetic yet realistic interaction poses; and collision awareness in generative approaches, for which we propose a self-supervised loss based on a collision resolution technique using volumetric proxies to ensure physically correct interactions. We extensively evaluate the capabilities of our model, and demonstrate a wide variety of plausible and physically correct interactions, not possible to generate with current state-of-the-art methods.
Synthesizing hand motion that matches the full body motion and the semantic labels is a difficult task due to their high degrees of freedom and the lack of semantic labels. To cope with this issue, we propose a prior-first, condition-second framework for body-conditioned hand motion completion. Our framework first learns a generic body-hand kinematic prior from large-scale unstructured and unlabeled motion data, capturing the intrinsic coordination between global body dynamics and hand articulation. Semantic control is then introduced through lightweight adaptation on top of the frozen prior, avoiding the need to relearn kinematic structure for each control interface. Our framework centers on a streaming, autoregressive body-hand prior that generates coherent, kinematically consistent hand motion from body dynamics in real time, using structured kinematic modeling to maintain mechanical body-hand coupling. To enable practical controllability under limited supervision, we introduce semantically-layered adapters that inject conditioning signals at appropriate kinematic levels, supporting both self-supervised attribute control and weakly supervised text-driven control with only a few hours of labeled data. Extensive evaluations demonstrate that our framework improves kinematic plausibility, robustness, and controllability compared to end-to-end conditioned baselines, particularly in low-resource and cross-dataset settings. We further showcase real-time inference and an interactive authoring workflow, highlighting the applicability to production animation pipelines. Homepage: https://AIGAnimation.github.io/HandPrior/
