Abstract Inverse rendering is the problem of recovering scene geometry, material properties, and illumination from one or more observed images. It has experienced rapid progress in recent years, driven by advances in differentiable rendering, neural scene representations, and generative priors. This paper presents a comprehensive review of inverse rendering methods published between 2020 and 2025. In contrast to prior surveys on neural rendering, which review methods primarily focused on novel‐view synthesis without explicit material and lighting decomposition, and intrinsic image decomposition surveys, which review approaches for reflectance and shading separation at the image level, we focus on methods that aim to solve the complete inverse rendering problem. We organize the surveyed approaches using a structured taxonomy that categorizes methods by input and output representations, computational strategies, and evaluation protocols. We further analyze commonly used datasets, evaluation strategies, and performance trade‐offs, and discuss the strengths and limitations of existing approaches in terms of accuracy, efficiency, and generalization. Finally, we identify open challenges, including the ill‐posed nature of material and illumination disentanglement, the lack of standardized benchmarks for joint evaluation, and the limited exploration of downstream applications such as relighting and scene editing.
Abstract Structured sheet materials such as 3D‐printed rod networks, multi‐material thin shells, and multi‐layer laminates exhibit diverse mechanical behaviors. To avoid the computational burden of native‐scale simulations, data‐driven homogenization offers a promising alternative. This process involves probing a representative patch of material—a unit cell—with a set of stretching and bending tests subject to periodic boundary conditions. Because macro‐scale bending moments are not directly available from native‐scale simulations, existing methods exclusively rely on elastic energy data. Unfortunately, using only elastic energy from uniaxial tests is not sufficient for capturing the full moment‐curvature relationship, and imposing biaxial curvature states would necessarily break periodicity. We present a moment‐based homogenization method that infers curvature coupling using only uniaxial bending tests. Our method computes macro‐scale bending moments from native‐scale simulations for a wide range of mechanical models. To this end, we translate internal deformations into elastic stresses and then integrate these stresses through the thickness and across the unit‐cell patch. We use the resulting homogenized bending moments along with energy data to fit neural bending energy density functions. We demonstrate our method on a diverse set of materials, including multi‐material shells, rod networks, and multi‐layer sheets. Our results show improved accuracy compared to existing approaches and realistic double‐curvature behavior when applied to larger samples.
Abstract Compositional 3D indoor scene generation is a long‐standing problem and a rapidly evolving area of research spanning computer graphics, 3D computer vision, and machine learning. The goal is to model the complex relationships among objects and their spatial and functional arrangements within a scene, enabling the creation of rich, diverse and useful 3D environments for a wide range of applications. This survey offers a comprehensive overview of the state of the art, formulating a unifying framework for analyzing scene generation systems and systematically categorizing existing methods according to their approaches to key components. We review recent progress, analyze the strengths and limitations of different paradigms, and highlight both major advances and open challenges. Our survey aims to serve as a resource for researchers and practitioners, offering insights into the current landscape and inspiring new ideas for future work in this area.
Abstract Unsigned distance fields (UDF) are a versatile, implicit representation of geometry. They can represent surfaces that are not bounding a solid or contain points or curves that are not manifold, for example several sheets meeting along a common curve. Contouring the implicit representation, i.e. turning it into an explicit one, requires finding the zero level set. This is challenging because of the lacking sign information. We present an adaptive re‐sampling approach based on regular triangulations that allows efficiently querying the UDF function at the most important locations. Using a dual contouring approach, information on the topology of the reconstructed surface patches is available during refinement, enabling to increase the resolution in the more critical non‐manifold regions.
Abstract The well‐established As‐Rigid‐As‐Possible (ARAP) energy has various forms. For surface deformation, commonly used energies contain an implicit bending penalty. We present a natural, continuous generalization that incorporates multiple ARAP versions with an implicit, user‐controllable bending penalty. We discretize an intuitive variant of the energy and demonstrate that it is independent of mesh resolution and produces comparable results to those of competing methods that include an explicit bending penalty term. We validate our method and demonstrate that, despite its computational overhead, it converges among the fastest of all ARAP variants.
Abstract We present the first closed‐form, per‐element eigensystems for 2D deformation energies in the primary domain of spatial coordinates. Previous analyses have only been able to find such eigensystems in proxy coordinates such as the deformation gradient, but as the spatial positions directly correspond to global degrees of freedom, our analysis opens up avenues for novel algorithms that were not possible before. First, we propose a spectral preconditioner that approximates global near‐nullspace modes using local analytical eigenvectors and clusters global eigenvalues using a low‐rank correction. Second, we derive a unified projected Newton formulation that combines the element eigensystem with the mass and damping terms, yielding a superior eigenvalue filter. We test our preconditioner on surface parameterization problems and our projected Newton framework on dynamic simulation examples. The results show significant efficiency gains over previous methods. In particular, our physics‐aware filtering strategy reduces eigenvalue clamping by up to 30×, which in turn decreases the Newton iteration count by roughly 2×.
Abstract We propose a first method to generate neural unsigned or signed distance fields (SDFs) that are guaranteed to be conservative with respect to a given 3D shape. This means the true distance is never overestimated and the zero‐level set is a bounding volume for the shape. The method makes use of neural network architectures that ensure Lipschitz continuity by design in combination with a novel tailored training data selection scheme and constrained training strategy. We demonstrate that this yields both theoretical and empirical benefits over previous approaches to conservativeness (for non‐distance neural implicits), allowing for tighter approximation and additionally providing the valuable distance information.
Abstract Transformers are sequence‐to‐sequence architectures originally designed to handle structurally rigid and order‐sensitive data, such as text and images. At their core, they exploit the attention mechanism, which is permutation‐equivariant and relies on computing token‐to‐token relationships. These models have been applied to 3D geometry in several instances, achieving discrete success across tasks such as shape generation, segmentation, classification, shape matching, and registration. While existing 3D geometry methods use transformers as traditional learners, we present an approach that reinterprets the transformer as an optimization pipeline for shape correspondence. By fitting the model directly to a shape pair, our method eliminates the need for large training datasets, providing a category‐agnostic solution. In particular, we focus on the use of attention weights, tailored to encode token‐to‐token information, to inject and extract point‐to‐point information during the processing of one or more meshes. We demonstrate, that self‐ and cross‐attention mechanisms can, by design, serve as feature extractors and matching solvers, respectively. Furthermore, instead of deriving correspondence from the final output of the network, we exploit the cross‐attention weights directly as the permutation matrix. This framework not only achieves robust shape matching and registration but also provides a theoretically grounded, interpretable approach to attention for unstructured 3D data. Notably, our work represents an approach that leverages the transformer architecture as an end‐to‐end pipeline for shape correspondence, operating effectively without requiring additional training data.
Abstract We present ParaMG, a surface multigrid method that restores classical multigrid structure on curved surfaces by expressing all hierarchy levels in a single globally consistent planar domain. Existing surface multigrid methods rely on composed local parameterizations or 3D projections for cross‐level transfer, requiring repeated per‐level local approximations that can degrade convergence. Our key insight is that a global conformal parameterization with cone singularities provides a shared coordinate system with low, controllable distortion, where prolongation reduces to exact planar barycentric interpolation. We construct the hierarchy directly in this domain using a seam‐aware simplification strategy, yielding sparse prolongation matrices with three entries per row. ParaMG converges in fewer V‐cycles than prior surface multigrid methods and delivers substantial speedups in applications with changing linear systems, including geometric flows, thin‐shell simulation, and polycube deformation.
Abstract Inside–outside classification is widely used for geometry processing tasks such as surface reconstruction, geometry completion, and calculating signed distance fields. We introduce a new integral formulation of this problem, which assigns confidence scores that points are inside or outside, given incomplete boundary geometry. Even though our geometric construction does not appear in previous work, we show that it is unexpectedly linked to both the well‐established generalized winding number (GWN) and pseudonormal methods for geometry completion, and it provably reduces to either one of them for specific values of a control parameter. The results obtained with our method frequently outperform screened Poisson surface reconstruction (PSR), GWN, and the pseudonormal method in terms of quality, and are at least on par with them on all of our examples. Unlike these methods, our algorithm naturally extends to the multi‐label setting, in which regions with an arbitrary number of colors or physical materials can be reconstructed, and non‐manifold features such as T‐junctions may appear in the interface and boundary geometry.
Abstract Neural implicit surfaces are a powerful tool for encoding a surface as the zero level set of a neural function. Trained using gradient‐descent based optimizers, these methods however suffer from a low‐frequency bias that prevents them to fit fine details of the surface. Solutions to break this bias exist but often damage the regularity of the long range implicit function, making geometrical queries more difficult. In this work, we combine a 1‐Lipschitz neural function with a detail field made of compactly supported radial basis functions. While the neural function handles long‐range queries with low surface accuracy, the detail field is fitted to perturb the neural function so that its zero level set precisely goes through sampled points of the surface and locally accounts for small‐scale details. As it is compactly supported, regions where the 1‐Lipschitz property breaks are finely controlled. We therefore achieve the best of both worlds by greatly increasing the representation power of the implicit function near the zero level set while maintaining robustness far from the zero level set.
Abstract We propose a novel framework for the statistical modeling and analysis of the spatio‐temporal shape variability in articulated 4D (i.e., 3D + time) shapes such as human bodies and animals. We treat articulated 3D shapes, represented using parametric models such as SMPL or its variants, as elements of the product space of shape and pose parameters. 4D shapes can then be seen as trajectories in this space, which has a nonlinear Riemannian structure. Our key contribution is to treat these trajectories as elements of a Riemannian shape space and propose computational tools that ( 1 ) perform temporal alignment of such trajectories to account for variations in their execution rates, ( 2 ) compute geodesics between trajectories, and thus 4D shapes, even when they exhibit different execution rates, and ( 3 ) statistically model the spatio‐temporal variability of collections of 4D shapes, enabling us to compute statistical summaries such as means and principal modes of variation. We derive a simple, yet efficient, framework for characterizing populations of 4D shapes using statistical models, which in turn can be used as a generative model for synthesizing novel 4D shapes by sampling from these distributions. We demonstrate the effectiveness of the proposed framework using publicly available 4D human and animal datasets, and show that it outperforms the state‐of‐the‐art both in terms of accuracy and computational efficiency. Our code, dataset, and videos that illustrate the results are available at https://arti4d.github.io/Arti4D/ .
Abstract 3D Gaussian Splatting (3DGS) has recently enabled real‐time, high‐quality novel view synthesis. However, its formulation treats each Gaussian independently, lacking explicit information exchange between neighboring elements and limiting its ability to capture local structure. This leads to suboptimal trade‐offs between reconstruction quality and representation compactness. A natural extension is to introduce interactions among neighboring Gaussians, yet naive message passing often results in training instability and over‐smoothing, degrading fine details. In this work, we present FlowGS, a graph‐based interaction framework that explicitly models and controls information flow among Gaussians. Our key insight is that neighborhood interaction in 3DGS is fundamentally a problem of controlling information flow, rather than simply enabling feature propagation. To this end, we represent Gaussians as nodes in a spatial graph to define the flow topology, and enable content‐aware feature aggregation for information exchange. Crucially, we introduce a learnable dynamic gating mechanism that adaptively controls, for each Gaussian, how much information to preserve from itself and how much to incorporate from its neighbors. We further incorporate lightweight regularization and a simple yet effective stabilization strategy to ensure robust and stable optimization. Extensive experiments show that, under the same training budget as vanilla 3DGS, our method consistently achieves higher reconstruction quality while using significantly fewer Gaussians, with up to 6.74 dB PSNR gain (on the Deep Blending playroom scene) and an average reduction of approximately 64.3% in Gaussian count. These results demonstrate that controlling information flow is key to enabling efficient and high‐quality point‐based scene representations.
Abstract We present a constructive method for generating field‐aligned, volume‐filling curves in watertight 3D volumes. Existing approaches rely on slow gradient‐based optimization and often struggle to simultaneously achieve good field alignment, regular spacing, and scalability. To overcome this limitation, we adopt a constructive strategy: a set of well‐spaced, aligned curves is constructed and then stitched together to form a single closed curve. Our core idea is to represent the intermediate curves as the intersection of two 3D wavefronts orthogonal to the tangent field. The wavefronts are the isosurfaces of two wave fields that can be optimized efficiently. This formulation enables the generation of high‐quality volume‐filling curves with improved alignment and spacing compared to prior work, while scaling to outputs with more than 10 million vertices. We further demonstrate robustness on 4398 solids from the Thingi10K dataset. Our current method assumes watertight inputs whose smallest geometric feature exceeds twice the target curve spacing.
Abstract We introduce stylized phase manifolds—a compact, interpretable latent representation that disentangles motion content (e.g. “jumping”, “walking”), the temporal structure (e.g. motion cycle frequency, gait timing), and style (i.e. how the motion is performed). Learned in an unsupervised manner and inherently low‐dimensional, the manifold offers intuitive and flexible editing. Building on this representation, we develop a diffusion‐based motion generator that enables fine‐grained control over semantic, temporal, and stylistic aspects of motion. To connect high‐level intent with low‐level motion, we treat the stylized manifold as an intermediate representation—a structured bridge between natural language and motion. By first mapping text into this manifold, our two‐stage pipeline improves the control over for text‐based motion generation, while producing high‐quality, diverse motion outputs.
Abstract This paper proposes t‐Regina, a novel iterative scheme that automates the manipulation of grid handles in free‐form deformation (FFD) for deformable registrations of cartoon character sprites. First, we build a data‐driven FFD (dFFD) that enables users to handle locations of FFD handles from lower‐dimensional parameters. To prepare training data, we implement an FFD‐based puppet tool and recorded the user‐designed locations of grid handles. Second, we iteratively optimize the parameters of dFFD by using a locally optimal block matching algorithm with almost‐uniform sampling strategies. t‐Regina is effective and easy to integrate into existing drawing systems. This paper shows some examples of deformation results to demonstrate the robustness of t‐Regina.
Abstract Cell migration is a biological process involved in various physiological and pathological phenomena. Advances in microscopy have generated large, multimodal datasets on cellular movement, whose complexity challenges analysis. Data visualization offers powerful tools to explore such data, but the development of effective tools is often hindered by the lack of structured analytical requirements and a shared vocabulary between visualization specialists and domain experts. This article addresses this gap by providing a structured, task‐oriented overview of data visualization for cell migration analysis. To this end, we developed and validated a task taxonomy for cell migration analysis in collaboration with domain experts. Following the PRISMA protocol, we conducted a systematic review to map the main functionalities of existing computational tools to the tasks they support and the visualization techniques they employ. Our contributions are: (i) a validated task taxonomy that provides a common framework for organizing analytical needs and guiding tool development; (ii) a systematic review that synthesizes the current landscape of visualization‐based tools for cell migration analysis; and (iii) a research agenda that identifies current gaps and outlines future opportunities for the data visualization field. This work provides a structured reference for cell migration data analysis, facilitating collaboration, guiding tool design, and advancing research in this fascinating domain.
Abstract In recent years, the Smoothed Particle Hydrodynamics (SPH) approach has been increasingly used for multiphase simulations involving interactions between diverse materials. A critical component of an SPH simulator is the pressure solver, which not only facilitates the simulation of compressible or incompressible fluids but also handles contact by preventing penetration between different materials. Currently, most SPH simulations in computer graphics employ implicit dual pressure solvers such as PBF, IISPH, or DFSPH. However, these solvers often exhibit instability when simulating high density ratios. Furthermore, they are difficult to strongly couple with many existing methods for non‐pressure forces, which typically utilize primal formulations. Consequently, pressure and non‐pressure solvers are often only weakly coupled, which can lead to stability issues. We present a novel implicit primal SPH pressure solver designed for multiphase simulations. Our method enables stable simulation of multiple interacting materials with large density ratios. We show that our solver robustly handles ratios of up to 1:1000 (e.g., air‐water interactions) which was not possible with previous implicit SPH pressure solvers. Moreover, we demonstrate how our solver allows for strong coupling with existing implicit simulation methods for viscosity, elasticity, and surface tension. Overall, our strong coupling significantly improves stability in complex multiphase simulations involving fluids, highly viscous materials, and deformable solids.
Abstract We propose a new next‐event estimation (NEE) technique for Gaussian Process Implicit Surfaces (GPISes). We show that the distribution of surface normals available for sampling at a GPIS hit point collapses from 2D to 1D as the GPIS realizations approach heightfields, causing existing NEE to fail entirely in this limit. We analyze this collapse and develop a new strategy that directly importance samples the resulting 1D arc of valid scattering directions restricted to the spherical cap towards a light source. Our technique enables NEE on heightfield GPISes for the first time and provides substantial variance reduction for highly anisotropic near‐heightfield configurations. We combine our strategy with existing techniques via multiple importance sampling for robust performance across all degrees of anisotropy, and support arbitrary emitter shapes via sphere proxies.
Abstract Path tracing uses Monte Carlo integration to solve the rendering equation by evaluating the integrand at random sampling points. The convergence rate of the error can be significantly improved by using correlated instead of random sampling, especially on smooth integrands. However, on integrands with discontinuities due to, e.g., occlusion, the improvement is less pronounced. Prior work has shown that the variance of the estimator is equal to the product of the power spectrum of the integrand and the expected power spectrum of the sampling pattern. Discontinuous integrands have anisotropic power spectra that exhibit high energies along the directions of the discontinuities, which need to match the low‐energy directions of the sampling pattern to reduce variance. However, existing anisotropic sampling patterns have at most two low‐energy directions. Therefore, we propose an optimization‐based algorithm to synthesize two‐dimensional correlated sampling patterns with spectra that have more than two low‐energy directions, leading to improved convergence behavior. Further, we propose a practical and sample‐efficient algorithm that estimates the directions of discontinuities in the power spectra of two‐dimensional integrands. We show that our algorithm can reliably estimate these directions, allowing us to align the low‐energy directions of anisotropic correlated sampling patterns with the predicted directions. We demonstrate in an offline path tracer with light source sampling that our aligned sampling patterns improve the convergence rate on two‐dimensional integrands with multiple discontinuities compared to existing anisotropic sampling patterns and thus reduce the error more quickly.
