High-quality geometry reconstruction from multi-view images with subsequent appearance decomposition into the physical shading components could enable a seamless integration of neural reconstructions into the modern rendering workflow. While 3D reconstruction techniques have steadily improved, the task of inverse rendering by decomposing an appearance into lighting effects and material properties remains fundamentally ill-posed and highly ambiguous. We show that current state-of-the-art inverse rendering approaches fail to accurately recover material properties, significantly impacting relighting quality. Furthermore, we demonstrate that existing evaluation methods, which rely on image-based metrics, do not adequately capture the reconstruction quality in novel lighting conditions. Our findings illustrate the dependence of current systems on simplified setups with predefined illumination, which are necessary to reliably disentangle light and material contributions and to ultimately achieve convincing relighting.
Measuring road appearance in situ requires either very specific hardware difficult to implement, or extracting core samples from used roads. This is why reflectance estimation from virtual samples constructed from non-invasive acquisitions are useful in a wide range of applications. This article proposes an analysis of several combinations for merging geometric and individual reflectance data, in order to estimate the resulting uncertainty, with various distributions.
Cameras make images collecting per-pixel measurements of light reflected by the objects in the world. Commonly, these measurements undergo a transformation so that they become values in a standardized color space, such as the sRGB space. This makes it possible to send the values to a display device and produce in a human a visual sensation as close as possible to what would have been caused by the original scene. In this work we aim to explore the difficulties and opportunities that arise in devising such non-bijective transformations, visualizing differences between device vision and human vision. In particular we are interested in the practical impact of observer metamerism: different camera devices and human observers can distinguish a different set of spectral stimuli presented to them. When characterizing a camera, this is usually ignored, missing potential to increase chromatic acuity where the camera can see more than the human observer. A question that arises is whether the metameric stimuli involved here do actually appear in practice in relevant cases. We run numeric experiments to investigate these questions.
3D printing has been extensively used for over two decades by various practitioners and professionals in the industry. This technique, which involves adding material from melted filament layer by layer based on CAD model geometry, imparts a unique appearance to the printed objects. The layering structure generates specific directional reflectance patterns on printed surfaces, leading to anisotropic highlights. Due to slight inaccuracies in the printing setup, the appearance of individual layers is not seamless and exhibits sparkle-like effects along the highlight. In this paper, we conducted a psychophysical experiment to analyze human perception of the printed objects, focusing on the intensity and width of the anisotropic highlights. We discovered that the contrast near the highlights and the variability of pixel intensities along the highlights are highly correlated with human ratings. Lastly, we present a straightforward method utilizing these computational features to enhance the visualization of 3D printed objects.
While colorimetry is traditionally measuring point colors, there is an increasing need to quantify colors of 3D objects in real-world scenes. 3D objects, especially translucent ones, exhibit high spatio-temporal variation in color. This raises multiple questions on how to measure color of 3D translucent objects, how to describe their color appearance, and how to quantify color differences among them. Or are these ill-posed problems in the first place? We discuss the first steps on this topic and suggest the future directions for color and appearance research.
Physically-based shaders benefit from physically measured material parameters. The Reflectometer is a simple-to-build, portable device that allows measurement of diffuse albedo, smoothness, and specular F-0 in the field.
Block compression is a widely used technique to compress textures in real-time graphics applications, offering a reduction in storage size. However, their storage efficiency is constrained by the fixed compression ratio, which substantially increases storage size when hundreds of high-quality textures are required. In this paper, we propose a novel block texture compression method with neural networks, Neural Texture Block Compression (NTBC). NTBC learns the mapping from uncompressed textures to block-compressed textures, which allows for significantly reduced storage costs without any change in the shaders. Our experiments show that NTBC can achieve reasonable-quality results with up to about 45% less storage footprint, preserving real-time performance with a modest computational overhead at the texture loading phase in the graphics pipeline.
Translucency is an appearance attribute used to characterize materials with some degree of subsurface light transport. Although translucency as a radiative transfer inside the medium is relatively well understood, translucency as a perceptual attribute leaves much room for interpretation. Our understanding of the translucency perception mechanisms of the human visual system remains limited. No agreement exists on how to quantify perceived translucency, how to compare translucency of multiple objects and materials, how translucency relates to transparency and opacity, and what are the perceptual dimensions of it. We highlight the challenges in perception research arisen by these ambiguities and argue for the need for standardization.
SVBRDF (spatially varying bidirectional reflectance distribution function) recovery is concerned with deriving the material properties of an object from one or more images. This problem is particularly challenging when the images are casual rather than calibrated captures. It makes the problem highly under specified, since an object can look quite different from different angles and from different light directions. Yet many solutions have been attempted under varying assumptions, and the most promising solutions to date are those which use supervised deep learning techniques. The network is first trained with a large number of synthetically created images of surfaces, usually planar, with known values for material properties and then asked to predict the properties for image(s) of a new object. While the results obtained are impressive as shown through renders of the input object using recovered material properties, there is a problem in the accuracy of the recovered properties. Material properties get entangled, specifically the diffuse and specular reflectance behaviors. Such inaccuracies would hinder various down stream applications which use these properties. In this position paper we present this property entanglement problem. First, we demonstrate the problem through various property map outputs obtained by running a state of the deep learning solution. Next we analyse the present solutions, and argue that the main reason for this entanglement is the way the loss function is defined when training the network. Lastly, we propose potential directions that could be pursued to alleviate this
Modern photorealistic rendering simulates spectral behaviour of light. Since many assets are still created in different RGB color spaces, spectral upsampling of the RGB colors to a spectral representation is required to use them in a spectral renderer. Limiting the upsampled spectra to physically valid and natural, i.e. smooth, spectra results in a more realistic image, but decreases the size of the gamut of colors that can be recreated. In order to upsample wide gamut color spaces with colors outside the gamut of physically valid reflectance spectra, a previous approach added fluorescence to create accurate and physically valid representations. We extend this approach to increase the realism and accuarcy while considering memory and computation time.
In this paper, we present a novel heterogeneous subsurface scattering (sss) representation, which is based on a combination of Singular Value Decomposition (SVD) and genetic optimization techniques. To find the best transformation that is applied to measured subsurface scattering data, we use a genetic optimization framework, which tries various transformations to the measured heterogeneous subsurface scattering data to find the fittest one. After we apply the best transformation, we compactly represent measured subsurface scattering data by separately applying the SVD per-color channel of the transformed profiles. In order to get a compact and accurate representation, we apply the SVD on the model errors, iteratively. We validate our approach on a range of optically thick, real-world translucent materials. It’s shown that our genetic algorithm based heterogeneous subsurface scattering representation achieves greater visual accuracy than alternative techniques for the same level of compression.
There is a general shortage of standardized comparisons in the field of appearance modeling. We therefore introduce a benchmark for assessing the quality of reflectance models on a dataset of high quality material measurements obtained with a commercial appearance scanner. The dataset currently consists of 56 fabric materials which are measured as radiometrically calibrated HDR images together with a precise surface geometry. We pose a public challenge to attract further participation and spark new research. Participants evaluate their models on provided directional light and view sampling to recreate the appearance of a set of unseen images per material. The results are automatically evaluated under various image metrics and ranked in a public leaderboard. Our benchmark provides standardized testing and thus enables fair comparisons between related works. We also release baseline SVBRDF material fits.
We propose a taxonomy and terminology for rendering engineers to use in describing the main categories of mathematical lobes that are combined to implement bidirectional scattering distribution functions (BSDFs). Bringing consistent language to this area will increase clarity in API names, textbooks
In this article we demonstrate robust estimation of the model parameters of a fully-linear data-driven BRDF model from a reflectance map under known natural lighting. To regularize the estimation of the model parameters, we leverage the reflectance similarities within a material class. We approximate the space of homogeneous BRDFs using a Gaussian mixture model, and assign a material class to each Gaussian in the mixture model. We formulate the estimation of the model parameters as a non-linear maximum a-posteriori optimization, and introduce a linear approximation that estimates a solution per material class from which the best solution is selected. We demonstrate the efficacy and robustness of our method using the MERL BRDF database under a variety of natural lighting conditions, and we provide a proof-of-concept real-world experiment.
The accurate reproduction of layered materials is an important part of physically-based rendering applications. Since no exact analytical model exists for any configuration of layer stacks, available models make approximations. In this paper, we propose to evaluate them with a numerical approach: we simulate BRDFs and BTDFs for layered materials in order to compare existing models against a common reference. We show that: (1) no single model always outperforms the others and (2) significant differences remain between simulated and modeled materials. We analyse the reasons for these discrepancies and introduce immediate corrections.
With the global trend in customer preference towards achromatic car colors, color designers in coating industry strive to create novel design-critical appearances based on novel effect pigments. At microscopic scale, the pigment particles allow to create specific optical effects like sparkle under directed lighting along with a specific texture in diffuse lighting, while at a macroscopic scale they create the appearance of angle-dependent color and a strong luminance contrast. Although individual particles in effect coatings exhibit anisotropic behavior, the majority of effect coatings exhibit isotropic appearance at a macroscopic scale due to a random orientation of the particles which can be explained with the manufacturing process of the coating. This paper demonstrates an visual appearances achievable by using anisotropic effect coatings based on magnetic pigments. In a psychophysical study, we assessed visual attractiveness of these coatings on a car-like shape for different viewing angles.
Microfacet shading models in film and game production have long used a simple approximation to the Fresnel equations, published by Schlick in 1994. Recently a growing number of film studios have transitioned to using the full Fresnel reflectance equations in lieu of Schlick’s approximation. This transition has been facilitated by Gulbrandsen’s 2014 parameterization which uses reflectance and edge tint instead of eta and kappa. Our recent investigations have found some unexpected drawbacks to this approach. In this presentation, we will show that in the context of RGB rendering (still by far the most common modality in film production), the Fresnel equations are actually less physically principled than Schlick’s approximation. In addition, they are less robust in practice and less amenable to authoring. Most surprisingly, as commonly used the Fresnel equations result in less accurate matches to measured materials, compared to Schlick’s approximation. The presentation primarily discusses metal reflectance, since our investigations so far have focused on metals.
In a recent journal paper, we introduced a technique to represent reflectance spectra by an arbitrary number of Fourier coefficients. As a special case, we converted tristimulus data to three Fourier coefficients. After summarizing this work, we introduce the Fourier sRGB color space. It is defined in terms of Fourier coefficients but designed to behave similar to sRGB. Textures stored in Fourier sRGB support efficient spectral rendering but can be compressed with techniques designed for sRGB textures. Compression errors are similar to sRGB.
