We use plenoptic measurements of visible, infrared, and THz radiation to locate and image objects that are hidden from direct view by detecting their passive radiation scattered from rough surfaces.

The visual fidelity of realistic renderings in Computer Graphics depends fundamentally upon how we model the appearance of objects resulting from the interaction between light and matter reaching the eye. In this paper, we survey the research addressing appearance modelling of living human tissue. Among the many classes of natural materials already researched in Computer Graphics, living human tissues such as blood and skin have recently seen an increase in attention from graphics research. There is already an incipient but substantial body of literature on this topic, but we also lack a structured review as presented here. We introduce a classification for the approaches using the four types of human tissues as classifiers. We show a growing trend of solutions that use first principles from Physics and Biology as fundamental knowledge upon which the models are built. The organic quality of visual results provided by these Biophysical approaches is mainly determined by the optical properties of biophysical components interacting with light. Beyond just picture making, these models can be used in predictive simulations, with the potential for impact in many other areas.

Given a single reference stimulus, test stimuli can be sorted with respect to perceptual similarity to this anchor stimulus. Aggregated ranks can then be computed from multiple sort sequences. This ordinal scaling provides an estimate of perceptible differences and can be used to develop and test predictive models. In this paper we propose the use of graph-based methods visualizing experimental data and computing aggregated ranks. Specifically, perceptual similarity is expressed as a sort sequence graph in which nodes are stimuli and weighted edges are the frequency of the corresponding ranks. This graph is also oriented in that it has a start, the reference stimuli, and an end, the least similar stimuli. The Schulze method or the 'strongest path' computation is used for rank aggregation. This analysis is explored in the context of two appearance experiments: the first using solid colors and the second using renderings of 3D printed stimuli varying in multiple appearance attributes. For the second experiment with the renderings of 3D printed stimuli we then use Kendall Tb values to assess a simple model based on mean CIELAB color differences. We find that the underlying sorting task is efficient and intuitive. Furthermore, the graph-based formulation of perceptual similarity allows the application of network analysis and graph theory to the study of visual appearance. New analyses are also possible, such as outlier detection using the sort sequences that are the inverse of the Schulze solution or approximately the 'wrongest path'.

A method is presented for perceptually characterizing appearance non-uniformities that result from 3D printing. In contrast to physical measurements, the model is designed to take into account the human visual system and variations in observer conditions such as lighting, point of view, and shape. Additionally, it is capable of handling spatial reflectance variations over a material's surface. Motivated by Schrödinger's line element approach to studying color differences, an image-based psychophysical experiment that explores paths between materials in appearance space is conducted. The line element concept is extended from color to spatially-varying appearances-including color, roughness and gloss-which enables the measurement of fine differences between appearances along a path. We define two path functions, one interpolating reflectance parameters and the other interpolating the final imagery. An image-based uniformity model is developed, applying a trained neural network to color differences calculated from rendered images of the printed non-uniformities. The final model is shown to perform better than commonly used image comparison algorithms, including spatial pattern classes that were not used in training.

Surface geometry can play an important role in our ability to understand and interpret material appearance and properties. This property ranges from large-scale shape changes impacting our identification of reflections to visible surface roughness affecting gloss perception. In this work, the authors present a user study that examines numerous surface geometries that are defined at the mesoscale: small enough to be considered indicative of the material and not object geometry, but large enough to be visible from a distance with the naked eye. Models of perceived brightness were compared against sparsely collected brightness judgments from the study and used to densely compare many generated mesoscale surface patterns. Averaging incoming luminance over a spatially varying surface proved effective at modeling brightness judgments. The effects of the mesoscale structure on perceived brightness were not directly correlated to parameters such as shape, size, or depth of the bumpy texture elements. (C) 2017 Society for Imaging Science and Technology.

An image-based relighting algorithm has been extended so that it can accommodate environment based lighting. Camera mounted flash photographs, employed in the original relighting algorithm, are also used to achieve the environment map based relighting results. In addition to preserving the simple equipment and setup utilized in the original relighting approach, the new method allows professional studio lighting effects, simulation of museum gallery illumination, and outdoor lighting at particular times of the day and year.

A novel image-based rendering system is proposed for documenting cultural heritage artifacts. The system utilizes backscattering photography to acquire the initial pictures and derives estimates for the object's diffuse albedo, surface normals, and the specular reflectivity from the images. A projective texture mapping technique is used to create a novel view of the artifact by blending the original photographs and projecting them onto a mesh that is also derived from the photos. By weighting the images according to how they best depict the manner in which a virtual light source illuminates the artifact's surface, object relighting is also achieved.

Surface geometry can play an important role in our ability to understand and interpret material appearance and properties. This property ranges from large-scale shape changes impacting our identification of reflections to visible surface roughness affecting how glossy a material appears. In this work we present a user study that examines numerous surface geometries that are defined at the mesoscale: small enough to be considered indicative of the material and not object geometry, but large enough to be visible from a distance with the naked eye. Subjects matched the perceived brightness of a ray-traced bumpy surface to a flat surface with adjustable intensity. Multiple classes of bumpy surface were generated and presented to subjects so that the effects of surface pattern on perceived brightness could be studied. We show that two predictive models of brightness are only conditionally accurate but that humans have a consistent means of measuring overall brightness.

We present an algorithm for morphing shape and view-dependent texture that utilizes an unstructured lumigraph representation to (1) gracefully handle topological changes, (2) create plausible appearance transformations, and (3) provide user control independent from the underlying mesh structure. It works by reducing the otherwise complex problem of a 3D shape and material morph into multiple simpler 2D morphs. The shape morph operates on 2D depth maps and then reconstructs a conventional mesh, making it insensitive to the input geometry quality. Our morphing algorithm robustly handles topology changes and outputs complete meshes and lumigraphs for later use. We show how 2D image morphs can be computed from 3D correspondences, while eliminating ambiguities that might result in the projected 2D morphs.

Environment map based lighting has proven to be an effective technique in simulations and real time rendering for adding realism and conveying complex illumination to the viewer. We present a novel algorithm for using environment maps to light and render polynomial texture maps. The technique is demonstrated with reflectance transformation images stored using both polynomial texture maps and hemispherical harmonics. The limitations of employing hemispherical harmonics in this context are discussed.

An experiment was performed to determine whether typical industrial automotive color paint comparisons made using real physical samples could also be carried out using a digital simulation displayed on a calibrated color television monitor. A special light booth, designed to facilitate evaluation of the car paint color with reflectance angle, was employed in both the real and virtual color comparisons. Paint samples were measured using a multi-angle spectrophotometer and were simulated using a commercially available software package. Subjects performed the test quicker using the computer graphic simulation, and results indicate that there is only a small difference between the decisions made using the light booth and the computer monitor. This outcome demonstrates the potential of employing simulations to replace some of the time consuming work with real physical samples that still characterizes material appearance work in industry.

A computer aided design system for determining the color appearance of metallic automotive coatings has been developed. A sketch based bidirectional reflectance distribution function design interface allows simple concept art to be used to style new metallic car colors. The final design is specified using industrial measurement standards for metallic color appearance, and paint formulations are determined by employing an automotive refinish system. A virtual collection of existing automotive paints, specified using the measurement standard, is provided, and tools for searching this database, for both design and manufacturing purposes, are described. The system is assessed by using it in industrial and educational design studios.

Robust statistical methods are employed to reduce the noise in Monte Carlo ray tracing. Through the use of resampling, the sample mean distribution is determined for each pixel. Because this distribution is uni‐modal and normal for a large sample size, robust estimates converge to the true mean of the pixel values. Compared to existing methods, less additional storage is required at each pixel because the sample mean distribution can be distilled down to a compact size, and fewer computations are necessary because the robust estimation process is sampling independent and needs a small input size to compute pixel values. The robust statistical pixel estimators are not only resistant to impulse noise, but they also remove general noise from fat‐tailed distributions. A substantial speedup in rendering can therefore be achieved by reducing the number of samples required for a desired image quality. The effectiveness of the proposed approach is demonstrated for path tracing simulations.

In this paper we present a technique for significantly improved rendering of objects scanned using photogrammetry techniques. We demonstrate the connection between photogrammetry and the unstructured lumigraph, a surface light field representation and rendering algorithm. We use our lumigraph rendering software to demonstrate improved results from photographically acquired cultural heritage artifacts. We discuss our improvements to the rendering of unstructured lumigraphs on modern hardware and offer this rendering tool as free, open source software.

Material appearance modeling should involve end user testing. These tests can be visual comparison experiments where decisions made with real physical samples are compared with choices done using computer graphic simulations. This type of evaluation is necessary to determine whether simulations derived from material appearance models can be used to accomplish such tasks as the design of new products or the sale of manufactured goods. Lighting conditions, viewing circumstances, decision criteria, and user expertise can vary widely in each situation where the results of material appearance modeling are used to make a choice. An experiment is currently underway to perform such tests for automotive paint simulation.

The propagation and reflection of electromagnetic waves in a three-dimensional environment is simulated, and realistic images are produced using the resulting light distributions and reflectance functions. A finite difference time domain method is employed to advance the electric and magnetic fields in a scene. Surfaces containing wavelength scaled structures are created, the interaction of the electromagnetic waves with these nano-structured materials is calculated, and the sub-surface interference and diffraction effects are modelled. The result is a reflectance function with wavelength composition and spatial distribution properties that could not have been predicted using classic computer graphic ray tracing approaches. The techniques are employed to reproduce demonstrations of simple interference and diffraction effects, and to create computer-generated pictures of a Morpho butterfly.

In this paper we demonstrate that rigorous high-order perturbation of surfaces (HOPS) methods coupled with analytic continuation mechanisms are particularly well-suited for the assessment and design of nanoscale devices (e.g., biosensors) that operate based on surface plasmon resonances generated through the interaction of light with a periodic (metallic) grating. In this connection we explain that the characteristics of the latter are perfectly aligned with the optimal domain of applicability of HOPS schemes, as these procedures can be shown to be the methods of choice for low to moderate wavelengths of radiation and grating roughness that is representable by a few (e.g., tens of) Fourier coefficients. We argue that, in this context, the method can, for instance, produce full and precise reflectivity maps in computational times that are orders of magnitude faster than those of alternative numerical schemes (e.g., the popular "C-method," finite differences, integral equations or finite elements). In this initial study we concentrate on the description of the basic principles that underlie the solution scheme, including those that relate to analytic continuation procedures. Within this framework, we explain how, in spite of conventional wisdom to the contrary, the resulting perturbative techniques can provide a most valuable tool for practical investigations in plasmonics. We demonstrate this with some examples that have been previously discussed in the literature (including treatments of the reflectivity and band gap structure of some simple geometries) and extend this to demonstrate the wider applicability of the proposed approach.

A tooth is a heterogeneously structured object of translucent materials, such as enamel and dentin. The correct simulation of the appearance of teeth is useful in the field of dental restoration to detect the correct color and to develop materials for restoration. However, conventional surface reflection models do not capture the appearance of translucent materials accurately, because they assume that the rays of light are just reflected off the surface. For translucent objects, light not only bounces off the surface, but also interacts with the material under the surface. Therefore, physically correct rendering of teeth must take this into account to understand and model subsurface scattering. This paper discusses the optical characteristics of enamel and dentin. In addition, rendered images using volume photon mapping with Monte Carlo photon tracing are presented for enamel, dentin and biomaterials.

The computer graphic simulation of a common spray painting artifact, called orange peel, is discussed. Orange peel distorts surface reflections and is commonplace in product design applications. The orange peel measurements from a standard industrial instrument are used to construct a height field, and this surface is rendered using traditional normal mapping techniques. Comparisons are made between real panels with orange peel and simulations of those panels. A simple visual model for detecting the presence of orange peel is also presented and evaluated. User testing of the model confirms that orange peel is more visible on dark paint colors than on light paint colors. The latter outcome suggests that to minimize application time, but still keep orange peel below visual threshold, paint application systems should be designed to take paint color into account.