This work presents a characterization method for cameras with trichromatic RGB color filter array and polarization layer (RGB-P) sensor-based imaging devices. Such sensors enable the reconstruction of color and polarization of registered scene elements, which is an important requirement in computer vision. We will present spectral responsivity measurements, which reveal different sensitivities for various color and polarization channels. Furthermore, we will discuss and model an observed chromaticity shift in registered camera signals for polarized irradiance. Both lead to inaccurate estimation of color and polarization features. In order to overcome these issues, we will present a neural-network-based model for color and polarization feature reconstruction. Essentially, it considers spectral sensitivity for polarized irradiance. Furthermore, the model takes into account that, for visualization, the color signals have to be a linear combination of polarization channels. Models were trained for selected natural and synthetic reflectance sets, as well as commonly used lighting. We evaluated the resulting performance, which yielded robust results. The method can be employed for an estimation of color and polarization features for RGB-P imaging devices. Applications can be found in photography, as well as machine and computer vision, in which object surface color rendering plays a major role.

A large number of modern video background modeling algorithms deal with computational costly minimization problems that often need parameter adjustments. While in most cases spatial and temporal constraints are added artificially to the minimization process, our approach is to exploit Dynamic Mode Decomposition (DMD), a spectral decomposition technique that naturally extracts spatio-temporal patterns from data. Applied to video data, DMD can compute background models. However, the original DMD algorithm for background modeling is neither efficient nor robust. In this paper, we present an equivalent reformulation with constraints leading to a more suitable decomposition into fore- and background. Due to the reformulation, which uses sparse and low-dimensional structures, an efficient and robust algorithm is derived that computes accurate background models. Moreover, we show how our approach can be extended to RGB data, data with periodic parts, and streaming data enabling a versatile use.

Frequency-based decomposition of time series data is used in many visualization applications. Most of these decomposition methods (such as Fourier transform or singular spectrum analysis) only provide interaction via pre- and post-processing, but no means to influence the core algorithm. A method that also belongs to this class is Dynamic Mode Decomposition (DMD), a spectral decomposition method that extracts spatio-temporal patterns from data. In this paper, we incorporate frequency-based constraints into DMD for an adaptive decomposition that leads to user-controllable visualizations, allowing analysts to include their knowledge into the process. To accomplish this, we derive an equivalent reformulation of DMD that implicitly provides access to the eigenvalues (and therefore to the frequencies) identified by DMD. By utilizing a constrained minimization problem customized to DMD, we can guarantee the existence of desired frequencies by minimal changes to DMD. We complement this core approach by additional techniques for constrained DMD to facilitate explorative visualization and investigation of time series data. With several examples, we demonstrate the usefulness of constrained DMD and compare it to conventional frequency-based decomposition methods.

We present a GPU-based technique for efficient selection in interactive visualizations of large particle datasets. In particular, we address multiple attributes attached to particles, such as pressure, density, or surface tension. Unfortunately, such intermediate attributes are often available only during the simulation run. They are either not accessible during visualization or have to be saved as additional information along with the usual simulation data. The latter increases the size of the dataset significantly, and the required variables may not be known in advance. Therefore, we choose to compute intermediate attributes on the fly. In this way, we are even able to obtain attributes that were not calculated by the simulation but may be relevant for data analysis or debugging. We present an interactive selection technique designed for such attributes. It leverages spatial regions of the selection to efficiently compute attributes only where needed. This lazy evaluation also works for intelligent and data-driven selection, extending the region to include neighboring particles. Our technique is evaluated by measurements of performance scalability and case studies for typical usage examples.

Dynamic Mode Decomposition(DMD)is a data-driven and model-free decomposition technique.It is suitable for revealing spatio-temporal features of both numerically and experimentally acquired data.Conceptually,DMD performs a low-dimensional spectral decomposition of the data into the following components:the modes,called DMD modes,encode the spatial contribution of the decomposition,whereas the DMD amplitudes specify their impact.Each associated eigenvalue,referred to as DMD eigenvalue,characterizes the frequency and growth rate of the DMD mode.In this paper,we demonstrate how the components of DMD can be utilized to obtain temporal and spatial information from time-dependent flow fields.We begin with the theoretical background of DMD and its application to unsteady flow.Next,we examine the conventional process with DMD mathematically and put it in relationship to the discrete Fourier transform.Our analysis shows that the current use of DMD components has several drawbacks.To resolve these problems we adjust the components and provide new and meaningful insights into the decomposition:we show that our improved components capture the spatio-temporal patterns of the flow better.Moreover,we remove redundancies in the decomposition and clarify the interplay between components,allowing users to understand the impact of components.These new representations,which respect the spatio-temporal character of DMD,enable two clustering methods that segment the flow into physically relevant sections and can therefore be used for the selection of DMD components.With a number of typical examples,we demonstrate that the combination of these techniques allows new insights with DMD for unsteady flow.

Source URL: https://www.researchgate.net/publication/282908037_A_Spectral_Database_of_Commonly_Used_Cine_Lighting Abstract In recent years, multiple methods have been published that describe how to estimate the spectral response of a camera system or object reflectance spectrum. These methods are becoming increasingly important for the color processing pipeline in movie productions. Knowing the spectral power distribution of light sources used in a movie production is essential for a spectral based approach to ensure a correct color reproduction in movie data creation. We present a thoroughly documented data set of spectral power distributions of typical film and TV lighting. The data set can be used to create more precise camera characterizations and help to develop spectral color processing pipelines. Furthermore, the spectral power distribution data set can be employed to detect and validate over-aged or damaged lighting, by comparing their spectra to our reference measurement.

We present a method to simulate fluid flow on evolving surfaces, e.g., an oil film on a water surface. Given an animated surface (e.g., extracted from a particle-based fluid simulation) in three-dimensional space, we add a second simulation on this base animation. In general, we solve a partial differential equation (PDE) on a level set surface obtained from the animated input surface. The properties of the input surface are transferred to a sparse volume data structure that is then used for the simulation. We introduce one-way coupling strategies from input properties to our simulation and we add conservation of mass and momentum to existing methods that solve a PDE in a narrow-band using the Closest Point Method. In this way, we efficiently compute high-resolution 2D simulations on coarse input surfaces. Our approach helps visual effects creators easily integrate a workflow to simulate material flow on evolving surfaces into their existing production pipeline.

Various neural network based methods are capable of anticipating human body motions from data for a short period of time. What these methods lack are the interpretability and explainability of the network and its results. We propose to use Dynamic Mode Decomposition with delays to represent and anticipate human body motions. Exploring the influence of the number of delays on the reconstruction and prediction of various motion classes, we show that the anticipation errors in our results are comparable to or even better for very short anticipation times (<0.4 s) than a recurrent neural network based method. We perceive our method as a first step towards the interpretability of the results by representing human body motions as linear combinations of previous states and delays. In addition, compared to the neural network based methods large training times are not needed. Actually, our methods do not even regress to any other motions than the one to be anticipated and hence it is of a generic nature.

We present a novel technique to correct errors introduced by the discretization of a fluid body when animating it with smoothed particle hydrodynamics (SPH). Our approach is based on the Shepard correction, which reduces the interpolation errors from irregularly spaced data. With Shepard correction, the smoothing kernel function is normalized using the weighted sum of the kernel function values in the neighborhood. To compute the correction factor, densities of neighboring particles are needed, which themselves are computed with the uncorrected kernel. This results in an inconsistent formulation and an error-prone correction of the kernel. As a consequence, the density computation may be inaccurate, thus the pressure forces are erroneous and may cause instabilities in the simulation process. We present a consistent formulation by using the corrected densities to compute the exact kernel correction factor and, thereby, increase the accuracy of the simulation. Employing our method, a smooth density distribution is achieved, i.e., the noise in the density field is reduced by orders of magnitude. To show that our method is independent of the SPH variant, we evaluate our technique on weakly compressible SPH and on divergence-free SPH. Incorporating the corrected density into the correction process, the problem cannot be stated explicitly anymore. We propose an efficient and easy-to-implement algorithm to solve the implicit problem by applying the power method. Additionally, we demonstrate how our model can be applied to improve the density distribution on rigid bodies when using a well-known rigid-fluid coupling approach.

Spectral imaging has many uses in the field of conservation of cultural heritage, medical imaging, etc. It collects spectral information at each location of an image plane as an image cube. Among various approaches, snapshot multispectral imaging techniques measure the cube within one integration period. Previous work has addressed the issue of optical design, while recent developments have shifted the focus towards computation. In this paper, we present a snapshot multispectral imaging technique with a computational camera and a corresponding image restoration algorithm. The main characteristics are: (1) transferring spectral information to the spatial domain by engineering user-defined PSFs; (2) measuring spectral images by computationally inverting the image formation. The design of our computational camera is based on a phase-coded aperture technique to generate spatial and spectral variant PSFs. The corresponding algorithm is designed by adapting single-channel and cross-channel priors. We show experimentally the viability of our technique: it reconstructs high resolution multispectral images from a snapshot. We further validate that the role of PSF design is critical.

Dynamic Mode Decomposition (DMD) is a data-driven decomposition technique extracting spatio-temporal patterns of time-dependent phenomena. In this paper, we perform a comprehensive theoretical analysis of various variants of DMD. We provide a systematic advancement of these and examine the interrelations. In addition, several results of each variant are proven. Our main result is the exact reconstruction property. To this end, a new modification of scaling factors is presented and a new concept of an error scaling is introduced to guarantee an error-free reconstruction of the data.

Dynamic Mode Decomposition (DMD) is a data-driven decomposition technique extracting spatio-temporal patterns of time-dependent phenomena. In this paper, we perform a comprehensive theoretical analysis of various variants of DMD. We provide a systematic advancement of these and examine the interrelations. In addition, several results of each variant are proven. Our main result is the exact reconstruction property. To this end, a new modification of scaling factors is presented and a new concept of an error scaling is introduced to guarantee an error-free reconstruction of the data.

Measuring the spectral responsivity of a camera using a monochromator is time-consuming and expensive. This work evaluates afast responsivity measurement method, where diffraction spectrum images are captured and then used for estimating camera responsivity. An error was noticed in the previously proposed measurement method that was caused by spectroradiometer measurement errors and vignetting effects from the camera's lens and sensor. Therefore, a correction step using chromaticity error minimisation is presented to adjust the initial responsivity estimate. It requires a chart to be captured under a known illumination. The chromaticity error of the improved procedure is approximately one order of magnitude smaller than the original error. This enhanced method was employed to create a dataset of spectral responsivities for machine vision, photographic, and movie cameras, which is presented here.

We propose a novel computational imaging system that enables the generation of Point Spread Functions (PSFs) of userspecified geometry. Key ingredient of our system is a phase-coded aperture which manipulates the phase distribution of the pupil function by inserting a phase modulator. We use a reflective phaseonly liquid crystal-based Spatial Light Modulator (SLM) for phase modulation. Via encoding a grayscale image on the SLM, the refractive index of each cell can be altered. Phase patterns of PSFs with different shapes are optimized by the Gerchberg-Saxton algorithm. A number of non-trivial, complex shaped PSFs has been captured. We further demonstrate how such a system can realize refocusing through encoding a Fresnel lens phase pattern to shift the focal plane.

In cloth simulation, the behavior of textiles largely depends on initial conditions, parameters, and simulation techniques. Usually, several combinations of those aspects are altered until a simulation setting is found to create a satisfying animation. However, if an initial condition, such as a collision object, is changed afterward or the cloth behavior is transferred to a different scene, the existing set of simulation parameters could no longer be suitable for the desired look. In this case, it is difficult to find a new configuration by changing parameters manually and to determine if it conforms the desired properties. This article introduces a feature vector that is used as a motion-shape signature to capture the spatiotemporal shape characteristics of cloth and can be applied as a similarity measure for physics-based cloth animations.

Smoothed particle hydrodynamics (SPH) is a popular mesh-free, particle-based fluid simulation approach for a wide range of applications. There are several numerical variants of SPH along with a variety of models for aspects such as boundary conditions, compressibility or incompressibility, and surface tension. Different combinations of these models lead to varying effects that occur during simulation, and their analysis is a critical challenge for fluid mechanics. In this paper, we address this challenge by presenting a visual debugging application for simulations, which allows users to evaluate the properties of the models and to detect possible computational errors. Our multi-view application uses a combination of interactive 3D visualization of the particles and non-spatial visualizations from the field of information visualization, namely scatter plots and parallel coordinates plots. Our visual debugging environment thus enables a quantitative analysis of the multidimensional simulation attributes, including internal and physical properties contributing to the simulation process. All views support brushing and linking, that is, selections of interesting value ranges in the plots are directly visible in the 3D view and, conversely, the selection of particles in the 3D view highlights the corresponding data points in the plots. Since typical SPH simulations come with large numbers of data points, we employ stochastic subsampling to reduce visual clutter in the non-spatial views and accelerate the rendering speed. We discuss four real-world use cases for visual debugging of fluid simulations that showcase how our visual debugging environment is instrumental for identify code errors and increases the understanding of the simulation models. We also show how the combination of coupled views can reveal internal details, thus serving to improve simulation results.

We present a novel method for fully asynchronous time integration of particle-based fluids using smoothed particle hydrodynamics (SPH). With our approach, we allow a dedicated time step for each particle. Therefore, we are able to increase the efficiency of simulations. Previous approaches of locally adaptive time steps have shown promising results in the form of increased time steps, however, they need to synchronize time steps in recurring intervals, which involves either interpolation operations or matching time steps. With our method, time steps are asynchronous through the whole simulation and no global time barriers are needed. In addition, we present an efficient method for parallelization of our novel asynchronous time integration. For both serial and parallel execution, we achieve speedups of up to 7.5 compared to fixed time steps and are able to outperform previous adaptive approaches considerably.

We present a direct raytracing method for implicitly described fluid surfaces that takes into account the effects of capillary solid coupling at the boundaries. The method is independent of the underlying fluid simulation method and solely based on distance fields. We make use of the closed-form solution of the meniscus shape at the fluid interface to achieve the effect of surface tension exerted by the solid object. The shape of the liquid at these boundaries is influenced by various physical properties such as the force of gravity and the affinity between the liquid and the solid material. We generate contact angles at the boundaries without the need for computationally intensive small-scale simulation. At render time, we combine the closed-form solution for a small-scale effect with the numerical solution of a large-scale simulation. Our method is applicable for any implicit representation of the fluid surface and does not require an explicit extraction of the surface geometry. Therefore, it is especially useful for particle-based simulations. Furthermore, the solution is guaranteed to yield the correct contact angle and, for certain scenarios, it delivers the entirely correct solution throughout the interface; even in general scenarios, it yields plausible results. As for an example, we implemented and tested the proposed method in the setting of a smoothed particle hydrodynamics (SPH) fluid simulation.

We present a robust and efficient method for the two-way coupling between particle-based fluid simulations and infinitesimally thin solids represented by triangular meshes. Our approach is based on a hybrid method that combines a repulsion force approach with a continuous intersection handling to guarantee that no penetration occurs. Moreover, boundary conditions for the tangential component of the fluid's velocity are implemented to model the different slip conditions. The proposed method is particularly useful for dynamic surfaces, like cloth and thin shells. In addition, we demonstrate how standard fluid surface reconstruction algorithms can be modified to prevent the calculated surface from intersecting close objects. For both the two-way coupling and the surface reconstruction, we take into account that the fluid can wet the cloth. We have implemented our approach for the bidirectional interaction between liquid simulations based on Smoothed Particle Hydrodynamics SPH and standard mesh-based cloth simulation systems.

We evaluate surface tension models in particle-based fluid simulation systems using smoothed particle hydrodynamics (SPH) with a benchmark test. Our benchmark consists of three experiments and a set of analysis methods that are useful for the comparison of surface tension models. Although visual quality is of major interest and is considered as well, we suggest quantification methods for the properties of these models. The goal is to identify if a certain model is suitable for a given scenario and to be able to control the results in the creation of animations. We apply the proposed evaluation methods to three existing surface tension models in combination with different SPH techniques (WCSPH, PCISPH, and IISPH) and perform systematic tests to show the influence of different settings and parameter choices. The surface tension models are chosen from different classes: a pure inter-particle force model, a model based on surface curvature, and a model using a combination of these. Additionally, we present a simple modification to improve the quality of inter-particle force models.