In the context of the development of MPEG-I standard for immersive video compression ISO/IEC 23090-12 (MIV), the need of handling scenes with non-Lambertian materials arose. This class of material is omnipresent in natural scenes, but violates all the assumptions on which depth image-based rendering (DIBR) is based. In this paper, we present a view-synthesizer designed to handle non-Lambertian objects with DIBR, replacing the classical depth maps by multi-coefficients non-Lambertian maps. We report the results of the exploration experiments on Future MIV designed to test this rendering method against the classical DIBR approaches, and demonstrate promising results on all the tested sequences.
In this work, we enhance a professional end-to-end volumetric video production pipeline to achieve high-fidelity human body reconstruction using only passive cameras. While current volumetric video approaches estimate depth maps using traditional stereo matching techniques, we introduce and optimize deep learning-based multi-view stereo networks for depth map estimation in the context of professional volumetric video reconstruction. Furthermore, we propose a novel depth map post-processing approach including filtering and fusion, by taking into account photometric confidence, cross-view geometric consistency, foreground masks as well as camera viewing frustums. We show that our method can generate high levels of geometric detail for reconstructed human bodies.
Static light fields are an image-based technology that allow for the photorealistic representation of inanimate objects and scenes in virtual environments. As such, static light fields have application opportunities in heterogeneous domains, including education, cultural heritage and entertainment. This paper contributes the design, implementation and performance evaluation of a web-based static light field consumption system. The proposed system allows static light field datasets to be adaptively streamed over the network and then to be visualized in a vanilla web browser. The performance evaluation results prove that real-time consumption of static light fields at AR/VR-compatible framerates of 90 FPS or more is feasible on commercial off-the-shelf hardware. Given the ubiquitous availability of web browsers on modern consumption devices (PCs, smart TVs, Head Mounted Displays, . . . ), our work is intended to significantly improve the accessibility and exploitation of static light field technology. The JavaScript client code is open-sourced to maximize our work’s impact.
Numerous stereoscopic 3D movies are released every single year to movie theaters and they evidently generate large revenues. Despite the notable improvements in stereo capturing and 3D video post-production technologies, stereoscopic artefacts continue to appear even in high-budget films. Existing automatic 3D video quality measurement tools can detect distortions in stereoscopic images and videos, but they fail to determine the viewer’s subjective perception of those arte-facts, and how these distortions affect their choices and the overall visual experience. In this paper, we introduce a novel recommendation system for stereoscopic 3D movies based on a latent factor model that meticulously analyzes the viewer’s subjective ratings and the influence of 3D video distortions on their personal preferences. To the best knowledge of the authors, this is definitely a first-of-its-kind model that recommends 3D movies based on quality ratings. It takes the correlation between the viewer’s visual discomfort and the perception of stereoscopic artefacts into account. The proposed model is trained and tested on the benchmark Nama3ds1-cospad1 and LFOVIAS3DPh2 S3D video quality assessment datasets. The experiments highlight the practical efficiency and considerable performance of the resulting matrix-factorization-based recommendation system.
In this paper, we analyse and compare four different color transfer algorithms, which were originally developed for 2D images, for 3D point clouds. In particular, we transfer the color distribution of a given reference 3D point cloud to a source 3D point cloud in order to change the illumination of the scene. Color transfer of 3D models might become an important task in AR and VR applications where e.g., an existing 3D model needs to be updated in real-time according to scene changes like displaced objects and illumination changes. In order to better compare the results of the color transfer algorithms, we created a data set of 3D point clouds consisting of reconstructions of an indoor scene under different lighting conditions, and applied two comparison methods for an objective evaluation, namely histogram comparison and voxel comparison, which will be described in detail in this paper.
