Geometric estimation is required for scene understanding and analysis of panoramic 360u00B0 images. Current methods usually predict a single feature, such as depth or surface normal. These methods can lack robustness, especially when dealing with intricate textures or complex object surfaces. We introduce a novel multi-task learning (MTL) network that simultaneously estimates depth and surface normals from 360u00B0 images. Our first innovation is our MTL architecture, which enhances predictions for both tasks by integrating geometric information from depth and surface normal estimation, enabling a deeper understanding of 3D scene structure. Another innovation is our fusion module, which bridges the two tasks, allowing the network to learn shared representations that improve accuracy and robustness. Experimental results demonstrate that our MTL architecture significantly outperforms state-of-the-art methods in both depth and surface normal estimation, showing superior performance for complex and diverse scenes. Our model's effectiveness and generalizability, particularly in handling intricate surface textures, establish it as a new benchmark for 360u00B0 image geometric estimation. The code and model are available at
Surface reconstruction from raw point clouds has been a pivotal focus in computer graphics for numerous years, primarily due to its integral role in modeling and rendering applications. A key approach to tackling this challenge involves establishing local geometries to align with the local curve. However, conventional methodologies typically resort to constructing either a local plane or a polynomial curve. While a local plane may result in the loss of sharp features, and boundary errors on open surfaces, utilizing a polynomial curve can pose difficulties when using neural networks due to issues with local coordinate consistency. In response to these limitations, we introduce a groundbreaking concept known as anglehedral surfaces to represent local geometry. This innovative idea offers enhanced flexibility in portraying sharp features and surface boundaries on open surfaces without necessitating a local coordinate system, which is crucial for neural network integration. Specifically, we leverage normals to construct anglehedral surfaces, encompassing dihedral and trihedral surfaces controlled by 2 and 3 normals, respectively. As a result, our approach adeptly accommodates sharp features on turning surfaces and accurately captures boundaries on open surfaces. We demonstrate superior performance on three widely-used datasets: ShapeNetCore, ABC, and ScanNet. Code is available at http://gitee.com/HuiTian1/3fold.
Point clouds are now a popular data representations in 3D vision research. However, due to the limited performance of sensors, the raw data usually suffer from sparsity, sensing noise, and incompleteness. This poses great challenges to downstream point cloud processing tasks. In recent years, deep-learning-based geometric point cloud enhancement methods are gaining tremendous research attention; they use deep neural networks with the aim of achieving dense, clean, and complete point clouds from low-quality raw point clouds. This paper presents a comprehensive survey of deep-learning-based geometric point cloud enhancement methods. It covers three main kinds of geometric point cloud enhancement: (i) denoising to achieve clean data, (ii) completion to recover unseen data, and (iii) upsampling to obtain dense data. Our survey presents a new taxonomy for recent state-of-the-art methods and systematic experimental results on standard benchmarks. In addition, we share our observations, and suggest future research directions for geometric point cloud enhancement with deep learning.
Inverse rendering from multi-view photographs is an ill-posed problem because it must jointly disentangle geometry, reflectance, and illumination. Existing 3DGS pipelines either optimize all factors in a single stage, failing to model view-dependent occlusion, or split the task into multiple stages, forfeiting end-to-end optimality. We present a one-stage 3DGS inverse-rendering framework that recovers geometry, BRDFs, and view-dependent occlusion in real time. Each Gaussian stores 4th-order spherical-harmonic visibility coefficients learned from splat transparencies, eliminating ray tracing. A prototype-based material prior decouples lighting from reflectance, and deferred split-sum shading keeps the pipeline fully differentiable while enabling HDR relighting. Experiments on indoor and outdoor benchmarks demonstrate that our approach improves inverse-rendering objectives, while remaining competitive on novel-view synthesis compared to strong reconstruction baselines such as 3DGS.
