3D Gaussian splatting (3DGS) has emerged as a promising approach for high-fidelity 3D scene representation. However, relighting and composition of Gaussian splatting remain challenging because path tracing is not directly applicable. Existing relighting methods for Gaussian splatting typically adopt either approximate rendering formulations or rely on Gaussian ray tracing, yielding low relighting performance and low rendering efficiency. To address these limitations, we propose Gaussian hybrid path tracing (GHPT), a three-stage framework to acquire relightable Gaussian splatting models. The first stage utilizes planar-based Gaussian splatting reconstruction representation (PGSR) to enable multi-view consistent depth rendering and reconstruct the surface mesh of a scene. The second stage performs physically-based differentiable rendering on the obtained mesh to reconstruct the material maps and the environment map. The third stage utilizes factorized inverse path tracing (FIPT) on the G-buffer rendered by the PGSR, and visibility and indirect illumination are evaluated by hardware-accelerated ray tracing on the mesh with the material maps and the environment map reconstructed in the second stage. Experiments demonstrate that the relighting performance of GHPT outperforms the baselines, and our method can perform real-time relighting and composition of Gaussian splatting.

Point cloud registration plays a crucial role in preservation and digitization of cultural heritage by accurately aligning multiple point cloud datasets to create a complete 3D model. However, the complexity and diversity of objects lead to low-overlap and significant variations, posing challenges in achieving high accuracy, robustness and generalizability. This study proposes a Cross-Domain Multi-Channel Transformer (CDMCT) to address these challenges. The multi-channel dynamic encoding to enhance model’s sensitivity to local structures and angular relationships. The cross-domain convergence network preserves the global relationships within point cloud and the overall structural information. The integration of graph networks allows for flexible handling of local feature variations. We trained on 3DMatch and KITTI, and validated on cultural heritage datasets Terracotta Warriors and WHU-TLS ancient buildings. Experimental results show that CDMCT achieves significant improvements in registration accuracy and robustness, demonstrating its broad application potential in the digital preservation of cultural heritage.

Radiance fields, such as neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS), are the new primitives to represent 3D scenes. Relighting and composition of radiance fields are critical for modeling the complex 3D world in computer graphics. However, it is difficult to relight and composite radiance fields because traditional physically-based rendering techniques, such as path tracing, cannot be directly applied to radiance fields. We propose a physically-based relighting and composition method for radiance fields with proxy meshes. A unified framework is presented to enable us to use radiance fields as the traditional assets in computer graphics. We generate proxy meshes of the radiance fields by reconstructing the geometries of the scenes using Gaussian-based surface reconstruction and the materials using physically-based differentiable rendering. We leverage differential rendering, which is previously used in augmented reality (AR) and mixed reality (MR), to evaluate the radiance change on the proxy meshes introduced by the changing lighting condition, the inserted radiance fields, or the inserted mesh models. Proxy meshes can help us utilize hardwareaccelerated ray tracing to perform real-time path tracing. Experimental results show that our method outperforms the baselines in terms of relighting performance and can achieve photorealistic relighting and composition of radiance fields in real-time.

The manufactured sand concrete of ballastless track needs to bear high frequency fatigue load during the service life. This study investigates the fatigue damage mechanism of manufactured sand concrete under frequencies ranging from 30 Hz to 45 Hz. Digital Image Correlation (DIC) was employed to monitor strain evolution and crack propagation, while X-ray computed tomography (X-CT) was utilized to analyze microscopic crack networks and pore characteristics of fatigue damaged concrete. The results demonstrate that concrete fatigue life decreases rapidly with increasing loading frequencies, showing a 45.5 % reduction when the loading frequencies from 30 Hz to 45 Hz. Both maximum and minimum fatigue strains exhibit progressive amplification accompanied by accelerated damage evolution rates and increased fatigue crack widths. Microscopic analysis reveals exponential growth trends in both volume and surface area of microcracks compared to sound specimen. The porosity displays a linear increase pattern at speeds lower than 40 Hz, but experiences a dramatic surge to 3.63 % at 45 Hz, representing a 46.4 % increase relative to 30 Hz. Notably, higher loading frequencies reduce pore aspect ratio while increasing throat volumes and enhancing pore connectivity. The deterioration of microstructure is identified as the primary mechanism governing concrete fatigue failure under these loading conditions.

For 3D imaging using fringe projection profilometry (FPP), temporal phase unwrapping (TPU) able to robust absolute phase recovery is essential for accurately measuring complex scenes with surface discontinuities. In the presence of systematic errors and other random noises, the most efficient dual-frequency TPU method generally restricts the frequency of high-frequency phases to about 16, compromising the 3D measurement accuracy. Employing the reference phases based on depth constraint allows for unwrapping the phase map with a higher frequency, albeit at the cost of a narrower measured depth range. In this paper, we present a fast and long-range 3D shape measurement method using reference-phase-based number-theoretical temporal phase unwrapping with a MEMS Projector. By introducing the reference phases into traditional number-theoretical TPU, the proposed method with the aid of the optimal bi-frequency scheme has the ability to efficiently and accurately remove the phase ambiguities of high-frequency fringes, while theoretically circumventing the limitations of the measurement range. Furthermore, simulations and experiments have been carried out to evaluate the absolute phase measurement performance of three groups of classical TPU algorithms using the reference phases, including multi-frequency approach, multi-wavelength approach, and number-theoretical approach. Additionally, the feasibility of our method is validated in two developed 3D imaging sensors using MEMS-based miniaturized projectors for various application scenarios. Owing to the more complex calibration processes and lower projection quality of MEMS projectors in comparison to DLP projectors, experimental results demonstrate that the proposed method is highly available for 3D imaging systems with MEMS projectors, which enhances the efficiency and accuracy of absolute phase measurement, achieving fast, wide-field-of-view, and long-range 3D imaging.

This paper presents an optimal multiple fringe pattern composition method for 3D shape measurement of highdynamic-range (HDR) objects using fringe projection profilometry (FPP). With the inverse variance weighting theory, we take the square of the modulation intensities of the fringe pattern images with different intensity levels as the weights to obtain the composited phase of fringe patterns by weighted complex amplitude fusion, which improves the measurement precision of HDR objects. Additionally, we integrate HDR 3D shape measurement and temporal noise reduction into a unified framework by utilizing weighted complex amplitude fusion to completely measure translucent objects with specular reflections. Simulations and experiments demonstrate that our method can achieve higher measurement precision and is resistant to the time-varying ambient light.

According to the Entropy Rate Constancy (ERC) principle, the information density of a text is approximately constant over its length. Whether this principle also applies to nonverbal communication signals is still under investigation. We perform empirical analyses of video-recorded dialogue data and investigate whether listener gaze, as an important nonverbal communication signal, adheres to the ERC principle. Results show (1) that the ERC principle holds for listener gaze; and (2) that the two linguistic factors syntactic complexity and turn transition potential are weakly correlated with local entropy of listener gaze.

Utilizing Generative Adversarial Networks (GANs) to generate 3D representations of the Terracotta Warriors offers a novel approach for the preservation and restoration of cultural heritage. Through GAN technology, we can produce complete 3D models of the Terracotta Warriors’ faces, aiding in the repair of damaged or partially destroyed figures. This paper proposes a distillation model, DIGAN, for generating 3D Terracotta Warrior faces. By extracting knowledge from StyleGAN2, we train an innovative 3D generative network. G2D, the primary component of the generative network, produces detailed and realistic 2D images. The 3D generator modularly decomposes the generation process, covering texture, shape, lighting, and pose, ultimately rendering 2D images of the Terracotta Warriors’ faces. The model enhances the learning of 3D shapes through symmetry constraints and multi-view data, resulting in high-quality 2D images that closely resemble real faces. Experimental results demonstrate that our method outperforms existing GAN-based generation methods.

The current mainstream way of cultural relic display is static display, that is, the cultural relic entity is presented to the user in a single way, the information output efficiency is low, and the user experience is ordinary. Our system to improve the display effect of cultural relics is mainly reflected in the hardware design and front-end display part: In the hardware part, we designed the cultural relics virtual-real combination display cabinet, which is mainly composed of an LCD transparent display screen that can be opened and closed and cabinet body. Based on augmented reality technology, the system designs a display mode that combines virtual and real cultural relics, namely the combination of cultural relics entity and digital 3D model of cultural relics. The transparent LCD screen is used to map the 3D model of cultural relics to the front of the cultural relics entity, and at the same time displays the relevant background information of cultural relics, so that the cultural relics information can be displayed to users in an all-round and multi-angle way. Equipped with Leap Motion gesture sensor, gesture recognition is used to realize contact-free interactive operation, which improves users’ browsing experience on the premise of ensuring the safety of cultural relics.

Rendering environment lighting using ray tracing is challenging because many rays within the hemisphere are required to be traced. In this work, we propose discrete visibility fields (DVFs), which store visibility information in a uniform grid to speed up ray-traced low-frequency environment lighting for static scenes. In the precomputation stage, we compute and store the visibility and occlusion masks at the positions of the point samples of the scene using octahedral mapping. The visibility and occlusion masks of the point samples inside a grid cell are then merged by the logical OR operation. We also store the occlusion label indicating whether more than half of the pixels are occluded in the occlusion mask of each grid cell. At runtime, we exclude the rays occluded by the geometry or visible to the environment according to the information stored in the DVF. Results show that the proposed method can significantly speed up the rendering of ray-traced environment lighting and achieve real-time frame rates without sacrificing image quality. Compared to other environment lighting rendering methods based on precomputation, our method is free of tessellation or parameterization of the meshes, and the precomputation can be finished in a short time.

Soft shadows of environmental lighting provide important visual cues in realistic rendering. However, rendering of soft shadows of environmental lighting in real-time is difficult because evaluating the visibility function is challenging. In this work, we present a method to render soft shadows of environmental lighting at real-time frame rates based on hardware-accelerated ray tracing. We assume that the scene contains both static and dynamic objects. To composite the soft shadows cast by dynamic objects with the precomputed lighting of static objects, the incident irradiance occluded by dynamic objects, which is obtained by accumulating the occluded incident radiances over the hemisphere using ray tracing, is subtracted from the precomputed incident irradiance. Conical ray culling is proposed to exclude the rays that cannot intersect dynamic objects, which significantly improves rendering efficiency. Rendering results demonstrate that our proposed method can achieve real-time rendering of soft shadows of environmental lighting cast by dynamic objects.

We present a compressive parallel single-pixel imaging (cPSI) method, which applies compressive sensing in the context of PSI, to achieve highly efficient light transport coefficients capture and 3D reconstruction in the presence of strong interreflections. A characteristic-based sampling strategy is introduced that has sampling frequencies with high energy and high probability. The characteristic-based sampling strategy is compared with various state-of-the-art sampling strategies, including the square, circular, uniform random, and distance-based sampling strategies. Experimental results demonstrate that the characteristic-based sampling strategy exhibits the best performance, and cPSI can obtain highly accurate 3D shape data in the presence of strong interreflections with high efficiency.

Optical 3D shape measurements, such as fringe projection profilometry (FPP), are popular methods for recovering the surfaces of an object. However, traditional FPP cannot be applied to measure regions that contain strong interreflections, resulting in failure in 3D shape measurement. In this study, a method based on single-pixel imaging (SI) is proposed to measure 3D shapes in the presence of interreflections. SI is utilized to separate direct illumination from indirect illumination. Then, the corresponding points between the pixels of a camera and a projector can be obtained through the direct illumination. The 3D shapes of regions with strong interreflections can be reconstructed with the obtained corresponding points based on triangulation. Experimental results demonstrate that the proposed method can be used to separate direct and indirect illumination and measure 3D objects with interreflections.

Interreflections introduced by points in a scene are not only illuminated by the light source used but also by other points in the scene. Interreflections cause inaccuracy and the failure of 3D recovery and optical measurements. In this research, a novel method for separating interreflections through parallel single-pixel imaging (PSI) is proposed, which can decompose interreflections into 1st bounce light, 2nd bounce light, and a higher order light component. PSI is used in obtaining the light transport coefficients of each camera pixel, and light transport coefficients are used in decomposing the intensity distribution of a projector and the component of interreflections. Results show that the proposed method can separate the interreflections of a real static scene in a concave surface.

We present parallel single-pixel imaging (PSI), a photography technique that captures light transport coefficients and enables the separation of direct and global illumination, to achieve 3D shape reconstruction under strong global illumination. PSI is achieved by extending single-pixel imaging (SI) to modern digital cameras. Each pixel on an imaging sensor is considered an independent unit that can obtain an image using the SI technique. The obtained images characterize the light transport behavior between pixels on the projector and the camera. However, the required number of SI illumination patterns generally becomes unacceptably large in practical situations. We introduce local region extension (LRE) method to accelerate the data acquisition of PSI. LRE perceives that the visible region of each camera pixel accounts for a local region. Thus, the number of detected unknowns is determined by local region area, which is extremely beneficial in terms of data acquisition efficiency. PSI possesses several properties and advantages. For instance, PSI captures the complete light transport coefficients between the projector–camera pair, without making specific assumptions on measured objects and without requiring special hardware and restrictions on the arrangement of the projector–camera pair. The perfect reconstruction property of LRE can be proven mathematically. The acquisition and reconstruction stages are straightforward and easy to implement in the existing projector–camera systems. These properties and advantages make PSI a general and sound theoretical model to decompose direct and global illuminations and perform 3D shape reconstruction under global illumination.

Three-dimensional (3D) shape measurement with fringe projection technique and vertical scanning setup can alleviate the problem of shadow and occlusion. However, the shape-from-defocus based method suffers from limited sensitivity and low signal-to-noise ratio (SNR), whereas the projection-triangulation based is sensitive to the zero-phase detection. In this paper, we propose paraxial 3D shape measurement using parallel single-pixel imaging (PSI). The depth is encoded in the radial distance to the projector optical center, which is determined by the projection of light transport coefficients (LTCs). The third-order polynomial fitting is used for depth mapping and calibration. Experiments on 5 objects with different materials and textures are conducted, and standards are measured to test the accuracy. The results verified that the proposed method can achieve robust, dense reconstruction with depth accuracy at 20 μm while the root-mean-square error (RMSE) of plane fitting up to 43 μm.

In the modern industrial manufacturing, how to effectively obtain the three-dimensional data of the parts profile is the key component for precision test and subsequent analysis. A light-duty design scheme for optical vision probe, which can be installed with a PH10T motorized probe head in CMM, is discussed in this paper. The optical probe can overcome several defects of the traditional measurement mode of CMM, such as poor efficiency and sparse point cloud. Therefore, the problem of 3D measurement and quality analysis for complicated parts can be solved. To splice data in different fields of view, a registration method using a new designed artifact is proposed. Experiments demonstrated the feasibility of the designed non-contact CMM integrated with optical 3D probe for precise 3D shape measurement. The measurement uncertainty of the optical probe can reach 0.012mm within the measuring volume width 200mm and the measurement uncertainty of the global 3D measurement is less than 0.03mm in 1500mm.

Traditional optical 3D shape measurement methods, such as light stripe triangulation, binary coding, and fringe projection, cannot acquire complete and correct 3D measurement results in the presence of interreflections. In this research, a 3D shape measurement method in the presence of interreflections based on light stripe triangulation is presented. The wrong measurement results caused by interreflections are excluded by the geometric constraints introduced by an additional camera. Each 3D point reconstructed by light stripe triangulation is projected onto the image plane of the additional camera to determine whether the 3D point is correct measurement result. Experimental results demonstrate that the proposed method can measure 3D shape in the presence of interreflections.

When fringe projection profilometry is applied for real-time 3D shape measurement, several problems remain to be solved such as multi-wavelength heterodyne phase unwrapping is sensitive to motion and the computation cost is high. In this paper, a real-time 3D shape measurement method with optimized multi-wavelength heterodyne phase unwrapping and GPU parallel computing is proposed. Experimental results demonstrate that the proposed method can acquire 3D shape at 40 fps. Dynamic object with discontinuities can be measured and the phase unwrapping mistakes are eliminated by smoothing the phase of beat frequency during multi-wavelength heterodyne phase unwrapping.