Compositional Neural Distance Field with Latent Code Embedding for Dynamic Objects

In recent years, implicit neural representation of 3D scenes has evolved significantly and has been rapidly extended to multiple application scenarios. However, using this type of approach for highly dynamic urban scenes remains a challenging problem. Typical images of objects captured by cameras on board autonomous vehicles from one trajectory contains limited number of views, leading unsatisfactory reconstruction quality. To address this problem, we propose a novel hybrid network structure that decomposes the urban scene into a static background and multiple dynamic objects. The background model outputs a signed distance field with its accuracy enhanced by surface constraints. To deal with the issues of insufficient camera view we use a pre-training strategy to leverage shape and appearance prior to the object model from external datasets. We design an autoencoder architecture to encode a category of objects and employ an attention-based fusion module to better extract features from multiple object images. Furthermore, we introduce a “symmetric completion” approach to leverage the inherent symmetry property of normal cars. During experiments with data from Carla Platform, we find that our model can reconstruct scenes with high-fidelity, generate novel-view successfully and edit 3D scene freely.