Lstm Architectures For Facade Structure Completion

3D city models are often generated from oblique aerial images and photogrammetric point clouds. In contrast to roof surfaces, facades can not directly be reconstructed in a similar high level of quality from this data. Distortions of perspective might appear in images, due to the camera angle. Occlusions and shadowing occur as well. Objects, such as windows and doors, will have to be detected on such data if facades are to be reconstructed. Although one can use inpainting techniques to cover occluded areas, detection results are often incomplete and noisy. Formal grammars can then be used to align and add objects. However, it is difficult to find suitable rules for all types of buildings. We propose a post-processing approach based on neural networks to improve facade layouts. To this end, we applied existing Recurrent Neural Network architectures like Multi-Dimensional Long Short-term Memory Network and Quasi Recurrent Neural Network in a new context. We also propose a novel architecture, the Rotated Multi-Dimensional Long Short Term Memory. In order to deal with two-dimensional neighborhoods this architecture combines four two-dimensional Multi-Dimensional Long Short-term Memory Networks on rotated images. We could improve the quality of detection results on the Graz50 data set.