Geometric Image Segmentation via Multiscale TILT Clustering ∗

We present a novel algorithm to acquire and analyze rich 3D geometric features in single urban images. Traditional representation of 3D structures via local image features lack global geometric information to provide highquality image correspondence and 3D models. The new approach utilizes the low-rank representation technique to seek a new class of invariant features based on minimizing the matrix rank of image textures, which are more holistic with respect to global geometric information, invariant to camera distortion, and robust to pixel corruption. Based on the transform-invariant low-rank texture (TILT) representation, we first propose an efficient algorithm to detect TILT features in urban images where man-made, symmetric patterns are abundant. Second, we introduce a multiscale, topdown representation of TILT clusters as TILT complexes, each of which represents a dominant planar structure (e.g., building facades) in 3D space. Extensive experiments are conducted on the Pankrac building database to demonstrate the efficacy of the algorithm. The source code of the algorithm will be available for peer evaluation.