We present a large-scale dataset of Planes in 3D, Pi3D, of roughly 1000 planes observed in 10 000 images from the 1DSfM dataset, and HEB, a large-scale homography estimation benchmark leveraging Pi3D. The applications of the Pi3D dataset are diverse, e.g. training or evaluating monocular depth, surface normal estimation and image matching algorithms. The HEB dataset consists of 226 260 homographies and includes roughly 4M correspondences. The homographies link images that often undergo significant viewpoint and illumination changes. As applications of HEB, we perform a rigorous evaluation of a wide range of robust estimators and deep learning-based correspondence filtering methods, establishing the current state-of-the-art in robust homography estimation. We also evaluate the uncertainty of the SIFT orientations and scales w.r.t. the ground truth coming from the underlying homographies and provide codes for comparing uncertainty of custom detectors. The dataset is available at https://github.com/danini/homography-benchmark.

We sketch Friedrich Ackermann's research program following the concept of Imre Lakatos, with some historical key developments in the theory and application of aerotriangulation and image matching. The research program, with its core being statistical estimation theory, has decisively influenced photogrammetry since the 60s, is still fully alive, and a challenge for today's methods of image interpretation. We describe (1) Lakatos' concept of a scientific research program, with its negative and positive heuristics and (2) Ackermann's research program, clearly made explicit in his PhD, with its mathematical model, the ability to predict theoretical precision and reliability, the potential of analyzing rigorous and approximate method, and the role of testing. The development of aerotriangulation, later augmented by image matching techniques, is closely connected to Ackermann's successful attempts to integrate basic research and practical applications.

2 Image Matching 4 2.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 The estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Re ned correspondences . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3.1 Point and a ne correspondences . . . . . . . . . . . . . . . . . 10 2.3.2 Uncertain point correspondences . . . . . . . . . . . . . . . . . . 11

Local features e.g. SIFT and its affine and learned variants provide region-to-region rather than point-to-point correspondences. This has recently been exploited to create new minimal solvers for classical problems such as homography, essential and fundamental matrix estimation. The main advantage of such solvers is that their sample size is smaller, e.g., only two instead of four matches are required to estimate a homography. Works proposing such solvers often claim a significant improvement in run-time thanks to fewer RANSAC iterations. We show that this argument is not valid in practice if the solvers are used naively. To overcome this, we propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. We propose a method for refining the local feature geometries by symmetric intensity-based matching, combine uncertainty propagation inside RANSAC with preemptive model verification, show a general scheme for computing uncertainty of minimal solvers results, and adapt the sample cheirality check for homography estimation. Our experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times when following our guidelines. We make code available at https://github.com/danini/affine-correspondences-for-camera-geometry.

Estimating uncertainty of camera parameters computed in Structure from Motion (SfM) is an important tool for evaluating the quality of the reconstruction and guiding the reconstruction process. Yet, the quality of the estimated parameters of large reconstructions has been rarely evaluated due to the computational challenges. We present a new algorithm which employs the sparsity of the uncertainty propagation and speeds the computation up about ten times w.r.t. previous approaches. Our computation is accurate and does not use any approximations. We can compute uncertainties of thousands of cameras in tens of seconds on a standard PC. We also demonstrate that our approach can be effectively used for reconstructions of any size by applying it to smaller sub-reconstructions.

Bundle adjustment is a central part of most visual SLAM and Structure from Motion systems and thus a relevant component of UAVs equipped with cameras. This paper makes two contributions to bundle adjustment. First, we present a novel approach which exploits trifocal constraints, i.e., constraints resulting from corresponding points observed in three camera images, which allows to estimate the camera pose parameters without 3D point estimation. Second, we analyze the quality loss compared to the optimal bundle adjustment solution when applying different types of approximations to the constrained optimization problem to increase efficiency. We implemented and thoroughly evaluated our approach using a UAV performing mapping tasks in outdoor environments. Our results indicate that the complexity of the constraint bundle adjustment can be decreased without loosing too much accuracy.

We propose and analyse methods to efficiently register point clouds based on plane correspondences. This is relevant in man-made environments, where most objects are bounded by planar surfaces. Based on a segmentation of the point clouds into planar regions and matches of planes in different point clouds, we (1) optimally estimate the relative pose(s); (2) provide three direct solutions, of which two take the uncertainty of the given planes into account; and (3) analyse the loss in accuracy of the direct solutions as compared to the optimal solution. The paper presents the different solutions, derives their uncertainty especially of the suboptimal direct solutions, and compares their accuracy based on simulated and real data. We show that the direct methods that exploit the uncertainty of the planes lead to a maximum loss of 2.76 in accuracy of the estimated motion parameters in terms of the achieved standard deviations compared to the optimal estimates. We also show that the results are more accurate than the classical iterative closest point and iterative closest plane method, but the estimation procedures have a significantly lower computational complexity. We finally show how to generalize the estimation scheme to simultaneously register multiple point clouds.

Fisheye cameras offer a large field of view, which is important for several robotics applications as a larger field of view allows for covering a large area with a single image. In contrast to classical cameras, however, fisheye cameras cannot be approximated well using the pinhole camera model and this renders the computation of depth information from fisheye stereo image pairs more complicated. In this work, we analyze the combination of an epipolar rectification model for fisheye stereo cameras with existing dense methods. This has the advantage that existing dense stereo systems can be applied as a black-box even with cameras that have field of view of more than 180° to obtain dense disparity information. We thoroughly investigate the accuracy potential of such fisheye stereo systems using image data from our UAV. The empirical analysis is based on image pairs of a calibrated fisheye stereo camera system and two state-of-the-art algorithms for dense stereo applied to adequately rectified image pairs from fisheye stereo cameras. The canonical stochastic model for sensor points assumes homogeneous uncertainty and we generalize this model based on an empirical analysis using a test scene consisting of mutually orthogonal planes. We show that: (1) the combination of adequately rectified fisheye image pairs and dense methods provides dense 3D point clouds at 6-7 Hz on our autonomous multicopter UAV; (2) the uncertainty of points depends on their angular distance from the optical axis; (3) how to estimate the variance component as a function of that distance; and (4) how the improved stochastic model improves the accuracy of the scene points.

Deriving semantic 3D models of man-made environments hitherto has not reached the desired maturity which makes human interaction obsolete. Man-made environments play a central role in navigation, city planning, building management systems, disaster management or augmented reality. They are characterised by rich geometric and semantic structures. These cause conceptual problems when learning generic models or when developing automatic acquisition systems. The problems appear to be caused by (1) the incoherence of the models for signal analysis, (2) the type of interplay between discrete and continuous geometric representations, (3) the inefficiency of the interaction between crisp models, such as partonomies and taxonomies, and soft models, mostly having a probabilistic nature, and (4) the vagueness of the used notions in the envisaged application domains. The paper wants to encourage the development and learning of generative models, specifically for man-made objects, to be able to understand, reason about, and explain interpretations.

This chapter motivates and introduces homogeneous coordinates for representing geometric entities. We aim at exploiting the algebraic properties of the representations of geometric entities and at giving geometrically intuitive interpretations.

Bundle adjustment is a unified method to simultaneously estimate the internal and external camera parameters and the 3D coordinates of the scene points in a statistically optimal manner.

Online pose estimation and mapping in unknown environments is essential for most mobile robots. Especially autonomous unmanned aerial vehicles require good pose estimates at comparably high frequencies. In this paper, we propose an effective system for online pose and simultaneous map estimation designed for light-weight UAVs. Our system consists of two components: (1) real-time pose estimation combining RTK-GPS and IMU at 100 Hz and (2) an effective SLAM solution running at 10 Hz using image data from an omnidirectional multi-fisheye- camera system. The SLAM procedure combines spatial resection computed based on the map that is incrementally refined through bundle adjustment and combines the image data with raw GPS observations and IMU data on keyframes. The overall system yields a real-time, georeferenced pose at 100 Hz in GPS-friendly situations. Additionally, we obtain a precise pose and feature map at 10 Hz even in cases where the GPS is not observable or underconstrained. Our system has been implemented and thoroughly tested on a 5 kg copter and yields accurate and reliable pose estimation at high frequencies. We compare the point cloud obtained by our method with a model generated from georeferenced terrestrial laser scanner.

This chapter discusses geometric operations of geometric entities. It covers a wide range of constructions, constraints, and functions based on points, lines, planes, conics, and quadrics, including elements at infinity.

Our objective is the interpretation of facade images in a top-down manner, using a Markov marked point process formulated as a Gibbs process. Given single rectified facade images, we aim at the accurate detection of relevant facade objects as windows and entrances, using prior knowledge about their possible configurations within facade images. We represent facade objects by a simplified rectangular object model and present an energy model, which evaluates the agreement of a proposed configuration with the given image and the statistics about typical configurations, which we learned from training data. We show promising results on different datasets and provide a qualitative evaluation, which demonstrates the capability of complete and accurate detection of facade objects.

This chapter collects the basic terms from probability theory and statistics. It motivates the axiomatic approach for the concept of probability, introduces the concept of a random variable, describes the key properties of the main distributions of random variables occurring when modelling observational uncertainties and testing hypotheses, and provides an introduction to stochastic processes.

This chapter discusses the basic geometry and orientation of image triplets. The higher redundancy caused by observing the scene in three instead of only two images, as before, leads to a number of advantages, so it is useful to treat the image triplet in detail.

This chapter discusses the representation of uncertain homogeneous. We introduce a representation of the uncertainty which is minimal, thus does not contain singular covariance matrices, and develop methods for the estimation of geometric elements and transformation parameters.