What Can Be Seen in Three Dimensions with an Uncalibrated Stereo Rig

This paper addresses the problem of determining the kind of three-dimensional reconstructions that can be obtained from a binocular stereo rig for which no three-dimensional metric calibration data is available. The only information at our disposal is a set of pixel correspondences between the two retinas which we assume are obtained by some correlation technique or any other means. We show that even in this case some very rich non-metric reconstructions of the environment can nonetheless be obtained.Specifically we show that if we choose five arbitrary correspondences, then a unique (up to an arbitrary projective transformation) projective representation of the environment can be constructed which is relative to the five points in three-dimensional space which gave rise to the correspondences.We then show that if we choose only four arbitrary correspondences, then an affine representation of the environment can be constructed. This reconstruction is defined up to an arbitrary affine transformation and is relative to the four points in three-dimensional space which gave rise to the correspondences. The reconstructed scene also depends upon three arbitrary parameters and two scenes reconstructed from the same set of correspondences with two different sets of parameter values are related by a projective transformation.Our results indicate that computer vision may have been slightly overdoing it in trying at all costs to obtain metric information from images. Indeed, our past experience with the computation of such information has shown us that it is difficult to obtain, requiring awkward calibration procedures and special purpose patterns which are difficult if not impossible to use in natural environments with active vision systems. In fact it is not often the case that accurate metric information is necessary for robotics applications for example where relative information is usually all what is needed.