Central to many problems in scene understanding based on using a network of tens, hundreds or even thousands of randomly distributed cameras with on-board processing and wireless communication capability is the “efficient” reconstruction of the 3D geometry structure in the scene. What is meant by “efficient” reconstruction? In this paper we investigate this from different aspects in the context of visual sensor networks and offer a distributed reconstruction algorithm roughly meeting the following goals: 1. Close to achievable 3D reconstruction accuracy and robustness; 2. Minimization of the processing time by adaptive computing-job distribution among all the cameras in the network and asynchronous parallel processing; 3. Communication Optimization and minimization of the (battery-stored) energy, by reducing and localizing the communications between cameras. A volumetric representation of the scene is reconstructed with a shape from apparent contour algorithm, which is suitable for distributed processing because it is essentially a local operation in terms of the involved cameras, and apparent contours are robust to ourdoor illumination conditions. Each camera processes its own image and performs the computation for a small subset of voxels, and updates the voxels through collaborating with its neighbor cameras. By exploring the structure of the reconstruction algorithm, we design the minimum-spanning-tree (MST) message passing protocol in order to minimize the communication. Of interest is that the resulting system is an example of “swarm behavior”. 3D reconstruction is illustrated using two real image sets, running on a single computer. The iterative computations used in the single processor experiment are exactly the same as are those used in the network computations. Distributed concepts and algorithms for network control and communication performance are theoretical designs and estimates.

In this paper, the duality in differential form is developed between a 3D primal surface and its dual manifold formed by the surface's tangent planes, that is, each tangent plane of the primal surface is represented as a four-dimensional vector that constitutes a point on the dual manifold. The iterated dual theorem shows that each tangent plane of the dual manifold corresponds to a point on the original 3D surface, that is, the "dual" of the "dual" goes back to the "primal." This theorem can be directly used to reconstruct 3D surface from image edges by estimating the dual manifold from these edges. In this paper, we further develop the work in our original conference papers resulting in the robust differential dual operator. We argue that the operator makes good use of the information available in the image data by using both points of intensity discontinuity and their edge directions; we provide a simple physical interpretation of what the abstract algorithm is actually estimating and why it makes sense in terms of estimation accuracy; our algorithm operates on all edges in the images, including silhouette edges, self occlusion edges, and texture edges, without distinguishing their types (thus, resulting in improved accuracy and handling locally concave surface estimation if texture edges are present); the algorithm automatically handles various degeneracies; and the algorithm incorporates new methodologies for implementing the required operations such as appropriately relating edges in pairs of images, evaluating and using the algorithm's sensitivity to noise to determine the accuracy of an estimated 3D point. Experiments with both synthetic and real images demonstrate that the operator is accurate, robust to degeneracies and noise, and general for reconstructing free-form objects from occluding edges and texture edges detected in calibrated images or video sequences.

Although stereo vision is a widely used mature technology for 3D surface reconstruction, there are still situations where stereo vision is difficult to apply. For example: surfaces with spatially slowly varying intensity; semi-transparent surfaces; and mirror-like surfaces. For these surfaces, making the necessary surface correspondences across multiple images is difficult if not impossible in a general setting. In these cases, there is often high contrast between object surface and background which enables 2D object boundaries to be more reliably detected. These 2D boundaries (silhouettes), often referred to as apparent contours, observed in 2D image, have complicated relations with the 3D object surface we want to reconstruct. These relations have been studied for the past three decades and several approaches have been developed. However all these approaches have various significant limitations. Proposed and studies in this thesis is a powerful and innovative mathematical framework for recovering 3D shape from silhouettes observed in a sequence of 2D images. The approach involves estimating the geometry of a surface dual to the 3D object to be reconstructed. The dual surface represents the family of 3D planes, in the scene being observed, that are tangent to the object to be reconstructed. This approach is very attractive because each tangent plane in the original space is directly measurable from an image and becomes a point in the dual space. The resulting object reconstruction algorithm is computationally fast, accurate, and robust.

We introduce an algebraic dual-space method for reconstructing the visual hull of a three-dimensional object from occluding contours observed in 2D images. The method exploits the differential structure of the manifold rather than parallax geometry, and therefore requires no correspondences. We begin by observing that the set of 2D contour tangents determines a surface in a dual space where each point represents a tangent plane to the original surface. The primal and dual surfaces have a symmetric algebra: A point on one is orthogonal to its dual point and tangent basis on the other. Thus the primal surface can be reconstructed if the local dual tangent basis can be estimated. Typically this is impossible because the dual surface is noisy and riddled with tangent singularities due to self-crossings. We identify a directionally-indexed local tangent basis that is well-defined and estimable everywhere on the dual surface. The estimation procedure handles singularities in the dual surface and degeneracies arising from measurement noise. The resulting method has O(N) complexity for N observed contour points and gives asymptotically exact reconstructions of surfaces that are totally observable from occluding contours.

A heretofore unsolved problem of great archaeological importance is the automatic assembly of pots made on a wheel from the hundreds (or thousands) of sherds found at an excavation site. An approach is presented to the automatic estimation of mathematical models of such pots from 3D measurements of sherds. A Bayesian approach is formulated beginning with a description of the complete set of geometric parameters that determine the distribution of the sherd measurement data. Matching of fragments and aligning them geometrically into configurations is based on matching break-curves (curves on a pot surface separating fragments), estimated axis and profile curve pairs for individual fragments and configurations of fragments, and a number of features of groups of break-curves. Pot assembly is a bottom-up maximum likelihood performance-based search. Experiments are illustrated on pots which were broken for the purpose, and on sherds from an archaeological dig located in Petra, Jordan. The performance measure can also be an aposteriori probability, and many other types of information can be included, e.g., pot wall thickness, surface color, patterns on the surface, etc. This can also be viewed as the problem of learning a geometric object from an unorganized set of free-form fragments of the object and of clutter, or as a problem of perceptual grouping.

We present a linear approach to the 3D reconstruction problem from occluding contours using algebraic surfaces. The problem of noise and missing data in the occluding contours extracted from the images leads us to this approach. Our approach is based first on the intensive use of the duality property between 3D points and tangent planes, and second on the algebraic representation of 3D surfaces by implicit polynomials of degree 2 and higher.

A heretofore unsolved problem of great archaeological importance is the automatic assembly of pots made on a wheel from the hundreds (or thousands) of sherds found at an excavation site. An approach is presented to the automatic estimation of mathematical models of such pots from 3D measurements of sherds. The overall approach is formulated and described and some detail is provided on the elements of the procedure. The end result is a representation suitable for comparisons, geometric feature extraction, visualization and digital archiving. Matching of fragments and aligning them geometrically is based on matching break-curves (curves on a pot surface separating fragments), estimated axes and profile curves for individual fragments and groups of matched fragments, and a number of features of groups of break-curves. Pot assembly is a bottom-up maximum likelihood performance-based search. In our case, associated with subassemblies of fragments is a loglikelihood which is a sum of energy functions. Experiments are illustrated on pots which were broken for the purpose, and on sherds from an archaeological dig located in Petra, Jordan. The addressed problem and solution can be considered as problems in "geometric learning" and in "perceptual grouping," where subgroups of pot fragments at a site location are to be assembled into individual virtual pots and other fragments are to be discarded as clutter.