In this report we introduce a novel approach for determining correspondence in a sequence of images. We formulate a probabilistic framework that relates a feature's appearance and its position under relaxed statistical assumptions. We employ a Monte-Carlo approximation for the joint probability density of the feature position and its appearance that uses a flexible noise and motion model to generate random samples. The joint probability density is modeled by a Gaussian Mixture. The feature's position given its appearance is then determined by maximizing its posterior. We evaluate our method using real and synthetic sequences and compare its performance with leading or popular algorithms from the literature. The noise robustness of our algorithm is superior under a wide variety of conditions. The method can be applied in the context of optical flow, tracking and any application that needs feature point matching.

A general method for optical flow computation is presented that uses an affine model for the flow field. The method uses a hierarchy of filters and it is stable. It can work with or without stabilizers like smoothness terms. An extreme variety of filters can be used but we found the Gabor filters to be particularly good. Care has been taken to address the issue of the differentiation and preconditioning. The method was tested on real and synthetic images and compared with other methods. The performance was excellent for images with or without discontinuities, large or subpixel flow and rotating or dilating image sequences.

Man yc lustering problems in Computer Vision group data points that are the result of statistical estimation and these data points can have a great amount of uncertainty . Motion segmentation by clustering of optical flo wi ss uch an example because very often optical flo wc annot be estimated without significant uncertainty .W ep resent a EM based clustering algorithm for incomplete data and we apply it to the problem of motion segmentation. The input to the algorithm are the velocity likelihoods and the number of clusters. The algorithm is mathematically very elegant because it does not impose any constraints on the velocity likelihood thus multi-modal likelihood is modeled without difficulty .C oupled with a sophisticated correlated image noise model, the algorithm can handle substantial deviations from the intensity constanc ya ssumption. Experiments with real image sequences sho we xcellent results. 1. Intr oduction The process of grouping pixels having similar motion characteristics is called motion segmentation .Ap opular approach for describing motion similarity within a se gment/layer [12] is by their optical flow. T he computation of optical flo wa tap ixel is an under-constrained problem and the classical solutions [2] almost exclusively use constraints from neighboring pixels by assuming one of the several smoothness constraints which usually do not hold on object boundaries. Motion segmentation based on optical flo wi st hus a chicken and egg problem: In order to compute flo wa ccurately ,w en eed to kno wm otion boundaries but locating the motion boundaries amounts to doing segmentation which requires flo wa si nput. Our approach subscribes to the paradigm [4] that does motion segmentation without computing the full optical flo wf irst.

We present a novel adaptive-sampling algorithm for spectral signature generation. Our algorithm is designed to increase inter-object discrimination and reduce feature-vector dimensionality. This algorithm is applied to a nonorthogonal-wavelet based multi-resolution object detection and recognition scheme. In this context we study and analyze the detection and identification of unknown objects in a complex background. Iterative optimization methods are employed to reduce computational demands during the learning phase. Our representation scheme takes into account all items in a given object library. It selects sample-point sets that maximize inter-object distance. Thus, the presented method increases identification robustness and can reduce the size of signature vectors.

We present a new algorithm that does motion segmentation by tracking small textured patches and then clustering them using EM. A small patch has the advantage that its motion is well modeled by uniform flow and runs a lower risk of boundary inclusion. Inherently, a small patch has less data so it is more susceptible to noise and it is not well suited to fit locally higher order flow models. To overcome these difficulties, we introduce a motion coherence detector to select only the best features and an efficient statistical technique to compute segment-wise affine flow from the EM clustering parameters. We incorporate a residual noise model without any statistical independence assumption and an efficient χ^2 test for the noise model to obtain dense segmentation. Computational efficiency is striven for within a rigorous mathematical framework. Experiments with real image sequences show good segments under a variety of conditions.

Two difficult issues in optical flow are motion discontinuities and large interframe motion. We present an algorithm that addresses both issues by first performing feature tracking and motion segmentation and then warping one of the images to reduce the interframe motion and avoid the motion discontinuities. The algorithm consists of three major phases: 1) feature selection, 2) feature tracking and segmentation 3) optical flow. We used the Lucas and Kanade algorithm to compute the flow. The experiments on real images as well as synthetic images with ground truth showed that this method is very accurate. Ke ywords: Motion, Segmentation, Tracking, Optical Flow

This report provides an review of Clustering using Mixture Models and the Expectation Maximization method[1] and then extends these concepts to the problem of clustering of unobserved data where we cluster a set of vectors u i for i = 1. . N for which we only know the probability distribution. This problem has several applications in Computer Vision where we want to cluster noisy data.

This paper presents a nove lt racking based motion segmen- tation algorithm. The tracking is done by f itting succes- sively more elaborate models of optical flo wo nt he tracked region and the segmentation is done by e xtracting the regions of the image that are consistent with the computed model of f low. T he method can track objects in image sequences with moving background, taken by a hand-held camera, tolerate up to 30 pix els interframe motion and takes 0.3 seconds per frame pair of size 320 x 240 pixels on a 500 Mhz Sun Blade 100 workstation. Ke ywords: Motion Segmentation, Tracking, Optical Flow 1. Related Work At racking algorithm measures and predicts the motion of am oving object ove rt ime. Contours (8, 13) correspond- ing to the silhouette of moving objects are commonly used feature for tracking. The coherence of a moving region (5, 1) corresponding to the projection of a surface of the mov- ing object is another good basis for tracking. Color (3, 9) of a moving object is also frequently used in tracking. Instead of tracking attributes belonging to the moving object, an orthogonal tracking approach is to find the mov- ing objects in a dynamic scene by performing image differ- ence on the image frames with known background (15). In all of the above approaches, an initial representation of the to-be-tracked object or its background is gi ve nt ot he tracker as input and the role of the tracker is to measure and predict the motion of the moving object representation ove rt ime. Meyer and Bouthemy (11) tracked the motion of regions computed by a motion segmentation algorithm ove rt ime assuming a model for the motion and change of

This paper presents a novel tracking based motion segmentation algorithm. The tracking is done by fitting successively more elaborate models of optical flow on the tracked region and the segmentation is done by extracting the regions of the image that are consistent with the computed model of flow. The method can track objects in image sequences with moving background, taken by a hand-held camera, tolerate up to 30 pixels interframe motion and takes 0.3 seconds per frame pair of size 320 x 240 pixels on a 500 Mhz Sun Blade 100 workstation. Ke ywords: Motion Segmentation, Tracking, Optical Flow

Recent growth in the number of digital images available motivates the development of image/video databases for the effective management of these ever-increasing images. A common image retrieval task requires retrieving all images in the database similar in image content to an example query image. In this paper, we develop a simple, fast and robust motion segmentation algorithm to separate images into independently moving objects and an indexing scheme that uses the trajectories, shapes and image flow vectors of the independently moving objects to insert/query the sequences in a database. In order to test our ideas, we developed a video database prototype and experimented with real images from scenes showing moving cars.

In this paper, we present an approach to autonomousrobot navigation in an unknown environment. We design and integratealgorithms to reconstruct the scene, locate obstacles and doshort-term field-based path planning. The scene reconstruction isdone using a region matching flow algorithm to recover imagedeformation and structure from motion to recover depth. Obstaclesare located by comparing the surface normal of the known floor withthe surface normal of the scene. Our path planning method is based onelectric-like fields and uses current densities that can guaranteefields without local minima and maxima which can provide solutionswithout the need of heuristics that plague the more traditionalpotential fields approaches. We implemented a modular distributedsoftware platform (FBN) to test this approach and we ran severalexperiments to verify the performance with very encouraging results.

Solutions to the structure from motion problem have been shown to be very sensitive to measurement noise and the respective motion and geometry configuration. Statistical error analysis has become an invaluable tool in analyzing the sensitivity phenomenon. This paper presents a unifying approach to the problems of statistical bias, correlated noise, choice of error metrics, geometric instabilities and information fusion exploring several assumptions commonly used in motion estimation and reviews several promising techniques for motion estimation. The techniques are based on a small number of principles of statistics and perturbation theory. The analyticity of the approach enables the design of alternatives overcoming the observed instabilities.

In this paper we study four fundamental image operators using Lagrange polynomials. These operators are interpolation, first and second derivative, and image reduction (or shrinking). We analyze each operation and compare it to a standard signal processing windowing approach using Gaussian window. The analysis shows the very simple mathematical relation between the two approaches for all four operations and provides a rigorous way to trade speed for accuracy depending on the available resources and requirements. Furthermore, the analysis suggests a way to get three orders of magnitude improvement in the computational efficiency for the Lagrange-based method over standard methods.

Several models of statistical estimation of motion from visual input are derived and analyzed theoretically and experimentally. We study a wide variety of models, ones that use least squares and ones that use maximum likelihood, with several different assumptions (dependent and independent noise, isotropic and non-isotropic noise), spherical and planar image surfaces, and different preprocessing (one based on correspondence and one based on disparity). We do all this analysis using only a few fundamental concepts from statistical estimation, so the relative merits and shortcomings of all the methods become evident. The experimental results provide a quantitative measure of these merits.

One of the main issues in the area of motion estimation given the correspondences of some features in a sequence of images is sensitivity to error in the input. The main way to attack the problem, as with several other problems in science and engineering, is redundancy in the data. Up to now all the algorithms developed either used two frames or depended on assumptions about the motion or the shape of the scene. We present in this paper an algorithm based on multiple frames that employs only the rigidity assumption, is simple and mathematically elegant and, experimentally proves to be a major improvement over the two-frame algorithms. The algorithm does minimization of the squared error which we prove equivalent to an eigenvalue minimization problem. One of the side effects of this mean-square method is that the algorithm can have a very descriptive physical interpretation in terms of the "loaded spring model."

It is known that due to the aperture problem motion cannot be recovered from two frames without additional assumptions. A method is proposed, that uses three frames and point or line correspondences to estimate 3-D motion. The algorithm is linearized, efficient and needs no assumptions other than rigidity. Using redundant points and lines the algorithm exhibits stability in the presence of noise. It has been tested with simulated data under a wide variety of conditions