Occlusions in motion processing

The instantaneous identification of independently mov- ing objects in a video is particularly difficult if the camera itself is moving, since the motion field on the image is cre- ated by the combined effect of camera motion (egomotion), structure (depth), and the independent motion of scene en- tities. For a camera with a restricted field of view undergo- ing a small motion between frames, there exists in general a set of 3D motions (translation, rotation) compatible with the observed flow field, even if only small amounts of noise are present. We address the instantaneous motion segmen- tation problem in the presence of this ambiguity in obtain- ing exact solutions for interframe 3D motion. We show that moving objects can be classified based on the amount of in- formation required for their detection: if separable clusters of solutions exist, motion based clustering alone will suffice for detecting the simplest class of moving objects. If only a single cluster is found, the positive depth (structure) con- straint can help us identify a second class of moving objects. A more difficult third class of moving objects is found by detecting conflicts between occlusions and structure from motion. Occlusions can not only reduce the ambiguity in 3D motion estimation but also help us identify an important class of moving objects. We underscore the observation that occlusions must not merely be identified, they must also be 'filled', so that ordinal depth may be deduced.