Handling Occlusions in Augmented Reality for Mobile Devices

The coherent display of the spatial relationship between real and virtual elements is a desirable feature for realistic Augmented Reality (AR) systems. The system must be able to automatically estimate the depths of the real and virtual elements that compose the scene and display in the foreground of the frame the element closest to the camera. In other words, real and virtual elements must hide (totally or partially) each other according to their depths in the frame. This paper presents a method for occlusion treatment of scene elements for mobile applications. The solution presented uses pixel proportions, relative to the total pixels of the frame, occupied by the real and virtual elements to estimate their depths. Then, a skin color model based segmentation method is applied to extract the real element in frames in which it is in the foreground of the scene and, as a result, must overlap the virtual object. The results of the evaluation of the proposed method showed its superiority to those found in the literature when a set of important features are compared.