Online motion smoothing for video stabilization via constrained multiple-model estimation

Video stabilization smooths camera motion estimates in a way that should adapt to different types of intentional motion. Corrective motion (the difference between smoothed and original motions) should be constrained so that black borders do not intrude into the (cropped) stabilized frames. Although offline smoothing can use all of the frames, online (real-time) smoothing can only use a small number of previous frames. In this paper, we propose an online motion smoothing method based on linear estimation applied to a constant-velocity model. We use estimate projection to ensure that the smoothed motion satisfies black-border constraints, which are modeled exactly by linear inequalities for general 2D motion models. We then combine the estimate projection with multiple-model estimation, which can adaptively smooth the camera motion in a probabilistic way. Experimental results show how the proposed algorithm can better smooth the camera motion and stabilize videos in real time.