Quadratic smoothing based video stabilization using spatio-temporal regularity flow

Camera motion such as jitter, hand sake, vibration are the common causes of video destabilization. They appear as noise in the form of undesired motion in the video frames. In this paper, we propose a video stabilization method that uses quadratic smoothing to remove undesired camera motion. Spatio-temporal regularity flow (SPREF) model is used to estimate the affine motion parameters of video frames by minimizing the flow energy. To obtain the smoothed affine motion parameters of a frame, the minimization of motion parameters is formulated as an unconstrained optimization problem with suitable penalty term. Selection of optimal penalty term to penalize the undesired camera motion and to preserve the desired camera motion is the key idea of the proposed method. Multiple penalty terms generate multiple stabilized videos. The video with maximum interframe transform fidelity (ITF) value is chosen as the most stabilized video. We also describe a method to detect irregular video frames or segments using flow curves and perform selective stabilization of these frames. This is in addition to the stabilization of all the frames in a destabilized video. Proposed video stabilization method is tested on various videos with different scene contents and conditions, and its performance comparison is made with several recent methods. An improved performance based on subjective and objective measures establish the effectiveness of the proposed method in different environments.