A Robust Traffic Video Stabilization Method Assisted by Foreground Feature Trajectories

This paper considers stabilizing traffic videos recorded by vehicle-mounted cameras. Compared with videos recorded by handheld cameras, traffic videos suffer from more challenges, such as dynamic scenes, dominant foreground objects, and significant parallax. As conventional video stabilization methods usually estimate a homography only from background feature trajectories and warp frames with that homography, they may not obtain enough number of background feature trajectories, especially in the existence of large moving vehicles or strong parallax, and cannot obtain a precise homography to stabilize videos. To resolve this issue, this paper proposes a novel method that makes use of foreground feature trajectories to assist background feature trajectories in stabilizing traffic videos. More specifically, the stabilized views of foreground trajectories are obtained through solving a spatial–temporal optimization. Then, background feature trajectories, together with foreground feature trajectories and their stabilized views, are used to compute the desired homography, which is implemented to stabilize videos. To further improve the stabilization performance, this paper proposes a block-based strategy, which divides each frame into a few equal blocks and attempts to maintain the balance of the numbers of feature trajectories of blocks by evenly extracting feature points and compensating discontinuous feature trajectories if necessary. The experimental results confirm that the proposed method is superior to several state-of-the-art methods in stabilizing traffic videos, especially when there are large moving objects and/or parallax.