Persistent 3D stabilization for aerial imagery

In this paper, we present a novel 3D stabilization method for aerial imagery. While most existing aerial surveillance applications rely on 2D stabilization, which generates parallax errors in urban scenes, we aim to compensate camera motion perfectly as if the scene had been captured by a static camera. It is not only a better way to visualize events but also a more reliable inputs for higher level surveillance tasks. We tackle this problem by proposing a novel approach which generates accurate long-term dense mapping and handles occlusion robustly. Accurate dense mapping from a given frame to the reference frame is achieved in two steps: 1. Identify and compensate outlier flow from independent moving objects. 2. Accumulate the compensated mapping in two paths to avoid drifting. To handle occlusion regions, we formulate occlusion segmentation as an optimization problem considering both mapping relation and smoothness. Seamless rendering is achieved by gradient guidance of a dynamic background model. We evaluate our method on real-world aerial sequences with more than 100 frames with strong parallax. Our method outperforms state-of-the-art 2D and 3D stabilization approaches both qualitatively and quantitatively. Moreover, we demonstrate that when using our stabilization result as an input, the number of false detection significantly reduces in an independent moving object detection method, which is widely applied to wide area aerial surveillance applications.