A layer extraction system based on dominant motion estimation and global registration

We describe a system that extracts layers from a video sequence based on a method estimating and stabilizing the dominant motion. Our approach performs pair-wise registration of images parameterized by 2D affine or projective transformation as the first step. This pair-wise registration is based on: (1) hierarchical parameter estimation and refinement; (2) feature-matching; (3) FFT (fast Fourier transformation)-based global matching; and (4) RANSAC-based parameter estimation. A global registration is then performed among overlapping multi-frames. The global registration is based on inferring the topology (i.e. spatio-temporal relation among frames) and the characterization of the likelihood within overlapping areas. As the last step, we extract the background layer that consists of static components of the video stream after motion estimation. Our background extraction is based on deriving a pixel-wise color distribution in time and provides a basis for a compact description of a video. The presented approach is illustrated by a set of challenging examples.