From video sequences to motion panoramas

We address the problem of constructing mosaics from video sequences taken by rotating cameras. In particular we investigate the widespread case where the scene is not only static but may also contain large dynamic areas, induced by moving or deforming objects. Most of the existing techniques fail to produce reliable results on such video sequences. For such alignment purposes, two classes of techniques may be used: feature-based and direct methods. We derive both of them in a unified statistical manner and propose an integrated framework to construct what we call motion panoramas, based on a mixed feature-based and direct approach. Experimental results are provided on large image sequences. In particular we consider sport videos where the moving and deforming athlete is visible in every frame of the sequence, thereby making the alignment task tricky.