Motion field modeling for video sequences

We propose a model for the interframe correspondences existing between pixels of an image sequence. These correspondences form the elements of a field called the motion field. In our model, spatial neighborhoods of motion elements are related based on a generalization of autoregressive (AR) modeling of the time-series. We also propose a joint spatio-temporal model by including spatial neighborhoods of pixel intensities in the motion model. A fundamental difference of our approach with most previous approaches to modeling motion is in basing our model on concepts from statistical signal processing. The developments in this paper give rise to the promise of extending well-understood tools of signal processing (e.g., filtering) to the analysis and processing of motion fields. Simulation results presented show the performance of our models in interframe prediction; specifically, on average the motion model performs 29% better in terms of the mean squared error energy over a commonly used pel-recursive approach. The spatio-temporal model improves the prediction efficiencies by 8% over the motion model. Our model can also be used to obtain estimates of the optical flow field as the simulations demonstrate.