A fast method of reconstructing high-resolution panoramic stills from MPEG-compressed video

Creating high quality still pictures from video presents a challenging problem due to the low spatial resolution of most video signals. Many algorithms have been proposed in the literature that utilize multiple video frames to increase spatial resolution. These algorithms depend on two critical assumptions: first, that the scene does not change significantly in the temporal vicinity of the frame of interest, and second that the motion estimation between video frames is extremely accurate. Noting that panoramic views are not only visually pleasing, but also fit the aforementioned assumptions, we propose the use of a scene change detection algorithm to locate scenes containing mainly pan/tilt types of motion. Since many digital video sequences are compressed using MPEG, it is desirable to perform all computations with minimal decompression. To this end, we also propose methods to locate pans from MPEG-compressed video. Once the pan segments are located, a number of highly accurate motion estimation methods can be successfully applied to the video segment. Given the resulting accurate motion, there exist various methods of attacking the resolution enhancement problem and creating a panoramic still image. These, for the most part, are computationally expensive. Therefore, we propose a fast method of obtaining enhanced resolution panoramas from the lower resolution video signal.