High-level semantic video annotation based on 3D scene structure analysis

The application of 3D scene reconstruction techniques in the area of automatic semantic annotation, search and retrieval of unedited video footage has become an interesting field of research for some specific type of video content. Usually static key-frames extracted from a sequence of images are analyzed in order to annotate the content. In the case of a moving camera, the temporal properties of the video can be exploited as well. Based on state of the art camera self calibration techniques a powerful analysis chain has been developed, which allows annotation with regard to specific properties of the 3D scene structure. It is demonstrated, that the reconstructed 3D scene information can be used to generate, accurate low level scene descriptors as well as meaningful medium and high level semantic information. The specific frame-based properties of the triangulated 3D scene contain a lot of potential for semantic annotation, which goes beyond standard 2D scene descriptors.