Online annotation of airborne surveillance and reconnaissance videos

A useful enrichment of videos captured by unmanned aerial vehicles (UAVs) is the annotation of image data or determining coordinates for objects in the video. This requires georeferencing of the video frames. For surveillance or reconnaissance applications, we propose georeferencing of UAV video frames with an ortho-photo. In this process, the challenge is to register temporally different images from different sensors that capture objects with quite different visual appearances. After an initial registration of the first video frame to the orthophoto, the subsequent frames are registered by image mosaicking. Whenever necessary, the mosaic is anchored by registration to the orthophoto. We explain the image-based registration approach and discuss its potential real-time processing capability. Internal performance evaluation and GPS/INS support is considered. The feasibility of the approach is demonstrated with real data sets captured by UAVs for georeferencing, video frame annotation, and motion detection.