Automatic event recognition for enhanced situational awareness in UAV video

Unmanned aerial vehicles (UAVs) have increasingly shown their usefulness in surveillance, and can greatly improve a commander's situational awareness. With deployment of increasingly larger numbers of UAVs, there is a corresponding increase in the number of operators and analysts required to fully manage these assets. Automatic analysis of UAV video data has the potential for reducing the number of operators required by providing a warning on the occurrence of situations and events of interest. But improvements over basic object recognition are required to provide the necessary levels of situational awareness. One also needs to be able to recognize the relationships between the objects and how those relationships evolve over time (occurrence of events). This paper describes research that is being done on the recognition of events that occur in UAV video, and some of the initial results of that research. Events and situations are described using the video event representation language (VERL), and these event representations are then implemented using Bayesian networks to recognize the event occurrence. The event recognition scheme uses the detection of objects and their parameters as the initial evidence for the event, it is also hierarchical in nature, so that recognized simple events become evidence for recognizing more complex events. Multiple simultaneous events involving multiple actors in the scene can be handled. The results of testing with a limited event recognition catalog are described. Additional event capability is being added to the catalog of recognizable events as the research work is continued.