A vision system for intelligent mission profiles of micro air vehicles

Recently, much progress has been made toward the development of small-scale aircraft, known broadly as Micro Air Vehicles (MAVs). Until recently, these platforms were exclusively remotely piloted, with no autonomous or intelligent capabilities, due at least in part to stringent payload restrictions that limit onboard sensors. However, the one sensor that is critical to most conceivable MAV missions, such as remote surveillance, is an onboard video camera and transmitter that streams flight video to a nearby ground station. Exploitation of this key sensor is, therefore, desirable, since no additional onboard hardware (and weight) is required. As such, in this paper we develop a general and unified computer vision framework for MAVs that not only addresses basic flight stability and control, but enables more intelligent missions as well. This paper is organized as follows. We first develop a real-time feature extraction method called multiscale linear discriminant analysis (MLDA), which explicitly incorporates color into its feature representation, while implicitly encoding texture through a dynamic multiscale representation of image details. We demonstrate key advantages of MLDA over other possible multiscale approaches (e.g., wavelets), especially in dealing with transient video noise. Next, we show that MLDA provides a natural framework for performing real-time horizon detection. We report horizon-detection results for a range of images differing in lighting and scenery and quantify performance as a function of image noise. Furthermore, we show how horizon detection naturally leads to closed-loop flight stabilization. Then, we motivate the use of tree-structured belief networks (TSBNs) with MLDA features for sky/ground segmentation. This type of segmentation augments basic horizon detection and enables certain MAV missions where prior assumptions about the flight vehicle's orientation are not possible. Again, we report segmentation results for a range of images and quantify robustness to image noise. Finally, we demonstrate the seamless extension of this framework, through the idea of visual contexts, for the detection of artificial objects and/or structures and illustrate several examples of such additional segmentation. This extension thus enables mission profiles that require, for example, following a specific road or the tracking of moving ground objects. Throughout, our approach and algorithms are heavily influenced by real-time constraints and robustness to transient video noise.