Cloud Detection System for UAV Sense and Avoid: Challenges and Findings in Flight Experiments

As part of Sense and Avoid, electro-optical cloud detection aims to enable unmanned aerial vehicle (UAV) operations according to visual flight rules (VFR) by meeting legal cloud minimums. In this paper, we focus on the specifics of cloud detection on real flight data collected during a flight test campaign. First, the approach to cloud detection is explained in detail, after which information on preparation and execution of the flight experiments are given. The task of sensor-based cloud detection under real flight conditions brings a number of challenges that are investigated. In this context, we show how cloud ground truth data is collected and what additional effects occur in real flight compared to simulation environments. Additionally, challenges for the application of feature detectors from an aerial perspective are pointed out. Findings from the flight experiments and the subsequent data analysis are finally described. This includes the development of a Convolutional Neural Network (CNN) model for cloud segmentation in monochromatic sensor images and the presentation of respective results. We also present a feature detection benchmark in which the parameters of different feature detectors were optimized on a real flight dataset. The resulting outcomes are discussed and compared between unsegmented and segmented sensor images. The process of triangulating cloud features to determine cloud distance relies on the sensor orientation being as accurate as possible, which requires synchronization of the sensor image and sensor pose. As an alternative solution to this, estimation of orientation from image sequences is proposed and described. Finally, first results of cloud distance estimation based on a cloud approach scenario are shown.