Live Detection of Foreign Object Debris on Runways Detection using Drones and AI

The removal of foreign objects and debris (FOD) on runways at both non-towered and tower-controlled airstrips is a universal problem. Current detection systems are relatively efficient at finding large foreign objects but are expensive to set up and tend to be inaccurate with small items. With recent developments in artificial intelligence and object detection, it has been proposed to use drones to collect images and then detect any FODs with a specific trained algorithm. This paper will discuss the required materials and method, the image collection process as well as the training and testing results analysis. The development and testing of a live detection model is also discussed. Six object tags have been selected for testing that are realistic representations of possible objects found on runways, these include Litter, Wood, Bolts, Metal, Rubber and Nuts. The collected image database provided a larger variety of images to increase the robustness of the model. The Microsoft Azure platform and Custom Vision module was selected for the deep learning and object detection component. Through the system development, two different models were created, the first with six individual classes and then a single object class model. Two different training algorithms were also used, the online general model and the local compact model. The compact model is able to rapidly analyse images however it loses overall model accuracy. The general model is significantly more accurate than the compact model, however it requires significantly more time to analyse. The single class general trained model has the lowest error rate of 10.5% with 51 correct objects detected out of 57. A live detection module was developed using a 4K HDMI video capture card to live stream the drone footage from the controller to the computer. The individual frames were then extracted and loaded through the detection module. The returned objects that have a probability above the set threshold are processed with a bounding box and given classification tag names. A GPS transmitter and receiver model captured the current coordinates of the drone and then synchronises it with the live video frame data. This was used to create a user-friendly data representation that is accurate to approximately 2.5m.