UAS-Assisted Corrosion Detection in Steel Using Combined Human and Machine Intelligence

Corrosion is one of the most common defects in steel infrastructures. Crewed visual inspection is the conventional method for corrosion detection in civil infrastructure, which can be dangerous, inconsistent, and labor-intensive. These limitations encouraged the researchers to explore the feasibility of using the $U$ ncrewed Aerial System (UAS) for autonomous real-time corrosion detection using artificial intelligence. A human-machine interface was implemented to take inspector input in sequential training of a corrosion detection model on visual imagery. The model was trained to detect corrosion using smart phone images. The model output was validated or corrected by the inspector after each inspection. The model was then retrained on the inspector output and used in the next inspection. Each dataset consisted of 225 x 225 pixels image tiles labeled as with corrosion and without corrosion. Six combinations of UAS inspection datasets were used to evaluate how the deep learning model performance changed in terms of true positive rate (TPR), true negative rate (TNR), false positive rate (FPR) and false negative rate (FNR). Before retraining 84.78^% of images with corrosion were correctly predicted by the model, and the TNR value was 82.75%. The result showed that the adapted deep learning model performance improved with more inspection as expected. In particular, the number reported false calls made by the model reduced. While the improvement was not always tangible due to corrosion image diversity in texture and color, however, same amount of training, regardless of their order, led to improved but comparable performance.