A Dynamic Transformer Network With Early Exit Mechanism for Fast Detection of Multiscale Surface Defects

Efficiency of surface defect detection is largely improved with the application of convolutional neural networks (CNNs). However, CNNs encounter difficulties in accurately modeling the multiscale features of defects due to the inherent locality of convolution. Additionally, existing methods for cross-level feature fusion introduce cumbersome computational complexity. Consequently, current networks struggle to reconcile location accuracy of multiscale defects with detection efficiency, resulting in poor performance in real-time detection. To this end, we proposed a dynamic transformer network for surface defect detection, utilizing self-attention-based transformers to precisely capture long-range semantic information within a routing space. Meanwhile, the network dynamically selected feature fusion paths according to inputs, enhancing its ability to aggregate multiscale defect features. Additionally, an early exit mechanism was designed by introducing a multitask loss function to induce defect-free inputs to exit from shallow layers of the network, thereby improving the detection speed. Furthermore, a novel learning algorithm was proposed with a consistent prediction loss function to strengthen the hierarchical relationship between the classification task and segmentation task. By performing experiments on self-built steel surface dataset with multiscale defects, the proposed framework outperformed all competing models in terms of detection accuracy and inference speed, demonstrating that the proposed network can be used for online accurate detection for multiscale surface defects.