Inspection of Imprint Defects in Stamped Metal Surfaces Using Deep Learning and Tracking

This article focuses on the automatic detection and classification of imprint defects on the surface of metal parts. This innovative research had collaboration with a multinational industry, which provided a system to capture images of vehicle parts as well as information about defects, their frequency, and quality requirements. As our main contribution, we propose a framework that combines detection, classification, and tracking in a synergistic way to assist the automotive industry. We use a state-of-the-art convolutional neural network, known as RetinaNet, to detect and classify imprint defects. We explore the temporal coherence in consecutive frames by tracking detected regions so as to reduce false alarms - unstable candidate regions that are rarely (re)detected many times - or to fix the classification of regions that are alternated classified as mild or severe imprint across frames. In our experiments, we achieve a mean average precision of 76% to detect and classify mild and severe defects, outperforming state-of-the-art detectors for static images. For severe imprints only, we achieve precision and recall values of 90% and 92%, respectively. These are promising results that could also benefit other industrial applications such as inspection of fissures, holes, wrinkles, and scratches, that also use image sequences.