Mel Spectrogram Analysis for Punching Machine Operating State Classification with CNNs

Data driven analysis and optimization of production processes has become a pivotal instrument to use enterprise resources more efficiently and to improve product quality. However, availability and quality requirements still limit the prevalence of big data and learning techniques in industrial applications. Therefore, retrofitting sensors to brownfield systems has been suggested as a solution to acquire relevant real-time process data. In this paper, a low-cost retrofit approach to analyze the operating state of manually operated punching machines based on sound analysis is presented. The machine operating states provide additional information about the metal forming process, required for the enterprise resource planning (ERP) system to optimally schedule orders in prefabrication and plan available resources. As an analysis tool, a transfer learning approach with a convolutional neural network was used to assess data accuracy and prediction results. The input data consists of Mel Spectrogram images acquired by sound sensors retrofitted to the punching machines. The experiments show that the adapted EfficentNet-B0 achieves an accuracy, sensitivity, and precision of approximately 98 % on unseen data in real environment thus demonstrating the applicability of the implemented system.