UIR-Net: Object Detection in Infrared Imaging of Thermomechanical Processes in Automotive Manufacturing

Thermomechanical processes (TMPs) such as resistance spot welding (RSW) and hot stamping are widely used in automotive manufacturing. Recent advancement in sensing technology has led to an increasing adoption of thermographic cameras to capture the infrared (IR) radiation of a metal part (or component of a part) during its thermomechanical processing or immediately after the process when the part is still hot. Detecting the object(s) of interest from raw IR images is an essential step in analyzing these data. Deep learning (DL) has been a recent success for object detection (OD), but the application of DL-based OD for industrial IR images in manufacturing is largely lagging behind. The major contribution of this work, which is also the distinction from previous OD studies, is the capability of building the OD model with unlabeled IR images, i.e., imaging data without accurate information indicating the object position. The architecture of Unsupervised IR Image Net (UIR-Net) is designed to accommodate the unique characteristics of IR images from TMPs in manufacturing. This study presents a novel method for OD in unlabeled IR images from TMPs. The proposed method, called UIR-Net, consists of two components: label generation and DL model construction. Two case studies from automotive manufacturing, RSW and hot stamping, are reported to demonstrate the feasibility and effectiveness of the proposed method. Note to Practitioners—This article was motivated by the problem of detecting objects such as weld nugget or metal piece in infrared (IR) imaging of thermomechanical processes (TMPs) in automotive manufacturing. The method is applicable to in situ IR images or videos that contain one or more objects to be detected. It only requires that the data are in image form and come from TMPs. Currently, there is no existing deep learning (DL)-based method for generic object detection (OD) in unlabeled IR images from TMPs. The proposed method takes advantages of the recent advancement in DL. This article suggests a systematic approach to build a DL-based OD model, named Unsupervised IR Image Net (UIR-Net), to extract objects from raw IR images collected for TMPs. A step-by-step procedure is given in this article to guide users through label generation, data quality evaluation, and model training to establish the proposed UIR-Net model. Results from resistance spot welding and hot stamping suggest that this approach is feasible and effective. It is one of the few generic OD works designed for manufacturing applications. Simple implementation, feasibility, and effectiveness make this method a suitable candidate for online data analytics and process monitoring in a wide range of manufacturing applications.