Weakly Supervised Instance Segmentation of Electrical Equipment Based on RGB-T Automatic Annotation

To address the problem of weakly supervised instance segmentation for electrical equipment using only a red, green, and blue (RGB) camera, an automatic annotation of masks of samples (AAMS) method based on thermal image guidance is proposed in this article. With only image-level label supervision, we exploit foreground segmentation results of thermal images to guide the instance mask extraction of electrical equipment in RGB images through the heterogeneous pixel registration algorithm between RGB-thermal (RGB-T) image pairs. It is realized to automatically annotate instance masks, which greatly improves efficiency and decreases costs. In addition, we further propose a progressively optimized model (POM) for instance segmentation, which first utilizes the fully connected conditional random field (CRF) and the constrain-to-boundary loss to specify fine-detailed boundaries of each object and to solve the difficulty of segmenting electrical equipment with complicated structures. This model also explores the self-paced learning technology to solve the issue of resolution differences between RGB-T image pairs for improving the generalization ability. By comparison to the other state-of-the-arts, experimental results show that our method can obtain by far the better performance on the electrical equipment data set.