A Multi-Signal Perception Network for Textile Composition Identification

Textile composition identification (TCI) is an essential basic link in the textile industry. Methods based on computer vision or near-infrared (NIR) signal processing have shown potential for the nondestructive TCI task. However, these methods ignore that the integration of NIR signals and visual information may help the model learn a better representation through information complementarity. This paper propose a Multi-Signal Perception Network (MSPNet) for nondestructive textile composition identification, allowing the model to benefit from the advantages of multimodal data. Firstly, a two-way feature extraction network is used to obtain multi-modal features. After that, we propose a multimodal signal fusion module to control the aggregation granularity among multimodal data. Specifically, the target areas of the image are perceived by a target area perception module (TAP). Then a bi-gated aggregation (Bi-GFA) is designed to capture consistent semantic information from signal to image and image to signal. The quantitative and qualitative results of the proposed MSPNet are significantly improved compared to both single and multimodal approaches.