TS-Net: two-stream network based on multispectral feature fusion

As artificial intelligence technology is rapidly developing, related industries that are based on pedestrian target detection technology are being adopted around the world in a variety of areas, such as autonomous vehicles and smart monitoring systems. However, there is a problem with traditional pedestrian target detection methods, a single visible spectrum can reduce the accuracy of pedestrian detection when it is affected by low light, shadows, rain, fog, or strong light during the day. Similarly, a single infrared spectrum may not be effective if strong light is present during the day. As a solution to the problem of pedestrian detection in complex lighting environments, this paper proposes a two-stream network TS-Net that utilizes multispectral feature fusion to process visible and infrared images simultaneously using two-stream parallel technology combined with the YOLOv4 algorithm. By combining the attention mechanism with the fusion module, the visibility and infrared image features can be fused more efficiently, and the Two-Stream Feature Fusion Module (TSFM) is proposed to enhance the respective features while reducing information loss in the fusion process while strengthening the respective features. For the purpose of demonstrating that this paper’s algorithm is superior to other algorithms available in the field of multispectral pedestrian detection, it is compared with various excellent algorithms in the field, as a result, the TS-Net network proposed in this paper performs better than other algorithms under complex lighting conditions.