A real-time load positioning and recognition system integrating image segmentation

Real-time positioning and recognition of loads lifted by tower cranes is essential for achieving intelligent tower crane operations in building construction process. Currently, the real-time collection system for load features employs a laser scanning technique, resulting in a large volume of data. It imposes high performance requirements on computing terminals, making it difficult to apply at a large scale in practical construction projects. Also, existing load positioning methods extract load features from the entire environmental dataset under complex construction site environments. It leads to difficulties in feature extraction due to the inclusion of redundant information. Furthermore, existing deep learning-based load recognition models are time-consuming to train. In this paper, a real-time load positioning and recognition system integrating image segmentation is proposed. Firstly, lifted loads of tower crane in construction sites is captured using a spherical network camera, and is transmitted to a computing terminal via the RTSP protocol in real-time. Secondly, the YOLOv5s algorithm is employed to eliminate environmental interference in tower crane surroundings, enabling the load positioning and segmentation based on the hook's location. Furthermore, the MobileNetV2 algorithm is utilized for load recognition to reduce computational cost. Lastly, a remote visual prototype system for load targets is designed and developed, providing real-time remote display and automatic recognition of tower crane loads. The proposed system has demonstrated excellent performance when tested on an engineering site dataset, focusing on tower crane usage in building construction for material handling.