Rubber Tyred Gantry Automatic Alignment System based on Computer Vision

The existing rubber tyred gantry (RTG) mainly adopts methods such as traditional laser, magnetic nail or other technologies to orient. In view of the shortcomings of the technology, such as single use scenario, limited information and high construction cost, a visual automatic alignment system based on Mask-RCNN is proposed to accomplish the integration of perception and decision-making. The system based on ROS (Robot Operating System) is mainly composed of target detection and segmentation module, motion decision module and visual human- machine interaction interface. The target detection and segmentation module is mainly used to perceive the target, including its foreground segmentation module based on Mask-RCNN and the target orientation module based on traditional image algorithm; The motion decision module is mainly used to control the movement of the tire crane, including motion modeling module and logic control module; The visual human-computer interaction interface is mainly used to adjust system parameters and monitor system operation status, including the back-end used for providing data and the front-end used for data interaction. Finally, the designed visual automatic alignment system has been tested on the tire crane, and the orientation accuracy and operation efficiency have been improved enormously.